tech-ed collisions

« Back to blog

Open source distributed search manager ready to go

For a number of years now education.au has been providing 'search' services to the education community in a number of its projects. The best known of these is 'edna'. edna's search functionality started out as a collection of metadata records describing web resources that would be of value to people in the education and training sector. With the explosive growth of the Web it soon became obvious that many more high-value and relevant resources were being created than could possibly be catalogued (using agreed metadata application profiles and controlled vocabularies) by a single organisation. At the same time a number of other, related organisations were populating their own repositories with similar records. edna, in agreement with the owners of those collections, started harvesting metadata from them and making all these resources discoverable from a single search. For a number of reasons, it was not practical or possible to harvest metadata from a number of other repositories however the notion of having a single search interface to many collections was, and is, very compelling. education.au set about developing software to achieve this. The result was a software application known as DSM (Distributed Search Manager). A number of our other stakeholders were facing the same challenges and DSM was deployed to address these challenges too. DSM is now used in a number of Australian education related search services and has also been used in New Zealand and other industry areas (such as natural resource management) where the same issues are being faced. The projects we have engaged in have enabled us to gain considerable experience with a number of repository interfaces and also standards/specifications that are commonly used in Search and repositories (especially in the education sector). The software has evolved to a point where we believe it could be of great benefit to other organisations facing the challenge of searching across multiple repositories, databases and search engines. We are very pleased to release this distributed search manager as open source and it is now available and known as openDSM. openDSM is located on GoogleCode at http://code.google.com/p/distributed-search-manager/. We encourage those that are interested to have a look at DSM, see it in use(the demo application simply accesses three open collections however we have developed adapters for approximately forty different collections so far), deploy it or even contribute towards the ongoing development of it. We hope that this software will be the first of a number of technologies that we can contribute as open source to the education community in particular.

technorati tags: , ,