Category Archive for: ‘Federated Search’

  • Is Google Good Enough for Medicine?

    Every morning I wake up to a number of Alerts generated by a number of our portals including Biznar, google mugMednar and Yesterday morning one alert with the title – Is Google good enough for Medicine – caught my attention.

    In their editorial commentary in the Journal of Neurology, Neurosurgery and Psychiatry, 3 medical professionals from down-under talk about how Google (now a verb in the Oxford English Dictionary) is changing the way that doctors practice medicine.

    Here’s one anecdote that Dr. Cindy Shin-Yi Lin and her colleagues relate that I found interesting (and scary):

    “In a recent letter, a rheumatologist describes a scene at rounds where a professor asked the presenting fellow to explain how he arrived at his diagnosis, ‘I entered the salient features into Google, and [the diagnosis] popped right up’.”

    The authors of the editorial also talk about:

    “Most clinicians will be familiar with the increasingly frequent scenario of a patient entering the consult room with a sizeable stack of printed webpages containing symptoms, pictures and a dreaded list of potential (and often grave) diagnoses that will undoubtedly commit the clinician to an arduous task of analyzing (and not infrequently, refuting) this information with the ‘cyberchondriac’ patient.”

    If this topic interests you check out this blog article that we published last year – Relying on Google for Science Information is Bad for your Health, and if you want to bring more authoritative stacks of paper to your physician in your next visit try out our freely available medical research site, Mednar which searches 40+ sources of quality medical information all at the same time.

    Oh, and I love that mug!!

  • Explorit Everywhere! Search Builder – Have Search Your Way

    Problem – Your Explorit Everywhere! solution (particularly at academic institutions) is typically setup to search  a wide spectrum of sources that aren’t always relevant to every researcher or student. Special groups (e.g. the Chemistry Department, the Psychology 101 course, or the High Energy Physics Division within a research lab) may prefer to search only a handful of resources, and don’t want to be overwhelmed by results from irrelevant sources. Sound familiar?

    Our elegant solution – Explorit Everywhere! Search Builder
    Each Explorit Everywhere! application comes loaded with Search Builder, a Search-Buildersearch engine creation tool designed to create custom search engines that only search the specific sources your user group needs. For example, an instructor teaching a class on Computer Programming may want students to research responsive design using four selected resources. Rather than tutoring the class in how to use their Explorit Everywhere! Advanced Search and limit the search to the four relevant resources, the instructor can simply create a search engine designed for his students that only search those resources the students should search. Then, he can place a search box to this new engine on his course page to make it even easier for his students.

    There are some Deep Web Technologies customers that create, use, and market only Search Builder engines engines that only search the specific sources your user group needs. For example, an instructor teaching a class on Computer Programming may want students to research responsive design using four selected resources. Rather than tutoring the class in how to use their Explorit Everywhere! Advanced Search and limit the search to the four relevant resources, the instructor can simply create a search engine designed for his students that only search those resources the students should search. Then, he can place a search box to this new engine on his course page to make it even easier for his students.

    Search Builder can create unlimited search engines for one person, groups, teams or even enterprises. Admins can build search engines for each department, students can create their own search portal with a personalized search box, professors can modify search engines for their classes, and teams can search the most relevant resources and delve into their critical materials most efficiently.

    Search Builder can be integrated into:

    • Learning Management Systems
    • LibGuides
    • Corporate Portal
    • Websites
    • Blogs
    • CRM Systems

     Help your researchers be be efficient – Use Search Builder. And, if your organization uses one of the pesky Discovery Services which don’t allow, or at least don’t make it easy to limit your search to relevant resources, Search Builder is a great reason to switch to Explorit Everywhere!

  • Is Federated Search Used in Pharma?

    I just came across this Fall 2014 survey conducted by one of our competitors who shall remain nameless (at least until you click on the link at the end of this blog post) which I found interesting and wanted to share with our readers.

    stethoscope lying on keyboard of a laptop

    Our competitor surveyed members of SLA (Special Library Association), mostly members of the PHT (Pharmaceutical & Health Technology) Division on whether they use federated search today and if not would they find federated search useful and what features would such a federated search solution need to have.

    Question 2 of the 6 question survey asked – “Does your information center provide a ‘federated search’ function that allows users to search *all* of your organization’s online content resources with a single query? 81% of the respondents said “No”.

    Explanations as to why federated search has not been more adopted in pharmaceutical companies, even though these companies have a wealth of content to access (and accessing this content is not so easy), seems to have to do at least to a large extent, with the lack of IT cooperation with the Knowledge Management or Information Services Group.

    Marydee Ojala, Editor-in-Chief of Online Searcher and one of my favorite people at Information points out that:

    “Searchability of electronic resources has long been piecemeal, but a federated search solution must take into account the IT infrastructure already in place.”

    I have certainly been on a number of prospect calls which included IT where the was a lack of understanding of the power of federated search and a reluctance to add yet another tool to the set of tools/software that IT needs to support. What I propose to many prospects is that we start with a solution that is hosted by us in the cloud, minimizing the involvement of IT. Once the value of our service is proven to our customer if they now want to add internal content to their Explorit Everywhere! subscription and public content that we are searching from the cloud then we can move our solution to servers sitting behind the customer’s firewall.

    When asked in question 5 of the survey, what respondents thought were the main drawbacks of not having federated search, the answers – which I was very happy to see – included:

    87.50 % — Time spent looking multiple places for information

    71.88 %  — Missed information / opportunities due to “inexpert searching”

    68.75 % — Reduced usage of online information sources

    65.63 %  — Over-reliance on search engines as primary research tool

    All-in-all this is a very interesting survey.  The survey results and analysis is available starting on page 18 of the Fall 2014 CapLits newsletter.

  • Co-opetition with the Discovery Services

    A couple of weeks ago I woke up to an email message from one of our partners in Europe asking if we handshakecould federate the enclosed list of sources for one of their prospects. Before I had a chance to respond to his message, my partner followed with a second email saying that he forgot to include the prospect’s EBSCO Discovery Service as one of the sources for us to federate. As I reviewed the list of sources that we would need to federate for this prospect I found that a couple of these sources were Ex-Libris Primo Discovery Services.

    What a great example of potential co-opetition, or “cooperating with one’s competitors”. In this case, co-opetition with EBSCO and Ex-Libris (now part of ProQuest) to build a comprehensive solution for a customer that provides one-stop access to content from 3 different Discovery Services as well as some additional sources, something that neither EBSCO nor Ex-Libris could do. This use case gives new meaning to my earlier blog post on Federating the Un-Federatable.

    Taking a look at the major Discovery Services we find that Summon has always been a pure Discovery Service, choosing not to complement their Discovery Service with federated search (even though they acquired two federated search companies – WebFeat and Serials Solutions). EDS and Primo have been hybrid services, enabling, although not so well, federation of sources not available in their indices. We’ve seen both EDS and Primo de-emphasize federated search in their Discovery Service, perhaps because federated search is not so easy to do well if it is not your product’s primary focus. OCLC’s WorldCat Discovery Service does not incorporate federation as part of their Discovery Service.

    So this opportunity to build a solution for this project that federates 3 Discovery Services and some other sources has energized me, David, to reach out once again to the Discovery Services Goliaths.  I have had numerous conversations with customers and prospects where I have heard repeatedly that important content is missing from their Discovery Service. I want to see if now is a good time to do some co-opetition that is a win-win for everyone, especially for the user who wants one-stop access to all of the content that they need and doesn’t care how that content is aggregated.


  • – a Cut Above the Web just received yet another pat on the back from the blog Inside Science Resources. WorldWideScience a Cut above the World Wide Web – Inside Science ResourcesKhue Duong, Science Librarian at California State University, Long Beach, posted a review titled, “WorldWideScience, a Cut above the World Wide Web.” Yes, the title summarizes it well.

    Users are often surprised to learn that the engine behind is a federated search engine, bursting the myth that federated search is an antiquated, dusty bit of technology that doesn’t aggregate source results and is dreadfully slow.  Here’s the first sentence of the review:

    Are you still looking for a reliable federated search tool that goes beyond the run-of-the-mill results?

    Enter Deep Web Technologies’ next-generation federated search, Explorit Everywhere!, the powerhouse behind In the review, Khue Duong explored the search functionality (including Advanced Search) on, and performed four searches to test the application results. Three of the tests performed well, returning relevant results:

    Without displaying the additional results, searching “Isle Royale National Park” as Full Record yields 66 papers and 49 data sources. Regarding papers (document type: articles, reports, etc.), the first 40 displayed results seem to be to the point; the rest probably has some mentioning of the search term in its document. The first twenty data results has some mentioning of Isle Royale National Park in the title. The rest, such as the DNA sequences from the DNA Data Bank of Japan, shows that the sample originates from Isle Royale.

    One of the four searches didn’t yield as many relevant results.

    Note that adding an additional layer of parentheses in the search of (“isle royale national park” AND (wolf OR wolves)) also yields many misleading results in all three categories: papers, multimedia and data.

    This is hardly surprising. On public search portals like, many sources do not support title search well. Trying the above search as a full-record search brought back many good results, including the top-ranked results containing all of the search terms in the title.

    We would also like to point out that the additional search results should be displayed when the search completes as these additional results may contain some of the most relevant results available. Also displaying the additional results should significantly reduce the variability in the results returned by running the same search multiple times.

    Khue Duong, however, still recommends as a science resource for researchers:

    Overall is another one-stop-shopping platform that one should consider when searching for publications, data, and media files from governmental and international scientific domains.

    Over the last year, has garnered lots of attention as not only an excellent search portal for scientific information, but a multilingual search portal as well. In January 2015, Microsoft published a Case Study about A few months later, Deep Web Technologies published an article in magazine. And isn’t the only portal using our federated search. Explorit Everywhere! is used for several other public search portals, such as and

    We’re proud of and proud to support the DOE Office of Scientific and Technical Information (OSTI) in bringing this free, publicly available, multilingual search portal to science and technology researchers around the world.

  • What Am I Searching (And Do I Care?)

    Indexed discovery services are a lot like Google: you can search simply and get results quickly, but the sources you are rocky-trailsearching tend to be mysterious. Do you know if you are searching specialized sources or generic sources? Authoritative or with an agenda? When a researcher pushes the search button, they get whatever results are deemed relevant from whatever sources are included, and they can’t limit their search to only the sources that matter to them.

    For some researchers, knowing the sources behind the search really makes no difference to them at all. To these researchers, often undergraduates, it’s the results that count. Most results nowadays do show the source, publication or journal that result is from. This makes it somewhat easier to eyeball a page of results, disregard those from irrelevant sources, or select results as appropriate if they are from an authoritative source. But that research methodology seems inefficient to say the least.

    Serious researchers, on the other hand, want to know what they are searching. If they know that their information will most likely be in three or four specific resources out of the 20 or 30 their organization subscribes to, then why should they wade through a massive results list or spend one iota of extra time filtering out the extra sources to view their nuggets of information? (Quick answer, they shouldn’t.)

    To begin to combat this resource transparency problem, libraries are creating separate web pages of source lists and descriptions for serious researchers. Who is the provider? What is the resource and what information exactly does the resource provide? These pages also include the categories of sources searched such as ebooks, articles, multimedia collections, etc. While these lists of resources are certainly helpful to document, is it fair to ask researchers to reference a separate web page to understand what digital content is included in their search, particularly when they are urgently trying to find something from a particular source of information? Or should we ask researchers to disregard knowing what sources they are searching and to just pay attention to the results?  Neither of these seems appropriate in this day and age.

    Most of us know the benefits of a single search of all resources. One search improves the efficiency of searching disparate sources, makes comparing and contrasting results faster, and provides an opportunity to save, export or email selected results. However, Explorit Everywhere! goes one step further by lending transparency of sources to researchers so they can search even faster. One of our customers mentioned that they moved from a well known discovery service because they were frustrated with all of the news results that were returned. It didn’t help that their researchers couldn’t select specific sources to search, particularly when their searches always seemed to bring back less than relevant results.

    Explorit Everywhere! helps to narrow a search up front with not only the standard Advanced Search fields, but a list of sources to pick and choose from. A researcher looking to search in four different sources doesn’t want to run a search against 25 sources. They can narrow the playing field to hone in on the needle in the haystack faster. And from the results page, they can limit to each individual source to view only those results, in the order that the source ranked them.  A serious researcher’s dream? That’s what we’ve heard

    Not all researcher’s care about drilling down into individual sources like this. But in Explorit Everywhere! the option is there to search the broad or narrow path. We even filter out the rocks.

    In a future blog article we’ll talk about the other side of this question: What content am I missing?

  • Search’s Unsatisfactory Progress

    Simon Bain, CEO of SearchYourCloud recently wrote an article for IT ProPortal, “The search continues: A progresshistory of search’s unsatisfactory progress.” In his article, Simon expands on a survey conducted by SearchYourCloud in which a third of respondents said they “spend between five and 25 minutes searching every time they want to find a document, while only one in five searches is correct the first time. The search for corporate information is eating into workplace productivity. Only 20 per cent of respondents reported first time successful searches. Other key findings from the survey include that it takes workers up to eight searches to find the right document and information, according to 80 per cent of respondents.”

    Search, Simon comments, hasn’t really improved over the last 20 years. Companies such as Microsoft and Autonomy, while assuming the mantle of usable search through the acquisition of search veterans like Verity and FAST, actually focused their efforts on consultancy rather than on improving results retrieval.

    The introduction of Big Data, however, has offered a twist to the information retieval conundrum. Piles of data, lots of repositories, different security levels and multiple devices all funnel down to researchers who must adapt their search to the search interface of the individual database or repository, meaning they are often searching databases one by one by one.

    We call this serial searching, and it is hardly efficient. Add to this that not all researchers know where to look, know the search syntax of the databases they are searching, or have time to search all of their databases.

    Luckily, federated search have arrived on scene, assessed the situation and now offer alternatives to serial searching.

    With the advent of federated search, the ability to search across multiple repositories has improved. Moreover, with federated de-duplicated results, users do not receive thousands of irrelevant documents or emails. Users can simultaneously search across applications. It is best to take a non-repository processing approach and keep the existing data silos separate. A large repository can be kludgy with inherent security risks and to combine multiple silos may create problems in reconciling different processing power and security levels.

    Deep Web Technologies’ customers see increased efficiency every day using Explorit Everywhere!, so we second Simon’s evaluation of how federated search improves the overall picture of modern search. Our customers tell us that they find exactly what they are looking for in seconds or minutes. We are constantly evaluating our efforts to rapidly return the right results from across multiple repositories. And as data grows, so do we.

    Federated Search is still moving search along with it’s adaptable infrastructure. It’s like it was made for modern, dynamic, fast-growing data streams.

    Read the full article by Simon Bain here.

  • MyLibrary – Explorit Everywhere!’s Filing Cabinet

    At Deep Web Technologies we think that Explorit Everywhere! is the simplest way for researchers to perform a single search for results MyLibraryacross their various information sources, find relevant results and seamlessly process those results. But it’s that final part – processing results – that DWT just made simpler for our researchers through our new MyLibrary feature. MyLibrary has evolved over the years from Citations Manager, to My Selections, and now to the more full-featured MyLibrary.

    A better way to think of MyLibrary, however, is as your library – A place to store your digital results, reference and organize your data, or share information with others. It’s a place to build your research world.

    Whether collaborative or solitary, your research can consume big chunks of time out of a day. If you’re moving fast, you may browse dozens of results rapidly and set some aside to explore further. Those results might hold some interest for you on a topic separate from what you are currently researching and you want explore them at a later date. They might be relevant, but need additional follow up. Or maybe your workday has ended, and you’ll resume researching again tomorrow. Whatever the reason, MyLibrary allows you to put your research on hold without missing a beat.

    Your results can be filed away similar to organizing folders in a filing cabinet. Once you’ve selected your results, MyLibrary lets you tag and categorize those results in a system that makes sense to you. Reference those saved results whenever you want – they are there until you delete them. Just log in to your Explorit Everywhere! application and click MyLibrary. When you are ready, you can print, email or export any or all of those results from any folder, at any time.

    Oh, and don’t think our ambitious engineers could resist tweaking a few other things while they were fiddling with MyLibrary in Explorit Everywhere!. Now, once you log in, you can save results preferences and language preferences in localized applications. Have an idea to make MyLibrary even better? Let us know!

  • Google Is All I Need…Oh Really?

    When we’re young and naive, we think that Google knows everything, from climate change to competitor data, from media impressions to military intelligenceGoogle-myth. If it’s not in Google, then we won’t find it anywhere. Thanks, and have a nice day.

    This unfortunate misconception is the plight many libraries face today: How do you reclaim your library from the overpowering presence of Google? This question isn’t just a philosophical question.  Authoritative sources of information that libraries subscribe to (yes, actually spend money on) are searched by uniformed researchers 16% of the time. This is a paltry number compared to 94% of researchers who will begin their search (and often complete their search) on Google or another generic search engine when looking for authoritative information.

    Brainwashing powers aside, Google has captured researchers with speedy results that seem to be the “right” results every time. And, to give credit where credit is due, Google is an amazing, ever evolving search engine that is the perfect search for everyday queries. It also is an excellent place to start broad research that doesn’t need to be supported by vetted information.

    But there comes a time when researchers outgrow Google…

    Libraries are central to a researcher’s quests for knowledge. They should be a font of information, housing books, magazines and catalogs, and directing researchers to the external resources to fuel and deliver on these information requests.  Maturing and expert researchers who realize that Google falls short of their information needs shouldn’t be limited by a library that isn’t vehemently countering the Google myth. A self-aware library will be asking these questions:

    • How do we support our researchers in their quest for authoritative information?
    • How can we make it simple for researchers to transition from Google to more authoritative sources?
    • How do we ensure that our authoritative information sources are getting found and used?
    • If we have a single search of all of our information sources, are they being ranked in an unbiased way (unlike Google with popular ranking)?

    Google has made the search and retrieval process easy. Good search engines follow that pattern too – simple and elegant, yet robust.

    If you’re a librarian facing Google encroachment in your libary, consider resurrecting yourself to a position of authority. Find a way to go the distance for your researchers.

    Click here for a presentation on “Taking the Library Back from Google.” 

  • Have It Your Way

    What makes a search app easy to use? Lots of things, such as intuitive navigation, a sleek interface, tooltips, and…personalization. Sometimes spending the extra time making an app 2000px-Chat_bubbles.svg“yours” is exactly what it takes to make the app more “usable.”

    For example, the first thing I do when I find a new start page or website that I’ll be using robustly every day is to personalize it. I add in the widgets that I know will help me out on my quest for efficiency. I change the colors to reflect my mood. I alter the text to cue me to next actions. When I tailor apps to my tastes, I inevitable use them more because they are more comfortable to use.

    At Deep Web Technologies, we want our customers to feel comfortable with their Explorit Everywhere! applications. We tailor your application to your organization’s style, administrative and researcher’s needs – make it the search application your organization needs to use it comfortably and use regularly.

    We have many customers who spend the time to tailor their applications to their researcher’s preferences or organization’s style.

    Here’s an example of a unique customization recently implemented:
    Our customer, the Virginia Department of Transportation (VDOT), wanted to install a chat widget into their application that appears on the right side of the page and connects users with a librarian. It was a simple piece of javascript to install on the back-end, but on the front end allows users to get immediate answers from library staff, is connected to a ticketing system for issue tracking and personalizes the application so that their users can connect quickly and the librarians can monitor questions.

    I had the opportunity to try it out. Within seconds, a librarian was online chatting with me. When done, I received a transcript of our conversation and was pleasantly surprised at the “human” element of the search page. VDOT had personalized their Explorit Everywhere! application in a way that created comfort, provided excellent service and immediately connected the user to the library.

    “We were pleased that Deep Web Technologies was responsive to our need to integrate another vendor’s chat software into our federated search solution.  We tested in a developmental environment first, saw no problems, and brought it into our live site right on schedule,” said Ken Winter, Associate Library Director, VDOT Research Library. “The result is that if our patrons using Deep Web have a question, they can initiate a live chat with library staff and get answers in seconds.”

    Our customer successes are our successes. Whether it’s adding google analytics to track users, a custom thesaurus, plugging in different widgets for document delivery or a chat widget, we’ll help you to tailor your application so that your users can best take advantage of Explorit Everywhere! to find information quickly.

Page 2 of 5«12345»