Deep Web Tech Blog

  • Include Libraries in Learning Management Systems, Don’t EXCLUDE Them!

    Learning Management Systems (LMS) are web based technologies used to offer online classes. The most common LMS are Blackboard, Canvas, Moodle, and Brightspace (previously known as Desire2Learn). And there are many more edging into the market because the idea of providing an online educational portal is really taking off. It’s one of the fastest-growing markets out there!learn-868815_960_720

    These LMS are used not only for online classes, but also for the traditional, face-to-face classrooms as an online portal for managing course materials, for offering bulletin boards for online discussions, and for turning in assignments. Given this growing, ubiquitous need for any classroom (including K-12, corporate, etc.) to cultivate an online presence, you would think that one of the central features of these LMS would be to allow for easy incorporation of the library into their online portals since every learning setting has one. But alas, this is far from the truth. In fact, it’s just the opposite. (See Libraries in the Learning Management System, Farkas, 2015)

    For most of these LMS, they have no easy user interface modules that allow instructors to include library tools such as access to librarians (e.g., Ask a Librarian), subject focused web pages (e.g., LibGuides), much less a module for students to search the school catalog and the numerous available research databases. Often the best the instructors can offer is a link to the library website as an additional resource. The goal, however, should be to bring the library into these online portals, not have the student leave them.

    At Deep Web Technologies, we have been researching how to improve this situation for our customers since the primary goal of our federated search tool, Explorit Everywhere!, is to help librarians provide students with easy, quick, and direct access to their subscription content. We found that for most LMS, if they have tools to support third-party content, it is an additional cost to access these features to incorporate library materials. Then, after the school does add these additional LMS features, the librarian has to do further work to get access to the course management interfaces and do the work to develop these modules. Once all of this is in place, the librarians will most likely need to work with the instructors to coordinate adding their library tools to their course management pages. Obviously, such an ordeal does not scale for the hundreds of instructors at larger institutions.

    Every LMS should automatically support incorporating the library as one of their base features. For any LMS, it should be easy for an instructor to configure an Explorit Everywhere! Search widget so students have easy access to the library’s catalog and subscription content. Moveover, the instructor can use the Explorit Everywhere! Search Builder to customize their search widget to search specific course-related databases. Librarians work hard to make their library useful and patron-friendly. Learning Management Systems need to work with them, not put up more barriers.

  • Promoting Explorit Everywhere! at the University of the Arts London

    This past weekend I was writing an email message to one of our new customers. I was sending this customer, as promised, UAL
    some example uses of Search Builder, our Explorit Everywhere! tool that makes it easy to create custom search widgets that can be dropped on any web page.

    I started my quest for Search Builder widgets by going to the library website of one of our favorite customers, the University of the Arts, London (UAL). UAL is one of our favorite customers, in part, because they do a lot to promote Explorit Everywhere! to their faculty and students across their 6 colleges.

    I digress, but last December (2015) I had the pleasure to attend the User Conference for one of our partners, PTFS Europe, and at this conference Paul Mellinger, Discovery Manager at the UAL Library gave a wonderful presentation on their implementation of Explorit Everywhere! and followed that with a guest article on the Customer Corner section of our company blog.

    Back to the task at hand, I went to UAL’s Library Subject Guides page and found that they have created 39 subject guides on subjects such as Beauty and Cosmetic Sciences, Graphic Design and Jewellery, subjects that parallel the way that they have organized the sources that we federate for them. If you go into many of these subject guides then click on the e-Resources tab you’ll see at the top left of the page an Articles Plus Search widget (this is what UAL calls Explorit Everywhere!). Each of the widgets on the various e-Resources pages searches a different, appropriate subset of all the sources that we federate for them.

    Then to top it off, I came across the Articles Plus Help subject guide (which I had not seen before), a wonderful guide, nicely organized into tabs for Simple Search, Advanced Search, Search Results and Tools and Navigation. Also added recently to this subject guide is a 3 and a half minute video tutorial on Articles Plus. This is why we love UAL!!

  • Is Google Good Enough for Medicine?

    Every morning I wake up to a number of Alerts generated by a number of our portals including Biznar, google mugMednar and Yesterday morning one alert with the title – Is Google good enough for Medicine – caught my attention.

    In their editorial commentary in the Journal of Neurology, Neurosurgery and Psychiatry, 3 medical professionals from down-under talk about how Google (now a verb in the Oxford English Dictionary) is changing the way that doctors practice medicine.

    Here’s one anecdote that Dr. Cindy Shin-Yi Lin and her colleagues relate that I found interesting (and scary):

    “In a recent letter, a rheumatologist describes a scene at rounds where a professor asked the presenting fellow to explain how he arrived at his diagnosis, ‘I entered the salient features into Google, and [the diagnosis] popped right up’.”

    The authors of the editorial also talk about:

    “Most clinicians will be familiar with the increasingly frequent scenario of a patient entering the consult room with a sizeable stack of printed webpages containing symptoms, pictures and a dreaded list of potential (and often grave) diagnoses that will undoubtedly commit the clinician to an arduous task of analyzing (and not infrequently, refuting) this information with the ‘cyberchondriac’ patient.”

    If this topic interests you check out this blog article that we published last year – Relying on Google for Science Information is Bad for your Health, and if you want to bring more authoritative stacks of paper to your physician in your next visit try out our freely available medical research site, Mednar which searches 40+ sources of quality medical information all at the same time.

    Oh, and I love that mug!!

  • Explorit Everywhere! Search Builder – Have Search Your Way

    Problem – Your Explorit Everywhere! solution (particularly at academic institutions) is typically setup to search  a wide spectrum of sources that aren’t always relevant to every researcher or student. Special groups (e.g. the Chemistry Department, the Psychology 101 course, or the High Energy Physics Division within a research lab) may prefer to search only a handful of resources, and don’t want to be overwhelmed by results from irrelevant sources. Sound familiar?

    Our elegant solution – Explorit Everywhere! Search Builder
    Each Explorit Everywhere! application comes loaded with Search Builder, a Search-Buildersearch engine creation tool designed to create custom search engines that only search the specific sources your user group needs. For example, an instructor teaching a class on Computer Programming may want students to research responsive design using four selected resources. Rather than tutoring the class in how to use their Explorit Everywhere! Advanced Search and limit the search to the four relevant resources, the instructor can simply create a search engine designed for his students that only search those resources the students should search. Then, he can place a search box to this new engine on his course page to make it even easier for his students.

    There are some Deep Web Technologies customers that create, use, and market only Search Builder engines engines that only search the specific sources your user group needs. For example, an instructor teaching a class on Computer Programming may want students to research responsive design using four selected resources. Rather than tutoring the class in how to use their Explorit Everywhere! Advanced Search and limit the search to the four relevant resources, the instructor can simply create a search engine designed for his students that only search those resources the students should search. Then, he can place a search box to this new engine on his course page to make it even easier for his students.

    Search Builder can create unlimited search engines for one person, groups, teams or even enterprises. Admins can build search engines for each department, students can create their own search portal with a personalized search box, professors can modify search engines for their classes, and teams can search the most relevant resources and delve into their critical materials most efficiently.

    Search Builder can be integrated into:

    • Learning Management Systems
    • LibGuides
    • Corporate Portal
    • Websites
    • Blogs
    • CRM Systems

     Help your researchers be be efficient – Use Search Builder. And, if your organization uses one of the pesky Discovery Services which don’t allow, or at least don’t make it easy to limit your search to relevant resources, Search Builder is a great reason to switch to Explorit Everywhere!

  • Is Federated Search Used in Pharma?

    I just came across this Fall 2014 survey conducted by one of our competitors who shall remain nameless (at least until you click on the link at the end of this blog post) which I found interesting and wanted to share with our readers.

    stethoscope lying on keyboard of a laptop

    Our competitor surveyed members of SLA (Special Library Association), mostly members of the PHT (Pharmaceutical & Health Technology) Division on whether they use federated search today and if not would they find federated search useful and what features would such a federated search solution need to have.

    Question 2 of the 6 question survey asked – “Does your information center provide a ‘federated search’ function that allows users to search *all* of your organization’s online content resources with a single query? 81% of the respondents said “No”.

    Explanations as to why federated search has not been more adopted in pharmaceutical companies, even though these companies have a wealth of content to access (and accessing this content is not so easy), seems to have to do at least to a large extent, with the lack of IT cooperation with the Knowledge Management or Information Services Group.

    Marydee Ojala, Editor-in-Chief of Online Searcher and one of my favorite people at Information points out that:

    “Searchability of electronic resources has long been piecemeal, but a federated search solution must take into account the IT infrastructure already in place.”

    I have certainly been on a number of prospect calls which included IT where the was a lack of understanding of the power of federated search and a reluctance to add yet another tool to the set of tools/software that IT needs to support. What I propose to many prospects is that we start with a solution that is hosted by us in the cloud, minimizing the involvement of IT. Once the value of our service is proven to our customer if they now want to add internal content to their Explorit Everywhere! subscription and public content that we are searching from the cloud then we can move our solution to servers sitting behind the customer’s firewall.

    When asked in question 5 of the survey, what respondents thought were the main drawbacks of not having federated search, the answers – which I was very happy to see – included:

    87.50 % — Time spent looking multiple places for information

    71.88 %  — Missed information / opportunities due to “inexpert searching”

    68.75 % — Reduced usage of online information sources

    65.63 %  — Over-reliance on search engines as primary research tool

    All-in-all this is a very interesting survey.  The survey results and analysis is available starting on page 18 of the Fall 2014 CapLits newsletter.

  • Co-opetition with the Discovery Services

    A couple of weeks ago I woke up to an email message from one of our partners in Europe asking if we handshakecould federate the enclosed list of sources for one of their prospects. Before I had a chance to respond to his message, my partner followed with a second email saying that he forgot to include the prospect’s EBSCO Discovery Service as one of the sources for us to federate. As I reviewed the list of sources that we would need to federate for this prospect I found that a couple of these sources were Ex-Libris Primo Discovery Services.

    What a great example of potential co-opetition, or “cooperating with one’s competitors”. In this case, co-opetition with EBSCO and Ex-Libris (now part of ProQuest) to build a comprehensive solution for a customer that provides one-stop access to content from 3 different Discovery Services as well as some additional sources, something that neither EBSCO nor Ex-Libris could do. This use case gives new meaning to my earlier blog post on Federating the Un-Federatable.

    Taking a look at the major Discovery Services we find that Summon has always been a pure Discovery Service, choosing not to complement their Discovery Service with federated search (even though they acquired two federated search companies – WebFeat and Serials Solutions). EDS and Primo have been hybrid services, enabling, although not so well, federation of sources not available in their indices. We’ve seen both EDS and Primo de-emphasize federated search in their Discovery Service, perhaps because federated search is not so easy to do well if it is not your product’s primary focus. OCLC’s WorldCat Discovery Service does not incorporate federation as part of their Discovery Service.

    So this opportunity to build a solution for this project that federates 3 Discovery Services and some other sources has energized me, David, to reach out once again to the Discovery Services Goliaths.  I have had numerous conversations with customers and prospects where I have heard repeatedly that important content is missing from their Discovery Service. I want to see if now is a good time to do some co-opetition that is a win-win for everyone, especially for the user who wants one-stop access to all of the content that they need and doesn’t care how that content is aggregated.


  • – a Cut Above the Web just received yet another pat on the back from the blog Inside Science Resources. WorldWideScience a Cut above the World Wide Web – Inside Science ResourcesKhue Duong, Science Librarian at California State University, Long Beach, posted a review titled, “WorldWideScience, a Cut above the World Wide Web.” Yes, the title summarizes it well.

    Users are often surprised to learn that the engine behind is a federated search engine, bursting the myth that federated search is an antiquated, dusty bit of technology that doesn’t aggregate source results and is dreadfully slow.  Here’s the first sentence of the review:

    Are you still looking for a reliable federated search tool that goes beyond the run-of-the-mill results?

    Enter Deep Web Technologies’ next-generation federated search, Explorit Everywhere!, the powerhouse behind In the review, Khue Duong explored the search functionality (including Advanced Search) on, and performed four searches to test the application results. Three of the tests performed well, returning relevant results:

    Without displaying the additional results, searching “Isle Royale National Park” as Full Record yields 66 papers and 49 data sources. Regarding papers (document type: articles, reports, etc.), the first 40 displayed results seem to be to the point; the rest probably has some mentioning of the search term in its document. The first twenty data results has some mentioning of Isle Royale National Park in the title. The rest, such as the DNA sequences from the DNA Data Bank of Japan, shows that the sample originates from Isle Royale.

    One of the four searches didn’t yield as many relevant results.

    Note that adding an additional layer of parentheses in the search of (“isle royale national park” AND (wolf OR wolves)) also yields many misleading results in all three categories: papers, multimedia and data.

    This is hardly surprising. On public search portals like, many sources do not support title search well. Trying the above search as a full-record search brought back many good results, including the top-ranked results containing all of the search terms in the title.

    We would also like to point out that the additional search results should be displayed when the search completes as these additional results may contain some of the most relevant results available. Also displaying the additional results should significantly reduce the variability in the results returned by running the same search multiple times.

    Khue Duong, however, still recommends as a science resource for researchers:

    Overall is another one-stop-shopping platform that one should consider when searching for publications, data, and media files from governmental and international scientific domains.

    Over the last year, has garnered lots of attention as not only an excellent search portal for scientific information, but a multilingual search portal as well. In January 2015, Microsoft published a Case Study about A few months later, Deep Web Technologies published an article in magazine. And isn’t the only portal using our federated search. Explorit Everywhere! is used for several other public search portals, such as and

    We’re proud of and proud to support the DOE Office of Scientific and Technical Information (OSTI) in bringing this free, publicly available, multilingual search portal to science and technology researchers around the world.

  • What Am I Searching (And Do I Care?)

    Indexed discovery services are a lot like Google: you can search simply and get results quickly, but the sources you are rocky-trailsearching tend to be mysterious. Do you know if you are searching specialized sources or generic sources? Authoritative or with an agenda? When a researcher pushes the search button, they get whatever results are deemed relevant from whatever sources are included, and they can’t limit their search to only the sources that matter to them.

    For some researchers, knowing the sources behind the search really makes no difference to them at all. To these researchers, often undergraduates, it’s the results that count. Most results nowadays do show the source, publication or journal that result is from. This makes it somewhat easier to eyeball a page of results, disregard those from irrelevant sources, or select results as appropriate if they are from an authoritative source. But that research methodology seems inefficient to say the least.

    Serious researchers, on the other hand, want to know what they are searching. If they know that their information will most likely be in three or four specific resources out of the 20 or 30 their organization subscribes to, then why should they wade through a massive results list or spend one iota of extra time filtering out the extra sources to view their nuggets of information? (Quick answer, they shouldn’t.)

    To begin to combat this resource transparency problem, libraries are creating separate web pages of source lists and descriptions for serious researchers. Who is the provider? What is the resource and what information exactly does the resource provide? These pages also include the categories of sources searched such as ebooks, articles, multimedia collections, etc. While these lists of resources are certainly helpful to document, is it fair to ask researchers to reference a separate web page to understand what digital content is included in their search, particularly when they are urgently trying to find something from a particular source of information? Or should we ask researchers to disregard knowing what sources they are searching and to just pay attention to the results?  Neither of these seems appropriate in this day and age.

    Most of us know the benefits of a single search of all resources. One search improves the efficiency of searching disparate sources, makes comparing and contrasting results faster, and provides an opportunity to save, export or email selected results. However, Explorit Everywhere! goes one step further by lending transparency of sources to researchers so they can search even faster. One of our customers mentioned that they moved from a well known discovery service because they were frustrated with all of the news results that were returned. It didn’t help that their researchers couldn’t select specific sources to search, particularly when their searches always seemed to bring back less than relevant results.

    Explorit Everywhere! helps to narrow a search up front with not only the standard Advanced Search fields, but a list of sources to pick and choose from. A researcher looking to search in four different sources doesn’t want to run a search against 25 sources. They can narrow the playing field to hone in on the needle in the haystack faster. And from the results page, they can limit to each individual source to view only those results, in the order that the source ranked them.  A serious researcher’s dream? That’s what we’ve heard

    Not all researcher’s care about drilling down into individual sources like this. But in Explorit Everywhere! the option is there to search the broad or narrow path. We even filter out the rocks.

    In a future blog article we’ll talk about the other side of this question: What content am I missing?

  • Search’s Unsatisfactory Progress

    Simon Bain, CEO of SearchYourCloud recently wrote an article for IT ProPortal, “The search continues: A progresshistory of search’s unsatisfactory progress.” In his article, Simon expands on a survey conducted by SearchYourCloud in which a third of respondents said they “spend between five and 25 minutes searching every time they want to find a document, while only one in five searches is correct the first time. The search for corporate information is eating into workplace productivity. Only 20 per cent of respondents reported first time successful searches. Other key findings from the survey include that it takes workers up to eight searches to find the right document and information, according to 80 per cent of respondents.”

    Search, Simon comments, hasn’t really improved over the last 20 years. Companies such as Microsoft and Autonomy, while assuming the mantle of usable search through the acquisition of search veterans like Verity and FAST, actually focused their efforts on consultancy rather than on improving results retrieval.

    The introduction of Big Data, however, has offered a twist to the information retieval conundrum. Piles of data, lots of repositories, different security levels and multiple devices all funnel down to researchers who must adapt their search to the search interface of the individual database or repository, meaning they are often searching databases one by one by one.

    We call this serial searching, and it is hardly efficient. Add to this that not all researchers know where to look, know the search syntax of the databases they are searching, or have time to search all of their databases.

    Luckily, federated search have arrived on scene, assessed the situation and now offer alternatives to serial searching.

    With the advent of federated search, the ability to search across multiple repositories has improved. Moreover, with federated de-duplicated results, users do not receive thousands of irrelevant documents or emails. Users can simultaneously search across applications. It is best to take a non-repository processing approach and keep the existing data silos separate. A large repository can be kludgy with inherent security risks and to combine multiple silos may create problems in reconciling different processing power and security levels.

    Deep Web Technologies’ customers see increased efficiency every day using Explorit Everywhere!, so we second Simon’s evaluation of how federated search improves the overall picture of modern search. Our customers tell us that they find exactly what they are looking for in seconds or minutes. We are constantly evaluating our efforts to rapidly return the right results from across multiple repositories. And as data grows, so do we.

    Federated Search is still moving search along with it’s adaptable infrastructure. It’s like it was made for modern, dynamic, fast-growing data streams.

    Read the full article by Simon Bain here.

  • A Whole Day’s Search Resolved in Minutes

    Customer stories are the lifelines of Deep Web Technologies. We revel in hearing how an unassuming
    researcher shortened their workday and redirected their energy toward other tasks. Or how a Nursing Instructor had allotted an hour for an article search and his task was complete in sixty seconds! The story below comes from our friend Anita Wilcox at the University College Cork Library. stopwatch

    I met a friend of mine in the library last week who I haven’t seen for ages! In the course of catching up on gossip she told me that she was studying for a Health & Safety course in UCC and had been looking for a case report relevant to her group project. She had gone through all the printed law reports and had searched all the major law databases individually, but couldn’t find the case! So I gave her a one minute quick tour of Explorit Everywhere! (we call it Searcher here). She didn’t know the name of the case or when it was heard/reported. I showed her how to do an Advanced Search limiting only to law specific databases and using keywords, and then limiting to topics on the results screen. I was in a hurry, so I left. Within a minute I got a text from her saying she found the case! Now, that’s what I call a miracle! A whole day’s search was resolved within minutes by searching through Explorit Everywhere!.

    Read more from Anita in her blog post on Why a Federated Search.

    Our customers are often surprised that they can limit to specific sources or categories of sources to know exactly what they are searching. Researchers have control over their searches, rather than searching “everything” and wading through ambiguously ranked results, weighted to a publisher’s content. Explorit Everywhere!, through a transparent interface and search of information puts our researchers into the driver’s seat. It’s a powerful tool, for power-researchers.

Page 1 of 1012345»...Last »