- Product Trial
Editor’s Note: This is a guest article by Michelle Powers, Director of Library Services at Career Education Corporation (CEC). The Colorado Technical University and a number of other schools owned and operated by CEC are Explorit Everywhere! customers.
In 2015, the librarians of Colorado Technical University wanted to investigate alternative options to our discovery search tool. While we were not unhappy with the system we had in place, we were often stymied by the inability to incorporate competing vendors into the system’s platform and create a truly seamless experience for our students who relied heavily on the ability to search multiple subscription resources at one time.
In short, we wanted a system that provided results from truly everything we had, based on relevancy, in a completely seamless fashion, through an easy-to-use interface.
To begin, it’s important to understand that CTU has 3 campuses: 1 online campus which serves students in a completely online environment and 2 ground campuses in which students can access a traditional library facility but rely heavily on the electronic database collection for academic research.
Each campus has a unique campus portal; therefore, there are 3 separate library portals. There are also some database differences, based on campus programs and other factors.
Our initial decision criteria included the following:
- We needed a system that we could implement across multiple campuses—giving our students and faculty who work in multiple campus environments the same experience
- We needed a system that would allow individual branding for our separate campuses
- The system needed to work well with a multitude of vendors
- They system needed to allow us to continue to track database usage from each campus
- We needed a system that could be embedded into our campus pages, which is behind a firewall and only available to authorized users
- We needed a system that would not require additional authentication from our students, such as a student ID, after they log onto the campus portals
- We needed the system to be intuitive for our users
- The system had to be affordable
We were at first hesitant to move back to a federated system, which is what we had in place prior to the implementation of the discovery system. Our federated system had burned images in our collective memory of clunky interfaces and groupings of results which confused users. However, after reviewing how Deep Web Technologies met the criteria outlined above we invited Abe to give us a demo.
We were impressed with Explorit Everywhere!’s easy-to-understand interface, and features like the Search Builder, the ability to categorize resources on the search page, the ability to embed the search widget into our LibGuides, and more. What we liked most about the product though was that it was vendor neutral and promised to incorporate all of our resources in a way our previous system did not.
In early 2016 we decided to make the switch and launch the Explorit Everywhere! search tool in April at CTU and multiple other institutions owned and operated by CTU’s parent company, Career Education Corporation. This meant an implementation of Explorit Everywhere! on nearly 50 campus portals in less than 3 months!
Deep Web Technologies’ team provided clear instructions of what needed to be done on our end, met regularly with our IT team and library leadership to ensure our timeline was met, created systems on the back end to allow for our requested search features, and created a method of providing the library with statistics.
Our launch was astoundingly…. quiet. No upset student responses, no confusion or dismay at the new interface. Our students and faculty took to the new system like fish in water, which reinforced the library’s own opinion that the system was easy to use, and satisfied the needs of our users.
Abe and his team including Christy Ziemba, Ellee Wilson, and Susan Martin have been awesome at responding to our queries and resolving any situations we’ve encountered with such a massive change.
The library is still gathering statistics to accurately calculate changes to usage with the implementation of Explorit Everywhere! We did immediately have increased usage in an e-book database that was unavailable through our previous system, and are expecting to find an overall increase in database usage, especially in the other resources unavailable in the previous system
Librarians smarter and better than me can argue the pros and cons of a federated system vs a discovery system. I can say that our students ultimately benefit by the comprehensive search feature Deep Web Technologies has offered us.
Disclaimer: CTU cannot guarantee employment or salary. Find employment rates, financial obligations, and other disclosures at www.coloradotech.edu/disclosures.
I was pleasantly surprised and pleased when I woke up one recent morning to an email message from Nick Dimant, Managing Director of our partner PTFS Europe. My company and PTFS Europe were partners-in-crime in a unique (hopefully to be repeated many more times) collaboration at the University of the Arts, London (UAL).
Nick had sent me a copy of – An innovative approach to discovery (available here), a feature article in the June 2016 issue of Update, the monthly magazine of CILIP (Chartered Institute of Library and Information Professionals) by Karen Carden, Resources & Systems Manager, Library Services, UAL and by Jess Crilly, Associate Director, Content and Discovery, Library Services, UAL.
Carden and Crilly explain in their article in detail their justification and approach to implementing their Library Search solution which “brings together two separate products into (what looks like) a single interface for the user where they can search across our print and e-resources.”
Carden and Crilly discuss their selection of Explorit Everywhere! back in 2013 (which of course I love) in:
“After a great deal of research, discussion and testing we opted for an unusual – especially in the UK – next generation federated search tool. Like most libraries in the sector we had experienced first generation federated search, but found that this was quite a different experience.”
The authors describe UAL as a specialist university. What this means to me is that as a specialist university focused on the arts, a lot of the databases that UAL subscribes to are not mainstream databases and thus not included in the Discovery Services but easily federated by Explorit Everywhere!.
We give another example in Federating the Unfederatable of a specialist library, this time a defense/international policy focused university where Explorit Everywhere! provides the one-stop discovery of all the sources important to the library patrons, many not available through the Discovery Services.
If you’d like to read further on our Explorit Everywhere! solution at UAL check out these blog articles: Customer Corner – Paul Mellinger presentation, Promoting Explorit Everywhere! at UAL, and Faceted Navigation – UAL example.
The Deep Web fascinates most of us and scares some of us, but is used by almost all of us. While over the past couple of years, more and more information has surfaced about the Deep Web, finding reputable information in those depths is still shrouded in mystery. Abe Lederman, CEO of Deep Web Technologies, wrote a guest article for Refer Summer 2016, republished in part below. Refer is an online journal published three times a year for the Information Services Group of the Chartered Institute of Library and Information Professionals (CILIP).
The Web is divided into 3 layers: the Surface Web, the Deep Web and the Dark Web. The Surface Web consists of several billion web sites, different subsets of which are crawled by search engines such as Google, Yahoo and Bing. In the next layer of the Web, the Deep Web consists of millions of databases or information sources – public, subscription or internal to an organization. Deep Web content is usually behind paywalls, often requires a password to access or is dynamically generated when a user enters a query into a search box (e.g. Netflix), and thus is not accessible to the Surface Web search engines. This content of the Deep Web is valuable because, for the most part, it contains higher quality information than the Surface Web. The bottom-most layer, called the Dark Web, gained a lot of notoriety in October 2013 when the FBI shut down the Silk Road website, an eBay-style marketplace for selling illegal drugs, stolen credit cards and other nefarious items. The Dark Web guarantees anonymity and thus is also used to conduct political dissent without fear of repercussion. Accessing the gems that can be found in the Deep Web is the focus of this article.
Michael Bergman, in a seminal white paper published in August 2001 entitled – The Deep Web: Surfacing Hidden Value, coined the term “Deep Web”. The Deep Web is also known as the Hidden Web or the Invisible Web. According to a study conducted in 2000 by Bergman and colleagues, the Deep Web was 400-550 times larger than the Surface Web, consisting of 200,000 websites, 550 billion documents and 7,500 terabytes of information. Every few years while writing an article on the Deep Web, I search for current information on the size of the Deep Web and I’m not able to find anything new and authoritative. Many articles that I come across still, like this article, refer to Bergman’s 2001 white paper.
Many users may not be familiar with the concept of the Deep Web. However If they have searched the U.S. National Library of Medicine PubMed Database, if they have searched subscription databases from EBSCO or Elsevier, if they have gone and searched the website of a newspaper such as the Financial Times or went to purchase a train ticket online, then they have been to the Deep Web.
If you are curious about what’s in the Deep Web and how can to find some good stuff, here are some places you can go to do some deep web diving…
Congratulations to Dr. Ellen Wilson (Ellee), our VP of Professional Services and Engineering, and a new PhD recipient. Ellee walked the PhD ceremony at her alma mater, Pacifica Graduate Institute, on Sunday, May 29, no doubt breathing a huge sigh of relief as she wrapped up this milestone achievement.
Deep Web Technologies hires smart people; we have smart engineers, smart project managers, smart leaders. But Ellee just may be at the top of the list. While her day job may be to guide the ebb and flow of DWT’s team, projects, and software development, her real passion is depth psychology, the exploration of theories and ideas delving into the relationship between consciousness and the unconscious.
Ellee’s dissertation didn’t just add a new set of letters behind her name; her line of exploration is unique to the realm of depth psychology. She presents credible research and ideas about how the world imprints itself on people, tying the order of mathematics throughout history and the present day to the chaos of a more sensory-based world. Are you interested how the development of non-Euclidean geometry created a diversion in the linear focus at the time and contributed to a multiplistic expression of human thought and experience? So is Dr. Ellen Wilson.
Don’t think that receiving her PhD is the end of the line for Ellee. From a very young age, Ellee has been driven to explore what makes us tick. She has multitudinous degrees, ranging from Women’s Studies, Mathematics, and Computer Engineering to Library and Information Science, and now, Depth Psychology. Her PhD may just be her intellectual fulcrum, harnessing her past and funneling it into a rich, erudite future.
DWT is proud to have Ellee on our staff.
This past weekend I was writing an email message to one of our new customers. I was sending this customer, as promised,
some example uses of Search Builder, our Explorit Everywhere! tool that makes it easy to create custom search widgets that can be dropped on any web page.
I started my quest for Search Builder widgets by going to the library website of one of our favorite customers, the University of the Arts, London (UAL). UAL is one of our favorite customers, in part, because they do a lot to promote Explorit Everywhere! to their faculty and students across their 6 colleges.
I digress, but last December (2015) I had the pleasure to attend the User Conference for one of our partners, PTFS Europe, and at this conference Paul Mellinger, Discovery Manager at the UAL Library gave a wonderful presentation on their implementation of Explorit Everywhere! and followed that with a guest article on the Customer Corner section of our company blog.
Back to the task at hand, I went to UAL’s Library Subject Guides page and found that they have created 39 subject guides on subjects such as Beauty and Cosmetic Sciences, Graphic Design and Jewellery, subjects that parallel the way that they have organized the sources that we federate for them. If you go into many of these subject guides then click on the e-Resources tab you’ll see at the top left of the page an Articles Plus Search widget (this is what UAL calls Explorit Everywhere!). Each of the widgets on the various e-Resources pages searches a different, appropriate subset of all the sources that we federate for them.
Then to top it off, I came across the Articles Plus Help subject guide (which I had not seen before), a wonderful guide, nicely organized into tabs for Simple Search, Advanced Search, Search Results and Tools and Navigation. Also added recently to this subject guide is a 3 and a half minute video tutorial on Articles Plus. This is why we love UAL!!
Problem – Your Explorit Everywhere! solution (particularly at academic institutions) is typically setup to search a wide spectrum of sources that aren’t always relevant to every researcher or student. Special groups (e.g. the Chemistry Department, the Psychology 101 course, or the High Energy Physics Division within a research lab) may prefer to search only a handful of resources, and don’t want to be overwhelmed by results from irrelevant sources. Sound familiar?
Our elegant solution – Explorit Everywhere! Search Builder
Each Explorit Everywhere! application comes loaded with Search Builder, a search engine creation tool designed to create custom search engines that only search the specific sources your user group needs. For example, an instructor teaching a class on Computer Programming may want students to research responsive design using four selected resources. Rather than tutoring the class in how to use their Explorit Everywhere! Advanced Search and limit the search to the four relevant resources, the instructor can simply create a search engine designed for his students that only search those resources the students should search. Then, he can place a search box to this new engine on his course page to make it even easier for his students.
There are some Deep Web Technologies customers that create, use, and market only Search Builder engines engines that only search the specific sources your user group needs. For example, an instructor teaching a class on Computer Programming may want students to research responsive design using four selected resources. Rather than tutoring the class in how to use their Explorit Everywhere! Advanced Search and limit the search to the four relevant resources, the instructor can simply create a search engine designed for his students that only search those resources the students should search. Then, he can place a search box to this new engine on his course page to make it even easier for his students.
Search Builder can create unlimited search engines for one person, groups, teams or even enterprises. Admins can build search engines for each department, students can create their own search portal with a personalized search box, professors can modify search engines for their classes, and teams can search the most relevant resources and delve into their critical materials most efficiently.
Search Builder can be integrated into:
- Learning Management Systems
- Corporate Portal
- CRM Systems
Help your researchers be be efficient – Use Search Builder. And, if your organization uses one of the pesky Discovery Services which don’t allow, or at least don’t make it easy to limit your search to relevant resources, Search Builder is a great reason to switch to Explorit Everywhere!
Indexed discovery services are a lot like Google: you can search simply and get results quickly, but the sources you are searching tend to be mysterious. Do you know if you are searching specialized sources or generic sources? Authoritative or with an agenda? When a researcher pushes the search button, they get whatever results are deemed relevant from whatever sources are included, and they can’t limit their search to only the sources that matter to them.
For some researchers, knowing the sources behind the search really makes no difference to them at all. To these researchers, often undergraduates, it’s the results that count. Most results nowadays do show the source, publication or journal that result is from. This makes it somewhat easier to eyeball a page of results, disregard those from irrelevant sources, or select results as appropriate if they are from an authoritative source. But that research methodology seems inefficient to say the least.
Serious researchers, on the other hand, want to know what they are searching. If they know that their information will most likely be in three or four specific resources out of the 20 or 30 their organization subscribes to, then why should they wade through a massive results list or spend one iota of extra time filtering out the extra sources to view their nuggets of information? (Quick answer, they shouldn’t.)
To begin to combat this resource transparency problem, libraries are creating separate web pages of source lists and descriptions for serious researchers. Who is the provider? What is the resource and what information exactly does the resource provide? These pages also include the categories of sources searched such as ebooks, articles, multimedia collections, etc. While these lists of resources are certainly helpful to document, is it fair to ask researchers to reference a separate web page to understand what digital content is included in their search, particularly when they are urgently trying to find something from a particular source of information? Or should we ask researchers to disregard knowing what sources they are searching and to just pay attention to the results? Neither of these seems appropriate in this day and age.
Most of us know the benefits of a single search of all resources. One search improves the efficiency of searching disparate sources, makes comparing and contrasting results faster, and provides an opportunity to save, export or email selected results. However, Explorit Everywhere! goes one step further by lending transparency of sources to researchers so they can search even faster. One of our customers mentioned that they moved from a well known discovery service because they were frustrated with all of the news results that were returned. It didn’t help that their researchers couldn’t select specific sources to search, particularly when their searches always seemed to bring back less than relevant results.
Explorit Everywhere! helps to narrow a search up front with not only the standard Advanced Search fields, but a list of sources to pick and choose from. A researcher looking to search in four different sources doesn’t want to run a search against 25 sources. They can narrow the playing field to hone in on the needle in the haystack faster. And from the results page, they can limit to each individual source to view only those results, in the order that the source ranked them. A serious researcher’s dream? That’s what we’ve heard.
Not all researcher’s care about drilling down into individual sources like this. But in Explorit Everywhere! the option is there to search the broad or narrow path. We even filter out the rocks.
In a future blog article we’ll talk about the other side of this question: What content am I missing?
Simon Bain, CEO of SearchYourCloud recently wrote an article for IT ProPortal, “The search continues: A history of search’s unsatisfactory progress.” In his article, Simon expands on a survey conducted by SearchYourCloud in which a third of respondents said they “spend between five and 25 minutes searching every time they want to find a document, while only one in five searches is correct the first time. The search for corporate information is eating into workplace productivity. Only 20 per cent of respondents reported first time successful searches. Other key findings from the survey include that it takes workers up to eight searches to find the right document and information, according to 80 per cent of respondents.”
Search, Simon comments, hasn’t really improved over the last 20 years. Companies such as Microsoft and Autonomy, while assuming the mantle of usable search through the acquisition of search veterans like Verity and FAST, actually focused their efforts on consultancy rather than on improving results retrieval.
The introduction of Big Data, however, has offered a twist to the information retieval conundrum. Piles of data, lots of repositories, different security levels and multiple devices all funnel down to researchers who must adapt their search to the search interface of the individual database or repository, meaning they are often searching databases one by one by one.
We call this serial searching, and it is hardly efficient. Add to this that not all researchers know where to look, know the search syntax of the databases they are searching, or have time to search all of their databases.
Luckily, federated search have arrived on scene, assessed the situation and now offer alternatives to serial searching.
“With the advent of federated search, the ability to search across multiple repositories has improved. Moreover, with federated de-duplicated results, users do not receive thousands of irrelevant documents or emails. Users can simultaneously search across applications. It is best to take a non-repository processing approach and keep the existing data silos separate. A large repository can be kludgy with inherent security risks and to combine multiple silos may create problems in reconciling different processing power and security levels.”
Deep Web Technologies’ customers see increased efficiency every day using Explorit Everywhere!, so we second Simon’s evaluation of how federated search improves the overall picture of modern search. Our customers tell us that they find exactly what they are looking for in seconds or minutes. We are constantly evaluating our efforts to rapidly return the right results from across multiple repositories. And as data grows, so do we.
Federated Search is still moving search along with it’s adaptable infrastructure. It’s like it was made for modern, dynamic, fast-growing data streams.
Customer stories are the lifelines of Deep Web Technologies. We revel in hearing how an unassuming
researcher shortened their workday and redirected their energy toward other tasks. Or how a Nursing Instructor had allotted an hour for an article search and his task was complete in sixty seconds! The story below comes from our friend Anita Wilcox at the University College Cork Library.
I met a friend of mine in the library last week who I haven’t seen for ages! In the course of catching up on gossip she told me that she was studying for a Health & Safety course in UCC and had been looking for a case report relevant to her group project. She had gone through all the printed law reports and had searched all the major law databases individually, but couldn’t find the case! So I gave her a one minute quick tour of Explorit Everywhere! (we call it Searcher here). She didn’t know the name of the case or when it was heard/reported. I showed her how to do an Advanced Search limiting only to law specific databases and using keywords, and then limiting to topics on the results screen. I was in a hurry, so I left. Within a minute I got a text from her saying she found the case! Now, that’s what I call a miracle! A whole day’s search was resolved within minutes by searching through Explorit Everywhere!.
Read more from Anita in her blog post on Why a Federated Search.
Our customers are often surprised that they can limit to specific sources or categories of sources to know exactly what they are searching. Researchers have control over their searches, rather than searching “everything” and wading through ambiguously ranked results, weighted to a publisher’s content. Explorit Everywhere!, through a transparent interface and search of information puts our researchers into the driver’s seat. It’s a powerful tool, for power-researchers.