Deep Web Tech Blog

The “Deep Web” is not all Dark

In a highly cited September 2001 article, The Deep Web: Surfacing Hidden Value, Michael Bergman coined the term “Deep Web” and wrote:

Searching on the Internet today can be compared to dragging a net across the surface of the ocean. While a great deal may be caught in the net, there is still a wealth of information that is deep, and therefore, missed. The reason is simple: Most of the Web’s information is buried far down on dynamically generated sites, and standard search engines never find it.

In February, 2002 just a few months after Michael Bergman published this article I saw the huge potential of the “Deep Web” for providing access to a wealth of high-quality content not available via search engines such as Google, so incorporated Deep Web Technologies that year. The “Deep Web” was a more accurate term for what had been referred to for a number of prior years as the “Hidden Web” or the “Invisible Web”. I’m not sure who eventually coined the term “Dark Web” or when. One early reference I found was to a chapter in a book on Intelligence and Security Informatics published in 2005: “The Dark Web Portal Project: Collecting and Analyzing the Presence of Terrorist Groups on the Web.

Everything was mostly good until October 2013 when the FBI shut down the Silk Road website, a Dark Web eBay-style marketplace for selling illegal drugs, stolen credit cards and other nefarious items. Since the take-down of Silk Road there have been a plethora of articles published which refer to the Dark Web as the Deep Web and lead to a lot of confusion and heartache for the CEO of one company in particular, Deep Web Technologies.

On November 2013, following a cover story in Time Magazine, on the Secret Web, which soon was referenced as the Deep Web, I wrote a letter to the Editor of Time and followed it with the blog article – The Deep Web isn’t all drugs, porn and murder to no avail.

In the past few months following the announcement of DARPA’s Memex project which states as its goal, “Creation of a new domain-specific indexing and search paradigm will provide mechanisms for improved content discovery, information extraction, information retrieval, user collaboration, and extension of current search capabilities to the deep web, the dark web, and nontraditional (e.g. multimedia) content,” there have been many more articles published equating the “deep web” and the “dark web” such as the following article about NASA’s efforts to leverage the memex efforts: NASA has big plans for DARPA’s scary “Deep Web.Deep Web Iceberg

What prompted me to write this blog article is that I learned a few days ago that Epix has produced a documentary, that is going to be released on May 31, 2015, titled Deep Web.

“Extending far beyond the confines of Google and Facebook, there is a vast section of the World Wide Web that is a hidden alternate internet. Appropriately named the Deep Web, this mysterious and complex cyberspace serves as an outlet for anonymous communication and was home to Silk Road, the online black market notorious for drug trafficking. The intricacies of this concealed cyber realm caught the attention of the general public with the October 2013 arrest of Ross William Ulbricht – the convicted 30-year-old entrepreneur accused to be ‘Dread Pirate Roberts,’ the online pseudonym of the Silk Road leader. Making its World Television Premiere this spring, Deep Web – an EPIX Original Documentary written, directed and produced by Alex Winter (Downloaded) – seeks to unravel this tangled web of secrecy, accusations, and criminal activity, and explores how the outcome of Ulbricht’s trial will set a critical precedent for the future of technological freedom around the world.”

Clearly Dark Web would be a more appropriate title for this documentary and might attract a bigger audience than Deep Web, but I’m not so fortunate. What am I to do?