History of Search Engines and Directories

  • 1993: The first widely acclaimed search engine, the World Wide Web Wanderer, appears. Created to measure the growth of the Web, it performs its job through 1997. Statistics that this search engine compiled are still available on the Web today.
  • 1994: WebCrawler comes on the scene. The original WebCrawler database contained just 6,000 websites. AOL bought WebCrawler in 1995 but sold it just two years later to Excite. Infospace, its current owner, bought WebCrawler when Excite declared bankruptcy.
  • 1994: Another powerhouse, the Lycos search engine, launches with 54,000 indexed documents. The Lycos search engine is still a player today, but it’s changed hands several times. Currently, it’s a subsidiary of the Indian-based company Ybrant Digital.
  • 1995: AltaVista is the first search engine to include multilingual search capabilities. AltaVista eventually becomes the property of Yahoo. (We’ll discuss Yahoo a bit later.) AltaVista is the search king until the rise of Google. In 2013, their service shuts down and the domain now redirects to Yahoo’s own search site.
  • 1998: Larry Page and Sergey Brin introduce the world to Google, which quickly shoots to the top of search engine rankings. (The name comes from the word googol, which is the name for the number 1 followed by 100 zeros.)
  • 2006: Google becomes such a part of our culture that several dictionaries, including Merriam-Webster and the Oxford English Dictionary, add “to google” as a verb.
  • 2009: Microsoft launches Bing, which introduces the use of suggested searches along with search results.
  • 2014: For the first time in history, more searches are performed using mobile devices than using desktop browsers.
  • 2015: Over half of Google searches are performed using mobile devices. Google announces that they’ll start using mobile-friendly factors in its mobile search results, meaning they actively list sites that aren’t mobile-friendly lower in mobile searches.
That’s where things stand now, with Google by far the most-used search engine on the Web. Google now enjoys greater than 50% of total search engine traffic. This means that a top Google ranking will yield more traffic to your site than a top ranking with any other search engine.Google’s popularity is due largely to speed and quality of search results. Both are possible because of a worldwide network of more than 1 million servers, which house Google’s index. The sheer number of servers and the speed at which they communicate with each other is unparalleled in the search industry. However, companies are constantly buying, selling, creating, and improving search engines. As a result, you’ll want to keep tabs on which search engines are gaining or losing popularity.In the Supplementary Material section of this lesson, you’ll find information about excellent search engine research companies that will provide their findings to you for free.Search Engines Versus DirectoriesAre you wondering why we didn’t talk about Yahoo in the last section? That’s because Yahoo began as a directory, not a search engine.Computers compile search engines’ indexes, but humans compile directories, which are categorical lists of websites. Before listing a site in a directory, someone (or a team of people) scrutinizes the site and places it in a specific category.Some people argue that directories are limited because they offer fewer search results than search engines do. However, I think you’ll find that directories’ results can often be much more useful than search engines’, depending on the type of information you’re interested in.Until recently, one of the most popular directories on the Web was Yahoo, which David Filo and Jerry Yang founded in 1994. They started Yahoo on a couple of computers in a campus trailer at Stanford, initially using it to track their own interests. It surprised them by taking off quickly, and they incorporated it in 1995 with an initial investment of almost $2 million. By the way, Yahoo is an acronym for Yet Another Hierarchical Officious Oracle—but I promise not to test you on that!In an effort to diversify, Yahoo decided to use Google’s engine to supply users with primary search results. Then, in 2004, Yahoo unveiled its own crawler for organic search results. In 2010 (for reasons that remain unclear to many) Yahoo allowed the Microsoft search engine Bing to supply its primary search results. Today, Bing still supplies search results to Yahoo, but Yahoo continues to develop its own search engine technology.The Open Directory Project is the most popular directory on the Web today. Technically, AOL owns the project, though it’s compiled by more than 90,000 volunteers who’ve indexed over 4 million sites. The directory contains just over 1 million categories, all of which you can search by keyword or category.
Brian Holzberger

Brian Holzberger

Leave a Reply