Dynamics of Search Engine Rankings - A Case Study.

Abstract

The objective of this study was to characterize the changes in the rankings of the top-n results
\nof major search engines over time and to compare the rankings between these engines. We
\nconsidered only the top-ten results, since users usually inspect only the first page returned by
\nthe search engine, which normally contains ten results. In particular, we compare rankings of
\nthe top ten results of the search engines Google and AlltheWeb on identical queries over a
\nperiod of three weeks. The experiment was repeated twice, in October 2003 and in January
\n2004 in order to assess changes to the top ten results of some of the queries during a three
\nmonths period. Results show that the rankings of AlltheWeb were highly stable over each
\nperiod, while the rankings of Google underwent constant yet minor changes, with occasional
\nmajor ones. Changes over time can be explained by the dynamic nature of the Web or by
\nfluctuations in the search engines’ indexes (especially when frequent switches in the rankings
\nare observed). The top ten results of the two search engines have surprisingly low overlap.
\nWith such small overlap (occasionally only a single URL) the task of comparing the rankings
\nof the two engines becomes extremely challenging, and additional measures are needed to
\nassess rankings in such situations.