Web Business Intelligence: Mining the Web for Actionable Knowledge

TLDR

The Web contains over seven billion static pages and potentially many more dynamic pages, yet only roughly 20 % of the static pages are indexed by search engines. The paper asks whether users can obtain all needed information from the Web for specific purposes and introduces Web Business Intelligence (WBI) as a solution. WBI is built on an architecture that surveys relevant technologies and demonstrates value through a detailed e‑finance example. While casual users generally find sufficient information, business and research users still need substantial effort, and the paper concludes by discussing the future of WBI.

Abstract

It is estimated that over seven billion static pages exist in the Web today, and backend databases can potentially produce at least three times as many dynamic pages. However, the best search engines index only approximately 20% of the static pages. So the real question is: While the Web is certainly the most amazing and comprehensive information source ever created, are you really getting all the information you need for your specific purpose? The answer to this question is mostly “yes” for the individual user, who uses the Web as an information source for casual purposes. However, for an individual who uses the Web as an essential and comprehensive source of information—for business or research—the answer is quite the opposite. Even a sophisticated Web user requires a significant amount of time and effort to find all of the information needed for a given task. In this paper the concept of Web Business Intelligence (WBI) is introduced, an emerging class of software that leverages the unprecedented content on the Web to extract actionable knowledge in an organizational setting. The contributions include an architecture for WBI, a survey of technologies relevant to the various components of the architecture, and illustration of the value of WBI by means of a detailed example from the e-finance domain. This article concludes with a discussion on the future of WBI.