A hybrid machine-crowdsourcing system for matching web tables

TLDR

Web pages contain abundant HTML tables that can be integrated into a knowledge repository, but discovering semantic correspondences between table columns is challenging because conventional schema‑matching techniques struggle with incomplete data. The authors propose a two‑pronged approach to effectively address these difficulties in web table matching. They map each table column to the best concept in a knowledge base, then employ a hybrid machine‑crowdsourcing framework that assigns the most beneficial column‑to‑concept tasks to humans within a budget and uses the crowd results to infer matches for the remaining columns, validated on two real‑world datasets. Experimental results demonstrate that the two‑pronged approach outperforms existing schema‑matching techniques while incurring only a low crowdsourcing cost.

Abstract

The Web is teeming with rich structured information in the form of HTML tables, which provides us with the opportunity to build a knowledge repository by integrating these tables. An essential problem of web data integration is to discover semantic correspondences between web table columns, and schema matching is a popular means to determine the semantic correspondences. However, conventional schema matching techniques are not always effective for web table matching due to the incompleteness in web tables. In this paper, we propose a two-pronged approach for web table matching that effectively addresses the above difficulties. First, we propose a concept-based approach that maps each column of a web table to the best concept, in a well-developed knowledge base, that represents it. This approach overcomes the problem that sometimes values of two web table columns may be disjoint, even though the columns are related, due to incompleteness in the column values. Second, we develop a hybrid machine-crowdsourcing framework that leverages human intelligence to discern the concepts for "difficult" columns. Our overall framework assigns the most "beneficial" column-to-concept matching tasks to the crowd under a given budget and utilizes the crowdsourcing result to help our algorithm infer the best matches for the rest of the columns. We validate the effectiveness of our framework through an extensive experimental study over two real-world web table data sets. The results show that our two-pronged approach outperforms existing schema matching techniques at only a low cost for crowdsourcing.