Reconstructing the Giant: On the Importance of Rigour in Documenting the Literature Search Process

TLDR

Literature reviews are essential for cumulative science, and their quality hinges on the rigor of the literature search process, which serves as the foundational skeleton for reconstructing accumulated knowledge. The authors aim to demonstrate that literature search procedures must be transparently described so that readers can evaluate exhaustiveness and other scholars can reliably reuse findings. They examined literature review articles in ten leading information systems journals, assessing how thoroughly each documented its search strategy and identifying gaps in methodological rigor. Their analysis shows that inadequate documentation hampers assessment of review exhaustiveness, prompting a call for greater rigor and the provision of guidelines for crafting literature reviews and searches in the IS domain.

Abstract

Science is a cumulative endeavour as new knowledge is often created in the process of interpreting and combining existing knowledge. This is why literature reviews have long played a decisive role in scholarship. The quality of literature reviews is particularly determined by the literature search process. As Sir Isaac Newton eminently put it: If I can see further, it is because I am standing on the shoulders of giants. Drawing on this metaphor, the goal of writing a literature review is to reconstruct the giant of accumulated knowledge in a specific domain. And in doing so, a literature search represents the fundamental first step that makes up the giant's skeleton and largely determines its reconstruction in the subsequent literature analysis. In this paper, we argue that the process of searching the literature must be comprehensibly described. Only then can readers assess the exhaustiveness of the review and other scholars in the field can more confidently (re)use the results in their own research. We set out to explore the methodological rigour of literature review articles published in ten major information systems (IS) journals and show that many of these reviews do not thoroughly document the process of literature search. The results drawn from our analysis lead us to call for more rigour in documenting the literature search process and to present guidelines for crafting a literature review and search in the IS domain.