Artificial Intelligence in Innovation: How to Spot Emerging Trends and Technologies

TLDR

Firms use strategic foresight to detect discontinuous technological changes early and plan actions for superior performance, yet much of this work remains manual and resource‑intensive. This article introduces an AI‑based data‑mining model that automates the spotting of emerging topics and trends. The modular model comprises query generation, data collection, preprocessing, topic modeling, analysis, and visualization, with self‑adaptive updates and supervised‑learning‑derived thresholds, enabling minimal manual setup. Applied to an independent test set, the model identified emerging technologies in three case studies before their first appearance in the Gartner Hype Cycle, demonstrating its effectiveness as an early warning system and offering theoretical and practical implications for firms.

Abstract

Firms apply strategic foresight in technology and innovation management to detect discontinuous changes early, to assess their expected consequences, and to develop a future course of action enabling superior company performance. For this purpose, an ever-increasing amount of data has to be collected, analyzed, and interpreted. Still, a major part of these activities is performed manually, which requires high investments in various resources. To support these processes more efficiently, this article presents an artificial-intelligence-based data mining model that helps firms spot emerging topics and trends at a higher level of automation than before. Its modular structure consists of components for query generation, data collection, data preprocessing, topic modeling, topic analysis, and visualization, combined in such a way that only a minimum amount of manual effort is required during its initial set up. The approach also incorporates self-adaptive capabilities, allowing the model to automatically update itself once new data has become available. The model parameterization is based on latest research in this area, and its threshold parameter is learnt during supervised training using a training data set. We have applied our model to an independent test data set to verify its effectiveness as an early warning system. By means of a retrospective analysis, we show in three case studies that our model is able to identify emerging technologies prior to their first publication in the Gartner Hype Cycle for Emerging Technologies. Based on our findings, we derive both theoretical and practical implications for the technology and innovation management of firms, and we suggest future research opportunities to further advance this field.