Dissecting the compensation conundrum: a machine learning-based prognostication of key determinants in a complex labor market

Abstract

Purpose Amidst the turbulent tides of geopolitical uncertainty and pandemic-induced economic disruptions, the information technology industry grapples with alarming attrition and aggravating talent gaps, spurring a surge in demand for specialized digital proficiencies. Leveraging this imperative, firms seek to attract and retain top-tier talent through generous compensation packages. This study introduces a holistic, integrated theoretical framework integrating machine learning models to develop a compensation model, interrogating the multifaceted factors that shape pay determination. Design/methodology/approach Drawing upon a stratified sample of 2488 observations, this study determines whether compensation can be accurately predicted via constructs derived from the integrated theoretical framework, employing various cutting-edge machine learning models. This study culminates in discovering a random forest model, exhibiting 99.6% accuracy and 0.08° mean absolute error, following a series of comprehensive robustness checks. Findings The empirical findings of this study have revealed critical determinants of compensation, including but not limited to experience level, educational background, and specialized skill-set. The research also elucidates that gender does not play a role in pay disparity, while company size and type hold no consequential sway over individual compensation determination. Practical implications The research underscores the importance of equitable compensation to foster technological innovation and encourage the retention of top talent, emphasizing the significance of human capital. Furthermore, the model presented in this study empowers individuals to negotiate their compensation more effectively and supports enterprises in crafting targeted compensation strategies, thereby facilitating sustainable economic growth and helping to attain various Sustainable Development Goals. Originality/value The cardinal contribution of this research lies in the inception of an inclusive theoretical framework that persuasively explicates the intricacies of a machine learning-driven remuneration model, ennobled by the synthesis of diverse management theories to capture the complexity of compensation determination. However, the generalizability of the findings to other sectors is constrained as this study is exclusively limited to the IT sector.