Kitaev-Heisenberg model in a magnetic field: Order-by-disorder and commensurate-incommensurate transitions

Abstract

We present a theoretical study of field-induced magnetic phases in the honeycomb Kitaev-Heisenberg model, which is believed to describe the essential physics of Mott insulators with strong spin-orbit coupling such as ${A}_{2}{\mathrm{IrO}}_{3}$ and $\ensuremath{\alpha}\text{\ensuremath{-}}{\mathrm{RuCl}}_{3}$. We obtain a rich finite temperature phase diagram in which the competition between the Zeeman coupling and thermal fluctuations gives rise to both collinear zigzag phases and noncoplanar magnetic orders. Our large-scale classical Monte Carlo simulations also unveil intriguing commensurate-incommensurate transitions and multiple-$\mathbf{Q}$ incommensurate phases at high field. Experimental implications are also discussed.