Competition of Dzyaloshinskii-Moriya and Higher-Order Exchange Interactions in Rh/Fe Atomic Bilayers on Ir(111)

Using spin-polarized scanning tunneling microscopy and density functional theory we demonstrate the occurrence of a novel type of noncollinear spin structure in $\mathrm{Rh}/\mathrm{Fe}$ atomic bilayers on Ir(111). We find that higher-order exchange interactions depend sensitively on the stacking sequence. For fcc-$\mathrm{Rh}/\mathrm{Fe}/\mathrm{Ir}(111)$, frustrated exchange interactions are dominant and lead to the formation of a spin spiral ground state with a period of about 1.5 nm. For hcp-$\mathrm{Rh}/\mathrm{Fe}/\mathrm{Ir}(111)$, higher-order exchange interactions favor an up-up-down-down ($\ensuremath{\uparrow}\ensuremath{\uparrow}\ensuremath{\downarrow}\ensuremath{\downarrow}$) s…