The ability of climate models in the Intergovernmental Panel on Climate Change Fourth Assessment Report to capture the temporal structure of the annular modes is evaluated. The vertical structure and annual cycle of the variability is quantified by the e‐folding time scale of the annular mode autocorrelation function. Models vaguely capture the qualitative features of the Northern and Southern Annular Modes: Northern Hemisphere time scales are shorter than those of the Southern Hemisphere and peak in boreal winter, while Southern Hemisphere time scales peak in austral spring and summer. Models, however, systematically overestimate the time scales, particularly in the Southern Hemisphere summer, where the multimodel ensemble average is twice that of reanalyses. Fluctuation‐dissipation theory suggests that long time scales in models could be associated with increased sensitivity to anthropogenic forcing. Comparison of model pairs with similar forcings but different annular mode time scales provides a hint of a fluctuation‐dissipation relationship.