This study compared expert yoga practitioners (n = 18) with a metabolically matched group (n = 17) and a group with comparatively higher cardiovascular fitness (n = 19) to investigate autonomic effects of yoga compared with physical exercise. Mean interbeat interval (in milliseconds) and heart rate variability were recorded in response to physical and mental challenges. Results showed that yoga males had different patterns of autonomic responding to speech task compared with all other participants. Yoga males showed a larger increase in mean interbeat interval and a corresponding decrease in heart rate variability in response to the speech task, followed by a larger decrease in mean interbeat interval and a corresponding increase in heart rate variability during speech recovery. This was interpreted as evidence of situation-appropriate autonomic regulation and suggests long-term yoga practice could be particularly beneficial for males in terms of autonomic flexibility.