Mid-Holocene climate was characterized by strong summer solar heating that decreased Arctic sea ice cover. Motivated by recent studies identifying Arctic sea ice loss as a key driver of future climate change, we separate the influences of Arctic sea ice loss on mid-Holocene climate. By performing idealized climate model perturbation experiments, we show that Arctic sea ice loss causes zonally asymmetric surface temperature responses especially in winter: sea ice loss warms North America and the North Pacific, which would otherwise be much colder due to weaker winter insolation. In contrast, over East Asia, sea ice loss slightly decreases the temperature in early winter. These temperature responses are associated with the weakening of mid-high latitude westerlies and polar stratospheric warming. Sea ice loss also weakens the Atlantic meridional overturning circulation, although this weakening signal diminishes after 150-200 years of model integration. These results suggest that mid-Holocene climate changes should be interpreted in terms of both Arctic sea ice cover and insolation forcing.