Obesity in humans is associated with poorer health outcomes after infections compared with non-obese individuals. Here, we examined the effects of white adipose tissue and obesity on T cell responses to viral infection in mice. We show that lymphocytic choriomeningitis virus (LCMV) grows to high titer in adipose tissue. Virus-specific T cells enter the adipose tissue to resolve infection but then remain as a memory population distinct from memory T cells in lymphoid tissues. Memory T cells in adipose tissue are abundant in lean mice, and diet-induced obesity further increases memory T cell number in adipose tissue and spleen. Upon re-challenge infection, memory T cells rapidly cause severe pathogenesis, leading to increases in lipase levels, calcification of adipose tissue, pancreatitis, and reduced survival in obese mice but not lean mice. Thus, obesity leads to a unique form of viral pathogenesis involving memory T cell-dependent adipocyte destruction and damage to other tissues.