The aim of this talk is to introduce the subject of spin glasses, and more generally the statistical mechanics of quenched disorder, as a problem of general interest to physicists and mathematicians from multiple disciplines and backgrounds. Despite years of study, the physics and mathematics of quenched disorder remains poorly understood, and represents a major gap in our understanding of the condensed state of matter. While there are many active areas of investigation in this field, I will narrow the focus of this talk to our current level of understanding of the low-temperature equilibrium structure of realistic (i.e., finite-dimensional) spin glasses. I will begin with a brief review of the basic features of spin glasses and what is known experimentally. I will then turn to the problem of understanding the nature of the spin glass phase --- if it exists. The central question to be addressed is the nature of broken symmetry in these systems. Parisi's replica symmetry breaking approach, now mostly verified for mean field spin glasses, attracted great excitement and interest as a novel and exotic form of symmetry breaking. But does it hold also for real spin glasses in finite dimensions? This has been a subject of intense controversy, and although the issues surrounding it have become more sharply defined in recent years, it remains an open question. I will explore this problem, introducing new mathematical constructs such as the metastate along the way. The talk will conclude with an examination of how and in which respects the statistical mechanics of disordered systems might differ from that of homogeneous systems.