Brain imaging and electrophysiological recording studies in humans have reported discrete cortical regions in posterior ventral temporal cortex that respond preferentially to faces, buildings, and letters. These findings suggest a category-specific anatomically segregated modular organization of the object vision pathway. Here we present data from a functional MRI study in which we found three distinct regions of ventral temporal cortex that responded preferentially to faces and two categories of other objects, namely houses and chairs, and had a highly consistent topological arrangement. Although the data could be interpreted as evidence for separate modules, we found that each category also evoked significant responses in the regions that responded maximally to other stimuli. Moreover, each category was associated with its own differential pattern of response across ventral temporal cortex. These results indicate that the representation of an object is not restricted to a region that responds maximally to that object, but rather is distributed across a broader expanse of cortex. We propose that the functional architecture of the ventral visual pathway is not a mosaic of category-specific modules but instead is a continuous representation of information about object form that has a highly consistent and orderly topological arrangement.