Neuroimaging studies have shown that the lateral occipito-temporal cortex is involved in multisensory object processing. However, how essential this region is for non-visual object processing remains a matter of controversy. In this preregistered fMRI-guided repetitive transcranial magnetic stimulation (rTMS) study, we examined the causal involvement of the extrastriate body area (EBA) in multisensory object recognition. Participants used either vision or haptics to identify three object categories: hands, teapots (tools), and cars (control objects). Continuous theta-burst stimulation (cTBS) was applied over the left EBA, right EBA, or vertex (control site). Performance for visually perceived hands and teapots (relative to cars) was more strongly disrupted by cTBS over the left EBA than over the vertex, whereas no such object-specific effect was observed in haptics. Instead, exploratory analyses showed that the effect of impaired recognition for all object categories was greater when cTBS was applied over the right EBA compared to the vertex. The simulation of the induced electric fields confirmed that the cTBS affected not only EBA but also the adjacent object-sensitive region. These results indicate that the lateral occipito-temporal cortex is functionally relevant for haptic and visual object processing, though the rTMS differently affected object recognition between the two sensory modalities.