To date, no measures of sexual fantasies and behavior have been tested using modern structural equation modelling techniques. A total of 4,280 adults from the US, UK, Canada, and Ireland completed a measure of diverse (paraphilic and normophilic) sexual fantasies and behaviors. Data were randomly split in half for a two-part analysis. First, an exploratory factor analysis (EFA) was performed to reduce the item pool and determine general factor structure. Second, we tested several models using confirmatory factor analysis (CFA) and exploratory structural equation modelling (ESEM). These were followed by tests of measurement invariance (based on sex and sexual orientation) and criterion validity. For both the fantasies and behaviors, Bifactor ESEMs were the most appropriate models. Similar specific factors emerged: 1) normophilia, 2) rough sex, 3) interest of intrusion, 4) assuming power, and 5) relinquishing power. Findings suggest that sexual interests show a hierarchical measurement structure. Males and non-heterosexuals had higher general fantasy scores; females and non-heterosexuals had higher general behavior scores. Heterosexuals generally scored lower than non-heterosexuals. Fantasy and behavior scores were positively related to Dark Triad Traits and sociosexuality; there was little to no relationship with depression and anxiety. Results support the psychometric validation of the Sexual Fantasies and Behaviors Inventory.