Background: Some studies have found higher rates of childhood sexual abuse in sex offenders while others have failed to find such relationships. Method: This study reviews the sexual and physical abuse histories of 156 male sex offenders with intellectual disability (ID), 126 non-sexual male offenders with ID and 27 female offenders with ID. Results: Sexual offenders reported a higher rate of sexual abuse in childhood than non-sexual male offenders (32.6% vs. 17.8%), while the opposite was true for non-accidental injury (16.0% vs. 32.5%). Female offenders reported the highest rates of sexual abuse in childhood (59.3%). Conclusions: Some specificity of childhood abuse was evident in the male cohorts suggesting limited evidence of a developmental pathway to offending, while the women were a group highly vulnerable to all forms of abuse.