Sex differences for chronic back pain (cBP) have been reported, with females usually exhibiting greater morbidity, severity and poorer response to treatment. Genetic factors acting in an age-specific manner have been implicated but never comprehensively explored. We performed sex- and age-stratified GWAS and SNP-by-sex interaction analysis for cBP defined as “Back pain for 3+ months” in 202,077 males and 237,754 females of European ancestry from UK Biobank. Two and seven non-overlapping genome-wide significant loci were identified for males and females, respectively. A male-specific locus on chromosome 10 near SPOCK2 gene was replicated in four independent cohorts. Four loci demonstrated SNP-by-sex interaction, although none of them were formally replicated. SNP-explained heritability was higher in females (0.079 vs 0.067, p = 0.006). There was a high, although not complete, genetic correlation between the sexes (r = 0.838±0.041, different from 1 with p = 7.8E-05). Genetic correlation between the sexes for cBP decreased with age (0.858±0.049 in younger people vs 0.544±0.157 in older people; p = 4.3E-05). There was a stronger genetic correlation of cBP with self-reported diagnosis of intervertebral disc degeneration in males than in females (0.889 vs 0.638; p = 3.7E-06). Thus, the genetic component of cBP in the UK Biobank exhibits a mild sex- and age-dependency. This provides an insight into the possible causes of sex- and age-specificity in epidemiology and pathophysiology of cBP and chronic pain at other anatomical sites.