Testosterone replacement for older men is increasingly common, with some observations suggesting a protective effect on cognitive function. We examined the association of endogenous testosterone with cognitive function among older men in a Mendelian randomization study using a separate-sample instrumental variable (SSIV) analysis estimator to minimize confounding and reverse causality. A genetic score predicting testosterone was developed in 289 young Chinese men from Hong Kong, based on selected testosterone-related single nucleotide polymorphisms (rs10046, rs1008805 and rs1256031). The association of genetically predicted testosterone with delayed 10-word recall score and Mini-Mental State Examination (MMSE) score was assessed at baseline and follow-up using generalized estimating equation among 4,212 older Chinese men from the Guangzhou Biobank Cohort Study. Predicted testosterone was not associated with delayed 10-word recall score (−0.02 per nmol/L testosterone, 95% confidence interval (CI) −0.06–0.02) or MMSE score (0.06, 95% CI −0.002–0.12). These estimates were similar after additional adjustment for age, education, smoking, use of alcohol, body mass index and the Framingham score. Our findings do not corroborate observed protective effects of testosterone on cognitive function among older men.