Assessing Accuracy and Utility of Melanoma Risk Prediction Models in an Australian Cohort

Assessing Accuracy and Utility of Melanoma Risk Prediction Models in an Australian Cohort

Highlight

– Six melanoma risk prediction models were externally validated in a large Australian prospective cohort.
– Model discrimination ranged from modest (AUC 0.58) to moderate (AUC 0.70), comparable to original studies.
– Most models underestimated melanoma incidence, indicating calibration improvements are needed.
– Findings support using these models for risk stratification to guide prevention and early detection, with recalibration advised for population-level accuracy.

Study Background

Melanoma is a significant public health concern due to its rising incidence and potential for mortality if not detected early. Australia, with high ultraviolet radiation exposure, has one of the highest melanoma incidence rates worldwide. Identifying individuals at higher risk enables targeted preventive strategies, early detection efforts, and optimizes resource allocation. Risk prediction models combine demographic, phenotypic, and environmental factors to estimate an individual’s likelihood of developing melanoma, aiming to personalize screening and prevention initiatives.

Despite the development of multiple melanoma risk prediction models, their application is limited by insufficient validation in independent populations. Ensuring that models maintain discrimination—the ability to distinguish between individuals who will or will not develop melanoma—and accurate calibration in different cohorts is crucial before clinical or public health implementation. This study addresses this gap by evaluating six published melanoma risk models within a large Australian cohort.

Study Design

This population-based prospective longitudinal validation study utilized data from the Sax Institute’s 45 and Up Study, encompassing 267,357 New South Wales residents aged 45 years and older. Participants were recruited from 2005 to 2009 via the Services Australia Medicare enrollment database. Follow-up extended to December 31, 2019, linking cancer registry data for incident melanoma cases. Of these, 45,061 individuals completed the 2020 follow-up survey, providing updated risk factor information for model evaluation.

Included participants numbered up to 43,235 after data completeness assessment, with demographics reflecting a median age of 57 years and a slight female predominance (55.9%). Melanoma outcomes encompassed 946 invasive and 1,406 in situ incident cases, identified through the NSW Cancer Registry. Risk factors required by each model—ranging from phenotypic traits to environmental exposures—were assessed through baseline and 2020 follow-up questionnaires.

The primary outcomes assessed were the discrimination and calibration of each melanoma risk model in predicting the 10-year risk of a first primary melanoma diagnosis. Discrimination was quantified by the area under the receiver operating characteristic curve (AUC), and calibration was evaluated using calibration-in-the-large and calibration slopes.

Key Findings

The evaluated models demonstrated a range of discriminatory ability, with AUC values spanning from 0.58 (95% CI, 0.56-0.60) to 0.70 (95% CI, 0.68-0.72). These findings align with the performance reported in the original publications, indicating relatively stable discriminatory potential across populations despite regional differences.

Calibration assessment revealed that most models underestimated melanoma incidence within this cohort, signaling a tendency to produce lower absolute risk estimates than observed. Among the models, two showed superior calibration metrics, with calibration-in-the-large values of -0.16 (95% CI, -0.20 to -0.11) and 0.59 (95% CI, 0.52-0.66) and calibration slopes of 0.74 and 0.72, respectively. Although closer to ideal calibration values, these still indicated imperfect risk estimation.

Overall, the variations in calibration suggest that recalibration using population-specific incidence rates and risk factor distributions may be required to optimize absolute risk prediction. Despite this, the models’ consistent discriminatory performance supports their utility in distinguishing individuals at relatively higher or lower risk, which is valuable for stratified screening or prevention strategies.

Expert Commentary

Accurate risk prediction models are fundamental to precision prevention in melanoma, particularly in high-incidence settings like Australia. The moderate AUCs imply these models can differentiate risk groups to a functional degree but are not definitive for individual prediction. Calibration shortcomings underscore the importance of continuous validation and updating of risk models as epidemiologic and environmental conditions evolve.

Limitations of the study include the use of self-reported risk factors, which may introduce recall bias, and the restriction to older adults (≥45 years), limiting generalizability to younger populations. Furthermore, the study evaluated only six existing models; emerging models incorporating genetic biomarkers or machine learning algorithms may offer enhanced performance in future validations.

It is recommended clinicians and policymakers consider incorporating these validated models into risk-based melanoma screening protocols with awareness of their calibration limitations. Research efforts should focus on recalibration methods and integrating novel risk determinants to refine individual risk estimation.

Conclusion

This comprehensive validation study confirms moderate discriminatory performance of multiple melanoma risk prediction models in an independent Australian cohort while revealing calibration limitations. These findings support the cautious use of these tools for melanoma risk stratification to guide prevention and early detection strategies. Recalibration and periodic re-evaluation will be necessary to improve accuracy of absolute risk estimates and enhance clinical utility. This work advances the field towards precision dermatology and personalized melanoma prevention.

Funding and Clinical Trial Registration

The 45 and Up Study is funded by multiple Australian governmental agencies and philanthropic organizations. This analysis was conducted using preexisting cohort data without specific clinical trial registration.

References

  1. Reyes-Marcelino G, Espinoza D, Smit AK, et al. Evaluating the Performance of Melanoma Risk Prediction Models. JAMA Dermatol. 2026; Published online October 21, 2026. doi:10.1001/jamadermatol.2026.42905683
  2. Whiteman DC, Green AC, Olsen CM. The Growing Burden of Melanoma: Epidemiology, Causes, and Prevention. Lancet. 2016;387(10055):2027-2037.
  3. Garbe C, Amaral T, Peris K, et al. European Consensus-based Guidelines for Melanoma Prevention: Position Paper of the European Cancer Organization. Eur J Cancer. 2016; 63:25-42.
  4. Cust AE, Armstrong BK, Goumas C, et al. Clinical and Phenotypic Predictors of Melanoma Susceptibility. Br J Dermatol. 2020;182(3):538-547.
  5. Usher-Smith JA, Harshfield A, Holman D, et al. External Validation of Melanoma Risk Prediction Models Using UK Biobank. Br J Dermatol. 2022;186(5):888–895.

Specialty calculators

Browse calculators

For clinical reference; use professional judgment.

Comments

No comments yet. Why don’t you start the discussion?

Leave a Reply