Evaluating the Difference in Diagnostic Accuracy on Common Dermatologic Conditions in Skin of Colour and the Impact of Artificially Processed Images

To the Editor,

Accurate identification of dermatologic conditions across varying skin phenotypes is a global concern, especially among immigrant and diverse populations, and where skin of colour (SoC) remains underrepresented in education and image databases, posing a challenge to equitable care. ¹ Artificial intelligence (AI) is increasingly being used to support dermatologic diagnosis, but effectiveness can be limited by biased training data. ²

Models trained predominantly on lighter skin tones tend to perform worse on SoC images, exacerbating disparities in care. ³ Although generative AI has the potential to create synthetic skin images to improve SoC representation, this technology is nascent. AI’s speed in evolution and adoption necessitates careful understanding of AI performance, and we explore how image manipulation to alter colour distributions impacts accuracy, using AIPDerm’s AI system for benchmarking.

Fifteen common dermatologic conditions were identified based on their global burden and prevalence in SoC. Three subgroups, each containing 10 images, were curated: “Fitzpatrick I to III,” “Fitzpatrick IV to VI,” and “Processed.” The “Processed” group was generated by manipulating the “Fitzpatrick I to III” images to resemble richly pigmented skin through image darkening or intensity adjustment (Supplementary Figure 1). Three hundred images were obtained from verified clinical sources and reviewed by 2 dermatologists to ensure diagnostic accuracy and subgroup uniformity, in terms of severity and location, for the AI’s performance benchmarking.

The drop in performance from 87.3% in “Fitzpatrick I to III” was consistent across “Fitzpatrick IV to VI” (82.7%, P = .363) and “Processed” (82.0%, P = .027) subgroups (Supplementary Table 1). Conditions with consistent manifestations across skin tones—pityriasis versicolor, melasma, and hidradenitis suppurativa—demonstrated the highest top-1 sensitivity in each subgroup, while those with variable skin tone representations—atopic dermatitis, basal cell carcinoma (BCC), and squamous cell carcinoma—had the lowest sensitivity and posed the greatest diagnostic challenges (Supplementary Figure 2). For example, in SoC, erythema in atopic dermatitis appears more violaceous and ashy grey, ⁴ and the majority of BCCs are pigmented, unlike the pearly, pink nodules classically present in non-SoC. ⁵

While the AI demonstrated some diagnostic bias for non-SoC images, the performance drop was not statistically significant. The colour manipulation in the “Processed” subgroup was able to reduce the diagnostic performance of the “Fitzpatrick I to III” images, while ensuring its subgroup performance insignificantly differs from “Fitzpatrick IV to VI” (P = .887). The impact of colour manipulation was expectedly varied across individual conditions; however, the goal of manipulating images to determine overall impact through a representative subgroup accuracy was achieved, indicating that skin tone itself can be a key variable affecting performance. This narrower image manipulation was viable in establishing a baseline of impact of skin tones and colour distributions, and as a tool for future consideration in dataset augmentation and SoC performance.

These findings underscore the need to increase SoC representation in databases and datasets. While AI tools hold promise for equitable care, attention must be paid to AI performance and how SoC is represented to avoid reinforcing disparities. Colour manipulation confirmed the impact of skin tones as a baseline while instilling the need for vigilant educational and research efforts to incorporate unique manifestations of conditions in richly pigmented skin.

Supplemental Material

sj-docx-1-cms-10.1177_12034754251408483 – Supplemental material for Evaluating the Difference in Diagnostic Accuracy on Common Dermatologic Conditions in Skin of Colour and the Impact of Artificially Processed Images

Supplemental material, sj-docx-1-cms-10.1177_12034754251408483 for Evaluating the Difference in Diagnostic Accuracy on Common Dermatologic Conditions in Skin of Colour and the Impact of Artificially Processed Images by Katrina D. Cirone, Mohamed Akrout, Latif Abid Beng, Stephen Solis-Reyes, Rachel S. Simpson and Norbert Kiss in Journal of Cutaneous Medicine and Surgery

Supplemental Material

sj-docx-2-cms-10.1177_12034754251408483 – Supplemental material for Evaluating the Difference in Diagnostic Accuracy on Common Dermatologic Conditions in Skin of Colour and the Impact of Artificially Processed Images

Supplemental material, sj-docx-2-cms-10.1177_12034754251408483 for Evaluating the Difference in Diagnostic Accuracy on Common Dermatologic Conditions in Skin of Colour and the Impact of Artificially Processed Images by Katrina D. Cirone, Mohamed Akrout, Latif Abid Beng, Stephen Solis-Reyes, Rachel S. Simpson and Norbert Kiss in Journal of Cutaneous Medicine and Surgery

Supplemental Material

sj-docx-3-cms-10.1177_12034754251408483 – Supplemental material for Evaluating the Difference in Diagnostic Accuracy on Common Dermatologic Conditions in Skin of Colour and the Impact of Artificially Processed Images

Supplemental material, sj-docx-3-cms-10.1177_12034754251408483 for Evaluating the Difference in Diagnostic Accuracy on Common Dermatologic Conditions in Skin of Colour and the Impact of Artificially Processed Images by Katrina D. Cirone, Mohamed Akrout, Latif Abid Beng, Stephen Solis-Reyes, Rachel S. Simpson and Norbert Kiss in Journal of Cutaneous Medicine and Surgery