AI Skin Analysis Across Diverse Skin Tones

The problem with AI and skin tone begins before any algorithm enters the room. It begins with how we measure skin color itself.

Skin color is primarily determined by eumelanin content and distribution in the epidermis. More eumelanin, darker phototype. That part is settled biology. What is far less settled is how we classify it, and that gap has consequences that ripple through every system built on top of it.

The Fitzpatrick scale has been the dominant clinical instrument since 1975. Six skin types, originally developed to gauge UV response and sunburn risk in lighter-skinned European patients. It was never designed to comprehensively classify skin color across global populations, and dermatologists have been saying so for years. It neither accurately nor adequately assesses sunburn risk in skin of color, and it creates a false sense of reassurance about skin cancer risk for patients whose phototype the scale was never built to represent.

Two alternatives are gaining traction. The Eumelanin Human Skin Color Scale grounds classification in actual pigment biology rather than UV behavior. Google's Monk Skin Tone Scale uses ten tones instead of six, was validated for broader demographic representation, and international dermatology literature has documented that its adoption improves categorization across a wider spectrum of patients.

Why does any of this matter for AI? If a model's training data is labeled using an inadequate scale, the model learns an inadequate representation of the world. Skin types 4 through 6 on the Fitzpatrick scale are more prone to post-inflammatory hyperpigmentation, eczema, and keloid scarring. A model misclassified at the labeling stage is more likely to miss those conditions at the prediction stage. The measurement instrument is contested, and that instability carries forward into every system built on top of it.

What AI skin analysis tools are actually trained on, and what that means for darker skin

Here is something that took me time to sit with: the bias in dermatology AI does not originate with the algorithm. It originates upstream, in the clinical and educational infrastructure that feeds the data.

The ISIC dataset, maintained by the International Skin Imaging Collaboration, is where most dermatology AI models begin. It skews heavily toward fair skin, with lighter skin tones often comprising more than 70% of its images. It leans toward dermoscopic images of confirmed malignancies and substantially under-represents inflammatory conditions, uncommon diseases, and diverse skin tones.

But where does that skew come from? A 2025 review of melanoma randomized controlled trials published in top dermatology journals found that only about half of eligible articles even recorded skin color in their demographic data, with no statistically significant improvement across decades of published research. Medical textbooks and case presentations used to train dermatologists have historically lacked images of darker skin. An analysis of New England Journal of Medicine image challenges found significantly fewer questions featuring dark skin. The image gap exists in medical education long before AI enters the conversation.

What a model trained on skewed data actually learns is pattern recognition optimized for the majority class. Lesion features that present differently on darker skin become underweighted, sometimes invisible to the model. This is not a design flaw in the narrow engineering sense. The model is accurately reflecting the data it encountered, which is itself skewed. That distinction matters when you are trying to fix it, because the solution is not a better algorithm applied to the same data. It is different data, full stop.

A 2025 evaluation of four generative AI models found that 89.8% of the 4,000 dermatological images they produced depicted light skin, even when explicitly prompted to generate diverse dermatological references. Adobe Firefly was the only model to produce output meaningfully aligned with U.S. demographic composition. Generative AI reproduces the skew it was trained on and will not self-correct just because you ask it to.

And there is a baseline clinical disparity the AI inherits before it even gets to the image problem. A 2024 Komodo Health analysis found that Black patients received an incorrect diagnosis before a correct psoriasis diagnosis 18% of the time, compared to 10% for White patients. Hispanic patients came in at 14%, Asian patients at 15%. AI tools trained on clinical data absorb that skew rather than correct for it.

How large the performance gap is when AI tools are tested on darker skin

Seven percentage points sounds manageable in the abstract. A 2025 meta-analysis found AI diagnostic systems achieved a pooled AUROC of 0.89 for lighter skin tones versus 0.82 for darker ones. Survivable in isolation, perhaps. But translate that to population scale and you are looking at a large number of missed diagnoses concentrated in the patients who can least afford them.

The Stanford Diverse Dermatology Images dataset makes the picture harder to dismiss. Testing on that dataset found one widely cited model drop from an AUROC of 0.72 on the lightest skin tones to 0.57 on the darkest, approaching the floor of meaningful discrimination. A second model dropped to 0.50 on dark skin. That is statistically indistinguishable from random guessing. A 2024 evaluation of ChatGPT-4o found significantly lower sensitivity, specificity, and accuracy for melanoma detection in darker skin tones. The difference was not modest.

Then there is the MIT-led study that found dermatologists correctly classified 38% of skin images versus 19% for general practitioners. Both groups lost roughly four percentage points in accuracy on darker skin images, a drop the researchers described as statistically significant. AI assistance improved overall accuracy in that study. But the improvements were larger for lighter skin. AI, as currently built and deployed, can amplify the disparity it was supposed to narrow.

The pattern holds across all of these studies. Any AI system evaluated primarily on fair-skin benchmarks will appear more capable than it actually is for a substantial share of real patients. And because the output of these tools does not announce its own uncertainty, a patient with darker skin has no way of knowing from the result alone that the model is performing near its lower bound.

Why consumer skin analysis apps sit in a different, and riskier, regulatory category than clinical AI

Only three FDA-approved AI-powered dermatology devices exist globally for skin cancer detection, and all of them use specialized spectroscopic hardware, not smartphone cameras. There are no FDA-approved AI-based dermatology mobile applications. The entire consumer app market operates outside that regulatory framework.

It is worth being precise about what FDA clearance does and does not mean. Tools like MelaFind and DermaSensor achieved regulatory approval for specific, circumscribed use cases and have shown limited specificity in real-world practice. Clearance establishes a safety threshold, not a clinical effectiveness guarantee. But at least regulated clinical devices are beginning to face requirements around subgroup performance transparency. Consumer apps face no equivalent validation obligation. A tool with near-random performance on darker skin can be sold without any disclosure of that limitation, and the interface will look exactly as confident as one that works.

The population this affects most acutely is not arbitrary. Patients most likely to rely on low-cost consumer apps rather than dermatologist access are often those with the most constrained healthcare access, and that population overlaps substantially with the groups most affected by AI's performance gap on darker skin. The regulatory vacuum and the clinical disparity do not just coexist; they reinforce each other.

Regulatory direction is slowly moving toward transparency. The U.S. Office of the National Coordinator's 2024 HTI-1 rule requires algorithmic decision-support tools to expose their inputs, logic, and subgroup performance to clinicians. The EU AI Act classifies AI used to inform clinical decisions as high-risk, triggering conformity assessment, transparency requirements, and human-oversight obligations. The gap between what these frameworks demand of clinical AI and what the consumer market currently discloses is where the most immediate patient risk lives.

Three levers that actually reduce the performance gap: diverse data, validated benchmarks, and clinician oversight

These are not speculative interventions. Each is grounded in what existing evidence shows actually moves the needle.

Diverse training data

The root cause is dataset skew, so the fix has to begin there. Proactively collecting labeled clinical images across Fitzpatrick types IV through VI, and ideally across the broader Monk scale, from clinical partners who serve underrepresented populations is a prerequisite, not an enhancement. Volume alone is insufficient; a condition must appear on darker skin in training data, not just in aggregate representation. Where model quality is high, AI assistance has been shown to improve diagnostic accuracy among junior dermatologists from a modest baseline to a substantially higher level. That ceiling rises when the training data is representative enough to actually support it.

Validated, skin-tone-stratified benchmarks

ISIC and similar benchmarks were not designed with subgroup fairness as a primary goal, and performance reported against those datasets overstates real-world accuracy for patients with darker skin. The Stanford DDI dataset exists specifically to identify where models break down. Its adoption as a standard evaluation requirement, not an optional audit, is consequential. A systematic review found pooled AI sensitivity of 0.91 and AUROC of 0.88 across tens of thousands of test images, but aggregate figures obscure subgroup performance. Disaggregated reporting by skin tone must become standard practice, and evaluation frameworks built on Monk-based labeling rather than Fitzpatrick produce more granular, actionable subgroup results.

Clinician oversight as a structural requirement, not a disclaimer

The MIT finding established that AI helps clinicians more with lighter skin. An oversight layer designed without awareness of that asymmetry will perpetuate rather than correct it. One architecture worth examining: a clinical trial registered in December 2024 (NCT06724627) uses AI to provide reference images only, with no diagnoses, keeping clinical judgment with the clinician. That structure holds the model in a supporting role.

Asynchronous teledermatology, where AI assists and a licensed specialist reviews before findings reach the patient, extends access without removing the clinician from the decision loop. AI as triage and reference, clinician as decision-maker. That is not an idealistic arrangement. It is the structural design most likely to catch what a model alone will miss, particularly for patients whose skin presentation the model was least exposed to during training.

What a patient with darker skin should ask before trusting any AI skin tool

A tool that performs near-randomly on darker skin does not announce that failure. It returns an output that looks like confidence. That gap between internal uncertainty and external presentation is the core practical risk, and it is not something the interface will resolve for you.

Before trusting any AI skin tool, a few questions are worth asking explicitly.

Has the tool been tested on diverse skin tones, with that data published? Not "diverse patients" in a marketing description. Published, stratified accuracy or AUROC by skin tone. If the company cannot point you to that data, you are operating without it.

Is a licensed clinician reviewing the AI output before a recommendation reaches you, or is the AI the endpoint? These are architecturally different products with meaningfully different safety profiles.

Is the tool FDA-cleared, and for what specific use case? Clearance for melanoma detection on a specific spectroscopic device is not the same thing as validation for your skin concern on a smartphone camera.

Does the platform disclose its training data composition? That question alone will filter out most consumer apps quickly.

What good AI-assisted dermatology care looks like in practice is a model where AI surfaces patterns, supports clinical thinking, and routes findings to a licensed clinician who can contextualize what the model is underweighting. That clinician should recognize that darker skin presentations differ and should avoid relying solely on erythema-based severity scoring for inflammatory conditions where redness is not visibly apparent. Asynchronous consultations with licensed clinicians, the kind that make specialist-level review accessible without requiring an in-person appointment, are a meaningful structural improvement over AI-only consumer tools, particularly for patients with limited dermatology access.

Platforms like Nolla, built around multimodal clinical AI trained on labeled clinical cases and supervised by licensed clinicians, represent one architecture designed around this principle. Not because the AI is infallible, but because the human oversight layer is structurally integrated rather than appended as a disclaimer. The distinction between oversight as architecture and oversight as footnote is the most important thing to look for when evaluating any of these tools.

The gap between what AI currently does well and what patients with darker skin actually need is real and documented. It is also closeable. Knowing what to ask is a reasonable place for patient agency to begin.