Multimodal AI Models in Dermatology Assessment

Dermatologists are scarce relative to the U.S. population,, and that shortage is concentrated exactly where patients can least afford it: rural counties, small towns, entire regions with one overbooked practice serving a hundred-mile radius. Multimodal AI models, systems that read a skin photo alongside a patient's symptoms and history, are being built and tested against that shortage. This piece walks through what these models actually do, where the published numbers hold up, and where a clinician still has to be in the loop.

Start with the wait itself. Surveys of physician appointment times have consistently found dermatology among the longest waits of any specialty. Research on patients waiting for dermatology appointments has found that a substantial share report their symptoms worsening before they are seen. In that gap, people reach for over-the-counter creams or herbal remedies, not out of preference but because there's nothing else on offer.

The shortage isn't a temporary staffing hiccup. It's structural. The Balanced Budget Act of 1997 froze the number of Medicare-funded residency slots, which capped how many new specialists the system could produce for decades afterward. The Association of American Medical Colleges has projected that the country could face a substantial physician shortfall in the coming decade. Dermatology has already adapted around the gap: advanced practice providers have taken on a growing share of dermatology prescribing. The specialty has been operating past its traditional staffing model for a while now. That's the backdrop against which multimodal AI is being developed, tested, and argued about. Not as a novelty, but as a response to patients already making decisions without a specialist anywhere near the conversation.

What "multimodal" actually means in a dermatology AI system

A multimodal system takes in more than one kind of input at once. Usually that means a photo (a clinical picture or a dermoscopic image taken with a magnifying attachment) paired with text: symptom history, patient demographics, answers to an intake questionnaire, sometimes lab results. An image-only model just looks at the picture. A multimodal model reasons across what it sees and what it's told about the patient.

That distinction isn't academic, and the numbers make it obvious. ChatGPT-4 reached 87.7% Top-1 diagnostic accuracy when it had both clinical images and patient metadata from a teledermatology platform to work with. Given clinical images alone, pulled from the DermNet image library, its accuracy dropped to 23%. That's not a small gap. It's the clearest single piece of evidence that "multimodal" is describing a real functional difference, not a marketing label stuck onto a photo classifier.

How do researchers actually build these things? Two approaches show up often in the literature. One pairs a deep learning image encoder, a deep learning image encoder paired with a transformer-based language model to pull meaning out of the text. The other uses a two-stage training pipeline: teach the model to describe what it sees in a lesion first, then teach it to reason toward a diagnosis. Nolla, an AI-guided skincare telehealth service, sits at a different point on that continuum, routing AI-assisted intake to licensed clinicians who make the final call. SkinGPT-4 is built this way, and it's one of the few multimodal models designed specifically for dermatology rather than adapted from a general-purpose system.

Hardware plays a role too. Facial skin analysis devices used in aesthetic dermatology clinics capture image, texture, and pigmentation data all at once and feed it into machine-learning models. Different form factor, same underlying idea: more than one data stream, reasoned over together.

One problem researchers are still working through is what's been called "diffuse attention": a general-purpose vision-language model tends to treat the background skin around a lesion as just as relevant as the lesion itself, which muddies its output. A newer architecture called SkinFlow, developed by Baichuan Inc. along with Tsinghua University and Peking University First Hospital, tackles this with dynamic visual encoding and staged reinforcement learning. On the Fitzpatrick17k benchmark, SkinFlow's 7-billion-parameter model beat much larger general-purpose models by 12.06 percentage points on Top-1 accuracy and 28.57 points on Top-6 accuracy. The architecture, in other words, decides what clinical context the model is even capable of absorbing, and that shapes both how well it performs and how it fails.

How these models perform when tested against real clinical cases

Zoom out to the population level and the picture gets more nuanced. A 2025 systematic review and meta-analysis, pooling results across more than 70,000 test images, found sensitivity of 0.91 and specificity of 0.64, with an area under the curve of 0.88. Sensitivity is the number that matters most for screening: missing a melanoma is a far worse mistake than flagging something benign as suspicious. But a specificity of 0.64 means these models are also generating a fair amount of noise, false alarms that someone still has to sort through.

A 2025 retrospective study at Indiana University offers a sharper comparison. Researchers tested DermFlow, a dermatology-trained multimodal model, against a general-purpose model on 59 patients with 68 biopsy-proven pigmented lesions. DermFlow hit 47.1% accuracy on the exact top diagnosis, but 92.6% accuracy when any of its listed possibilities included the right answer. Its F1 score was 0.948, sensitivity 93.9%, specificity 89.5%, balanced accuracy 91.7%. Worth noting: DermFlow had both patient history and images to work with, while the general-purpose model only got images. That's not an apples-to-apples comparison in the strictest sense, but it does reflect how these tools actually get deployed in practice.

Controlled studies often report deep learning classification accuracy above 90%, according to a 2025 review. But controlled benchmarks use curated images, shot under good lighting, at decent resolution. Real patient photos, taken on a phone in a bathroom with mixed lighting, don't look like that. So how much of that 90%-plus figure survives contact with an actual smartphone camera at 11pm?

There's a more encouraging data point buried in a study by Li et al., which tested junior dermatologists diagnosing lupus erythematosus subtypes and other inflammatory skin diseases. Their baseline accuracy was 66.9% (give or take 6.9 points). With AI assistance, that jumped to 81.3% (give or take 4.2 points). That baseline matters, because a junior dermatologist working without much specialist backup is closer to what most patients outside major cities actually encounter. The AI isn't outperforming a seasoned expert here. It's raising the floor for someone who's still learning.

All of these figures, vendor-reported or academic, should be read as ceilings, not averages. Real-world image quality, population diversity, and the mess of an actual clinical workflow all pull performance down from what a controlled study can show.

Where multimodal AI still falls short: the skin tone equity problem

AI dermatology models trained on imbalanced image sets perform worse on darker skin tones, and the failure mode that matters most is a higher false negative rate for melanoma, a diagnosis where a miss can be fatal.

A July 2025 study out of the University of Florida, published in Cureus, tested ChatGPT-4o on 324 biopsy-confirmed dermatologic images spanning the Fitzpatrick skin type scale. For types III and IV, it dropped to 29%. For types V and VI (darker skin), it landed at 43%. Diagnostic accuracy followed the same pattern: 71% for the lightest skin types, 42% for the darkest. The study found these differences statistically significant, not just noise.

Why does this happen? Follow the problem upstream, to the training data itself. A 2025 study that looked at 4,000 AI-generated dermatological images found only 10.2% depicted dark skin, and only 15% of images overall accurately showed the condition they were supposed to represent. The bias isn't a flaw the model invented on its own. It's a reflection of what it was shown.

This isn't a rounding error to be patched with a software update later. A model that misses melanoma more often in patients with darker skin risks reproducing, and possibly deepening, the very access gap AI is supposed to help close. Researchers studying this problem consistently call for more representative datasets, but existing regulatory frameworks have not kept pace with the skin tone equity problem these models present. For patients with darker skin tones specifically, the practical takeaway is straightforward: AI-assisted assessment belongs in a supportive role, not a decisive one, and clinician review matters more, not less, in exactly the populations where specialist access is already thinnest.

How the FDA regulates AI dermatology tools and what is still unregulated

The regulatory field here is narrow. A 2025 comprehensive review counted fifteen regulatory-approved AI devices worldwide for dermatology, including three approved by the FDA in the U.S. All three rely on specialized hardware, spectroscopy devices built for skin cancer detection, not software apps a patient downloads.

A meaningful milestone came on January 17, 2024, when the FDA authorized DermaSensor, the first AI-enabled medical device cleared for skin cancer detection in a primary care setting. That approval opened a regulatory pathway for AI and machine learning tools in dermatology that didn't really exist before.

But step outside that narrow lane and the picture changes fast. As of 2025, there are no FDA-approved AI-based dermatology apps for consumers. Most of the direct-to-consumer AI skin tools on the market, including general-purpose models people use informally to analyze a photo, have neither regulatory approval nor the clinical evidence to back their claims. Anyone using one of these tools is operating entirely outside a reviewed safety framework, whether they realize it or not.

The FDA's most recent move, guidance issued in December 2024 on managing "Predetermined Change Control Plans" for AI-enabled device software, is about how an already-approved model can be updated over time. It's a framework for managing change, not a signal that approvals are about to expand broadly. Patients seem to sense the gap themselves: patient hesitancy toward AI diagnoses that don't involve a clinician in the loop is a recurring theme in the literature. That hesitation isn't irrational. It lines up with what the evidence actually shows. The practical split to keep in mind: an FDA-reviewed device used in a clinical setting is a different category of thing entirely from a consumer app that analyzes a photo on a phone. Treating them as equivalent overstates the evidence behind either one.

How teledermatology carries multimodal AI into actual clinical workflows

Teledermatology runs on two models. Real-time, synchronous video visits work like a video call with a specialist. Store-and-forward is asynchronous: a patient submits images and history, and a clinician reviews the case later. Store-and-forward is also the natural place for AI to plug in, since a model can process the images and intake data before a clinician ever opens the file, which can speed up how cases get triaged.

The infrastructure for this kind of care exploded fast. Medicare telehealth visits went from 840,000 in 2019 to nearly 52.7 million in 2020, a 63-fold jump. Dermatology AI tools are being built on top of that infrastructure, not from scratch.

What does this look like in an actual clinic? LMU University Hospital Munich ran 629 teledermatology consultations between July 2023 and December 2024. The most common diagnoses were acne (6.4%), atopic dermatitis (5.4%), and hand eczema (3.3%), and inflammatory skin conditions made up 42.7% of all cases. Median response time was 21.4 hours, and 65.3% of consultations ended in a prescription. These are exactly the kinds of cases where a clinician can act on a photo plus a symptom history, without needing the patient in the room. That's the workflow multimodal AI is built to support.

Does this actually widen access, or just move the bottleneck around? A Penn Medicine study found that access to a store-and-forward teledermatology triage service cut the need for in-person visits by 27% and reduced emergency department visits by 3%. That's a real, measurable reduction in pressure on a system that's already stretched thin.

Teledermatology has limits, and most dermatologists agree it isn't a stand-in for a full-body skin exam. A blurry photo or a shaky connection can hide detail that a clinician needs to see clearly. Neither AI nor a virtual visit works when what's actually called for is a hands-on exam or a biopsy. The conditions that show up again and again in teledermatology literature, such as acne, atopic dermatitis, and inflammatory skin diseases, are the same conditions where a photo and a good history are usually enough to work with.

What clinician oversight actually looks like in an AI-assisted skin assessment

Strip it down to the mechanics: a multimodal model takes in an image and structured intake information, and produces a list of possible diagnoses or flags something as concerning. It does not diagnose. A licensed clinician reads that output against the full picture of the patient in front of them, and makes the call.

Why does that step matter so much, given everything above? Put the pieces together: the skin tone bias data, a population-level specificity of just 0.64, and the gap between domain-trained and general-purpose model performance all point in the same direction. AI output without a clinician reading it is least reliable in exactly the situations where reliability matters most.

But the augmentation effect is real too, and worth holding alongside the caution. Junior dermatologists in the Li et al. (2024) study went from 66.9% to 81.3% accuracy with AI assistance. The operative word there is "with." The AI raised the floor for a less experienced reviewer. It didn't replace the reviewer.

Asynchronous care with a clinician reviewing the AI's output, patient submits photos and history, the model pre-processes the case, a licensed provider reviews and responds, is a clinically defensible structure for most everyday skin concerns. It's the setup that makes fast, affordable dermatology assessment trustworthy rather than just fast.

So what should a patient actually ask before trusting one of these tools? A few questions cut through most of the marketing:

• Is a licensed clinician reviewing the AI's output, or is the AI's answer the final word?
• Has this tool been tested on skin tones similar to the patient's?
• Does this platform handle data under HIPAA-compliant standards?
• What happens if the AI flags something urgent? Is there a clear next step to in-person care?

The honest ceiling for AI-assisted teledermatology right now is a first step: something that improves triage, surfaces a reasonable set of possibilities, and gets a patient connected to a clinician faster than they'd otherwise manage. It is not a replacement for a biopsy, a full-body exam, or a specialist follow-up when the case actually calls for one.