This AI Can Detect Any Disease from a Photo

Various AI health apps and startups have marketed themselves as being able to diagnose skin conditions, eye diseases, and other health issues from a smartphone photo. Some claim accuracy rates of 95% or higher, comparable to or exceeding dermatologists and ophthalmologists.

AI diagnostic tools have shown genuine promise in controlled studies, particularly for skin cancer detection and diabetic retinopathy screening. However, the gap between benchmark performance and real-world reliability is significant.

Study conditions use high-quality, standardized images with clear pathology. Real-world photos are taken in variable lighting, at different angles, with consumer cameras, on diverse skin tones. Many studies that report high accuracy were tested on the same type of data they were trained on, which inflates performance.

Skin condition AI tools have shown significant accuracy disparities across skin tones, performing worse on darker skin. This is a critical limitation that is often absent from marketing claims.

The claims are overblown because they present benchmark accuracy as clinical reliability. A tool that is 95% accurate in a lab setting might be 70% accurate in real-world conditions. In healthcare, that gap can mean missed diagnoses, unnecessary anxiety, and delayed treatment.

AI diagnostic tools are valuable as screening aids that help clinicians prioritize cases, not as standalone diagnostic systems. The responsible framing is "AI-assisted diagnosis" not "AI diagnosis." No credible medical authority recommends using consumer AI apps as a substitute for professional medical evaluation.

Nature Medicine: AI in dermatology - https://www.nature.com/nm/

FDA: AI/ML-based medical devices - https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-aiml-enabled-medical-devices

JAMA Dermatology: Bias in dermatology AI - https://jamanetwork.com/journals/jamadermatology