Explainable AI in Medical Diagnostics: A Comparative Evaluation of Interpretability Methods

Abstract

As deep learning models gain traction in medical diagnostics, concerns have arisen regarding their
interpretability and trustworthiness—especially in high-stakes domains like radiology and pathology. This
paper provides a comparative evaluation of leading explainability techniques applied to convolutional neural
networks (CNNs) used in medical image classification. We implement and analyze three popular
interpretability methods—Gradient-weighted Class Activation Mapping (Grad-CAM), Layer-wise Relevance
Propagation (LRP), and SHAP (SHapley Additive exPlanations)—on CNN models trained on chest X-rays
from the NIH ChestX-ray14 dataset to detect conditions such as pneumonia and cardiomegaly. Each method
is evaluated based on fidelity, localization accuracy (compared to radiologist-annotated regions), and clinician
interpretability via a structured survey with 12 medical professionals. Grad-CAM exhibits the best visual
coherence with pathological regions, while LRP provides more granular relevance maps. SHAP offers superior
feature-level insights, especially when used with auxiliary patient metadata. However, the added complexity of
SHAP explanations made them harder for clinicians to interpret without training. All three methods improve
clinician trust compared to black-box outputs alone. The study concludes that no single method is universally
superior—each has strengths depending on the task, data modality, and end-user profile. Our findings suggest
a hybrid approach, combining region-based and feature-based explanations, may offer the most robust path
toward explainable AI in healthcare. This work informs both developers and clinical stakeholders seeking to
safely integrate AI into diagnostic workflows.