What key performance metrics should clinicians understand when evaluating AI models?

examchapter-15

Key metrics include: Sensitivity/recall (proportion of true positives correctly identified — critical for screening), Specificity (proportion of true negatives — important where false positives carry harm), AUROC (summary discrimination metric from 0.5 to 1.0), Calibration (agreement between predicted probabilities and observed outcomes), and Net benefit/decision curve analysis. An AUROC of 0.90 is meaningless if the model is poorly calibrated, biased, or tested only on unrepresentative data.