???Explainability??? Is a Poor Band-Aid for Biased AI in Medicine

This scenario is closer than you think. In fact, scenarios like this are already happening in health systems around the country, sometimes in pilot programs, sometimes with more full-fledged integration. But the point remains, at some point clinician’s diagnoses are going to be “aided” by AI.

What’s the problem with AI predictions? Well, people often complain it’s a “black box” — sure it may tell me it thinks the diagnosis is heart failure, but I don’t know WHY it thinks that. To make AI work well with clinicians, it needs to explain itself.

Website