If a computer gives you a diagnosis, should it also give you an explanation?

Jurors from previous Citizens’ Juries

by Malcolm Oswald

Artificial intelligence (AI) plays an increasing role in our daily lives. Computers are being trained how to do many things, including making medical diagnoses. For example, AI can diagnose skin cancer from skin images as reliably as dermatologists, and this clever software is only going to get better. But how do we know whether a diagnosis we get is accurate? If we are given it by a human doctor, we can ask for an explanation. However, the most advanced AI systems are very complex and do not just act according to pre-defined rules, but continue to “learn”, and it may not be possible to explain to a patient how the computer reached its diagnosis.

If you were given a diagnosis by a computer, and were given the choice, would you always prefer to be given an explanation of how the computer reached its diagnosis even if that meant the computer’s diagnosis was likely to be a little less accurate?

That is one question being put to two “citizens’ juries” being commissioned by the NIHR Greater Manchester Patient Safety Translational Research Centre (Greater Manchester PSTRC) in early 2019. Citizens Juries c.i.c. will recruit 18 people from around Manchester – chosen to represent a cross-section of the public – to come together for five days to hear expert evidence and tackle difficult questions concerning how AI should be used within healthcare. The process will then be repeated with 18 different people from around Coventry to see whether they reach the same conclusions.

The Greater Manchester PSTRC is collaborating on this project with the Information Commissioner’s Office which has the challenging task of regulating the use of AI. The results of the juries will feed directly into national guidance that the Information Commissioner’s Office has to produce on citizens’ rights to an explanation when decisions that affect people are made using AI.

For more information about citizens’ juries, see the Citizens Juries c.i.c. website, or if you have a specific enquiry about this project, email Dr Malcolm Oswald or the principal investigator, Prof Niels Peek.