January 2011



Recent Posts



Carl Heneghan

Using brain scans to detect autism would be a huge expensive waste of money, says Carl Heneghan

The BBC, the Guardian and Reuters this week widely reported British researchers published in the Journal of Neuroscience have developed a brain scan which can detect autism in adults with 90% accuracy. 

Dr Christine Ecker, the lead author, showed her imaging technique was able to detect which people in her group had autism. “If we get a new case, we will also hopefully be 90% accurate,” she said.

Pretty simple then, you turn up, have the test, and you have a 90% chance of finding out whether you have autism.

Well, you couldn’t be any further from the truth.

To determine if a test is accurate, it might appear reasonable to recruit a disease positive group and a disease free group which is what happened in the brain scan study. An example of how this strategy raises false hopes is the story of carcino-embryonic-antigen (CEA) which was measured in 36 people with known advanced cancer of the colon or rectum. 35 patients (97%) showed elevated results, slightly more than the 90% in the autistic study. At the same time lower levels were found in people with other diseases and without cancer.

From these results it would seem CEA is a useful diagnostic test. However, later studies in patients with less advanced cancer, or those with similar symptoms to colon cancer, the accuracy of CEA plummeted and its use in diagnosis was abandoned.

The authors of the current study report: “The existence of an ASD biomarker such as brain anatomy might be useful to facilitate and guide the behavioural diagnosis. This would, however, require further extensive exploration in the clinical setting.”

To obtain a useful result, a diagnostic study needs to include a broad spectrum of the diseased, from mild to severe. A study also needs to have independent, blind comparison of test results (in this case the brain scan) with a reference standard (the current tests for autism) among a consecutive series of patients suspected (but not known) to have the target disorder and replication of studies in other settings.

But this isn’t my main concern with the reporting of the results. If they stand up to scrutiny and brain scans are adopted widely in the population it will be an expensive waste of money. In those with a positive test, autism will be diagnosed with an accuracy of only 5%, potentially leading to more harm than good.

Dr Ecker said she hoped the findings might result in a widely available scan to test for autism.

Wait a minute, what has happened? One minute the world news is reporting a test that has 90% accuracy, and I’m saying it is only 5% accurate.

Gerd Gigerenzer in his classic BMJ paper on simple tools for understanding risks tells us: “A glance at the literature shows a shocking lack of statistical understanding of the outcomes of modern technologies, from standard screening tests for HIV infection to DNA evidence.”

Read in Full:  http://www.guardian.co.uk/science/blog/2010/aug/12/autism-brain-scan-statistics

Leave a Reply