TTuberculosis kills 1.4 million people every year, mostly in places where poverty and deprivation conspire to make people especially vulnerable and unable to get lifesaving care in time.
Google is now joining a global fight to snuff out the disease, using AI to automate its detection — and speed up treatment — in communities where doctors are in short supply. A new study published Tuesday in Radiology, the journal of the Radiological Society of North America, found that its AI model performed as well as radiologists in detecting tuberculosis on chest X-rays.
Google isn’t the first to develop an AI system to detect TB, and its tool isn’t likely to make a dent in death rates anytime soon. But outside experts said its early results are particularly promising given their consistency across diverse patient populations. The model met or exceeded performance standards set by the World Health Organization when tested on historical patient data from China, India, the United States and Zambia.
– Advertising –
“Unlike most published AI data, the (Google) study was large and used different training sets, which showed their system to be robust,” said Edith Marom, head of the AI. chest imaging at Chaim Sheba Medical Center in Israel.
Marom, who was not involved in the research, added that Google’s tests did not match real-life circumstances, however. The datasets contained higher than normal rates of disease and were skewed towards patients who were younger and able to withstand upright X-rays – conditions that generally make images easier to interpret. Its performance also fell among sicker populations with more lung abnormalities, such as HIV-positive patients and a group of miners in South Africa.
“To be applicable worldwide, it would need to be tested in populations with low TB prevalence resembling typical patients” with multiple abnormalities in the chest, Marom said. “It will also have to be tested on an older population, generally encountered in a hospital setting. »
Effective treatments for tuberculosis, which usually attacks the lungs, have long been available. But 90% of cases occur in 30 countries that often lack the resources to effectively screen patients, isolate them and provide the care they need.
Google’s model is designed to be used for screening rather than diagnosis. He analyzes X-ray images to determine which patients should undergo follow-up molecular testing to confirm the presence of tuberculosis. Such tests are expensive, time-consuming and impractical in large groups, which is why the WHO has concluded that chest X-ray – augmented by AI – is an essential tool in the fight against the disease.
Google’s tool rates chest X-rays for detecting tuberculosis on a scale of 0 to 1, with a higher score equaling a greater likelihood that the disease is present. In the study, the company’s researchers calibrated the tool to recommend follow-up testing at a threshold of 0.45, which turned out to be the right choice, as it proved to be very sensitive in catching the disease without generating high rates of false positives.
The researchers said a primary goal of the work was to expose the AI to a variety of patients and clinical circumstances to ensure it would not be triggered by variations that normally occur in different areas. geography and care contexts.
“We tried to be very thorough with the validation of this one to explore different presentations of TB, different X-ray manufacturers and countries,” said Rory Pilgrim, product manager for Google’s AI team. Health and co-author of the study. He added that the model was trained on data where TB cases were confirmed by molecular testing, rather than radiological results whose diagnostic accuracy varies widely.
The researchers also used a training method known as ‘noisy student’ to help improve the model’s ability to recognize TB despite the different ways it can appear on X-ray images. This technique allowed them to expose the AI to data that was not explicitly labeled as TB positive or negative. Normally, models are only trained on data that is explicitly marked with the outcome of interest, so its trainers can tell the model whether its conclusions were right or wrong.
Under a noisy student, the model uses its training on past data to generate its own labels for new images it encounters. This allows him to iterate and improve without limiting his exposure to only labeled examples.
“It allowed us to take advantage of a lot more data from a TB screening population,” said Sahar Kazemzadeh, a Google software engineer who led the development of the model. The population in question consisted of data on South African miners, who had a higher level of lung disease, presenting the AI model with a higher degree of complexity.
“As the model sees more iterations and permutations, its generalizability increases,” Pilgrim said. “It becomes less narrow” and able to detect TB in a wider range of circumstances.
Google’s AI system, along with many other similar systems, could dramatically improve TB outcomes and reduce costs if it overcomes additional testing hurdles.
The next challenge is to examine its performance in a real environment. Google is currently pursuing a study at a clinic in Zambia, where the accuracy of the tools’ results will be measured against the results of molecular tests for each patient. The study should be completed by the end of the year.