March 2024: Digital Tools for Tuberculosis Management

An evaluation of five artificial intelligence algorithms

Lancet Digital Health, 2021

Z.Z.Qin et.al

This was a retrospective, evaluative study of 5 commercially available AI algorithms for tuberculosis triage conducted in Dhaka, Bangladesh. Three registered radiologists assessed chest X-rays and categorized them as normal or abnormal (3 categories of abnormal), which was then compared with the assessment done by the computer-aided detection (CAD) software to assess its accuracy and other key indicators for triaging tools.

“Our subanalysis showed that the performance of computer-aided detection varied wit demographic and clinical factors, as well as patient source, and therefore implied that variation exists in AI performance across different contexts and geographies.”

Key Takeaways:

• AI-powered software is increasingly being used to analyze medical images. In March 2021, WHO updated its TB screening guidelines to recommend CAD software for the analysis of digital chest X-rays.
• The study found that all 5 AI algorithms performed significantly better than the expert radiologists in detecting tuberculosis-related abnormalities, but with a relatively poorer ability to differentiate when it came to people over 60 years old and those with a history of TB.
• While AI algorithms are commonly evaluated to assess accuracy, the authors recommended a broader evaluation framework that includes implementation indicators; this can inform software choice selection as well as thresholds for abnormality scores.

Comment from the Center for Global Digital Health Innovation

While the study demonstrated that AI algorithms have the ability to triage TB cases, performance differences require further research to inform the scaling-up of such CAD software. Further, rapid innovation in AI performance means evaluations and implementation guidelines must keep pace with these changes. Other studies that aim to evaluate similar CAD software should adopt such evaluation frameworks and include indicators that aid implementers in making informed decisions. 

A Systematic Review and Meta-analysis

Open Forum Infectious Diseases, 2024

A. J.Scott et.al

This was a systematic review of studies(n=5) that evaluated the use of CAD as a screening tool to detect pulmonary tuberculosis with accuracy estimates as the primary outcome measure within Africa.

“In the context of active case finding, CAD has the potential to be a useful and cost-effective screening tool for TB in a resource-poor, HIV-endemic African setting, assisting active case finding strategies to break the TB transmission cycle.”

Key Takeaways:

• The study found that CAD showed promise as a screening tool for pulmonary TB with pooled accuracy just below the WHO-recommended target product profile. However, a quality assessment of the studies showed a high risk of bias due to several factors.
• The authors commented that setting a threshold value is an important consideration when evaluating CAD software, as more sensitive thresholds result in high costs due to the need for more confirmatory microbiological testing.
• Identifying active cases in low-resource settings can be difficult, and CAD software offers a potential solution with good accuracy and cost-effectiveness.

Comment from the Center for Global Digital Health Innovation

As in the first study, this study highlighted the need for further research using adaptive techniques to evaluate CAD software. Limited published data has evaluated the use of CAD for active case finding, even though a large number of TB cases are identified in asymptomatic patients who don’t visit health facilities for diagnosis. Additionally, there is scope to evaluate the accuracy of these tools by stratifying populations and clinical characteristics, such as for people living with HIV.

A stepped-wedge cluster-randomized trial in Uganda

PLoS Medicine, 2021

A.Cattamanchi et.al 

This was a step-wedge cluster-randomized trial (n=1913, control=1022, intervention=891) to assess improvement in treatment outcomes with 99DOTS, a low-cost digital adherence technology (DAT), in Uganda.

“Our trial confirms that 99DOTS—and likely other DATs—are unlikely on their own to substantially improve population-level TB treatment outcomes when implemented as part of routine care.”

Key Takeaways:

• The authors suggest that 99DOTS (and similar DAT) should “not be used as a universal replacement for directly observed therapy” (DOT) in tuberculosis management as their findings found similar treatment outcomes and completion rates between the intervention and control groups.
• However, DAT could be used to enable DOT for patients given greater convenience and lower costs—especially for patients with phone access who are interested in adopting the technology.
• A key limitation of the study was selection bias as practitioners could assign patients to the intervention arm.

Comment from the Center for Global Digital Health Innovation

This study was one of the first few that aimed to assess the effect of DAT on treatment outcomes, as opposed to only treatment completion rates or loss to follow-up. Future research can aim to assess the effectiveness of DAT on population-level treatment outcomes and whether supportive initiatives (e.g., providing cell phones to patients) could improve the uptake of the intervention. As DATs like 99DOTS help reduce the burden on patients in accessing therapy, there is a need to research how these tools can be implemented effectively.

A multicentre randomized controlled trial

Lancet, 2024

X. Wei et.al

This study was a multicenter randomized controlled trial (n=278, control=135, intervention=143) in Tibet (six counties in Shigatse) that assessed treatment adherence in TB management through a comprehensive package including electronic medication monitors, a linked smartphone for patient-provider communication, a free data plan, and a “treatment supporter.”

“All intervention components were co-designed with partners in Tibet to ensure good understanding of the cultural context. Indeed, systematic reviews suggest that digital adherence technologies alone will not improve treatment unless they also improve communication between patients and their care teams.”

Key Takeaways:

• There was a significant difference in rates of poor adherence between the intervention and control group with the intervention group having fewer missed doses, better treatment outcomes, and improved sputum conversion at the end of the second month.
• This study was one of the first trials to report such significant improvements in TB therapy adherence and treatment outcomes, perhaps due to comprehensive package design, newer electronic medication monitors providing real-time feedback, better uptake of the intervention among the study group, or a very low baseline adherence rate.
• One of the trial’s strengths was that the feasibility and acceptability were tested in a pilot, which enabled the intervention to be adapted to the local context—in turn, maximizing uptake and use.

Comment from the Center for Global Digital Health Innovation

As one of the first studies that showed promise in leveraging DAT for tuberculosis therapy success, this study provides several insights for developing interventions to improve drug adherence and treatment outcomes. Crucial to the success of the intervention were its multi-component, localized, and “comprehensive” intervention design and focus on implementation outcomes such as fidelity and acceptability. Future evaluations of DAT should adopt an implementation science lens for accurately assessing impact. 

A household randomized trial

ERJ Open Res (ERS), 2019

J. L.Davis et.al

The article describes a randomized trial (n=919, control=448, intervention=471) of home-based TB contact investigation and SMS-facilitated results sharing in an urban-based setting in Kampala, Uganda. 

“As improving the quality of TB care assumes greater priority on the global TB research and programme agenda, such multilevel, multimodal evaluations are critical to understanding the context, fidelity, and adaptability of new interventions.”

Key Takeaways:

• The intervention aimed to target the low percentage of household contacts of TB patients completing tuberculosis testing, as several trials have demonstrated the role of active case finding in reducing community transmission.
• The researchers found no significant difference in tuberculosis testing rates between the home-based arm versus the clinic-based control, with only one in five eligible household contacts completing the tuberculosis testing process.
• The failure was due to the roadblocks in implementation, with several challenges such as difficulty for asymptomatic individuals to provide samples, limited private space for sample collection, apprehensive community health workers who feared contracting TB, and barriers to mobile phone access for SMS communication.

Comment from the Center for Global Digital Health Innovation

While the study was designed with some input from the health workers and household contacts, it faced several challenges in implementation due to the complexities of the active case-finding process. A more user-centered approach would have helped overcome some of the barriers faced while deploying digital interventions in the community.