June 2023: Chatbots in Health—What Do We Know?

Parmar et al, Health-focused conversational agents in person-centered care: a review of apps, npj Digital Medicine, 2022

Summary & Takeaways: 

This scoping review reports the use of healthbot apps across 33 countries. Of the 78 apps reviewed on Apple iOS and Google Play stores, 47 (31%) apps were developed for a primary care domain area and 22 (14%) for a mental health domain. There were only six (8%) apps that utilized a theoretical or therapeutic framework underpinning their approach, and five of these six apps were focused on mental health. Only a few apps use machine learning and natural language processing approaches, despite marketing claims. Most apps allowed for a finite-state input, where the dialogue is led by the system and follows a predetermined algorithm. The Internet and health apps have become essential for accessing health information, with over 318,000 health apps available globally. Integrated person-centered health services enable individuals to make informed decisions about their health.

Comment from the Center for Global Digital Health Innovation:

Healthbot apps can fill healthcare gaps by complementing face-to-face care. The lack of a theoretical or therapeutic underpinning, however, is of concern. While healthbot technology is still in nascent stages and few use machine learning, the use of natural language processing can help further personalize the information chatbots are able to provide.

Abd-Alrazaq et al, Effectiveness and Safety of Using Chatbots to Improve Mental Health: Systematic Review and Meta-Analysis, JMIR , 2020

Summary & Takeaways: 

This systematic review and meta-analysis study included 12 studies examining the effect of using mental health chatbots on 8 outcomes – depression, anxiety, positive and negative effects, subjective psychological well-being, psychological distress, stress, acrophobia, and safety. Meta-analysis was included for 4 of those studies. The studies were conducted in more than 11 countries and published between 2015 and 2018. Some studies reported a significant decrease in the severity of depression. However, the results on most other outcomes were conflicting, and the associated studies had a high risk of bias resulting in weak conclusions.

Comment from the Center for Global Digital Health Innovation:

 The results highlight the need for more robust studies to assess the impact of chatbot interventions on health outcomes. However, chatbots may serve as an adjunct to already available interventions to encourage individuals to seek medical advice where appropriate. 

Wang et al 2022, An Artificial Intelligence Chatbot for Young People’s Sexual and Reproductive Health in India (SnehAI): Instrumental Case Study, JMIR

Summary & Takeaways: 

This case study applied the Gibson theory of affordances to analyze SnehAI, a Hinglish (Hindi + English) AI chatbot designed to educate and empower Indian adolescents and young adults on sexual and reproductive health, and advocate for the rights and health entitlements of women and girls. With 8.2 million messages exchanged over 5 months, the need and acceptability of such a resource suggests that SnehAI served as a trusted friend and mentor, addressing personal questions on sexual and reproductive health through entertaining and educational content. Its natural language processing system enhanced user experience through personalized responses. Gender disaggregation of the users showed that 93% of the users were male. Topics of conversation included safe sex practices (e.g., consent, oral and anal sex, etc.), family planning methods, female reproductive health (e.g., menstruation, infertility virginity, etc.), adolescent sexual health issues (e.g., masturbation, STIs,etc.), adolescent mental health (e.g., bullying, peer pressure), social issues (e.g., child marriage, gender equality, etc.).

Comment from the Center for Global Digital Health Innovation:

With a high-volume of exchanges and robust gender data, this study offered insights into the design of chatbots and their effective engagement with users, highlighting opportunities to address gaps in sexual and reproductive health education, particularly in harder-to-reach populations. 

Li et al, Feasibility and effectiveness of artificial intelligence-driven conversational agents in healthcare interventions: A systematic review of randomized controlled trials, International Journal of Nursing Studies, 2023

Summary & Takeaways: 

This systematic review examined 21 articles between 2013-2021 to assess the quality of information generated by AI-driven conversational agents. Approximately half were conducted in the United States, while the remaining studies took place in Europe, Australia, and Asia. It explored the practicality and effectiveness of interventions that utilized virtual conversational agents designed to mimic human interactions using machine learning, natural language processing programs and AI technology. While participants showed inconsistent effects on improving physical activity, functional capacity, and mental health, counseling sessions with the conversational agent led to reduced negative thoughts and emotions. 

Comment from the Center for Global Digital Health Innovation:

Conversational agent-based interventions have been found to be viable, well-received, and practical options in healthcare and nursing interventions. Future studies should evaluate conversational agent interventions in developing countries where there is a greater shortage of health professionals.

Lee et al, Effectiveness of Chatbots on COVID vaccine confidence and Acceptance in Thailand, Hong Kong, and Singapore (2023), npj Digital Medicine, 2023

Summary & Takeaways:

This multisite, parallel randomized controlled trial study, conducted in 2022 and including 748 adult guardians of children and seniors who were unvaccinated or had delayed vaccinations, found that chatbots were more effective in improving vaccine confidence and acceptance among unsure minorities and individuals with lower education levels. Among children, chatbot users, compared to non-users, had decreased confidence in vaccine effectiveness, decreased vaccine acceptance, and decreased confidence in vaccine safety. Among seniors, the Thailand invention group reported decreases in vaccine confidence for reducing severe conditions and misinformation awareness about COVID-19 vaccine clinical trials.

Comment from the Center for Global Digital Health Innovation:

The findings suggest inconclusive evidence regarding the ultimate effectiveness of chatbots on vaccine confidence, uptake across different populations, and the possibility of ‘backfire’ effects, where pro-vaccine messaging delivered on social media can be counterproductive. Given these results, evidence-led strategies for behavior change in the development of chatbots need to be incorporated in studies focused on using communication strategies to change behavior and improve health outcomes.

Q.J. To et al, Feasibility, Usability, and Effectiveness of a Machine Learning–Based Physical Activity Chatbot: Quasi-Experimental Study (2021, JMIR,  2021

Summary & Takeaways: 

This quasi-experimental design study took place over the course of six weeks and included 116 people in Australia wearing a chatbot-connected Fitbit Flex 1. The goal of this study was to investigate the feasibility, usability, and effectiveness of a machine learning–based physical activity chatbot, delivered via the Facebook Messenger app. The results showed that, for a little over half of the participants, the chatbot helped them increase their physical activity. About a third of the participants continued to use chatbots in the future and felt like the chatbot was very useful in helping them increase confidence and stay motivated. Most participants, however, experienced technical issues with the Facebook Messenger app and stopped receiving the chatbot messages during the study.

Comment from the Center for Global Digital Health Innovation:

This study highlights the importance of addressing technical limitations and ensuring uninterrupted functionality to enhance user experience. It also touches on considering the use of independent platforms to deliver interventions in a manner that does not disrupt the intervention, as occurred in this case due to a Facebook policy change.

Fadhil et al, Designing for Health Chatbots (2019), arXiv , 2019

Summary & Takeaways: 

This systematic literature review analyzed 103 articles on conversational interfaces and chatbots, resulting in a major survey of the domain. It noted that many researchers highlighted their focus on user experience (UX) design principles and interaction patterns for conversational interfaces in healthcare. The authors of this review also found that research goals largely appeared to be the support of practitioners in exploring opportunities and indications in interaction designs for conversational agents. As such, these studies mainly involved discussions on predicting user intent and how to personalize conversations to fit user preferences; designing for user engagement with the chatbot received less emphasis. 

Comment from the Center for Global Digital Health Innovation:

While there remains significant room for improvement in enhancing the dialogue of chatbots, this review serves as a valuable guideline for designing domain-specific interfaces for conversational agents. Additionally, empathic dimensions in the conversation can create more engaging and empathetic interactions.