Implementation of Machine Learning in Cluster for Reviews and Health Technology Assessments: Results for ML 3.0
Borge, Tiril Cecilie; Ames, Heather Melanie; Jardim, Patricia Jacobsen; Bergsund, Hans Bugge; Larsen, Martin Smådal; Rose, Christopher
Abstract
In 2020, the Cluster for Reviews and Health Technology Assessments (HTV) at the Norwegian Institute of Public Health (NIPH) established a dedicated machine learning (ML) team. The ML team has since become an international leader in integrating and implementing ML into evidence synthesis, achieving significant milestones, and securing official financing in November 2022, which contributed to much of the ML activities performed by ML 3.0. The overall goal of the ML team is to use ML in a way that best combines human intelligence and ML, to enhance human activities, by figuring out how best to integrate ML and workflow changes, throughout the review process. This report outlines the team's activities during its iteration, ML 3.0, covering implementation, peer-to-peer support, dissemination, evaluations, innovation, horizon scanning, and external networking and collaborations. ML Team 3.0 accomplished a variety of project deliverables, including providing ML support to six teams, conducting teaching sessions, implementing an ML reporting template, and implementing e-learning course. Dissemination efforts included presentations, poster sessions, and publications, while evaluations encompassed various projects, including a pilot on interrater agreement using ChatGPT. Innovations comprised development of a scalable e-learning course, a survey on ML attitudes and barriers, and qualitative interviews.