AMIA VIS Working Group Task Force on Evaluation

AMIA_logoThe healthcare domain has long been a data-driven enterprise. From point-of-care decisions made by clinicians based on a patient’s medical history, to longitudinal population studies that provide evidence for clinical practice guidelines (CPG), to individuals monitoring their own health through patient-generated health data (PGHD), the collection, organization, and utilization of information is at the center of nearly every aspect of modern medicine. This was the case in the era of paper charts, and continues as both the collection and utilization of data in medical practice has accelerated during the industry’s shift toward a more digital and modern health IT infrastructure. For example, in the United States, the Office of the National Coordinator for Health Information Technology now reports that 96% of hospitals have an electronic health record system (EHR).1 Reports suggest similar percentages of hospitals making progress toward meaningful use standards,2 a set of criteria designed to access the capture and use of clinical data from EHR systems to improve quality, safety, and efficiency.

vahc_taskforce

A framework developed by the TFoE to capture the design space for evaluation of visual analytics technologies within the healthcare domain.

This ongoing digital transformation is producing large amounts of digital data, and is sparking a broad range of research and development aimed at enabling new data-driven methods for improving the healthcare system. One critical aspect of this wave of innovation has been in the design and development of effective ways to communicate data that can ultimately generate new knowledge and enable more insightful actions. Both the medical informatics and visualization research communities have recognized the growing importance of this challenge and have identified visual analytics as a critical area for technological innovation.3,4 Visual analytics technologies support analytical reasoning about complex and large scale datasets using a combination of interactive visualization-based user interfaces and computational analysis. As such, these methods have the potential to help make data more interpretable and actionable for a range of healthcare user populations: from clinicians, to population health analysts, to patients, and to their caregivers and families.

However, despite the great promise of visual analytics to support more effective data analysis and decision making, it can be challenging to evaluate the benefits that a specific technology provides. This difficulty is recognized within the visualization community,5 but is an even more critical hurdle in medical informatics applications where technologies must be rigorously proven before they can be widely adopted.

In early 2016, the American Medical Informatics Association (AMIA) Visual Analytics Working Group (VIS WG) established a Task Force on Evaluation (TFoE) to investigate the state-of-the-art in visual analytics evaluation and to provide a report documenting recommendations for visual analytics evaluation within the context of the medical informatics domain. The TFoE includes David Gotz (chair), David Borland, Jesus Caban, Dawn Dowding, Brian Fisher, Vadim Kagan, and Danny Wu. This team has broad representation, with members from industry, government, and academia.

Publications

Danny T.Y. Wu, Annie T. Chen, John D. Manning, Gal Levy-Fix, Uba Backonja, David Borland, Jesus J. Caban, Dawn W. Dowding, Harry Hochheiser, Vadim Kagan, Swaminathan Kandaswamy, Manish Kumar, Alexis Nunez, Eric Pan, David Gotz. Evaluating Visual Analytics for Health Informatics Applications: A Systematic Review from the AMIA VIS Working Group Task Force on Evaluation. Journal of the American Medical Informatics Association (JAMIA) (Volume 26, Number 4, 2019).
[An open-access PDF for this article is available through the journal’s website.]

David Gotz, David Borland, Jesus Caban, Dawn Dowding, Brian Fisher, Vadim Kagan, and Danny T.Y. Wu. Evaluating Visual Analytics for Health Informatics Applications: A Progress Report from the AMIA VIS Working Group Task Force on Evaluation. Visual Analytics in Healthcare Workshop, Chicago, IL (2016).
[PDF, 260k]

Sch of Inform and Libr Science