Menu
Analytical Validation Library
The Analytical Validation Library is a one-stop-shop for digital health product developers, researchers, clinicians, regulators, payers, and patients to access peer-reviewed manuscripts reporting the performance of best-in-class analytical validation studies as defined by DiMe’s V3 Framework.
Access the Analytical Validation Library to discover ‘what good looks like’ in terms of high-quality analytical validation to implement these best practices when validating measures of digital health products.
The Study
The DATAcc team performed a systematic review of over 1,200 studies to assess the state of evidence for analytical validation of digital health measurement products. DATAcc’s systemic review benchmarked the state of the science describing analytical validation studies of connected, mobile, sensor-based digital health technologies (DHTs) and assessed the quality of study reporting by applying DiMe’s EVIDENCE Checklist to identified papers.
DHTs are electronic tools, systems, devices, and resources that generate, store, and process data in health care and include mobile health, wearable devices, telehealth, telemedicine, electronic health records, and patient portals. They are valuable tools in clinical research and across the care continuum.
There were 303 analytical validation studies identified, but only 208 were considered high-quality. The Analytical Validation Library is an open-access catalog of evidence capturing only the high-quality analytical validation studies returned by DATAcc’s systematic review.
DHTs are electronic tools, systems, devices, and resources that generate, store, and process data in health care and include mobile health, wearable devices, telehealth, telemedicine, electronic health records, and patient portals. They are valuable tools in clinical research and across the care continuum.
There were 303 analytical validation studies identified, but only 208 were considered high-quality. The Analytical Validation Library is an open-access catalog of evidence capturing only the high-quality analytical validation studies returned by DATAcc’s systematic review.
The Findings
Hundreds of analytical validation studies are being published across a broad spectrum of measures, disease types, and technology types. This is essential to advancing the use of DHTs for clinical decision-making in clinical research, patient care, and individual health promotion.
One hundred and twelve of the 1,200 studies were excluded from inclusion in the systematic review because they failed to report a reference standard or the reference standard was inadequate. Of the 303 studies that made it through the systematic review, only 208 are considered high-quality studies and were included in our new Analytical Validation Library.
Some of the scientific basics, like selecting a reference standard, are not up to standard, creating the risk that we make clinical decisions based on data that’s not fit for purpose.
- This is a fundamental part of the science and new research is needed to define how to select optimal reference studies.
Only 9% of the studies included reported race and/or ethnicity. This is absolutely unacceptable if we want to ensure that these new tools work equally well for everyone and improve – not worsen – racial disparities in health and health care.
- This finding puts us at risk. Lack of inclusion in studies fails to capture the opportunity to improve equity as we digitize healthcare and, in fact, creates new risks of inequity.
- We must insist upon inclusive approaches to the evaluation of DHTs, including, but not limited to, requiring the reporting of race and ethnicity in AV studies. DATAcc’s Toolkit for Digital Health Measurement Product Development provides the “how to” accomplish this.
DiMe’s V3 Framework has become the common evidentiary standard for evaluating DHTs across the field. This is great for innovation, innovators, clinical decision-makers (researchers and clinicians), and patients themselves who have a common standard to build to and benchmark against.
- Forthcoming regulatory guidance should codify V3 not just as the defacto industry standard but as the regulatory gold standard for DHT evaluation.
Read the full manuscript, including all findings and results, and learn more about the methodology used for the Analytical Validation Library.
Similar to DiMe’s Library of Digital Endpoints, the Analytical Validation Library will be maintained over time as the industry’s premier open-access resource benchmarking the state of the analytical validation science for DHTs.
