Home Healthcare, Hospice, and Information Technology Innovations Conference

Innovations in Home Healthcare, Hospice, and Information Technology

A forum towards achieving evidence-based diffusion and implementation of innovations

H3IT: Home Healthcare, Hospice, and Information Technology Conference Washington – D.C., 2014

Development of an eHealth Measures Compendium

Bonnie Wakefield

1

, Timothy Hogan

1

, Carolyn Turvey

2

, Stephanie Shimada

1

, Kim Nazi

3

C

onsistent and well-validated metrics of design, efficiency, and improved communication are necessary to deter-

mine the true benefit of any eHealth intervention without which healthcare organizations cannot 1) calculate

return on investment of eHealth technology; 2) effectively address barriers to adoption that stem from these

metrics (i.e. usability, accessibility of a technology); or 3) accurately estimate the likelihood of adoption. The

goal of this project is to create a compendium of potential metrics that could be used in any study using eHealth interventions

and create a standardized array of recommended metrics that will support both eHealth operations and research.

Methods: Working with an experienced health sciences librarian, an extensive list of search terms were developed address-

ing platforms (e.g., cell phone, patient portal), measurement (e.g., performance measurement, survey development) and

functions (e.g., health information seeking). To date, major healthcare literature databases have been searched including

Scopus, Pubmed, Cumulative Index of Nursing and Allied Health Literaterue (CINAHL), Health and Psychosocial Instru-

ments (HAPI), and PsychInfo. To build this compendium effectively, the literature search will extend beyond a review of

the medical literature and include research from the IEEE (Institute of Electrical and Electronics Engineers) and ACM

(Association for Computing Machinery) digital libraries. Reliability estimates will be explored using the range of current

statistics available (e.g. internal consistency, test-retest reliability, alternate forms reliability) as will validity estimates (con-

tent validity, construct validity, predictive validity, discriminant validity). Each metric will be described using a uniform

format. A brief overview of the instrument’s development, scoring procedures, psychometric properties, key references on

the development and/or use of the instrument, and the actual scale (if available) will be included. The final compendium

will be searchable by key words (using MeSH terms) so each metric will be cross-indexed by the topic/construct covered, the

types of technology the metric addresses, and populations where the metric has been used. The database will also include

references to articles or abstracts on use of the metric. Finally, at the conclusion of the search and review, we will upload

our information to the Grid-Enabled Measures (GEM) database, sp onsored by the National Cancer Institute.

Results: The study is in progress. We have developed and tested the review instrument using the uniform requirements

addressed above. To date, 15 instruments known to the investigators prior to the search have been reviewed. The search of

the healthcare literature resulted in 33,217 citations; approximately 70% have undergone a title/abstract review. Of those,

less than 1% describe an instrument and less than 4% describe use of an instrument that potentially could be included in

the compendium.

Discussion: Evaluation of eHealth is unique from evaluation of other interventions in three important ways that warrant

development of a unique compendium: 1) it must include evaluation of the technology platforms and functions in terms of

usability, functionality, and availability of the technology to target users; 2) eHealth applications are promoted to improve

efficiency and accessibility, but there are no uniform widely agreed upon metrics; and 3) eHealth interventions aim to improve

communication in one form or another, thus metrics are needed that quantify specifically the degree to which communication

is improved.

Conclusion: The results of this project will provide critical insights regarding existing eHealth measures and identify gaps

where new metrics are needed. The compendium can also inform future studies so that the results from multiple studies can

be compared and synthesized because they used the same handful of metrics. A white paper will be developed to provide a

critical synthesis and analysis of the current state of evaluation of eHealth in light of the strengths and weaknesses for each

of the domains covered.

1

VA Medical Center

2

VA University of Iowa

3

Veterans Health Administration

Copyright © 2014 by Maryland Health Information Technology LLC — Creative Commons License c b n d 3