Effectiveness of telemedicine: A systematic review of reviews

https://doi.org/10.1016/j.ijmedinf.2010.08.006Get rights and content

Abstract

Objectives

To conduct a review of reviews on the impacts and costs of telemedicine services.

Methods

A review of systematic reviews of telemedicine interventions was conducted. Interventions included all e-health interventions, information and communication technologies for communication in health care, Internet based interventions for diagnosis and treatments, and social care if important part of health care and in collaboration with health care for patients with chronic conditions were considered relevant. Each potentially relevant systematic review was assessed in full text by one member of an external expert team, using a revised check list from EPOC (Cochrane Effective Practice and Organisation of Care Group) to assess quality. Qualitative analysis of the included reviews was informed by principles of realist review.

Results

In total 1593 titles/abstracts were identified. Following quality assessment, the review included 80 heterogeneous systematic reviews. Twenty-one reviews concluded that telemedicine is effective, 18 found that evidence is promising but incomplete and others that evidence is limited and inconsistent. Emerging themes are the particularly problematic nature of economic analyses of telemedicine, the benefits of telemedicine for patients, and telemedicine as complex and ongoing collaborative achievements in unpredictable processes.

Conclusions

The emergence of new topic areas in this dynamic field is notable and reviewers are starting to explore new questions beyond those of clinical and cost-effectiveness. Reviewers point to a continuing need for larger studies of telemedicine as controlled interventions, and more focus on patients’ perspectives, economic analyses and on telemedicine innovations as complex processes and ongoing collaborative achievements. Formative assessments are emerging as an area of interest.

Introduction

Previous reviews of telemedicine have concluded that irrefutable evidence regarding the positive impact of telemedicine on clinical outcomes still eludes us. One review [1] of more than 150 articles concluded that potential effectiveness could only be attributed to teleradiology, telepsychiatry, transmission of echocardiographic images and consultations between primary and secondary health providers. Another systematic review [2] that assessed more than 1300 papers making claims about telemedicine outcomes found only 46 publications that actually studied at least some clinical outcomes. A review that analyzed the suitability of telemedicine as an alternative to face-to-face care [3] concluded that establishing systems for patient care using telecommunications technologies is feasible; however, the studies provided inconclusive results regarding clinical benefits and outcomes. A report on peer-reviewed literature for telemedicine services that substituted face-to-face services with ICT-based services at home and in offices or hospitals [4] identified 97 articles that met the inclusion criteria for analysis. The authors concluded that telemedicine is being used even if the use is not supported by high quality evidence. Reviews on cost outcomes have fared similarly. A study focused on cost-effectiveness interventions concluded that there is no good evidence that telemedicine is or is not a cost-effective means for delivering healthcare [5].

The quality of studies is a recurrent concern in these reviews [1], [2], [4], [5], [6]. There is also a debate about appropriate research methodologies. For example, economic analysis of telemedicine has not yet met accepted standards [5]; there is a relative lack of exploration of the socio-economic impact of telemedicine [7]; evidence on factors promoting uptake of telemedicine is lacking [8]; there is relatively undeveloped use, at the time, of qualitative methods [9]; many studies have not been well-designed [4], [10]; and, considering perceived difficulties of building a robust evidence base for recent innovations, researchers have argued that simulation modelling needs further development [11].

The lack of consensus raises questions about the quality of research evidence in terms not only of the data collected and analysed, but also in terms of the approaches to evaluation, that is, the underlying methodologies used, which may not be capable of addressing the questions to which different stakeholders seek answers. Others have noted that evaluation traditions do not sufficiently collaborate to cross borders and that a common language for evaluation is missing [12].

This paper reports on research funded under EU SMART 2008/0064, which sought to review the evidence on the effectiveness of telemedicine with particular reference to both outcomes and methodologies for evaluation. This paper focuses mainly on the evidence about effectiveness, and assesses the range of conclusions drawn by reviewers about the effectiveness of telemedicine and the gaps in the evidence base. A companion paper focuses on the methodological issues and recommendations [13].

Section snippets

Objectives

The objective of the work was to conduct a review of reviews on the impacts and costs of telemedicine services and consider qualitative and quantitative results, with the purpose of synthesizing evidence to date on the effectiveness of telemedicine. The key questions addressed were firstly, how are telemedicine services defined and described in terms of participants, interventions, comparisons and outcome measures; secondly, what are the reported effects of telemedicine: thirdly which

Methods

An initial search identified systematic reviews of telemedicine published from 1998. A systematic review was defined as an overview with an explicit question and a method section with a clear description of the search strategy and the methods used to produce the systematic review. The review should also report and analyse empirical data. In addition, reviews which described or summarised methods used in assessing telemedicine were included. Because of the large number of reviews retrieved, a

Results

We identified 1593 records through the searches and excluded 1419 following screening. We retrieved 174 potentially relevant articles in full text. We excluded 94 of these based on the pre-specified inclusion and exclusion criteria. The qualitative synthesis below relate to 73 of the 80 included articles.

The results of the 80 systematic reviews included are summarised in seven tables in Appendix 1. Tables one through six list populations, interventions, outcomes, results and conclusions for the

Telemedicine is effective

Twenty reviews (Table 1) concluded that telemedicine works and has positive effects. These include therapeutic effects, increased efficiencies in the health services, and technical usability.

Types of interventions that were found to be therapeutically effective include online psychological interventions [15]; programmes for chronic heart failure that include remote monitoring [16]; home telemonitoring of respiratory conditions [17]; web and computer-based smoking cessation programmes [18];

Telemedicine is promising

Nineteen reviews (Table 2) were less confident about the effectiveness of telemedicine, suggesting that it is promising, or has potential, but that more research is required before it is possible to draw firm conclusions. In some cases, in which the same conditions and interventions are discussed, these more tentative conclusions must temper those of authors who find conclusive evidence.

One review [35] for example found internet-delivered CBT to be a ‘promising’ and ‘complementary’ development,

Evidence is limited and inconsistent

Twenty-two reviews (Table 3) however concluded that the evidence for the effectiveness of telemedicine is still limited and inconsistent, across a wide range of fields.

In terms of therapeutic effectiveness, there is some limited evidence regarding telemonitoring for heart failure [50]; despite reviewers suggesting that electronic transfer of self-monitored results has been found to be feasible and acceptable in diabetes care, they find only weak evidence for improvements in HbA1c or other

Economic analysis

An important emerging issue from our review is the lack of knowledge and understanding of the costs of telemedicine (Table 4).

Several reviewers suggested that telemedicine seemed to be cost-effective, but few draw firm conclusions. One review found that 91% of the studies showed telehomecare to be cost-effective, in that it reduced use of hospitals, improved patient compliance, satisfaction and quality of life [67]. This was the clearest conclusion, with others being much more cautious:

Is telemedicine good for patients?

A second emerging issue concerns patient satisfaction with telemedicine, and indications that telemedicine may alter the relationships between patients and health professionals (Table 5).

One review found that health service users with ICT used in support, education and virtual consultation feel more confident and empowered, with better knowledge and improved health outcomes, as well as experiencing better nurse-patient relationships [73]. The reviewers call for more research on the mechanisms

Asking new questions

We have already noted the emergence of new topic areas in this dynamic and complex field. The focus on patient benefits however indicates a more basic development, namely that reviewers are starting to explore new questions beyond those of clinical and cost-effectiveness. Our review produced two key examples (Table 6). Firstly, a review that identified gender differences in computer-mediated communications relating to online support groups for people with cancer cautioned that studies are

Reflections on the methodology of our study

Our study is a review of systematic reviews. There are some inherent weaknesses in this approach. In general we have to rely on the information in the included reviews. The quality of the reviews may vary; the reviews may have done a poor job in specifying their inclusion and exclusion criteria, the searches may not be comprehensive, the review authors may not have assessed or extracted data from the primary studies adequately, nor analysed and synthesised the findings across the studies

Conclusions

Despite large number of studies and systematic reviews on the effects of telemedicine, high quality evidence to inform policy decisions on how best to use telemedicine in health care is still lacking. Large studies with rigorous designs are needed to get better evidence on the effects of telemedicine interventions on health, satisfaction with care and costs. As the field is rapidly evolving, different kinds of knowledge are also in demand, e.g. a stronger focus on economic analyses of

Acknowledgements

The study was funded by the EU under SMART 2008/0064 and was conducted as part of the MethoTelemed project. We acknowledge the support of our MethoTelemed colleagues, the group of external review experts, the workshop participants, the project officers at the Norwegian centre for integrated care and telemedicine, and Ingrid Harboe at the Norwegian Knowledge Centre for the Health Services, who did the literature searches.

References (93)

  • S. Lauriks et al.

    Review of ICT-based services for identified unmet needs in people with dementia

    Ageing Res. Rev.

    (2007)
  • P.K. Mo et al.

    Gender differences in computer-mediated communication: a systematic literature review of online health-related support groups

    Patient Educ. Couns.

    (2009)
  • M. Azar et al.

    Web-based management of diabetes through glucose uploads: has the time come for telemedicine?

    Diabetes Res. Clin. Pract.

    (2009)
  • G. Demiris

    The diffusion of virtual communities in health care: concepts and challenges

    Patient Educ. Couns.

    (2006)
  • J.C. Elliott et al.

    R Computer-based interventions for college drinking: a qualitative review

    Addict. Behav.

    (2008)
  • R. Roine et al.

    Assessing telemedicine: a systematic review of the literature

    CMAJ

    (2001)
  • D. Hailey et al.

    Systematic review of evidence for the benefits of telemedicine

    J. Telemed. Telecare

    (2002)
  • R. Currell et al.

    Telemedicine Versus Face to Face Patient Care: Effects on Professional Practice and Health Outcomes

    (2002)
  • W.R. Hersh et al.

    Telemedicine for the medicare population: update

    Evid. Rep. Technol. Assess (Full Rep.)

    (2006)
  • P.S. Whitten et al.

    Systematic review of cost effectiveness studies of telemedicine interventions

    BMJ

    (2002)
  • G. Demiris et al.

    An analysis of the specialized literature in the field of telemedicine

    J. Telemed. Telecare

    (2005)
  • P. Jennett et al.

    The socio-economic impact of telehealth: a systematic review

    J. Telemed. Telecare

    (2003)
  • E. Johnsen et al.

    Benefits from Telemedicine in Norway: An Examination of Available Documentation, HØYKOM Report No. 2006: 1. HØYKOM Report Series

    (2006)
  • E. Murphy et al.

    Qualitative research methods in health technology assessment: a review of the literature

    Health Technol. Assess.

    (1998)
  • W. Hersh et al.

    Clinical outcomes resulting from telemedicine interventions: a systematic review

    BMC Med. Informatics Decis. Making

    (2001)
  • J. Barlow et al.

    Meeting government objectives for telecare in moving from local implementation to mainstream services

    J. Telemed. Telecare

    (2005)
  • A.G. Ekeland, A. Bowes, S.A. Flottorp, Methodologies for assessing telemedicine: a systematic review of reviews (under...
  • R. Pawson et al.

    Realist review – a new method of systematic review designed for complex policy interventions

    J. Health Serv. Res. Policy

    (2005)
  • A. Barak et al.

    A comprehensive review and a meta-analysis of the effectiveness of Internet-based psychotherapeutic interventions

    J. Technol. Hum. Services

    (2008)
  • R.A. Clark et al.

    Telemonitoring or structured telephone support programmes for patients with chronic heart failure: systematic review and meta-analysis

    BMJ

    (2007 May 5)
  • M. Jaana et al.

    Home telemonitoring for respiratory conditions: a systematic review

    Am. J. Manag. Care

    (2009)
  • S.K. Myung et al.

    Effects of Web- and computer-based smoking cessation programs: meta-analysis of randomized controlled trials

    Arch. Intern. Med.

    (2009)
  • L. Neubeck et al.

    Telehealth interventions for the secondary prevention of coronary heart disease: a systematic review

    Eur. J. Cardiovasc. Prev. Rehabil.

    (2009)
  • G.M.K. Pineau et al.

    Telehealth: Clinical Guidelines and Technical Standards for Telepsychiatry

    (2006)
  • G.B. Prange et al.

    Systematic review of the effect of robot-aided therapy on recovery of the hemiparetic arm after stroke

    J. Rehabil. Res. Dev.

    (2006)
  • M.A. Reger et al.

    A meta-analysis of the effects of internet- and computer-based cognitive-behavioral treatments for anxiety

    J. Clin. Psychol.

    (2009)
  • V. Spek et al.

    Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis

    Psychol. Med.

    (2007)
  • K. Tran et al.

    Home Telehealth for Chronic Disease Management

    (2008)
  • M.H. van den Berg et al.

    Internet-based physical activity interventions: a systematic review of the literature

    J. Med. Internet Res.

    (2007)
  • S.E. Hyler et al.

    Can telepsychiatry replace in-person psychiatric assessments? A review and meta-analysis of comparison studies

    CNS Spectr.

    (2005)
  • J. Barlow et al.

    A systematic review of the benefits of home telecare for frail elderly people and those with long-term conditions

    J. Telemed. Telecare

    (2007)
  • M. Jaana et al.

    Home telemonitoring of patients with diabetes: a systematic assessment of observed effects

    J. Eval. Clin. Pract.

    (2007)
  • P. Azarmina et al.

    Remote interpretation in medical encounters: a systematic review

    J. Telemed. Telecare

    (2005)
  • G. Demiris et al.

    Technologies for an Aging Society: A Systematic Review of “Smart Home” applications. Yearbook of medical informatics

    (2008)
  • A. Martinez et al.

    A systematic review of the literature on home monitoring for patients with heart failure

    J. Telemed. Telecare

    (2006)
  • R. Gaikwad et al.

    The role of home-based information and communications technology interventions in chronic disease management: a systematic literature review

    Health Informatics J.

    (2009)
  • Cited by (783)

    View all citing articles on Scopus
    View full text