Effectiveness of telemedicine: A systematic review of reviews
Introduction
Previous reviews of telemedicine have concluded that irrefutable evidence regarding the positive impact of telemedicine on clinical outcomes still eludes us. One review [1] of more than 150 articles concluded that potential effectiveness could only be attributed to teleradiology, telepsychiatry, transmission of echocardiographic images and consultations between primary and secondary health providers. Another systematic review [2] that assessed more than 1300 papers making claims about telemedicine outcomes found only 46 publications that actually studied at least some clinical outcomes. A review that analyzed the suitability of telemedicine as an alternative to face-to-face care [3] concluded that establishing systems for patient care using telecommunications technologies is feasible; however, the studies provided inconclusive results regarding clinical benefits and outcomes. A report on peer-reviewed literature for telemedicine services that substituted face-to-face services with ICT-based services at home and in offices or hospitals [4] identified 97 articles that met the inclusion criteria for analysis. The authors concluded that telemedicine is being used even if the use is not supported by high quality evidence. Reviews on cost outcomes have fared similarly. A study focused on cost-effectiveness interventions concluded that there is no good evidence that telemedicine is or is not a cost-effective means for delivering healthcare [5].
The quality of studies is a recurrent concern in these reviews [1], [2], [4], [5], [6]. There is also a debate about appropriate research methodologies. For example, economic analysis of telemedicine has not yet met accepted standards [5]; there is a relative lack of exploration of the socio-economic impact of telemedicine [7]; evidence on factors promoting uptake of telemedicine is lacking [8]; there is relatively undeveloped use, at the time, of qualitative methods [9]; many studies have not been well-designed [4], [10]; and, considering perceived difficulties of building a robust evidence base for recent innovations, researchers have argued that simulation modelling needs further development [11].
The lack of consensus raises questions about the quality of research evidence in terms not only of the data collected and analysed, but also in terms of the approaches to evaluation, that is, the underlying methodologies used, which may not be capable of addressing the questions to which different stakeholders seek answers. Others have noted that evaluation traditions do not sufficiently collaborate to cross borders and that a common language for evaluation is missing [12].
This paper reports on research funded under EU SMART 2008/0064, which sought to review the evidence on the effectiveness of telemedicine with particular reference to both outcomes and methodologies for evaluation. This paper focuses mainly on the evidence about effectiveness, and assesses the range of conclusions drawn by reviewers about the effectiveness of telemedicine and the gaps in the evidence base. A companion paper focuses on the methodological issues and recommendations [13].
Section snippets
Objectives
The objective of the work was to conduct a review of reviews on the impacts and costs of telemedicine services and consider qualitative and quantitative results, with the purpose of synthesizing evidence to date on the effectiveness of telemedicine. The key questions addressed were firstly, how are telemedicine services defined and described in terms of participants, interventions, comparisons and outcome measures; secondly, what are the reported effects of telemedicine: thirdly which
Methods
An initial search identified systematic reviews of telemedicine published from 1998. A systematic review was defined as an overview with an explicit question and a method section with a clear description of the search strategy and the methods used to produce the systematic review. The review should also report and analyse empirical data. In addition, reviews which described or summarised methods used in assessing telemedicine were included. Because of the large number of reviews retrieved, a
Results
We identified 1593 records through the searches and excluded 1419 following screening. We retrieved 174 potentially relevant articles in full text. We excluded 94 of these based on the pre-specified inclusion and exclusion criteria. The qualitative synthesis below relate to 73 of the 80 included articles.
The results of the 80 systematic reviews included are summarised in seven tables in Appendix 1. Tables one through six list populations, interventions, outcomes, results and conclusions for the
Telemedicine is effective
Twenty reviews (Table 1) concluded that telemedicine works and has positive effects. These include therapeutic effects, increased efficiencies in the health services, and technical usability.
Types of interventions that were found to be therapeutically effective include online psychological interventions [15]; programmes for chronic heart failure that include remote monitoring [16]; home telemonitoring of respiratory conditions [17]; web and computer-based smoking cessation programmes [18];
Telemedicine is promising
Nineteen reviews (Table 2) were less confident about the effectiveness of telemedicine, suggesting that it is promising, or has potential, but that more research is required before it is possible to draw firm conclusions. In some cases, in which the same conditions and interventions are discussed, these more tentative conclusions must temper those of authors who find conclusive evidence.
One review [35] for example found internet-delivered CBT to be a ‘promising’ and ‘complementary’ development,
Evidence is limited and inconsistent
Twenty-two reviews (Table 3) however concluded that the evidence for the effectiveness of telemedicine is still limited and inconsistent, across a wide range of fields.
In terms of therapeutic effectiveness, there is some limited evidence regarding telemonitoring for heart failure [50]; despite reviewers suggesting that electronic transfer of self-monitored results has been found to be feasible and acceptable in diabetes care, they find only weak evidence for improvements in HbA1c or other
Economic analysis
An important emerging issue from our review is the lack of knowledge and understanding of the costs of telemedicine (Table 4).
Several reviewers suggested that telemedicine seemed to be cost-effective, but few draw firm conclusions. One review found that 91% of the studies showed telehomecare to be cost-effective, in that it reduced use of hospitals, improved patient compliance, satisfaction and quality of life [67]. This was the clearest conclusion, with others being much more cautious:
Is telemedicine good for patients?
A second emerging issue concerns patient satisfaction with telemedicine, and indications that telemedicine may alter the relationships between patients and health professionals (Table 5).
One review found that health service users with ICT used in support, education and virtual consultation feel more confident and empowered, with better knowledge and improved health outcomes, as well as experiencing better nurse-patient relationships [73]. The reviewers call for more research on the mechanisms
Asking new questions
We have already noted the emergence of new topic areas in this dynamic and complex field. The focus on patient benefits however indicates a more basic development, namely that reviewers are starting to explore new questions beyond those of clinical and cost-effectiveness. Our review produced two key examples (Table 6). Firstly, a review that identified gender differences in computer-mediated communications relating to online support groups for people with cancer cautioned that studies are
Reflections on the methodology of our study
Our study is a review of systematic reviews. There are some inherent weaknesses in this approach. In general we have to rely on the information in the included reviews. The quality of the reviews may vary; the reviews may have done a poor job in specifying their inclusion and exclusion criteria, the searches may not be comprehensive, the review authors may not have assessed or extracted data from the primary studies adequately, nor analysed and synthesised the findings across the studies
Conclusions
Despite large number of studies and systematic reviews on the effects of telemedicine, high quality evidence to inform policy decisions on how best to use telemedicine in health care is still lacking. Large studies with rigorous designs are needed to get better evidence on the effects of telemedicine interventions on health, satisfaction with care and costs. As the field is rapidly evolving, different kinds of knowledge are also in demand, e.g. a stronger focus on economic analyses of
Acknowledgements
The study was funded by the EU under SMART 2008/0064 and was conducted as part of the MethoTelemed project. We acknowledge the support of our MethoTelemed colleagues, the group of external review experts, the workshop participants, the project officers at the Norwegian centre for integrated care and telemedicine, and Ingrid Harboe at the Norwegian Knowledge Centre for the Health Services, who did the literature searches.
References (93)
- et al.
Visions and strategies to improve evaluation of health information systems. Reflections and lessons based on the HIS-EVAL workshop in Innsbruck
Int. J. Med. Inform.
(2004) - et al.
Virtual reality exposure therapy for anxiety disorders: a meta-analysis
J. Anxiety Disord.
(2008) - et al.
A systematic review of randomized control trials evaluating the effectiveness of interactive computerized asthma patient education programs
Ann. Allergy Asthma Immunol.
(2007) - et al.
Systematic review of home telemonitoring for chronic diseases: the evidence base
J. Am. Med. Inform. Assoc.
(2007) - et al.
Telemonitoring for patients with chronic heart failure: a systematic review
J. Card. Fail.
(2007) - et al.
Biomedical informatics applications for asthma care: a systematic review
J. Am. Med. Inform. Assoc.
(2006) - et al.
The effectiveness of web-based interventions designed to decrease alcohol consumption – a systematic review
Prev. Med.
(2008) - et al.
A review of computer and Internet-based interventions for smoking behavior
Addict. Behav.
(2006) - et al.
Health care consumers’ experiences of information communication technology – a summary of literature
Int. J. Med. Inform.
(2007) Home telehealth-current state and future trends
Int. J. Med. Inform.
(2006)
Review of ICT-based services for identified unmet needs in people with dementia
Ageing Res. Rev.
Gender differences in computer-mediated communication: a systematic literature review of online health-related support groups
Patient Educ. Couns.
Web-based management of diabetes through glucose uploads: has the time come for telemedicine?
Diabetes Res. Clin. Pract.
The diffusion of virtual communities in health care: concepts and challenges
Patient Educ. Couns.
R Computer-based interventions for college drinking: a qualitative review
Addict. Behav.
Assessing telemedicine: a systematic review of the literature
CMAJ
Systematic review of evidence for the benefits of telemedicine
J. Telemed. Telecare
Telemedicine Versus Face to Face Patient Care: Effects on Professional Practice and Health Outcomes
Telemedicine for the medicare population: update
Evid. Rep. Technol. Assess (Full Rep.)
Systematic review of cost effectiveness studies of telemedicine interventions
BMJ
An analysis of the specialized literature in the field of telemedicine
J. Telemed. Telecare
The socio-economic impact of telehealth: a systematic review
J. Telemed. Telecare
Benefits from Telemedicine in Norway: An Examination of Available Documentation, HØYKOM Report No. 2006: 1. HØYKOM Report Series
Qualitative research methods in health technology assessment: a review of the literature
Health Technol. Assess.
Clinical outcomes resulting from telemedicine interventions: a systematic review
BMC Med. Informatics Decis. Making
Meeting government objectives for telecare in moving from local implementation to mainstream services
J. Telemed. Telecare
Realist review – a new method of systematic review designed for complex policy interventions
J. Health Serv. Res. Policy
A comprehensive review and a meta-analysis of the effectiveness of Internet-based psychotherapeutic interventions
J. Technol. Hum. Services
Telemonitoring or structured telephone support programmes for patients with chronic heart failure: systematic review and meta-analysis
BMJ
Home telemonitoring for respiratory conditions: a systematic review
Am. J. Manag. Care
Effects of Web- and computer-based smoking cessation programs: meta-analysis of randomized controlled trials
Arch. Intern. Med.
Telehealth interventions for the secondary prevention of coronary heart disease: a systematic review
Eur. J. Cardiovasc. Prev. Rehabil.
Telehealth: Clinical Guidelines and Technical Standards for Telepsychiatry
Systematic review of the effect of robot-aided therapy on recovery of the hemiparetic arm after stroke
J. Rehabil. Res. Dev.
A meta-analysis of the effects of internet- and computer-based cognitive-behavioral treatments for anxiety
J. Clin. Psychol.
Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis
Psychol. Med.
Home Telehealth for Chronic Disease Management
Internet-based physical activity interventions: a systematic review of the literature
J. Med. Internet Res.
Can telepsychiatry replace in-person psychiatric assessments? A review and meta-analysis of comparison studies
CNS Spectr.
A systematic review of the benefits of home telecare for frail elderly people and those with long-term conditions
J. Telemed. Telecare
Home telemonitoring of patients with diabetes: a systematic assessment of observed effects
J. Eval. Clin. Pract.
Remote interpretation in medical encounters: a systematic review
J. Telemed. Telecare
Technologies for an Aging Society: A Systematic Review of “Smart Home” applications. Yearbook of medical informatics
A systematic review of the literature on home monitoring for patients with heart failure
J. Telemed. Telecare
The role of home-based information and communications technology interventions in chronic disease management: a systematic literature review
Health Informatics J.
Cited by (783)
Screening and Evaluation of Sleep Disturbances and Sleep Disorders in Children and Adolescents
2024, Psychiatric Clinics of North AmericaStakeholders’ preferences for the design and delivery of virtual care services: A systematic review of discrete choice experiments
2024, Social Science and MedicineInvestigating the Development and Psychometric Testing of the Information and Communication Technology Use Scale in Iranian Patients With Chronic Conditions
2024, Journal of Client-Centered Nursing CareThe effectiveness of continuity of care in patients with inflammatory bowel disease: a systematic review
2024, BMC Gastroenterology