The validity of searching routinely collected general practice computer data to identify patients with chronic kidney disease (CKD): a manual review of 500 medical records

Abstract

Background. We conducted a search of 12 practices’ routinely collected computer data in three localities across the UK and found that 4.9% of the registered population had an estimated glomerular filtration rate (GFR) of <60 ml/min/1.73 m² (equivalent to stages 3–5 CKD). Only 3.6% of these were known to have renal disease. Although UK general practice is computerized, important clinical data might be recorded in letters or free-text computer entries and might therefore be invisible to the standard computer search tools. We therefore manually searched through all the records of patients with stages 3–5 CKD in one practice, to test the validity of the computer generated diagnosis and to see if other relevant information was missed by the computer search.

Methods. We identified 492 people with stages 3–5 CKD using computer searching and then manually searched their computer records and written notes for any missed data. The dataset included cardiovascular morbidities and risk factors including diabetes; drugs which may impair renal function; known renal disease; and terminal diagnoses and dementia.

Results. The manual searches only added four renal diagnoses to the 36 already identified. Although heart failure and stroke appear to be over-estimated by computer searches, other cardiovascular diagnoses were reliably recorded. Cardiovascular risk factors and drug recording is a strength of general practice computer data. It is complete and contemporary, though most patients had scope to have their cardiovascular risk reduced further. Eighty-four percent had a haemoglobin estimation, and a higher proportion with reduced renal function were anaemic (P<0.001). Testing for proteinuria was less well recorded; negative stick tests were not recorded. Clinical diagnoses of prostatism and bladder outflow problems made these data hard to interpret.

Conclusions. Automated searching of general practice computer records could provide a reliable and valid way of identifying people with stages 3–5 CKD who could benefit from interventions readily available in primary care.

Introduction

Chronic kidney disease (CKD) is a relatively common condition with increasing mortality and morbidity [1–3]. Although CKD is a major predictor for end-stage renal disease, mortality in people with CKD is predominantly due to cardiovascular disease [2,4]. Consequently, earlier identification of CKD in primary care, better management of cardiovascular risk, avoiding medicines that impair renal function and specialist referral where appropriate may improve long-term outcomes. The prevalence of stages 3–5 CKD in the USA is 4.7% [people with a glomerular filtration rate (GFR) <60 ml/min/1.73 m²] [4] and we have recently described a similar prevalence of CKD in England. We found a population prevalence for stages 3–5 CKD of 4.9% [5]. However, there was a low level of awareness of CKD in our study. Only 3.6% of the patients with CKD were recorded in their general practitioner (GP) computer records as having renal disease [5]; this is in marked contrast to the USA where just under a quarter of patients with stage 3 (22%) and just under half with stage 4 (44%) are aware of their renal impairment [6].

Although UK general practice, in common with most European countries, is computerized and many practices regard themselves as paperless, we cannot be certain that computer searches find all the information contained within these medical records [7]. Computer searches only look at ‘structured’ data, often referred to in UK general practice as ‘Read coded’ data because most GPs use Read version 2 for clinical coding. The equivalent European system to Read is the International Classification for Primary Care (ICPC) [8]. Computer searches ignore the narrative or ‘free-text’ part of the computer record. Data are ‘coded’ by using picking lists (Figure 1), by entering data into forms or spreadsheets that automatically code the data entered, or when it is coded automatically when transferred into the computer system. The only widely used automatic coding system is the link to pathology laboratories. Test results, including serum creatinine, are transferred into the GP computer system, although the results do have to be checked and authorized before filing into patients' records. If data are not coded, computer search systems will not be able to retrieve them. This lack of information is likely to be even more profound in many European countries, which use ICPC, a much less detailed computer coding system than the Read system used in the UK [9].

Much of the computer record comprises narrative or ‘free-text’. There are numerous ways in which medical concepts can be represented in the free text record (e.g. kidney disease, renal impairment), and they often have complicated qualifiers (e.g. ‘Worried he might have …’, ‘Not …’, ‘His friend with the same symptoms had …’). Natural language processing has not yet reached the stage where it can extract medical meaning from free-text. Therefore information in free-text can only be searched manually. The scope of the free-text record is wide. It includes the free-text in the computer records, data within hospital letters and reports (whether stored on paper or scanned into computer systems) and the written notes.

It is possible that much of the pertinent data might be recorded in the narrative record and therefore be invisible to standard computer searches. We therefore decided to test the validity of the use of computer searches by searching manually all the records of the people with stages 3–5 CKD in one practice.

Subjects and methods

The sample of notes searched was drawn from one practice chosen to be representative of the whole study.

As a year had elapsed since the original study, repeat electronic searches were made of the GP computer system in the study practice. This search identified all the patients with a recorded serum creatinine (SCr) and also included the principal modifiable causes of kidney disease as in our previous study. These included: cardiovascular co-morbidities, cardiovascular risk factors and whether anaemia is present, drugs that can impair renal function, known renal disease, and diseases that might obstruct the urinary tract. A diagnosis of anaemia was based on the World Health Organization (WHO) definition (haemoglobin levels <12 g/dl for non-pregnant females and <13 g/dl for males and post-menopausal women) and the KDOQI [10] and European Best Practice Guidelines [11] threshold for consideration of treatment of anaemia (haemoglobin <11 g/dl). Many drugs can impair renal function or can be inappropriate to prescribe to patients with CKD under certain circumstances, e.g. angiotensin converting enzyme inhibitors in renal artery stenosis. However, almost all of these medications can be appropriate under some circumstances. It was therefore decided to record the number of prescriptions for drugs that may impair renal function to identify the number of people with CKD who may benefit from a medication review which took into account their renal disease.

Department of health data extraction software MIQUEST (Morbidity Information QUery and Export Syntax) was used to extract an up-to-date dataset. Data were extracted into a Microsoft Excel spreadsheet, setting out the variables as columns and a row for each patient. Within Excel, a macro was created that estimated GFR using the simplified MDRD (Modification of Diet in Renal Disease) equation [12]. Cases were then ranked by GFR and those retained where GFR <60 ml/min/1.73 m² (equivalent to stages 3–5 CKD [13]).

Re-running this data extract found a slightly different number of patients than the original study. The changes are due to patients leaving and joining the practice. Four hundred and ninety-two patients were identified with stages 3–5 CKD, and these are the sample used in this study.

Alongside the variable columns, new columns were created for the information that was to come from the manual search. Into these new columns a triplet was inserted containing the following: a numeric (e.g. 9.5 would be inserted into the haemoglobin column if the estimation was 9.5 g/dl); the date (that the test was performed, or diagnosis made, or if not available the date of data entry); and the source (clinicians free-text computer record, scanned-in letter or other non-coded data entered into the computer, or data from the written record). The manual searches were performed during October and November 2004.

Additional data were also collected to enable us to identify whether any of the people with stages 3–5 CKD might be inappropriate to treat. It was considered important to determine whether patients with advanced malignancies, those with dementia or those who might be susceptible to dehydration with an inter-current illness were being picked up on routine blood tests as having CKD, but who because of the nature of their problem would be inappropriate to refer.

The additional data made it possible to determine the positive predictive value of a diagnosis of stages 3–5 CKD, and the reliability of co-morbidity and risk factor recording for these patients.

Statistical and analysis methods

The study population was standardized [14] using the original study (which had an age–sex profile close to the England and Wales national average [15]). This was done because of the unusual age–sex profile of the practice with its excess of young working age people, especially men, so that comparisons could be made with the main study. χ² tests were used to compare whether proportions with a result or test were significant.

Ethical approval and informed consent

The study was approved by a research ethics committee in the locality where the practice included in the study was based. They approved the study without the need for contacting individual patients, as all individual patient data were anonymized.

Results

At the time of the original study, the practice had a list size of 10 979 compared with the median list size of the practices in the original study of 9146 (range, 3385–14 542). In the selected practice, 25.2% (n = 2771/10 979) had a SCr recording compared with a mean level of recording of 25.7% (n = 28 862/112 215) across the whole study sample. The practice had 5.1% (561/10 979) patients with stages 3–5 CKD; the main study had 4.9% (5449/112 215). The practice, like the main study, had minimal ethnicity data recording (<0.3% of patients) so no correction could be made when estimating GFR for those of Afro-Caribbean or Asian origin. The lack of ethnicity data is less significant in this practice; the 2001 census data suggests that this area has <1% black population, 2.1% Asian and 1.2% mixed race of which white-Asian are the largest group at 0.5% [16]. The age–sex profile of the selected practice differed slightly from the study and the national population: the female to male ratio was 0.93:1 (i.e. a small excess of males), and its proportion of working age population is higher than the national or main study average (see Figure 2).

The practice had 492 patients with stages 3–5 CKD identified from computer searches of practice records. The crude prevalence was 4.5% (492/10 975) of the practice population, and when standardized using the original study population (which approximates to the England and Wales population) the adjusted prevalence was 5.1%. Ethnicity was not recorded. Nearly three times as many women as men had CKD (female:male ratio: 2.7:1). The age–sex distribution and crude and standardized prevalence of CKD is shown in Table 1. The prevalence of CKD increased steeply with age. Over half of the females and a third of the males over 75 years, according to the computer search, had stages 3–5 CKD.

Nearly all of those with stages 3–5 CKD (97.1%; 477/492) were in stage 3, while 14 people were in stage 4 and only one in stage 5. Only 36 (7.3%) of these patients were identified on computer searches as known to the renal services or having a diagnosis of primary renal disease. The manual searches only revealed four additional cases missed by the computer search; i.e. there was actually a total of 40 (8%) patients with stages 3–5 CKD. This means that there were 452 (92%) who had undiagnosed stages 3–5 CKD. The one person with stage 5 CKD was known to the renal services; but only 8/14 (57.1%) of those with stage 4.

Cardiovascular co-morbidities were carefully checked for the reliability of the diagnosis. 11.8% (58/492) people had a computer diagnosis of heart failure. Thirty-one of these had one or more supporting investigations or recorded evidence of left ventricular hypertrophy (LVH), ECG evidence of LVH (9/31), echocardiogram evidence (8/31), chest X-ray evidence of cardiomegaly or heart failure (6/31), and the largest group had a diagnosis documented within discharge summaries and clinic letters (15/31). These data were evenly split between being recorded as structured (or Read coded) data, in free text in the computer record, or derived from computer summary data. Ninety-three percent of (54/58) people with heart failure had a recorded haemoglobin level. All but one of the most recent recordings were entered as structured data. The one value extracted from a hospital discharge letter was 15.1 g/dl. Thirty-seven percent (20/54) were anaemic according to the WHO definition; 9.3% (5/54) were anaemic using the 11 g/dl threshold.

Cardiovascular co-morbidities were common in patients with stages 3–5 CKD, although the study did not determine whether the CKD or CVD co-morbidity occurred first: 77.4% (381/492) had at least one diagnosis. 23.8% (117/492) had ischaemic heart disease (IHD), 11.2% (55/492) stroke or transient ischaemic attack (TIA), 3.9% (19/492) peripheral vascular disease (PVD), 62% (305/492) hypertension, and 18.3% (80/492) diabetes. The increasing prevalence of cardiovascular disease, and multiple cardiovascular diseases with age is shown in Table 2. Although the fall in prevalence in people over 85 years may be due to survival bias, these cardiovascular co-morbidities were well recorded and appeared to be reliable when the clinical record as a whole was examined. Six patients, two with IHD, one with a history of TIA and three with a diagnosis of hypertension were identified from hospital letters but were not coded in the patient records. This means that the sensitivity of these diagnoses were 98.3% (IHD), 98.2% (stroke and TIA) and 99% (hypertension), respectively.

Cardiovascular risk recording was complete and contemporary and manual searching did not improve data quality. The computer data quality was as follows. All but one patient had a blood pressure (BP) recording. Three BP dates were dates in the future (therefore invalid); 83% (406/492) had been recorded in the year prior to this data collection; 12% (50/492) from between 2 and 1 year previously; only eight BP readings dated from before the year 2000. Manual searching did not add to this data. Seventy-five percent (370/492) of patients had a cholesterol recording and 72% of these (268/370) dated from the previous year. Ninety-eight percent (481/492) of the patients had a smoking record of which 376 records (76%) had been collected in the year prior to this data collection. Of these 481 with a smoking record, 48% (n = 232) had never smoked, 10% were current smokers (n = 50) and 42% (n = 199) were ex-smokers. Table 3 shows the level of control of these risk factors; bearing in mind that tight control of BP and cholesterol is recommended, large numbers remain over target.

Repeat and acute prescription data were also complete and accurate. However, half (49%; 242/492) were prescribed medication that might impair renal function, although in many circumstances the benefits of these medications might outweigh the risk. Of these, half (47.3%; 106/242) were only prescribed one medication, but a third (34.8%; 78/242) were prescribed two, and 17.9% (40/242) were prescribed three or more. Roughly 70% of patients with heart failure and with diabetes, and 60% with IHD and hypertension were prescribed angiotensin converting enzyme inhibitors (ACEI). Fewer people were prescribed beta-blockers: 40% of those with IHD and hypertension, only 34% of those with diabetes and 20% of those with heart failure (Table 4). The two patients prescribed lithium both had hypertension, and two of the patients prescribed warfarin had GFR <30 ml/min/1.73 m². Twenty percent (100/492) of patients were prescribed a non-steroidal anti-inflammatory drug. This rate of prescription was consistent across all co-morbidities including co-existing circulatory disease. Five percent (24/492) were prescribed cortico-steroids. It was not possible from the information within the medical record to say whether these prescriptions were appropriate or not; however, there was no record of medication review taking into account renal function.

Data quality about potential causes of renal outflow obstruction was complicated and there was often a lack of diagnostic certainty. Eight percent (40/492) patients had clinical notes entries implying a general practice managed problem with their prostate. The commonest code used was the ‘Prostatism’ 1AA code. A further 6% (30/492) had a more definite diagnosis implying possible obstruction: a diagnosis of prostatic hypertrophy (n = 15), carcinoma of the prostate (n = 4), bladder neck obstruction (n = 2), or another obstructive problem. Two cases of renal calculi were identified, one case from a computer code, one from a hospital letter. The positive predictive value of definite diagnostic codes was 100%; however, the accuracy of the more vague GP diagnoses, e.g. ‘prostatism‘, was less.

7.3% (36/492) of those with stages 3–5 CKD had an established renal diagnosis or a code indicating that they had received treatment by a renal physician, dialysis or transplantation. A further four cases were identified from the manual notes search. Three of these additional cases had stage 3 and one stage 4 disease. One with an estimated GFR equivalent to stage 3 CKD had actually received a transplant; the other three had a diagnosis of chronic renal failure. Not surprisingly, there was a highly significant association between the proportion with stages 4 and 5 CKD and those known to the renal services (χ²P<0.001). Three transplanted patients had an estimated GFR equivalent to stage 3 disease (44.6, 45.6 and 48.5 ml/min/1.73 m²) and two equivalent to stage 4 disease (28.6 and 29.3 ml/min/1.73 m²). The positive predictive value of a diagnosis or treatment for renal disease was 100%, and its sensitivity was 90% (36/40).

Proteinuria testing might help determine whether there is renal damage; however, it was recorded for only 19% (93/492) of the patients as structured or coded data. Unfortunately, the code ‘4672’, ‘Urine protein test negative’, was not used at all. Where there was a code for proteinuria testing it was distributed as follows: trace 33.3% (31/93); ‘+’ 47% (44/93); ‘++’ 5.4% (5/93); ‘+++’ 8.6% (8/93); ‘++++’ 3.2% (3/93); and two non-specific codes. Manual searches revealed 142 additional records showing that urine had been tested. Nearly half (48.6%, 69/142) were negative, 1.4% (2/142) recorded a trace and 50% (71/142) were non-specific codes. However, seven of those recorded as negative manually were actually miscoded as trace (n = 2) and ‘+’ (n = 5). There were also three patients who had their manual ‘trace’ or ‘+’ inverted in what was computer coded, and a final seven patients who had non-specific manual notes entries but ‘trace’ or ‘+’ in their computer records. The positive predictive value that a coded computer record represents a valid measure of proteinuria was 97.8% (91/93) if ‘trace’ is included and 96.8% (61/63) if it is not. However, as there were so many proteinuria records that were not coded, the sensitivity of their being a valid proteinuria recording was only 56% (91/162). Microalbuminuria and microalbuminuria-creatinine testing did not add much to the picture; 17 people, all diabetics, were tested for the latter; 15 of these were also tested for microalbuminuria (the only recorded microalbuminuria tests in the sample); and six were also tested for proteinuria. Of the four people with undiagnosed stage 4 CKD, two had been tested and two had positive proteinuria tests recorded (one ‘trace’ and one ‘+’).

83.5% (411/492) had a haemoglobin level recorded, 33.8% were anaemic by the WHO definition, 6.1% of patients (25/492) had a haemoglobin level of <11 g/dl. Anaemia was more common with declining renal function, 5% (20/399) of those in stage 3 and 41% (5/12) in stage 4 had a haemoglobin level below 11 g/dl; this change in proportion was highly significant (P<0.001, χ² test). 39% (155/399) of those in stage 3 and 92% in stage 4 fell within the WHO definition of anaemia. Again, this change in proportion was highly significant (P = 0.001, χ² test).

Nineteen patients had (non-dermatological) diagnoses of cancer that required active management from an oncologist or other specialist. Nine of the patients had diagnoses of senile dementia or were requiring nursing home care. No patients were identified who might have had an intercurrent illness that may have made them susceptible to dehydration at the time of measurement of SCr.

Discussion

The principal finding of this paper is that computer searches are a useful and valid way of rapidly screening for CKD in general practice computer systems. The study confirmed that whilst 5% of the population had stages 3–5 CKD, only a small proportion of them (8%; 40/492) had a renal diagnosis or had been seen by a renal physician. Only four additional cases were found through manual searching of GP records. Earlier recognition of patients with CKD would be beneficial because its progression can potentially be slowed by early intervention, for example with ACE inhibitors, and would trigger assessment for other associated conditions, including unrecognized cardiovascular disease, and their subsequent management. It is also likely to be cost-effective because treatment of serious impaired renal function, particularly renal dialysis, is very costly to healthcare systems. More women were identified as having CKD than men. This is in line with previous studies [3–5]. A possible reason may be that women live longer than men in the UK. Renal function declines with age, so CKD will be more prevalent in older age groups, in which there are more women than men. Women also consult their GP more than men, and may, potentially, be more likely to be tested for creatinine levels, making detection rates of CKD higher.

It is possible that computer recorded diagnoses for heart failure and stroke may over-estimate the prevalence of these conditions. Other cardiovascular diagnoses and cardiovascular risk factor recording appears complete and contemporary. Problems with ambiguous smoking codes, found in our main study, were not found in this data collection [5]. It is possible that the new quality based contract for general practice in the UK (which awards financially incentivized quality points, based on computer data) [17] may have encouraged better recording. Prescribing data are complete though it is hard to tell from the records whether prescribers have taken into account the potential effect of medications on renal function. As GFR is not yet routinely estimated or recorded in patient records, the GPs in this practice would not have been aware of GFR or stage of CKD. The largest problem areas were around clinical diagnoses of ‘prostatism’ or ‘bladder outflow’ problems. It was hard to tell whether this represented significant pathology that might affect renal function or not. Failure to code normal urine reagent strip tests as structured data, and to make ambiguous records suggesting that reagent strip testing had been done (and was probably negative though it was not stated), was an obvious shortcoming. Clearly there may be inter-practice variation in data quality and what has been found in this practice may not be found in another [18,19]; though the new quality-based contract for general practice is causing improvement and convergence in coding practice in the areas included, its main focus is cardiovascular disease.

The implications for practice from this study are that there is much that could be done in primary care to reduce the rate of progression of CKD and associated co-morbidity. The majority of those with stages 3–5 CKD have sub-optimally managed cardiovascular risk factors, and many are anaemic. Primary care medication reviews need to take into account renal function so that drugs that impair renal function might be stopped or reduced. A move to routine GFR reporting by pathology laboratories is to start in the UK. This is outlined in a recent guideline designed to improve management of renal disease in England and Wales (Part 2 of the Renal National Service Framework) [20]. A new Read code, for estimated GFR, is being created so that biochemistry laboratories can use the MDRD equation to estimate GFR and use the pathology links system to post the result into the GP computer system (Read code 451E ‘GFR calculated, abbreviated MDRD’). Formalized out-reach from renal centres might be a way of educating primary care teams that there is a job to be done in their CKD populations. This could also be supplemented by web-based health professional and patient education, together with electronic prompting; as internationally general practice computer systems become increasingly integrated with hospital and other data [21].

Earlier detection of unrecognized CKD by searching general practice computer systems could be used in any countries with computerized primary care services to improve the prevention of progression of renal dysfunction, with associated benefits on CVD. The computerized coding system used in many other European countries, ICPC, although having a less detailed coding system than the Read system, has sufficient granularity to enable data such as used in this study to be collected, and therefore these findings are also relevant outside the UK. Other countries with well developed primary care computing systems, including the Scandinavian countries, the Netherlands, Australia and New Zealand, could readily carry out similar computer searches [22].

Conclusions

Whilst there are inevitable shortcomings in the data, the automated searching of general practice computer records seems to be a reliable and valid method of identifying the majority of patients with stages 3–5 CKD. A significant proportion of these could benefit from interventions that could be provided in primary care.

Sobana Anandarajah and Tracey Tai carried out the notes search as their final year medical student projects. We thank Nigel Hague, research fellow, for writing the MIQUEST queries; Neil Dhoul, who extracted the computer data; Jeremy van Vlymen, for help with SPSS; and Sue Mayor, who commented on the manuscript.

Conflict of interest statement. No funding was received to carry out this study. M. Walker is an employee of Roche, who manufacture a treatment for renal anaemia. S. de Lisignan, P. Stevens and D. O'Donoghue have received research grant funding from Roche for other renal work referenced in this study. This manuscript has not been submitted to any editorial scrutiny by Roche.

References

Author notes

1St George's and 2Primary Care Informatics, Division of Community Health Sciences, St George's, University of London, London SW17 0RE, UK, 3Department of Renal Medicine, Kent and Canterbury Hospital, Ethelbert Road, Canterbury, Kent CT1 3NG, UK, 4Department of Renal Medicine, Salford Royal Hospitals NHS Trust, Hope Hospital, Salford M6 8HD, UK and 5Roche Products Ltd, Welwyn Garden City, Hertfordshire, AL7 3AY, UK