This quality assurance project was designed to determine the reliability, completeness and comprehensiveness of the data entered into Niday Perinatal Database.
Quality of the data was measured by comparing data re-abstracted from the patient record to the original data entered into the Niday Perinatal Database. A representative sample of hospitals in Ontario was selected and a random sample of 100 linked mother and newborn charts were audited for each site. A subset of 33 variables (representing 96 data fields) from the Niday dataset was chosen for re-abstraction.
Of the data fields for which Cohen's kappa statistic or intraclass correlation coefficient (ICC) was calculated, 44% showed substantial or almost perfect agreement (beyond chance). However, about 17% showed less than 95% agreement and a kappa or ICC value of less than 60% indicating only slight, fair or moderate agreement (beyond chance).
Recommendations to improve the quality of these data fields are presented.
To examine the validity of case definitions for systemic autoimmune rheumatic diseases [SARD; systemic lupus erythematosus (SLE), systemic sclerosis (SSc), myositis, Sjögren's syndrome, vasculitis, and polymyalgia rheumatica] based on administrative data, compared to rheumatology records.
A list of rheumatic disease diagnoses was generated from population-based administrative billing and hospitalization databases. Subjects who had been seen by an arthritis center rheumatologist were identified, and the medical records reviewed.
We found that 844 Nova Scotia residents had a diagnosis of one of the rheumatic diseases of interest, based on administrative data, and had had = 1 rheumatology assessment at a provincial arthritis center. Charts were available on 824 subjects, some of whom had been identified in the administrative database with > 1 diagnosis. Thus a total of 1136 diagnoses were available for verification against clinical records. Of the 824 subjects, 680 (83%) had their administrative database diagnoses confirmed on chart review. The majority of subjects who were "false-positive" for a given rheumatic disease on administrative data had a true diagnosis of a similar rheumatic disease. Most sensitivity estimates for specific administrative data-based case definitions were > 90%, although for SSc, the sensitivity was 80.5%. The specificity estimates were also > 90%, except for SLE, where the specificity was 72.5%.
Although health administrative data may be a valid resource, there are potential problems regarding the specificity and sensitivity of case definitions, which should be kept in mind for future studies.
A prerequisite for using administrative data to study the care of critically ill patients in intensive care units (ICUs) is that it accurately identifies such care. Only limited data exist on this subject.
To assess the accuracy of administrative data in the Canadian province of Manitoba for identifying the existence, number, and timing of admissions to adult ICUs.
For the period 1999 to 2008, we compared information about ICU care from Manitoba hospital abstracts, with the criterion standard of a clinical ICU database that includes all admissions to adult ICUs in its largest city of Winnipeg. Comparisons were made before and after a national change in administrative data requirements that mandated specific data elements identifying the existence and timing of ICU care.
In both time intervals, hospital abstracts were extremely accurate in identifying the presence of ICU care, with positive predictive values exceeding 98% and negative predictive values exceeding 99%. Administrative data correctly identified the number of separate ICU admissions for 93% of ICU-containing hospitalizations; inaccuracy increased with more ICU stays per hospitalization. Hospital abstracts were highly accurate for identifying the timing of ICU care, but only for hospitalizations containing a single ICU admission.
Under current national-reporting requirements, hospital administrative data in Canada can be used to accurately identify and quantify ICU care. The high accuracy of Manitoba administrative data under the previous reporting standards, which lacked standardized coding elements specific to ICU care, may not be generalizable to other Canadian jurisdictions.
British Columbia's central prescription database, PharmaNet, is often used for both clinical and research applications. However, PharmaNet details prescription transactions, not actual medication consumption, resulting in many potential sources of inaccuracy when the information is assumed to reflect population or individual drug utilization.
To assess the accuracy of PharmaNet for adherence assessment in patients with heart failure who are taking beta-blockers.
A 6-month prospective, longitudinal assessment of adherence to the prescribed beta-blocker regimen was carried out using both PharmaNet data and the Medication Event Monitoring System (MEMS) for each patient enrolled. The limit of agreement between the 2 adherence assessment methods was assessed using the Bland-Altman approach.
Fifteen of 58 patients initially enrolled in the study were excluded, most due to misuse of MEMS or failure to return the MEMS vial despite thorough follow-up. For the 43 patients included in the final analysis, mean +/- SD adherence was 97.8 +/- 11.8% when assessed by PharmaNet and 97.1 +/- 7.3% when MEMS was used. However, the limit of agreement, reported as the mean of the differences +/- 2SD, was 6.8 +/- 18.5%, indicating a moderate-to-high level of agreement between the 2 methods when the confidence interval is taken into consideration.
These results suggest that PharmaNet data accurately reflect medication adherence for most patients. The MEMS system proved unreliable in several cases, illustrating the difficulty of identifying a gold standard for adherence assessment.
Clinical databases are increasingly being employed to evaluate the quality of treatments, including patients with peripheral vascular disease. Valid data is vital to the value of these analyses.
To assess the validity of clinical data in a population-based national vascular registry.
Traditional reproducibility study was supplemented by refilling of data by an independent observer, thereby creating three data sets for comparison.
Twenty prospectively recorded electronic forms from each department were selected randomly from the Danish National Vascular Registry. Data forms were refilled by the surgeons of the department concerned, and by an independent member of the board of the Danish National Vascular Registry. Refilling was performed blinded to the original forms.
A high degree of accuracy of clinical data can be achieved. An independent observer makes it possible to evaluate the classification of observer dependent parameters and explain differences in the reproducibility of data.
This study was designed to evaluate the accuracy of the Oncology Patient Information Systems (OPIS) database for patients with breast cancer and lymphoma. We conducted a detailed individual patient chart review of patients with lymphoma or breast cancer who were seen in consultation by an oncologist between July 1991 and June 1995. Information extracted directly from the patients' clinic charts was compared with information captured in the OPIS database with respect to demographics, staging, histological diagnosis, treatment, relapse status, date of relapse and survival. OPIS database failed to capture 14.4% and 23.4% of lymphoma and breast cancer patients seen over the four-year period. When compared to the clinic charts there were differences in staging in 31.5% and 8.1%, relapse status in 27.6% and 7.2%, and date of relapse in 56.4% and 14.7% of lymphoma and breast cancer patients respectively. The deficiencies and inaccuracies in the OPIS database emphasize the need for caution in basing administrative, policy, or practice decisions on this database.
To determine the validity of pregnancy variables recorded in administrative databases of Quebec using patient medical charts as the gold standard among asthmatic pregnant women.
Three administrative databases were linked and provided information on maternal, pregnancy and infant characteristics for 726 pregnant asthmatic women who delivered in 1990-2000. Algorithms were developed to measure variables that were not recorded directly in the databases or to minimize the number of missing values for variables recorded in two or more databases. Medical file data were collected by two trained research nurses in 43 hospitals. The validity of categorical variables was assessed with sensitivity, specificity, predictive positive values (PPVs) and predictive negative values (PNVs), whereas the validity of continuous variables was assessed with Pearson correlation using the medical chart as the gold standard.
The sensitivity of the sex of the baby, previous live birth and previous pregnancy ranged from 0.97 to 0.99. Corresponding figures were 0.92-0.98 for specificity. We also found high correlation coefficients, ranging from 0.875 to 0.999 for the length of gestation, dates of last menstruation and delivery, maternal age and birth weight.
Pregnancy-related variables recorded in administrative databases or derived from algorithms based on two or more databases were found to be highly valid as compared to the medical chart among asthmatic women.
There is little agreement on the philosophy of measuring clinical quality in health care. How data should be analyzed and transformed to healthcare information is an ongoing discussion. To accept a difference in quality between health departments as a real difference, one should consider to which extent the selection of patients, random variation, confounding and inconsistency may have influenced results. The aim of this article is to summarize aspects of clinical healthcare data analyses provided from the national clinical quality databases and to show how data may be presented in a way which is understandable to readers without specialised knowledge of statistics.