Patients with psoriasis referred to the Dermatology Service at Women's College Hospital and satisfying certain criteria relating to percentage body involvement, age, and the absence of serious coincident conditions were assigned at random to three weeks of day care and education at the Psoriasis Education and Research Centre (PERC) or to the Dermatology Service at Women's College Hospital (WCH) for normal hospital care. On admission, all study patients received normal history and physical examinations and were photographed by a standardized procedure that provided an accurate estimate of type and extent of body involvement. A functional history was taken from PERC patients that provided information concerning their ability to cope at home, at work, and socially; the extent and appropriateness of their self-care practices; and their knowledge concerning the pathophysiology and etiology of psoriasis and the names and actions of the medications they were using. Individualized patient education programs were designed with reference to the medical and functional information and implemented in the three weeks of day care. Photographic assessment and the functional history were repeated at three weeks, six months and twelve months. Hospital patients were reassessed at six and twelve months and a functional history was taken at six months. The functional status of PERC and hospital patients was compared at six months. The results of this study reinforced the belief that in the case of psoriasis, education coupled with treatment is more effective than treatment alone.
Since 1986, the Ontario Ministry of Health has provided a medical licensure preparation programme for international medical graduates. Because of the diversity in candidates' oral English proficiency, this competency has been viewed as a particularly important selection criterion.
To assess and compare the quality of ratings of oral English proficiency of international medical graduates provided by physician examiners and by standardized patients (SPs). PARTICIPANTS AND MATERIALS: The study samples consisted of 73 candidates for the Ontario International Medical Graduate (IMG) Program, and physician examiners and SPs in five 10-minute encounter objective structured clinical examination (OSCE) stations. Materials used were a seven-item speaking performance rating instrument prepared for the Ontario IMG Program.
Rating sheets were scanned and the results analysed using SPSS 9.0 for Windows.
Correlations between the physician and SP ratings on the seven items ranged from 0.52 to 0.70. The SPs provided more lenient ratings. Mean alpha reliability for the physicians' ratings on the seven items was 0.59, and for the SPs' 0.64. There was poor agreement between the two sets of raters in identifying problematic candidates.
Notwithstanding the sizable correlations between the ratings provided by the two rater groups, the results demonstrated that there was little agreement between the two groups in identifying the potentially problematic candidates. The physicians were less prone than the SPs to rate candidates as problematic. SPs may be better placed than the physician examiners to directly assess IMG candidates' oral English proficiency.
To assess the quality of ratings of interviewing skills and oral English proficiency provided on a clinical skills OSCE by physician examiners, standardized patients (SPs), and communication skills experts.
In 1998, 73 candidates to the Ontario International Medical Graduate (OIMG) Program completed a 29-station OSCE-type clinical skills selection examination. Physician examiners, SPs, and communication skills experts assessed components of oral English proficiency and interview performance. Based on these results, the frequency and generalizability of English-language flags, physician examiners' indications that spoken English skills were bad enough to significantly impede communication with patients; the reliability of the OIMG's Interview and Oral Performance Scales and generalizability of overall interview and oral performance ratings; and comparisons of repeated assessments by experts were calculated. Principal-components analysis was applied to the panels' ratings to determine a more economical expression of the language proficiency and interview communication skills results.
The mean number of English-language flags per candidate was 2.1, the median was 1.0, and Cronbach's alpha of the ratings was 0.63. Means, SDs, and alphas of the physician examiners' and SPs' ratings of the interview performance scale were 9.15/10, 0.43, 0.36, and 9.30/10, 0. 56, 0.50, respectively. Corresponding values for overall interview performance ratings were 3.08/4, 0.30, 0.33, and 3.34/4, 0.32, 0.47. Means, SDs, and alphas of the physician examiners' and SPs' ratings of the oral performance scale were 8.54/10, 0.74, 0.78, and 8.74/10, 1.00, 0.76. Corresponding values for overall ratings of oral performance were 3.85/5, 0.51, 0.68, and 4.08/5, 0.60, 0.68. For the two experts' ratings of two contiguous five-minute interview stations, internal consistencies were 0.88 and 0.78. For the two experts' ratings of standardized ten-minute interviews, internal consistencies were 0.81 and 0.92. Correlations between the mean values of the experts' ratings of the ten- and five-minute stations were 0.45 and 0.51. Three factors emerged from the PCA, language proficiency, physician examiners' ratings of interview proficiency, and SPs' ratings of interview proficiency.
Consistency between the physician examiners' and SPs' ratings of English proficiency was observed; less agreement was observed in their ratings of interviewing skills, and little agreement was observed between the experts' ratings. Communication skills results may be validly expressed by three measures: one overall global rating of language proficiency provided by physician examiners or SPs, and overall global ratings of interview proficiency provided separately by physician examiners and SPs.
The Medical Council of Canada has made use of examiners' pass/fail classifications of candidates' behaviors in objective structured clinical examination (OSCE) stations in defining cutting scores for these stations. This process assumes that there is consistency in the judgments of examiners employed in the same stations at different testing sites and in the cutting scores derived from these judgments. These assumptions were tested using the results of the fall 1993 administration of part 2 of the Medical Council of Canada's Evaluating Examination to 744 candidates. The results of this study provided evidence of the consistency of the pass/fail and cutting score definitions for the stations used across examiners.
A Discriminant Function Analysis on 233 graduates of foreign medical schools, from whom a group of 24 were selected that although several variables correlated significantly with selection, no combination of demographic variables could act as an efficient screen of clinically deficient candidates.
To apply differential item functioning (DIF) procedures to investigate station gender bias in multiple-station tests of clinical skills, and to compare these results with those obtained by comparing the station-score distributions of men and women examinees.
The data were from 23 stations used in the selection of seven successive cohorts (1987-1993) of candidates to the Ontario Pre-Internship Program for graduates of foreign medical schools. The stations had been used on at least three occasions, with a minimum sample of about 210 candidates per station. Each station's score was expressed as both a binary score and a continuous score, and DIF was assessed using the Mantel-Haenszel procedure with the binary scores and analysis of covariance with the continuous scores. For each station, DIF effect sizes were calculated and compared with the gender-group mean differences.
Using the binary scores, significant DIF was observed for three stations; using the continuous scores, significant DIF was observed for five stations. Significant gender differences were observed in the scores of nine stations. In eight, these differences favored women. Overall, in more stations the direction of DIF favored the men, while the women demonstrated higher levels of ability.
The results suggest the importance of using a DIF approach for controlling the "ability factor" in studies of this kind: although significant gender differences were observed in the continuous score distributions of nine stations, generally these differences were not indicative of station gender bias.