Headlines: July 18th, 2005

A new report points to a decline in performance by doctors taking a postgraduate medical exam between 1997 and 2001. The study has been conducted by the Royal Colleges of Physicians and is published today in the Open Access journal BMC Medicine. The fall follows more than a decade of improved performances.The report indicates that the overall performance of postgraduate medical students taking the Part 1 examination of the Membership of the Royal Colleges of Physicians on specific exam questions that were repeated across the years was 14.1 per cent lower in 2001 than in 1996. The exam is part of higher specialist training.

Professor Chris McManus from University College London, who wrote the report with colleagues from the Royal Colleges, said the reasons for the poorer performance were not clear but the finding had implications for medical education. Further studies were needed, he said in other postgraduate and undergraduate examinations.”

The author says the role of postgraduate examinations is to set standards of practice and to assure both the public and the medical profession that doctors have the knowledge and expertise needed to diagnose and treat patients and to progress in their medical careers.

The report looks at the performance of students on ‘marker questions’ – those repeated in different Royal College membership examinations held between 1985 and 2001. The study shows that overall the ability of examination candidates increased from 1985 to 1997 but then declined sharply.

The researchers also compared the performance of doctors in an examination held in 1996 and another that was held in 2001. The results from this part of the study show that fewer candidates who passed the examination in 2001 gave correct answers than their 1996 equivalents had done. The candidates’ mean score went down from 69.4 in 1996 to 59.6 in 2001.

The report says past reliance on ‘norm-referencing’ for setting the pass mark, which saw a fixed proportion of candidates allowed to pass each examination, meant that candidates achieving a pass at the end of this period had a somewhat lower standard. Norm referencing was abandoned for the exams in 2002 in favour of ‘criterion-referencing’. In this system examiners determine the pass mark according to levels of desirable knowledge based on individual questions. This means the standard required to pass all parts of the exam is fixed and is independent of the standard of the candidates sitting at any time.