Dear Editor, We read with interest the paper by Rovamo et al in which the authors assessed the resuscitation skills of physicians on a manikin using a standard simulation scenario of birth asphyxia.(1) They used a30-item checklist to score the technical skills of each participant. The cut-off point for passing or failing each of the 30 items was determined by experts using the Angoff method for absolute standards setting, as stated by the authors.(2) The results they obtained showed a high prevalence of inadequate skills in neonatal resuscitation. The Angoff method is based on the estimated percentage of borderline candidates who will adequately perform each assessment item, as judged by a large panel of experts in that field. The scores (or percentages) for all the items defining the whole assessment are summed up then averaged and that obtained average constitutes the final passing score.(2) It is widely used and recommended for standard setting of objective structured clinical examinations (OSCE) even when used in simulation on a manikin such in this study.(3) We have concerns regarding the way in which this method was used by the authors. It seems that the same score of 0.662 (or 66% of borderline participants) was estimated for each of the 30 items of neonatal resuscitation and the total passing mark was calculated accordingly. We feel that this methodology is intrinsically flawed and likely to have influenced the obtained results. This is because it is extremely unlikely and indeed implausible that exactly the same percentage of participants (66%) would be expected to adequately perform each of the separate 30 skills encompassing an extremely wide range of complexity, for example something as simple as drying the baby or as technically demanding as performing endotracheal intubation. Without adopting the proper standard setting methodology, the stated results cannot be taken at face value. There could be, worryingly, even far more physicians with inadequate resuscitation skills than stated by the authors, or, reassuringly, many more who can perform adequate neonatal resuscitation. The resulting implications for corrective strategies for neonatal resuscitation training are too important.

References

1. Rovamo L, Mattila MM, Andersson S, et al. Assessment of newborn resuscitation skills of physicians with a simulator manikin. Archives of disease in childhood Fetal and neonatal edition 2011;96:F383-9.

2. Angoff WH. Scales, Norms and Equivalent Scores. Educational Measurement. Washington, DC: American Council on Education, 1971:508-600.

3. Pell G, Fuller R, Homer M, et al. How to measure the quality of the OSCE: A review of metrics - AMEE guide no. 49. Medical teacher 2010;32:802-11

Conflict of Interest:

None declared