BACKGROUND: To explore whether clinical, teamwork, and interpersonal skills of nurses could be monitored inexpensively at
scale, we compared two simulation measurement tools at 160 primary health care facilities in Bihar, India. The
aim was to determine if the less expensive, less resource-intensive real-time assessment tool (RTAT) could be
used in place of the more expensive, more resource-intensive video coding tool.
METHODS: Embedded in a large-scale ongoing nurse mentoring program “AMANAT”, we conducted a cross-sectional
study comparing the measurement tools to assess clinical performance in normal and emergency obstetrical
and neonatal simulations. The RTAT consisted of a series of seven global questions related to clinical
identification, management, use of evidence-based practices, communication and teamwork. We also
analysed the simulations using a video coding tool, which included 80 clinical, teamwork and communication
indicators. We calculated the means and standard errors for each indicator using both tools. We calculated the
Cohen’s kappa coefficient to determine the level of agreement between the two tools.
RESULTS: We analysed paired data from the two tools for 222 simulations in three types of scenarios: normal
spontaneous vaginal deliveries, neonatal resuscitations and post-partum haemorrhages. The mean scores on
the RTAT were generally higher than those on the video coding tool. The kappa coefficients for all indicators
indicated no agreement between the two tools.
CONCLUSION: RTAT performed poorly against the video coding tool to measure the clinical skills of nurses in simulated
normal and complicated births at primary health care facilities in Bihar