Page 69 - JSOM Summer 2024

The Effect of Critical Task Auto-failure Criteria on
Medical Evaluation Methods in the Pararescue Schoolhouse

1
2
Ian P. Richardson, MS, NRP *; Michael J. Lauria, MD, NRP ;
4
Brian L. Gravano, NRP ; Jeffrey M. Swenson, NRP ; Stephen C. Rush, MD 5
3

ABSTRACT
Background: Medical training and evaluation are important Given the importance of medical simulation, how we eval-
for mission readiness in the pararescue career field. Because uate medical simulation is critical. One of the primary chal-
evaluation methods are not standardized, evaluation meth- lenges is the lack of standardized training evaluation metrics
ods must align with training objectives. We propose an alter- for tactical medicine, even though medical evaluations are
native evaluation method and discuss relevant factors when ubiquitous and requisite for pararescue combat mission read-
designing military medical evaluation metrics. Methods: We iness. The inherent complexity of combat medicine scenarios
4
compared two evaluation methods, the traditional checklist provides additional complexity when designing evaluation
(TC) method used in the pararescue apprentice course and an metrics. 5
alternative weighted checklist (AWC) method like that used at
the U.S. Army static line jumpmaster course. The AWC allows The traditional checklist (TC) evaluation counts a single in-
up to two minor errors, while critical task errors result in auto- structor assist as an error, with the second error resulting in
failure. We recorded 168 medical scenarios during two Appren- failure. All errors carry the same weight within the current
tice course classes and retroactively compared the two evalua- evaluation criteria, and any two errors result in failure. This
tion methods. Results: Despite the possibility of auto-failure evaluation format is common within Air Education and Train-
with the AWC, there was no significant difference between ing Command (AETC) and U.S. Air Force Special Warfare
the two evaluation methods, and both showed similar overall training. This evaluation method presents two problems.
6,7
pass rates (TC=50% pass, AWC=48.8% pass, p=.41). The two First, equal weight for all errors may result in unintended and
evaluation methods yielded the same result for 147 out of 168 inappropriate student focus. For instance, critical actions such
scenarios (87.5%). Conclusions: The AWC method strongly as correct tourniquet application carry the same weight as less
emphasizes critical tasks without significantly increasing fail- critical errors such as gross manipulation of fractures or docu-
ures. It may provide additional benefits by being more closely mentation errors. This weighting may result in students focus-
aligned with our training objectives while providing quantifi- ing too much on some aspects of care and too little on others.
able data for a longitudinal review of student performance. Second, this method provides little quantifiable information to
enable trending student performance over time.
Keywords: evaluation metrics; military medicine; training
techniques; trauma care; education; pararescue This investigation aimed to address these problems by modify-
ing the TC evaluation method to emphasize critical tasks while
maintaining a similar level of difficulty for the evaluation. We
propose an alternative weighted checklist (AWC) evaluation
Introduction
method, using a numeric score out of 100 possible points,
Training via medical simulation is crucial to readiness. There with 70 points required to pass (Figure 1). This evaluation
are ethical and logistical constraints to teaching novice learn- method is based on the major error/minor error system some-
ers using live human patients, and it is difficult to integrate times used in military training environments such as static line
military medics into civilian hospitals. Thus, medical simula- jumpmaster school. We hypothesize that the AWC will more
8
tion provides an important training substitute. There is strong appropriately emphasize critical tasks but that the possibility
1
evidence that medical simulation training is broadly effective of auto-failure using the AWC will increase failure rates.
in both civilian and military environments. In a review of 44
2
studies, Lynagh et al. found that 70% of studies reported skill Methods
laboratories or simulator training significantly improved pro-
cedural skills in medical students when compared with stan- The Pararescue Apprentice tactical medicine course is located
dard or no training. 3 at Kirtland Air Force Base, New Mexico, and trains students in
*Correspondence to ian.richardson.5@us.af.mil
1 Ian P. Richardson is the Program Manager of Tactical Medicine at the 351st Special Warfare Training Squadron, United States Air Force; Kirtland
Air Force Base, Albuquerque, NM. Dr. Michael J. Lauria is an Emergency Medical Services and Critical Care Fellow at the University of New
2
3
Mexico School of Medicine; Albuquerque, NM. Staff Sergeant Brian L. Gravano is a Pararescueman at the 103rd Rescue Squadron, United
4
States Air Force; Francis S. Gabreski Air National Base, Westhampton Beach, NY. Technical Sergeant Jeffrey M. Swenson is a Pararescueman and
Instructor at the 351st Special Warfare Training Squadron, United States Air Force, Kirtland Air Force Base, Albuquerque, NM. Lt Col Stephen
5
C. Rush is a flight physician at the 308th Rescue Squadron, United States Air Force; Patrick Space Force Base, FL; and is also affiliated with the
departments of Radiation, Oncology, and Neurosurgery, New York University; New York, NY.
67