Page 71 - JSOM Spring 2023

and US-guided vascular access, and regional anesthesia. A Correlations Between Motion Metrics and Expert Ratings
trained examiner was present at each station to grade partic- All metrics had significant moderate negative correlations with
ipants on a series of dichotomous (yes/no) and recall (probe expert ratings (Pearson correlation coefficient between 0.49
selection/intervention choice) questions. 9 and 0.64 and p < 0.001 for all; Table 2).

Observational and Motion-Based Feedback OSCE Scores
After each US exam, participants were advised verbally by The average scores ± standard deviations for the OSCE sta-
course instructors on how to improve their US imaging based tions are below:
on direct observation. Therefore, each participant was given
direct, individualized feedback at least twice daily. The latter • US-guided Regional Anesthesia: 94.3% ± 11.05%
three days of the course also included morning feedback ses- • Lung US: 100% ± 0%
sions, in which participants were coached based on trends in • Transthoracic Echocardiography: 99.6% ± 2.04%
their motion metrics. During feedback sessions, the instruc- • RUSH: 99.6% ± 1.86%
tors advised trainees on how they could better acquire each US • Ultrasound-Guided Vascular Access: 99.2% ± 4.08%
view and perform the complete RUSH exam. Instructors used
a combination of expert rating scores and motion metrics to Discussion
generate feedback unique to each medic and identify the most
challenging views to acquire. SOF combat and tactical medics demonstrated significant neg-
ative trends in all four motion metrics, indicating improved
Statistical Analysis performance and economy of motion. Motion metrics had
Motion data were post-processed using Excel (Microsoft, significant moderate negative correlations with expert rat-
https://www.microsoft.com/en-us/microsoft-365/excel). Statis- ings. Path length decreased as medics became familiar with
tical analyses were performed with Stata/Special Edition 13.1 the appropriate landmarks and intercostal spaces associated
(StataCorp LP, https://www.stata.com/). We analyzed the trend with each US view. Over time, medics’ ability to slide, tilt, and
of each metric (path length, translational motions, rotational rotate the probe to optimize their image improved, resulting
sum, and time) for the overall RUSH exam across the eight in gradual decreases in both path length and rotational sum.
trials using generalized estimating equations (GEE), taking Translational motions decreased following a reduction in the
into account medic variability, with trial as the independent number of extraneous movements required to locate and op-
variable. We used a Gaussian distribution, identity link func- timize each US view. Throughout the course, medics obtained
tion, and exchangeable correlation structure. Results of the each view more efficiently as well, decreasing their time taken.
GEE analysis are summarized as coefficients (95% confidence Individually, motion metrics can provide valuable insight into
interval). Given the multiple comparisons made (four metrics), the dexterity of a sonographer. In conjunction, certain trends
we applied a Bonferroni correction and considered a p-value may also reveal changes in performance. Although prior expe-
of < 0.0125 to be significant. rience (as depicted in Table 1) may have impacted the baseline
performance of medics, we still found a significant negative
To determine how motion metrics correlated with expert rat- trend in motion metrics after taking into account variability in
ings, we calculated the Pearson correlation coefficient between the medics in the GEE analysis. This may be evidence that even
each metric and item scored by experts. Because of multiple with previous experience, medics can still improve in these
comparisons (four metrics and five items), we applied a Bon- skills, which can be objectively detected with motion metrics.
ferroni correction and considered a p-value of < 0.0025 to be
significant. Anecdotally, the authors note that trends in motion metrics
have corresponded with patterns in image acquisition. For ex-
OSCE scores were averaged for each station and are reported ample, trainees who exhibit a particularly large path length
as average ± standard deviation. and small rotational sum tend to rely more on gross motor
movements, as opposed to minor adjustments, to optimize
their images. Early in the course, medics would slide over in-
Results
tercostal spaces several times before attempting to tilt or ro-
A total of 24 SOF combat and tactical medics participated in tate their probe to acquire an image. As training progressed,
the study and completed eight trials each. Participants’ prior they located image windows with less gross motor movement
experience with ultrasound is summarized in Table 1 (median (reduced path length) and more deliberately manipulated the
number of US training sessions before this course: 2–5; me- probe to optimize their images (reduced rotational sum).
dian number of US exams conducted independently before this
course: 1 –5). Motion metrics for one trial were corrupted and Previous studies have found moderate to strong correla-
therefore excluded from the analysis. tions amongst motion metrics and global rating scale (GRS)
scores. 15,20,21 Similarly, in this study motion metrics of the US
Trends of Motion Metrics for the RUSH Exam probe exhibited moderate correlations with expert ratings.
Average motion metric values of the RUSH exams are depicted
in Figures 2–5. There were significant negative trends in all In the three recent iterations of our course, we introduced mo-
metrics as the number of trials increased (path length: –216.77 tion analysis to objectively assess performance in RUSH ex-
[–256.18 to –177.36] cm per trial, translational motions: ams. Using these innovative metrics, our goal was to ensure
–54.77 [–65.36 to –44.17] translational motions per trial, ro- that all learners who complete our training program acquire
tational sum: –1457.27 [–1743.06 to –1171.48] degrees per the skills necessary to successfully perform RUSH exams using
trial, time: –23.82 [–28.13 to –19.52] seconds per trial; p < portable US probes. Beyond their use in summative skills as-
0.001 for all). sessment, we used motion metrics of the US probe to provide

Motion Analysis: POC Ultrasound | 69