Page 98 - Journal of Special Operations Medicine - Summer 2015

compared by using the sign test. Reliability analysis was Results
performed by grouping the scores according to reviewer Data were collected by study investigators for 32 pa-
type (clinician versus TNC) and assuming each group tients who received trauma-team activation during
consisted of scores by a single reviewer. Interrater agree- this period. Patient characteristics are shown in Table
ment was examined with the Bland–Altman limits of 2. The JTTR recorded “motor vehicle collision” as the
agreement method, with reproducibility defined as 95% mechanism of injury in three patients; however, two of
of differences lying within 2 standard deviations (SDs) of these were unequivocally casualties from motor vehicles
the mean. 22
struck by improvised explosive devices (IEDs). Four ad-
ditional records of “major trauma” casualties (ISS ≥16)
Interrater agreement and reliability were further exam- were found in the JTTR for this period. Although these
ined by using weighted κ and ICC statistics, respectively. would have been expected to meet inclusion criteria,
Weighted κ statistics treat the assigned scores as ordinal they had not been identified by the investigators. Conse-
data and provide an index of agreement between the quently, no ratings by clinical staff had been performed,
two raters (clinicians and TNCs). The κ estimates were and they could not be included in analysis.
based on squared weights: the squared distance from
perfect agreement determined the weight assigned to Table 2 Patient Characteristics
any disagreement between raters. 23
Characteristic Data
To examine consistency rather than absolute agreement, Male sex, no. (%) 30 (94)
and for comparison with similar studies, 16,17 interrater Age, median (IQR) [range], y 22 (21–27) [3–42]
reliability of maximum AIS(M), ISS, and NISS was as- Affiliation, no. (%)
sessed by using ICC estimates. We assumed cases were
drawn from a larger pool of casualties but that raters Afghan Security Forces 15 (47)
were fixed, and thus estimated single-measures ICC ISAF 11 (34)
using a one-way random-effects model. ICC estimates Civilian 6 (19)
were reported with 95% confidence intervals (CIs). Our Mechanism of injury, no. (%)
predetermined CI width was 95%, in keeping with com-
mon practice and for comparison with other published Explosion 19 (59)
16
results. We used the CI to reflect the level of uncertainty Gunshot 10 (31)
associated with the estimates, not to imply statistical sig- Burns 2 (6)
nificance; our study was not powered to do so. Motor vehicle collision 1 (3)
Note: IQR, interquartile range; ISAF, International Security Assistance
Both weighted κ and ICC were interpreted by using the Force
24
arbitrary method of Landis and Koch. Statistics were
calculated using R, version 3.10 (R Foundation for Sta- The median ISS recorded by investigators was 20 (inter-
tistical Computing; www.r-project.org) with the Meth- quartile range [IQR], 9–28) compared with 17.5 (IQR,
25
26
Comp and irr libraries. 9–31.5) for the JTTR (p = .7). Investigators recorded a
median NISS of 27 (IQR, 12–42) while the JTTR data
Data Collection had a median of 25.5 (IQR, 11.5–41; p = .7). The 32
During a 3-week period from 24 September 2012 to 16 casualties had sustained 214 injuries as recorded in the
October 2012, attending surgeons who operated on these JTTR, whereas the study investigators noted 212 inju-
patients were provided with a copy of the AIS(M) diction- ries. The percentage agreement for number of injuries
ary and asked to code each injury with which they had recorded was 19%, with a difference of up to 14 inju-
been involved. In cases where the patient underwent sur- ries per anatomic region for the entire cohort (Table 3).
gery, this coding took place as soon after the initial opera- Injuries were recorded for 98 separate body regions in
tion as was practical. ISS and NISS were calculated from at least one of the datasets. Percentage agreement for
these codes. In parallel (and independent of the study in- the highest regional AIS(M) was 51%. Study investiga-
vestigators), JTTR data collection continued as normal, tors recorded 75 injuries scored as “serious” or greater
undertaken by TNCs. JTTR scores were retrieved via a [AIS(M) ≥3], whereas the JTTR had 68 such injuries re-
standard request for release of data analyzed after data corded. Percentage agreement for the maximum AIS(M)
collection was complete. The following data were identi- assigned to cases [maxAIS(M)] was 66% (Table 4), with
fied from each source: AIS(M) for each injury, number of disagreement by one AIS(M) grade in 10 of 11 cases
injuries in each body region (by AIS definition), highest (91%). The remaining case differed by two grades.
AIS(M) in each body region, number of injuries for each
severity grade, total number of injuries, ISS, and NISS. All Percentage agreements for identical ISS and NISS were
matched cases were included in analysis. 34% and 28%, respectively. Reproducibility criteria were

88 Journal of Special Operations Medicine Volume 15, Edition 2/Summer 2015