Abstract:
Background: Radiologically-confirmed pneumonia (RCP) is a specific end-point used in
trials of Pneumococcal Conjugate Vaccine (PCV) to estimate vaccine efficacy. However,
chest radiograph (CXR) interpretation varies within and between readers. We measured
the repeatability and reliability of paediatric CXR interpretation using percent agreement
and Cohen's Kappa and the validity of field readings against expert review in a study of
the impact of PCV on pneumonia.
Methods: CXRs were obtained from 2716 children admitted between 2006 and 2014 to
Kilifi County Hospital, Kilifi, Kenya, with clinically-defined severe or very-severe
pneumonia. Five clinicians and radiologists attended a three-day training course on CXR
interpretation using a WHO standard. All CXRs were read once by two local primary
readers. Discordant readings and 13% of concordant readings were arbitrated by a panel
of three expert radiologists. To assess repeatability, a 5% median random sample was
presented twice. Sensitivity and specificity of the primary readers' interpretations was
estimated against the 'gold-standard' of the arbitrators' results.
Results: Of 2716 CXRs, 2 were uninterpretable and 159 were evaluated twice. The
percent agreement and Kappa for RCP were 89% and 0.68 and ranged between 84-97%
and 0.19-0.68, respectively, for all pathological findings. Intra-observer repeatability was
similar to inter-observer reliability. Sensitivities of the primary readers to detect RCP
were 69% and 73%; specificities were 96% and 95%.
Conclusion: Intra- and inter-observer agreements on interpretations of radiologicallyconfirmed pneumonia are fair to good. Reasonable sensitivity and high specificity make
radiologically-confirmed pneumonia, determined in the field, a suitable measure of
relative vaccine effectiveness.