Lung nodule measurement reliability in computed tomography: A retrospective study

Introduction. In the 21st century, medical imaging evolved from simple assessment of the presence of a pathological sign to quantitative assessment of detected changes. Despite the obvious dependence of diagnostic accuracy on the degree of measurement error, modern classifications of radiologic errors do not single out measurement error as a separate cause of discrepancies between specialists. We consider it necessary to study in detail the size of measurement error of radiologists in conditions close to routine practice and to determine the possibility of the influence of such an error on the formation of the correct patient management tactics.

Aim. To determine radiologist discrepancy in measuring the pulmonary nodule size during opportunistic screening in chest computed tomography (CT).

Materials and methods. A retrospective study was performed on a database containing the 3897 labeled native chest CT studies using a routine protocol. Experts determined the type and mean size of each pulmonary nodule. We used the following statistical methods: intraclass correlation coefficient to evaluate measurement reliability; limits of agreement (Bland-Altman method) to quantify measurement variability. The significance level for all statistical hypotheses was set to 0.05.

Results. The final analysis included 87 CT studies, each containing a single solid pulmonary nodule with a diameter ranging from 6 to 30 mm, independently measured by three radiologists. The median nodule size was 10 mm. The intraclass correlation coefficient was 0.95 with 95% CI (0.93; 0.97). The 95% CI for the maximum absolute difference between observers ranged between (1.5; 1.9) mm.

Conclusion. The measurement disperancy between three radiologists for solid pulmonary nodule in chest CT did not exceed the methodologically established threshold for pulmonary nodule growth.

1. Sullivan DC, Bresolin L, Seto B, Obuchowski NA, Raunig DL, Kessler LG. Introduction to metrology series. Stat Methods Med Res. 2015;24(1):3–8. https://doi.org/10.1177/0962280214537332.

2. Lederle FA, Wilson SE, Johnson GR, Reinke DB, Littooy FN, Acher CW et al. Variability in measurement of abdominal aortic aneurysms. J Vasc Surg. 1995;21(6):945–952. https://doi.org/10.1016/S0741-5214(95)70222-9.

3. Larici AR, Farchione A, Franchi P, Ciliberto M, Cicchetti G, Calandriello L et al. Lung nodules: size still matters. Eur Respir Rev. 2017;26:170025. https://doi.org/10.1183/16000617.0025-2017.

4. Николаев АЕ, Сучилова ММ, Коркунова ОА, Блохин ИА, Гончар АП, Решетников РВ, Морозов СП. Терминология описания органов грудной клетки – рентгенография и компьютерная томография. М.: ГБУЗ «НПКЦ ДиТ ДЗМ»; 2022. 84 с.

5. Bankier AA, MacMahon H, Goo JM, Rubin GD, Schaefer-Prokop CM, Naidich DP. Recommendations for Measuring Pulmonary Nodules at CT: A Statement from the Fleischner Society. Radiology. 2017;285(2):584–600. https://doi.org/10.1148/radiol.2017162894.

6. Chelala L, Hossain R, Kazerooni EA, Christensen JD, Dyer DS, White CS. Lung-RADS Version 1.1: Challenges and a Look Ahead, From the AJR Special Series on Radiology Reporting and Data Systems. Am J Roentgenol. 2021;216(6):1411–1422. https://doi.org/10.2214/AJR.20.24807.

7. Revel M-P, Bissery A, Bienvenu M, Aycard L, Lefort C, Frija G. Are Two-dimensional CT Measurements of Small Noncalcified Pulmonary Nodules Reliable? Radiology. 2004;231(2):453–458. https://doi.org/10.1148/radiol.2312030167.

8. Морозов СП, Кульберг НС, Гомболевский ВА, Ледихова НВ, Соколина ИА, Владзимирский АВ, Бардин АС. Свидетельство о государственной регистрации базы данных RU 2018620500. Тегированные результаты компьютерных томографий легких. 2018. Режим доступа: https://www.elibrary.ru/item.asp?id=39294439.

9. Morozov SP, Kuzmina ES, Vetsheva NN, Gombolevskiy VA, Lantukh ZA, Polishuk NS et al. Moscow Screening: Lung Cancer Screening With Low-Dose Computed Tomography. Probl Sotsialnoi Gig Zdravookhranenniiai Istor Med. 2019;27:630–636. https://doi.org/10.32687/0869-866X-2019-27-si1-630-636.

10. Владзимирский АВ, Новик ВП, Павлов НА, Андрейченко АЕ, Блохин ИА, Гомболевский ВА и др. Свидетельство о государственной регистрации базы данных RU 2023621103. MosMedData: КТ с признаками рака легкого. 2023. Режим доступа: https://elibrary.ru/item.asp?id=52121950.

11. Morozov SP, Gombolevskiy VA, Elizarov AB, Gusev MA, Novik VP, Prokudaylo SB et al. A simplified cluster model and a tool adapted for collaborative labeling of lung cancer CT scans. Comput Methods Programs Biomed. 2021;206:106111. https://doi.org/10.1016/j.cmpb.2021.106111.

12. Vasilev YA, Savkina EF, Vladzymyrskyy AV, Omelyanskaya OV, Arzamasov KM. Overview of modern digital diagnostic image markup tools. Kazan Medical Journal. 2023;104(5):750–760. (In Russ.) https://doi.org/10.17816/KMJ349060.

13. Kottner J, Audigé L, Brorson S, Donner A, Gajewski BJ, Hróbjartsson A et al. Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol. 2011;64(1):96–106. https://doi.org/10.1016/j.jclinepi.2010.03.002.

14. Abu-Arafeh A, Jordan H, Drummond G. Reporting of method comparison studies: a review of advice, an assessment of current practice, and specific suggestions for future reports. Br J Anaesth. 2016;117(5):569–575. https://doi.org/10.1093/bja/aew320.

15. Chow S-C, Shao J, Wang H, Lokhnygina Y (eds.). Sample size calculations in clinical research. Boca Raton: Taylor & Francis; 2017.

16. Jones M, Dobson A, O’Brian S. A graphical method for assessing agreement with the mean between multiple observers using continuous measures. Int J Epidemiol. 2011;40(5):1308–1313. https://doi.org/10.1093/ije/dyr109.

17. Blokhin IA, Kodenko MR, Shumskaya YF, Gonchar AP, Reshetnikov RV. Hypothesis testing using R. Digital Diagnostics. 2023;4(2):238–247. (In Russ.) https://doi.org/10.17816/DD121368.

18. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med. 2016;15(2):155–163. https://doi.org/10.1016/j.jcm.2016.02.012.

19. MacMahon H, Naidich DP, Goo JM, Lee KS, Leung ANC, Mayo JR et al. Guidelines for Management of Incidental Pulmonary Nodules Detected on CT Images: From the Fleischner Society 2017. Radiology. 2017;284(1):228–243. https://doi.org/10.1148/radiol.2017161659.

20. Marten K, Auer F, Schmidt S, Kohl G, Rummeny EJ, Engelke C. Inadequacy of manual measurements compared to automated CT volumetry in assessment of treatment response of pulmonary metastases using RECIST criteria. Eur Radiol. 2006;16(4):781–790. https://doi.org/10.1007/s00330-005-0036-x.

21. Zhao B, James LP, Moskowitz CS, Guo P, Ginsberg MS, Lefkowitz RA et al. Evaluating Variability in Tumor Measurements from Same-day Repeat CT Scans of Patients with Non–Small Cell Lung Cancer. Radiology. 2009;252(1):263–272. https://doi.org/10.1148/radiol.2522081593.

22. Ziefle M. Effects of Display Resolution on Visual Performance. Hum Factors. 1998;40(4):554–568. https://doi.org/10.1518/001872098779649355.

23. Al-Ekrish AA, Ekram MIH, Al Faleh W, Alkhader M, Al-Sadhan R. The validity of different display monitors in the assessment of dental implant site dimensions in cone beam computed tomography images. Acta Odontol Scand. 2013;71(5):1085–1091. https://doi.org/10.3109/00016357.2012.741709.

24. Garland LH. On the Scientific Evaluation of Diagnostic Procedures: Presidential Address Thirty-fourth Annual Meeting of the Radiological Society of North America. Radiology. 1949;52:309–328. https://doi.org/10.1148/52.3.309.

25. Waite S, Scott J, Gale B, Fuchs T, Kolla S, Reede D. Interpretive Error in Radiology. AJR Am J Roentgenol. 2017;208(4):739–749. https://doi.org/10.2214/AJR.16.16963.

26. Brady AP. Error and discrepancy in radiology: inevitable or avoidable? Insights Imaging. 2017;8(1):171–182. https://doi.org/10.1007/s13244-016-0534-1.

27. Uchevatkin AA, Yudin AL, Afanas’yeva NI, Yumatova EA. Shades of grey: how and why we make mistakes. Medical Visualization. 2020;24(3):123–145. (In Russ.) https://doi.org/10.24835/1607-0763-2020-3-123-145.

28. Aras M, Erdil TY, Dane F, Gungor S, Ones T, Dede F et al. Comparison of WHO, RECIST 1.1, EORTC, and PERCIST criteria in the evaluation of treatment response in malignant solid tumors. Nucl Med Commun. 2016;37(1):9–15. https://doi.org/10.1097/MNM.0000000000000401.

29. Schwartz LH, Litière S, De Vries E, Ford R, Gwyther S, Mandrekar S et al. RECIST 1.1 –Update and clarification: From the RECIST committee. Eur J Cancer. 2016;62:132–137. https://doi.org/10.1016/j.ejca.2016.03.081.

30. Yoon SH, Kim KW, Goo JM, Kim D-W, Hahn S. Observer variability in RECIST-based tumour burden measurements: a meta-analysis. Eur J Cancer. 2016;53:5–15. https://doi.org/10.1016/j.ejca.2015.10.014.

31. Pinto A. Spectrum of diagnostic errors in radiology. WJR. 2010;2(10):377. https://doi.org/10.4329/wjr.v2.i10.377.