Automatic speech recognition in voice-speech rehabilitation effectiveness evaluation in patients after laryngectomy

Introduction. Lost voice function compensation determines the personal and social life of laryngectomees. Automatic speech recognition and synthesis methods are widely used as apps for additional and alternative communication. One of the urgent tasks in clinical practice is voice restoration effectiveness evaluation.

Aim. To evaluate the effectiveness of voice rehabilitation results in laryngectomized patients using the automatic assessment of speech intelligibility.

Materials and methods. 3 groups of 30 laryngectomized patients depending on the method of voice rehabilitation (esophageal voice, tracheoesophageal voice, electrolarynx), and 14 patients after various surgical interventions on the larynx were included. All patients underwent pseudo-voice recording for further assessment of phrasal intelligibility using the automatic assessment software module, as well as by a trained and untrained listener.

Results and discussion. Comparative analysis shows a minimum intelligibility level in automatic recognition, apparently due to semantic and contextual recognition even in untrained listener. Alaryngeal speech demonstrates worse recognition, compared to organ-preserving operations on the larynx, in untrained listeners and in automatic recognition. Experienced listeners demonstrate a consistently high level of recognition of all types of substitute speech.

Conclusion. Objectification of the intelligibility assessment of substitute speech using automatic recognition systems allows leveling semantic and contextual recognition during assessment by both trained and untrained listeners. Automatic speech recognition and synthesys systems have application prospects in rehabilitation medicine, in particular, in patients with head and neck cancer.

1. Daikhes NA, Vinogradov VV, Reshulskii SS, Isaeva ML, Fedorova EB, Khabazova AM et al. Comprehensive rehabilitation of patients after total laryngectomy: мajor developmental milestones. Rossiiskaya Otorinolaringologiya. 2022;21(1):93–104. (In Russ.) https://doi.org/10.18692/1810-4800-2022-1-93-104.

2. Daikhes NA, Vinogradov VV, Reshulsky SS, Azizyan RI, Isaeva ML. Functional and oncological results of endolaryngeal laser resections of squamouscell carcinomas of the voice box. Opuholi Golovy i Sei. 2022;12(3):37–43. (In Russ.) https://doi.org/10.17650/2222-1468-2022-12-3-37-43.

3. Kozhanov AL. Current aspects of treatment and rehabilitation of patients with pharyngeal cancer. Opuholi Golovy i Sei. 2016;6(2):17–25. (In Russ.) https://doi.org/10.17650/2222-1468-2016-6-2-17-24.

4. Allegra E. Verbal performance of total laryngectomized patients rehabilitated with esophageal speech and tracheoesophageal speech: impacts on patient quality of life. Psychol Res Behav Manag. 2019;12:675–681. https://doi.org/10.2147/PRBM.S212793.

5. Kapila M, Deore N, Palav RS, Kazi RA, Shah RP, Jagade MV. A brief review of voice restoration following total laryngectomy. Indian J Cancer. 2011;48(1):99–104. https://doi.org/10.4103/0019-509X.75841.

6. Souza FGR, Santos IC, Bergmann A, Thuler LCS, Freitas AS, Freitas EQ. Quality of life after total laryngectomy: impact of different vocal rehabilitation methods in a middle income country. Health Qual Life Outcomes. 2020;18(1):92. https://doi.org/10.1186/s12955-020-1281-z.

7. Daikhes NA, Vinogradov VV, Reshul’skii SS, Isaeva ML Voice restoration possibilities and ways to optimize the choice of voice rehabilitation method in laryngectomized patients. Vestnik Oto-Rino-Laringologii. 2023;88(3):108–109. (In Russ.) Available at: https://www.elibrary.ru/xmbcoi.

8. Crosetti E, Fantini M, Arrigoni G, Salonia L, Lombardo A, Atzori A et al. Telephonic voice intelligibility after laryngeal cancer treatment: is therapeutic approach significant? Eur Arch Otorhinolaryngol. 2017;274(1):337–346. https://doi.org/10.1007/s00405-016-4217-9.

9. Hilgers FJ, Ackerstaf AH. Comprehensive rehabilitation after total laryngectomy is more than voice alone. Folia Phoniatr Logop. 2000;52(1-3):65–73. https://doi.org/10.1159/000021514.

10. Xiao X, Droppo J, Acero A. Information retrieval methods for automatic speech recognition. In: 2010 IEEE International Conference on Acoustics, Speech and Signal Processing. Dallas, TX, USA; 2010, pp. 5550–5553. https://doi.org/10.1109/ICASSP.2010.5495229.

11. Koren EE, Stepanova YuE, Mokhotaeva MV, Bakhilin VM. The acoustic analysis results voice patients with organic and functional dysphonias. Rossiiskaya Otorinolaringologiya. 2013;(3):70–74. (In Russ.) Available at: https://cyberleninka.ru/article/n/rezultaty-akusticheskogo-issledovaniyagolosa-bolnyh-s-funktsionalnymi-i-organicheskimi-disfoniyami/viewer.

12. Шевченко ТИ, Бурая ЕА, Галочкина ИЕ, Кузьмина МО, Сокорева ТВ, Федотова МВ. Методы анализа звучащей речи: новые измерения и результаты. Дубна: Феникс+; 2017. 248 с.

13. Focht KL, Martin-Harris B, Bonilcha HS. Stroboscopic parameters reported as voice outcome measures in patients treated for laryngeal cancer: a systematic review. J Med Speech Lang Pathol. 2013;21(3):5. Available at: https://pubmed.ncbi.nlm.nih.gov/25339842.

14. Dejonckere PH, Bradley P, Clemente P, Cornut G, Crevier-Buchman L, Friedrich G et al. A basic protocol for functional assessment of voice pathology, especially for investigating the efficacy of (phonosurgical) treatments and evaluating new assessment techniques. Eur Arch Otorhinolaryngol. 2001;258(2):77–82. https://doi.org/10.1007/s004050000299.

15. Jiang J, Zhang Y, McGilligan C. From Modeling to Measurement Chaos in Voice. J Voice. 2006;20(1):2–17. https://doi.org/10.1016/j.jvoice.2005.01.001.

16. Sprecher A, Olszewski A, Jiang JJ, Zhang Y. Updating signal typing in voice: addition of type 4 signals. J Acoust Soc Am. 2010;127(6):3710–3716. https://doi.org/10.1121/1.3397477.

17. Kostuchenko E, Novokhrestova D, Pekarskikh S, Shelupanov A, Nemirovich-Danchenko M, Choynzonov E, Balatskaya L. Assessment of Syllable Intelligibility Based on Convolutional Neural Networks for Speech Rehabilitation After Speech Organs Surgical Interventions. In: Salah A, Karpov A, Potapova R (eds.). Speech and Computer. SPECOM 2019. Lecture Notes in Computer Science, vol. 11658. Cham: Springer; 2019, pp. 359–369. https://doi.org/10.1007/978-3-030-26061-3_37.

18. Tkachenko GA, Podvyaznikov SO, Mudunov AM, Gusakova ЕV. Psychological assistance to patients after laryngectomy. Opuholi Golovy i Sei. 2020;10(1): 101–106. (In Russ.) https://doi.org/10.17650/2222-1468-2020-10-1-101-106.

19. Lorenz KJ. Rehabilitation after total laryngectomy – A tribute to the pioneers of voice restoration in the last two centuries. Front Med. 2017;4:81. https://doi.org/10.3389/fmed.2017.00081.

20. Daikhes NA, Osipenko EV, Orlova OS, Isaeva ML, Kotelnikova NM, Mikhalevskaya IA, Krivykh YuS. Methods of formation of oral speech in laryngectomized patients (literature review) Otorhinolaryngology. Eastern Europe. 2021;11(3):334–347. (In Russ.) https://doi.org/10.34883/PI.2021.11.3.022.

21. Новохрестова ДИ, Костюченко ЕЮ, Балацкая ЛН, Чойнзонов ЕЛ. Методы и модель оценки качества произношения слогов при речевой реабилитации. М.: НИЦ МИСИ; 2019. 69 с.

22. Singer S, Wollbrück D, Dietz A, Schock J, Pabst F, Vogel H-J. Speech rehabilitation during the first year after total laryngectomy. Head Neck. 2013;35(11):1583–1590. https://doi.org/10.1002/hed.23183.

23. Balatskaya LN, Krasavina EA, Mescheryakov RV, Chizhevskaya SYu, Choinzonov EL. Acoustic analysis at stages of combined treatment and vocal rehabilitation at sick of cancer of throat and drink throat. Neurocomputers. 2023;25(1):17–24. (In Russ.) Available at: http://radiotec.ru/ru/journal/Neurocomputers/number/2023-1/article/23266.

24. Дайхес НА, Виноградов ВВ, Решульский СС, Исаева МЛ, Исаев АВ. Программный модуль для автоматического распознавания, преобразования в текст оценки разборчивости субститутивной аларингеальной речи. Патент RU2023613477, 02.03.2023. Режим доступа: https://www.elibrary.ru/deynke.

25. Daikhes NA, Vinogradov VV, Reshulskii SS, Isaeva ML, Krasavina EA, Fedorova EB. Communication status assessment in patients after total laryngectomy. Otorhinolaryngology. Eastern Europe. 2024;14(2):176–191. (In Russ.) https://doi.org/10.34883/PI.2024.14.2.030.

26. Daikhes NA, Vinogradov VV, Reshulskii SS, Fedorova EB, Isaeva ML. Comparative characteristics of methods for formation of pharyngoesophageal anastomosis and tubeless tracheostomy in laryngectomized patients. Rossiiskaya Otorinolaringologiya. 2023;22(1):18–23. (In Russ.) https://doi.org/10.18692/1810-4800-2023-1-18-23.

27. Longobardi Y, Savoia V, Bussu F, Morra L, Mari G, Nesci DA et al. Integrated rehabilitation after total laryngectomy: a pilot trial study. Support Care Cancer. 2019;27(9):3537–3544. https://doi.org/10.1007/s00520-019-4647-1.

28. Bickford JM, Coveney J, Baker J, Hersh D. Support following total laryngectomy: Exploring the concept from different perspectives. Eur J Cancer Care. 2018;27(3):e12848 https://doi.org/10.1111/ecc.12848.

29. Bua BA, Pendletonc H, Westinb U, Rydellc R. Voice and swallowing after total laryngectomy. Acta Otolaryngol. 2018;138(2):170–174. https://doi.org/10.1080/00016489.2017.1384056.