Medical and social aspects of career guidance for disabled people: assessment of types of work activities using large language models



Cite item

Full Text

Open Access Open Access
Restricted Access Access granted
Restricted Access Subscription or Fee Access

Abstract

BACKGROUND: career guidance for people with disabilities requires comparable and reproducible criteria that make it possible to correlate persistent disorders of body functions with the requirements of specific types of work and factors of the working environment.

AIM: to form and validate a database of estimated indicators of types of work activities as a tool for ranking them in the framework of professional orientation.

METHODS: 435 types of labor activity at the level of primary groups of the All-Russian classifier of occupations were analyzed. For each of them, a profile has been formed based on 21 medical and social indicators, grouped into safety blocks, the ability to perform work functions and working conditions. The primary estimates were obtained using several large language models on a unified ten-point scale; postprocessing was performed for a single interpretation (“the higher, the higher the suitability”) and the calculation of integral values by averaging cross-model estimates. External validation was carried out on the basis of independent expert survey data (87 questionnaires): in each question, five professions were compared on a scale of 0–100; a total of 9135 ratings were processed. The comparison was performed after intramural min-max normalization; ranking consistency (Spearman, Kendall), the proportion of matched pairs, and the normalized rank difference were evaluated.

RESULTS: the share of informative comparison blocks was 92.3%; constant expert assessments were identified in 7.6% of cases. The consistency of the model's ranking with experts was moderate: the average Spearman p=0.416 and Kendall t-b=0.363, the proportion of matched pairs was 0.653, and the average normalized rank difference was 0.278. The highest validity was demonstrated by indicators related to neuromuscular, skeletal, and movement-related (statodynamic) functions (the need to use a wheelchair), while the indicator of emotional stress was characterized by minimal consistency.

CONCLUSION: the formed assessment base is applicable for profiling and pre-selection of types of work in career guidance for the disabled; for contextual factors of working conditions, operational definitions and expert calibration are required.

Full Text

Restricted Access

About the authors

Mikhail V. Ryabtcev

Albrecht Federal Scientific and Educational Center for Medical and Social Expertise and Rehabilitation, Saint Petersburg

Author for correspondence.
Email: ryabtcevmv@yandex.ru
ORCID iD: 0000-0001-5564-6093

Deputy Director of the Institute for Rehabilitation and Habilitation of Persons with Disabilities

Russian Federation, Saint Petersburg, Russia

Gennady N. Ponomarenko

Albrecht Federal Scientific and Educational Center for Medical and Social Expertise and Rehabilitation, Saint Petersburg; Mechnikov North-Western Medical University, Saint Petersburg

Email: ponomarenko_g@mail.ru
ORCID iD: 0000-0001-7853-4473
SPIN-code: 8234-7005

MD, Dr. Sci. (Medicine), Professor, Corresponding Member of the Russian Academy of Sciences

Russian Federation, Saint Petersburg, Russia; Saint Petersburg, Russia

Andrey V. Sokurov

Albrecht Federal Scientific and Educational Center for Medical and Social Expertise and Rehabilitation, Saint Petersburg

Email: ansokurov@yandex.ru
ORCID iD: 0000-0001-5621-0240

MD, Dr. Sci. (Medicine), Associate Professor

Russian Federation, Saint Petersburg, Russia

Viktor V. Mikhaylishin

Albrecht Federal Scientific and Educational Center for Medical and Social Expertise and Rehabilitation, Saint Petersburg

Email: mikhailishin_v@mail.ru
ORCID iD: 0000-0002-9518-1945

MD, Cand. Sci. (Medicine)

Russian Federation, Saint Petersburg, Russia

References

  1. Puzin SN, Shurgaya MA, Memetov SS, et al. Disability in the 21st Century: The State of the Problem of Medical and Social Rehabilitation and Habilitation of Disabled People in Modern Russia. Medical and Social Expertise and Rehabilitation. 2018;21(1–2):10–17. doi: 10.18821/1560-9537-2017-21-1-10-17 EDN: PJHCJH
  2. Goryainova MV, Karasaeva LA, Nurova AA, et al. Indicators of the Need for Disabled People in Professional Rehabilitation Measures. Medical and Social Expertise and Rehabilitation. 2021;24(2):21–27. doi: 10.17816/MSER65083 EDN: AKWVEP
  3. Ananyeva TN, Ilyukhina GI, Sazonova YuV. The system of vocational guidance and assistance in employment of disabled people and persons with limited health opportunities within the framework of the international movement “Abilympics”. Medical and Social Expertise and Rehabilitation. 2021;24(2):5–9. doi: 10.17816/MSER71135 EDN: DSKSZI
  4. Kalashnikov AI, Shamsheva EV, Salikova SI. Professional Rehabilitation of Disabled People in the Omsk Region: Current State and Development Prospects. Physical and Rehabilitation Medicine. 2019;1(4):49–55. doi: 10.26211/2658-4522-2019-1-4-49-55 EDN: YLIACA
  5. Levack WMM, Fadyl JK. Vocational interventions to help adults with long-term health conditions or disabilities gain and maintain paid work: an overview of systematic reviews. BMJ Open. 2021;11(12):e049522. doi: 10.1136/bmjopen-2021-049522
  6. Whitworth A, Baxter S, Cullingworth J, Clowes M. Individual Placement and Support (IPS) beyond severe mental health: an overview review and meta-analysis of evidence around vocational outcomes. Preventive Medicine Reports. 2024;43:102786. doi: 10.1016/j.pmedr.2024.102786
  7. Bond GR, Al-Abdulmunem M, Marbacher J, et al. A systematic review and meta-analysis of IPS supported employment for young adults with mental health conditions. Administration and Policy in Mental Health and Mental Health Services Research. 2023;50(1):160–172. doi: 10.1007/s10488-022-01228-9
  8. Weld-Blundell I, Shields M, Devine A, et al. Vocational interventions to improve employment participation of people with psychosocial disability, autism and/or intellectual disability: a systematic review. International Journal of Environmental Research and Public Health. 2021;18(22):12083. doi: 10.3390/ijerph182212083
  9. Nevala N, Pehkonen I, Teittinen A, et al. The effectiveness of rehabilitation interventions on the employment and functioning of people with intellectual disabilities: a systematic review. Journal of Occupational Rehabilitation. 2019;29:773–802. doi: 10.1007/s10926-019-09837-2
  10. Ponomarenko GN, editor. Rehabilitation of the disabled: a national guide. Moscow: GEOTAR-Media; 2018. 736 p. (In Russ.)
  11. De Baets S, Calders P, Schalley N, et al. Updating the evidence on functional capacity evaluation methods: a systematic review. Journal of Occupational Rehabilitation. 2018;28(3):418–428. doi: 10.1007/s10926-017-9734-x
  12. Safikhani P, Avetisyan H, Föste-Eggers D, Broneske D. Automated occupation coding with hierarchical features: a data-centric approach to classification with pre-trained language models. Discover Artificial Intelligence. 2023;3(1):6. doi: 10.1007/s44163-023-00050-y
  13. Clavié B, Soulié G. LLM4Jobs: unsupervised occupation extraction and standardization leveraging large language models. Knowledge-Based Systems. 2025;321:113526. doi: 10.1016/j.knosys.2025.113526
  14. Bethmann A, Schierholz M, Wenzig K, Zielonka M. Machine Learning for Occupation Coding — A Comparison Study. Journal of Survey Statistics and Methodology. 2021;9(5):1013–1034. doi: 10.1093/jssam/smaa033
  15. Tyao T, Yamazaki N, Otani M, et al. Application of large language models (LLM) for automatic classification of work accident text data: verification of accuracy and practicality. medRxiv. 2025. doi: 10.1101/2025.10.02.25337141
  16. Barth J, de Boer WEL, Busse JW, et al. Inter-rater agreement in evaluation of disability: systematic review of reproducibility studies. BMJ. 2017;356:j14. doi: 10.1136/bmj.j14
  17. Spanjer J, Groothoff JW, Brouwer S. Assessing work ability — a cross-sectional study of interrater agreement between disability claimants, treating physicians, and medical experts. Scandinavian Journal of Work, Environment & Health. 2014;40(5):493–501. doi: 10.5271/sjweh.3438
  18. Durand M-J, Loisel P, Poitras S, et al. The interrater reliability of a functional capacity evaluation: the Physical Work Performance Evaluation. Journal of Occupational Rehabilitation. 2004;14(2):119–129. doi: 10.1023/B:JOOR.0000018328.35521.e8
  19. Mecham J, Wargo J, Matheson LN. An evaluation of the inter-rater and intra-rater reliability of OccuPro's functional capacity evaluation. Work. 2018;60(3):465–473. doi: 10.3233/WOR-182754
  20. Matheson LN. The functional capacity evaluation. In: Demeter SL, Andersson GBJ, editors. Disability Evaluation. St. Louis: Mosby; 2003: 746–762.
  21. Escorpizo R, Reneman MF, Ekholm J, et al. A conceptual definition of vocational rehabilitation based on the ICF: building a shared global model. Journal of Occupational Rehabilitation. 2011;21(2):126–133. doi: 10.1007/s10926-011-9292-6
  22. Finger ME, Glässel A, Erhart P, et al. Identification of relevant ICF categories in vocational rehabilitation: a cross sectional study evaluating the clinical perspective. Journal of Occupational Rehabilitation. 2011;21(2):156–166. doi: 10.1007/s10926-011-9294-4
  23. Rigó M, Dragano N, Wahrendorf M, Siegrist J, Lunau T. Work stress on rise? Comparative analysis of trends in work stressors using the European working conditions survey. International Archives of Occupational and Environmental Health. 2021;94(3):459–474. doi: 10.1007/s00420-020-01593-8
  24. Giorgi G, Lecca LI, Ariza-Montes A, et al. The importance of contextualized psychosocial risk indicators in workplace stress assessment: evidence from the healthcare sector. International Journal of Environmental Research and Public Health. 2021;18(6):3263. doi: 10.3390/ijerph18063263
  25. Cendales-Ayala B, Useche SA, Gómez-Ortiz V, Bocarejo JP. Psychosocial work factors, job stress and strain at the wheel: validation of the Copenhagen Psychosocial Questionnaire (COPSOQ) in professional drivers. Frontiers in Psychology. 2019;10:1531. doi: 10.3389/fpsyg.2019.01531
  26. Lunau T, Siegrist J, Dragano N, Wahrendorf M. The association between education and work stress: does the policy context matter? PLoS ONE. 2015;10(3):e0121573. doi: 10.1371/journal.pone.0121573
  27. Negri L, Spoladore D, Fossati M, et al. Proposal for an ICF-based methodology to foster the return to work of persons with disability. Work. 2023;74(2):649–662. doi: 10.3233/WOR-211226
  28. Saltychev M, Kinnunen A, Laimi K. Vocational rehabilitation evaluation and the International Classification of Functioning, Disability, and Health (ICF). Journal of Occupational Rehabilitation. 2013;23(1):106–114. doi: 10.1007/s10926-012-9385-x
  29. Kottner J, Audigé L, Brorson S, et al. Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. Journal of Clinical Epidemiology. 2011;64(1):96–106. doi: 10.1016/j.jclinepi.2010.03.002
  30. Hager P, Jungmann F, Holland R, et al. Evaluation and mitigation of the limitations of large language models in clinical decision-making. Nature Medicine. 2024;30:2613–2622. doi: 10.1038/s41591-024-03097-1
  31. Benkirane K, Kay J, Perez-Ortiz M. How can we diagnose and treat bias in large language models for clinical decision-making? In: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024: 12345–12360. doi: 10.48550/arXiv.2410.16574
  32. Chen IY, Hoffmann J, Dinan E. The evaluation illusion of large language models in medicine. NPJ Digital Medicine. 2025;8(1):512. doi: 10.1038/s41746-025-01963-x

Supplementary files

Supplementary Files
Action
1. JATS XML
Download

Copyright (c) Eco-Vector

License URL: https://eco-vector.com/for_authors.php#07
СМИ зарегистрировано Федеральной службой по надзору в сфере связи, информационных технологий и массовых коммуникаций (Роскомнадзор).
Регистрационный номер и дата принятия решения о регистрации СМИ: серия ПИ № ФС 77 - 86505 от 11.12.2023 г. 
СМИ зарегистрировано Федеральной службой по надзору в сфере связи, информационных технологий и массовых коммуникаций (Роскомнадзор).
Регистрационный номер и дата принятия решения о регистрации СМИ: серия ЭЛ № ФС 77 - 80654 от 15.03.2021 г.