Author:
Connell Charlotte J. W.,Salkeld Alexander J,Wells Cameron,Verstappen Antonia C.,Poole Phillippa,Wilkinson Tim J,Bagg Warwick
Abstract
Abstract
Background
The Medical Schools Outcomes Database and Longitudinal Tracking Project (MSOD) in New Zealand is one example of a national survey-based resource of medical student experiences and career outcomes. Longitudinal studies of medical students are valuable for evaluating the outcomes of medical programs against workforce objectives. As a prospective longitudinal multiple-cohort study, survey response rates at each collection point of MSOD vary. This paper assesses the effects of participant non-response rates on MSOD data.
Methods
Demographic variables of MSOD respondents between 2012 and 2018 were compared to the distribution of the demographic variables in the population of all NZ medical graduates to ascertain whether respondent samples at multiple survey collection points were representative of the population. Analysis using logistic regression assessed the impact of participant non-response on variables at collection points throughout MSOD.
Results
2874 out of a total population of 2939 domestic medical students graduating between 2012 and 2018 responded to MSOD surveys. Entry and exit surveys achieved response rates around 80% and were broadly representative of the total population on demographic variables. Post-graduation survey response rates were around 50% of the total population of graduates and underrepresented graduates from the University of Auckland. Between the entry and exit and the exit and postgraduation year three samples, there was a significant impact of non-response on ascribed variables, including age at graduation, university, gender and ethnic identity. Between the exit and postgraduation year one sample, non-response significantly impacted ascribed and non-ascribed variables, including future practice intentions.
Conclusion
Samples collected from MSOD at entry and exit are representative, and findings from cross-sectional studies using these datasets are likely generalisable to the wider population of NZ medical graduates. Samples collected one and three years post-graduation are less representative. Researchers should be aware of this bias when utilizing these data. When using MSOD data in a longitudinal manner, e.g. comparing the change in career intentions from one collection point to the next, researchers should appropriately control for bias due to non-response between collection points. This study highlights the value of longitudinal career-tracking studies for answering questions relevant to medical education and workforce development.
Publisher
Springer Science and Business Media LLC
Subject
Education,General Medicine
Reference34 articles.
1. Poole P, Wilkinson TJ, Bagg W, Freegard J, Hyland F, Jo E et al. Developing New Zealand’s medical workforce: realising the potential of longitudinal career tracking. NZMJ [Internet]. 2019 [cited 2023 May 24];132:1495. Available from: www.nzma.org.nz/journal.
2. Morton SMB, Bandara DK, Robinson EM, Atatoa Carr PE. The 21st Century, what is an acceptable response rate? Aust N Z J Public Health. 2012 Apr;36(2):106–8.
3. Draugalis JR, Coons SJ, Plaza CM. Best practices for survey research reports: a synopsis for authors and reviewers. Am J Pharm Educ. 2008;72(1).
4. Taylor BV, Palmer A, Simpson S, Lucas R, Simmons RD, Mason D, et al. Assessing possible selection bias in a national voluntary MS longitudinal study in Australia. Multiple Scler J. 2013;19(12):1627–31.
5. Young AF, Powers JR, Bell SL. Attrition in longitudinal studies: who do you lose? Aust N Z J Public Health 2006 Aug 1;30(4):353–61.