Practical validation framework for competence measurement in VET: a validation study of an instrument for measuring basic commercial knowledge and skills in Switzerland

Empirical Research in Vocational Education and Training

Table. 3 Methodical approach and sources of validity evidence for simuLINCA

Methodical approach	Sample	Data	Sources of validity evidence
Methodical approach	Sample	Data	A) Test Content	B) Response Processes	C) Internal Structure	D) Other Variables	E) Con-sequences
I. Document analysis		Tasks and items allocated to contents of school and branch curricula	√
II. Pilot Study I:Interviews with apprentices in winter 2013	A^a	20 interviews at workplace (think alouds, ≈ 150 min), recorded on tape, no incentive 20 questionnaires	√	√
III. Pilot Study II: test application in winter 2013	B^b	Test data of 102 apprentices Qualitative feedback (open format)	√		√
IV. Main study in spring 2015	C^c	Test data of 1365 apprentices
A) SimuLINCA					√
B) Frequent tasks*			√
C) Context (CAT, marks)*					√	√
D) Feedback*							√

* Was assessed/collected during the study of the longitudinal project LINCA (Holtsch and Eberle 2018)
^aSample A: N = 20 apprentices, final year of training, employed by companies that are part of the ten largest branches. Age: 18.64 (SD = 1.74). Sex: 60% female. 70% E-Profile
^bSample B: N = 102 apprentices, 4 vocational schools, 6 classes, half a year before the end of the training. Sex: 61.8% female. 75.5% E-profile
^cSample C: N = 1,365 apprentices, 82 randomly drawn commercial classes. Age: 19.1 years (SD = 1.5). Sex: 63.2% female. 52.5% E-profile