Cognitive skills, personality traits and dropout in Dutch vocational education

Abstract

Designing effective educational programs to reduce dropout in higher and vocational education requires thorough understanding of the underlying mechanisms of study success. This study examines if first-year dropout is associated with cognitive ability and personality traits as measured by a formative entry test just before enrollment in Dutch vocational education. The results show that the formative entry test outcomes do not properly identify students at risk, the association between first-year dropout and, respectively, cognitive skills and personality traits is generally not significant. Consequently, and of importance for both students and vocational education institutes, students cannot be properly informed about whether their skills and personality traits match the required skills and personality traits of the program.

Introduction

Even though there is strong evidence that schooling outcomes (e.g. dropout, degree, grades) are associated with cognitive skills and personality traits, these relationships have hardly been examined for vocational education (Cawley et al. 2001; Heckman et al. 2006; Richardson and Norgate 2015; and references therein). This is somewhat surprising given that Dutch vocational training programs tend to have high drop-out rates and low study success (Dutch Inspection of Education 2015). Moreover, Eichhorst et al. (2015) showed in their review that low-ability youths and those working in low-skill jobs particularly benefit from vocational study completion, in that graduation fosters the transition to a more profitable employment and wage and employment outcomes in general.

Many studies have looked at the main drivers of study success. Heckman and Kautz (2012) mention that dropout and study success are simultaneously influenced by cognitive and personality drivers and Borghans et al. (2006) indicate that the relationship between personality traits and cognitive skills can exist for two reasons. First, a positive attitude towards work and performance might cause people to do their best at tests (irrespective of the rewards offered) and therefore might partly makes that cognitive skill outcomes are upward biased. Second, people might have an attitude to put effort in a task only when there are sufficient rewards but also have favorable behavioral or labor-market outcomes. This could serve as an explanation for people with a lower cognitive test score but still a successful career.

This study contributes to the current empirical literature through a better understanding of the drivers of first-year drop out in vocational education using entry-test data that provides an accurate reflection of the cognitive skills and personality traits of students when enrolling in the vocational training program. The simultaneous measure of personality traits and cognitive ability allows us to examine if and to what extent cognition and personality are substitutes or separate-sphere drivers of first-year dropout. Moreover, the focus on vocational education is interesting as there is much more cognitive and personality diversity between the vocational training programs as there is between higher education programs. Vocational programs rely on cognition to different degrees (identified in the Netherlands through four cognitive levels), and also rely to different degrees on personality skills. We therefore also investigated whether cognitive skills and personality traits drive dropout to different degrees at different levels and different kinds of vocational education.

Tests that measure cognition have been developed and refined over the past century. Many psychologists use IQ, achievement tests, and grades (grade-point average, GPA) to measure “cognitive ability” or “intelligence,” and this practice is also wide- spread in economics (Heckman and Kautz 2012). Often grades are used to describe academic performance and cognitive ability [e.g. (Mitrofana and Iona 2013)]. Research has generally supported the value of prior grades as predictors of study success, however, grades do not give insight in the underlying drivers that explain why a student is successful. Also several studies [e.g. (Noonan et al. 2005)] stated that the non-cognitive skills or personality traits should be taken into account.

Indeed, personality traits (often included under the broader banner non-cognitive skills) have strong effects on educational attainment and predict later-life outcomes with the same, or greater, strength as measures of cognition (Kautz et al. 2014). Several researchers found that personality (based on the Big Five) can be an important predictor of study success in higher education (Busato et al. 1999; Feltzer and Rickli 2009; Hakimi et al. 2011). The empirical literature indicated that conscientiousness and openness are correlated with future study success (Furnham et al. 2003), while agreeableness and neuroticism are not (Busato et al. 1999; O’Connor and Paunonen 2007). Extraversion is a factor that shows both positive and negative correlations with study success. Several variables such as age, level of education, family background and type of assessment seem to influence the nature of this correlation [e.g. (Furnham et al. 2003; Lundberg 2013; Poropat 2009)].

Even though previous studies have focused either on the relationship between study success and cognitive ability (Kappe 2011; Melse et al. 2012) or on the relationship between study success and personality traits (Feltzer and Rickli 2009; Furnham et al. 2003; Hakimi et al. 2011; Heckman et al. 2006), few studies have examined how study success is influenced by both factors (Kautz et al. 2014). To the best of our knowledge, there has even been no study that related study success to measures of personality traits and cognitive ability of which the data was collected just before educational enrollment.

Since 2012, Dutch vocational institutes organize an intake procedure, during which students take a formative entry test in which their cognitive skills are measured, together with the Big Five personality traits (AMN 2017). The formative entry test is referred to as the AMN test, which stands for Assess, Manage and Navigate. This points to the formative and guiding character it is supposed to have once students are enrolled in the educational program. The objective of the formative entry test was, first of all, to give students information about the match between the own and required skills of the vocational program and about what skills should be further developed to increase study success once being enrolled in the program. Secondly, this entry test gives educational institutes insights in the current cognitive and personality state at the moment of enrollment such that students at risk can be identified at an early stage and study counselling and guidance can be more effectively targeted.

The AMN test is administered to incoming students at about 60% of all Dutch vocational institutes. Important for this study is that it was not mandatory for vocational institutes to include this test in the intake procedure, but if institutes decided to do so students were required to take the test. Here, we use data from this entry test to study how first-year dropout in Dutch vocational education is related to cognitive skills and personality traits. Educational institutes are not allowed to select students based on this test. Students are allowed to subscribe to several educational programs and may eventually choose one program they want to follow. This free choice can nevertheless be influenced by the test scores, but sufficient variance remains to answer our research questions.

This paper proceeds as follows. “Dutch educational system and the formative entry test” section describes the Dutch education system, the formative entry test and empirical results on association between cognitive skills, personality traits and future study success. “Data and descriptive statistics” section describes the data and descriptive statistics. “Methods and results” and “Conclusion” sections present and discusses the empirical analysis and findings.

Dutch educational system and the formative entry test

The Dutch educational system

The Dutch school system (graphically illustrated in Fig. 1) tracks children into secondary education levels. In the final grade of primary education (grade six) children make a nationwide standardized test and are then tracked into three secondary education levels based upon this test and a primary school advice. About 50% of children are tracked into two pre-university or upper secondary general education, while the other 50% are assigned to the pre-vocational track (Dutch Ministry of Education Culture and Science 2013). Pre-vocational education prepares children for upper secondary education vocational education (vocational education in this paper). Vocational education is organized in large institutions and the educational programs educate students for specific professions. These programs are typically developed together with the industry in which the graduates will work after completing the program.

Fig. 1

(Figure is taken from Cornelisz and Van Klaveren (2017))

The Dutch education system. The international standard classification of education (ISCED) level and the Dutch acronym for the school level is printed in between brackets

Vocational education has four cognitive levels, which match with the International Standard Classification of Education (ISCED) levels two, three and four. Level 2 programs are comparable with the first stage of secondary education that builds on primary education, and typically represents a more subject-oriented curriculum. Level 3 programs is comparable with the second/final stage of secondary education and provide skills that are directly relevant to employment, usually with an increased range of subject options. Level 4 programs provides learning experiences that build on secondary education and prepare for labour market entry and/or tertiary education; the content is broader than secondary but not as complex as tertiary education.

Figure 2 shows the percentage of student dropout in the different educational tracks. The drop out is higher in higher professional education than in vocational education. Note that failure to gain qualifications at the vocational level 2 usually means that no qualification at all is attained, which can lead to a vicious cycle of unemployment, social exclusion and poverty (Cedefop 2016), whereas dropouts in other forms of higher education can usually rely on other qualifications to shield them from poverty (Dutch Ministry of Education Culture and Science 2013).

Fig. 2

The percentage of Dutch student dropout after 4 years, specified in educational levels. Numbers are retrieved from ‘ocwincijfers’, a website of the Dutch Ministry of Education, Culture and Science

Compulsory education, degree obligation and dropout in vocational education

The Dutch compulsory schooling law requires children to go to school until the age of 18 or until pre-university, upper secondary general education or upper secondary vocational education is successfully finished (i.e. if a degree is attained). For children enrolled in pre-university of upper secondary general education the law simply enforces that children are in education until they are 18 years old and in general these children manage to successfully finish the educational program. However, the situation is dramatically different for vocational education. Students who successfully finish pre-vocational education are, on average, 16 years old and obliged by law to continue with a vocational training program even if they do not prefer to do so. As a result of this law many children enrol in a vocational training program (level 2, 3 or 4) after successfully finishing pre-vocational education, but might drop out legitimately without a degree after 1 or 2 years when they become 18 years old. This context partly explains, at least for the Netherlands, why dropout in vocational education is relatively high.

The formative entry test AMN

Since 2012, approximately 60% of the Dutch vocational institutes require student applicants to take the formative AMN entry test during the intake procedure. The test measures a variety of cognitive skills and personality traits and the outcomes are used during an intake interview to evaluate with students the match between their own skills and required skills. For students this evaluation can be useful as it gives insights in the skills that should be further developed to be more successful once enrolled in the education program. For the institute the evaluation is useful because students who are potentially at risk (i.e. required and possessed skills differ much) can be identified at an early stage, and study counselling and guidance can be more effectively targeted. Institutes are not allowed to select students based on this test. Students are allowed to subscribe to several educational programs and institutes and may eventually choose one program they want to follow.

The process from application to enrollment is as follows. Students apply for a particular vocational program before May 1st and applicants are invited to participate in an intake procedure in the beginning of June. Importantly, applicants receive the results of the secondary school exit exams, which determines whether they are eligible to enroll in the vocational program, after the intake procedure. Conditionally on the outcomes of the exit exam and the intake procedure, students decide whether or not to enroll in the program they have applied for.

Psycho-diagnostic instruments issued in the Netherlands, like the formative entry test, are assessed by the independent Dutch Committee on Tests and Testing (COTAN, part of the National Institute of Psychologists). COTAN audits the quality of psychological tests with the objective to raise standards in the use of such tests. The formative entry test of AMN was validated by COTAN. Table 1 shows that it measures various cognitive skills and the Big Five personality traits.

Table 1 Measured cognitive skills and personality traits, with for the cognitive skills the subtest that is used to measure each skill

The AMN cognitive skill measures are based on the Cattell–Horn–Carroll model of intelligence (Alfonso et al. 2005). This model makes a distinction between fluid and crystallized intelligence. Fluid intelligence refers to logical flexibility (called logical reasoning in AMN) and involves the ability to reason and to visualize (spatial) information. Crystallized intelligence (called learning intelligence in AMN) refers to what has been learned in the course of life and involves the knowledge gained and the understanding of the world. In total 157 items measure the different cognitive skills on a 10-points scale (the higher the score, the higher the cognitive skills). There is a separate test for level 1–2 students and for level 3–4 students. Scores are compared to a norm group of peers to distinguish the level of the student in their age group.

Personality traits are measured using the Big Five taxonomy (John and Srivastava 1999; McCrae and Costa 2008). These five personality factors are extraversion (talkative, assertive, energetic), agreeableness (good-natured, cooperative, trustful), conscientiousness (orderly, responsible, dependable), emotional stability versus neuroticism (calm, not neurotic, not easily upset) and openness to experience (intellectual, imaginative, independent-minded). Each factor of the Big Five is measured by asking applicants whether 92 statements reflecting the various factors apply to them. This is measured on a 5-point scale where 1 means not applicable at all and 5 means very applicable. For the AMN, the Big Five dimensions are defined in such way that a “positive personality” (open, conscientious, extravert, stable and agreeable) conducive to success yields positive numbers (this entails a rescoring of the typical Big Five neuroticism trait to a positive stability trait).

Data and descriptive statistics

Data and participants

This study uses information of 14.875 students who entered or registered in a vocational program offered by ROC TOP between 2012 until 2016. ROC TOP is a medium-sized regional vocational institute that provides sixty vocational education programs at all levels of vocational education. Not all entering students in vocational education took the formative entry test, which can be caused by differences between vocational programs and because of late registration of students. For the students sample considered we observe the following three situations: (1) a formative entry test outcome but no program enrollment, (2) program enrollment but not formative entry test outcome, and (3) both a formative entry tests outcome and program enrollment. The main focus in the analysis will be on the student group for which we observe both enrollment and the formative entry tests outcome, but differences between the three groups will be outlines to acknowledge the potential selective nature of the sample of analysis.

Descriptives

Table 2 shows that 24.1% of the students only took the formative entry test and did not enrol in a vocational education program. This can either be explained by the fact that students chose a different institution than they applied for or because it was impossible to match the entry test results to the student registration data. To examine potential selective enrollment in terms of the student’s cognitive skills and personality traits, a probit analysis was conducted. The estimation results are shown in Table 3 and show only marginal differences for students who enrolled in a level 1–2 program, indicating that for these program levels there appears to be no selective enrolment. For level 3–4 programs enrollment is related to cognitive skills and extraversion, indicating that there is selective program enrollment in these characteristics. In the estimation models include a control variables which capture potential selection based on cognitive skill and personality trait variables (i.e. the inverse-mills ratio).

Table 2 Number of students for whom only a formative entry test result was available, for whom only data was available about their enrollment in a vocational program of ROC TOP, and for whom both were available

Table 3 Probit regression estimates for enrolment in a vocational program

About 40% of the students enrolled in a vocational education program without taking the formative entry test. This group contained students who enrolled in programs or in years in which the test was not offered or, as stated above, the entry test data could not be matched to the student registration data. Finally, we observe for 36% of the students both enrollment in vocational education (and consequently whether they drop out of the program) and the outcome of the formative entry test. For the 11.283 students who started a vocational program at ROC TOP in the observation period, 2344 students were still active in the educational program, 4223 students dropped out of the program and 4717 students received a diploma. Of the students who dropped out of the program, allmost than 80% did so in their first year. Since later dropout is a more ambiguous outcome in the sense that it may reflect success instead of failure (e.g., students may be employed and decide that they do not need a formal closure of their program), we decided to take first-year dropout as the dependent measure of the analyses.

Table 4 shows that the student population of ROC TOP differs from the student population of other vocational schools in the Netherlands. This is important with respect to the generalization of the empirical results of the study at hand. On average, the student population of ROC TOP has more students that come from poverty problem accumulation areas. These areas are defined by Statistics Netherlands, give an indication of the income of the area and derived from their annual regional income survey. Also a higher percentage of the students of ROC TOP have dropped out before and a lower percentage of students have a higher qualification than the minimally required qualification to start with a program.

Table 4 Student population of ROC TOP, another vocational school in Amsterdam and the national Dutch average

As explained in “The Dutch educational system” section, Dutch law requires that children go to school until the age of 18 (compulsory schooling law) or until pre-university, upper secondary general education or upper secondary vocational education is successfully finished (i.e. if a degree is attained). Dropout may thus increase after reaching the age of 18. This could mean that drop out might be influenced by age and it has to be concluded as a control variable in the analyses. When plotting the proportion of dropout in the first year as a function of age (Fig. 3), it becomes apparent that the proportion of drop out in the first year increases when a student turns 18 (cut-off point).

Fig. 3

The proportion of dropout in the first year as a function of student age. This Figure shows that before the age of 18 (before the cut-off point) the proportion of drop out is lower than after the student turned 18

Methods and results

Methods

To examine the extent to which cognitive skills and personality traits can be explained by common underlying factors we first performed a principal component analysis. If cognitive skills and personality traits are independent and separate dimensions then it is expected that there are two underlying factors. Based on the outcomes of this principal component analysis variables are constructed that capture the joint variation in cognitive skills and personality traits. We note that these factors are generated by computing weighted factor scores, such that the generated variables are more efficient compared to the situation when we would simply take the average of certain constructs.

The association between first year dropout and the identified common factors underlying cognitive skills and personality traits (i.e. the constructed variables from the factor analysis) are estimated using the following model:

$$y_{i} = \beta_{0} + \beta_{1} D_{ki} + \delta^{\prime}x_{i} + \varepsilon_{i} .$$

(1)

In this model the first-year dropout indicator of student i is represented by y_i, and D_k represents the k identified common factors underlying cognitive skills and personality traits. Matrix x_i represent the inverse Mills ratio to take account of a possible selection bias (Heckman 1979; Mills 1926), a set of student and control variables (including a program fixed effect) and ɛ_i represent the usual error term for which we assume that ɛ_i ~ iid(0, σ²).

By estimating Model (1) separately for the two lower and the two higher cognitive vocational tracks we can examine how differences in required cognitive skill-levels influence the estimated association. In other words, does the association between first-year dropout and cognitive skills depends on required cognitive skills?

In a similar fashion, we also examine how the estimated association between first-year dropout and common factors underlying personality traits depends on required personality traits by the educational program. A difficulty is that we do not observe required personality traits by educational program, which is why we asked six experts in vocational education to rank all educational programs offered by ROC TOP in accordance with the need of personality traits in the future occupation of the student (see Appendix B). The inter-rater reliability coefficient between the six participants was determined by estimating a two-way random-effects and the estimated coefficient turned out to be extremely high with 0.92. In order to take into account how expert opinions of required skills influence the estimated association between dropout and the identified common factors underlying cognitive skills and personality traits we estimate the following models separately for lower and higher vocational tracks:

$$y_{i} = \beta_{0} + \left( {\gamma_{0} + \gamma_{1} R_{0 - 25} + \gamma_{2} R_{25 - 50} + \gamma_{3} R_{50 - 75} } \right) \cdot D_{PT,i} + \gamma_{4} D_{CS,i} + \varepsilon_{i} .$$

(2)

In Model (2) indicator variables R refer to the personality-trait ranking of the educational program and subscript 0–25 refers to the first quartile, 25–50 to the second quartile and 50–75 to the third quartile. Variable D_PT (D_CS) represents to the underlying common factors that can be associated with personality traits (cognitive skills). The interpretation of the gamma parameters are of huge importance in this model. Parameter γ₀ represents the association between y and D_PT for educational programs that are ranked by the experts as educational programs which required students with the highest personality trait-levels. The other gamma-parameters indicate how differently ranked studies deviate from this baseline effect. By this we mean that the association between y and D_PT for studies ranked in the lowest quartile is estimated to be γ₀ + γ₁. If the parameter estimate γ₁ turns out to be negative, it implies that the coefficient for the R_0–25-studies is lower than the estimated coefficient for R_75–100-studies (with γ₁).

Results

Principal component analysis

A principal component analysis was conducted (with varimax rotation) to examine the extent to which there is simultaneous variation in cognitive skills and personality traits. Table 5 shows the outcomes of the principal component analysis; the two columns indicate that 48% observed variation in cognitive skills and personality traits can be explained by the two factors with an eigenvalue above the cutoff of 1.0 (χ²(55) = 1.1 * 10⁴ p <0.001; Cronbach’s alpha personality traits 0.75, Cronbach’s alpha cognitive skills 0.74). Factor 1 explains underlying variation in personality traits and Factor 2 explains variation in cognitive skills.

Table 5 Estimation results of factor analysis

Table 6 additionally shows the correlation matrix, such that it becomes more apparent how the various measured skills and traits are interrelated. The correlation matrix confirms that the significant correlations cluster in two groups, which is of course what the principal component analysis indicates.

Table 6 Correlation matrix

Importantly, the high correlation between the cognitive skill variables on the one hand and the personality trait variables on the other indicate that causal interpretation of the individual cognitive skills and personality traits in relation to first-year dropout is not possible. If we would find, for example a significant relationship between 3D-components and first-year dropout then this association also captures the effect of differences. As a result we would for example not be able to state that first-year dropout can be improved by structurally improving 3D-components as this may not be the causal relationship observed. However, based on Tables 5 and 6, we can conclude that Personality Traits and Cognitive Skills describe rather independent variation. Because the literature uses a categorization in which the two cognitive domains (crystallized and fluid intelligence) and the Big Five personality traits are considered separately (Rammstedt et al. 2018) we follow this categorization. By examining the relationship between first-year dropout and these separately considered cognitive and personality constructs we can examine how personality traits and cognitive skills generally influence are associated with first-year dropout.

Regression estimates

Tables 7 and 8 show the estimation results separately for level 1–2 and level 3–4 studies. We start departure from a model with only the cognitive and personality factors and a constant included in the model (the baseline model) and then include first the inverse mills ratio to control for selective enrollment, secondly student characteristics, and finally vocational program fixed effects. The student characteristics included in the model are age and gender. The chosen stepwise approach shows how the estimation parameters of interest change when controlling for these background characteristics, which gives information on whether the identified factors are independent from these characteristics and increases precision. We note that all factor variables are standardized such that the coefficients represent association measured in standard deviations.

Table 7 Baseline estimation results future dropout in the first year for level 1–2 vocational programs

Table 8 Baseline estimation results future dropout in the first year for level 3–4 vocational programs

The results are shown separately for low and high vocational levels, as these vocational levels have distinct formative entry tests. Because of this separate representation these baseline results immediately are informative for whether required cognitive skills influence the estimated association between dropout and the identified common factors. We expected for students enrolled in a level 1–2 program that cognitive skills and personality traits are relatively less important because students are trained for action-oriented occupations (such as supporting administrative professions or private security). Level three and four programs are more abstract and therefore cognitive skills and personality supposedly more important.

Table 7 shows for level 1–2 vocational studies that the constructs stability, openness and agreeableness are related to first-year dropout. The crystallized intelligence construct is also negatively and significantly related to dropout, but the estimated coefficient is small. The coefficient of the inverse Mills ratio variable, obtained from the probit equation, is statistically significant thereby supporting the relationship between dropout and selective enrollment. These estimation results reveal that first-year dropout for studies which require relatively low cognitive skills is not associated with the observed variation in personality traits and cognitive skills.

Table 8 shows for level 3–4 vocational studies that cognitive skills are not related to first-year dropout, but two personality traits are. The negative sign of the conscientiousness estimate suggest that higher levels of conscientiousness is associated with lower dropout, which is consistent with the currently existing empirical evidence. Because the Big Five constructs are highly correlated we cannot make any causal claims on the effects of conscientiousness on first-year dropout.

Personality traits in different vocational programs

The estimation results of Eq. 2 in “Methods” are shown in Table 9 and indicate whether vocational studies that require more personality traits according to the six experts show stronger associations between first-year dropout and personality trait levels. We note, however, that the results should be treated with caution, as experts opinions are not a valid measure of required personality traits. The inter-rater reliability between these six participants was determined using a two-way random-effects intra-class correlation coefficient (ICC). The average ICC of 0.92 was highly reliable with a 95% confidence of [0.86, 0.96] (F(30, 150) = 12.07, p < 0.001). The ranking is shown in Appendix B. We note that personality trait-measures can be compared between students of all different vocational program levels. Therefore we estimate Model 2 not only separately for the different cognitive levels, but also estimate it for a pooled data in order to reduce standard errors.

Table 9 Regression estimations future dropout in the first year based on personality traits ranking

Personality in this matter is a continuum and we labelled (as AMN did) one end of the continuum as positive and summed scores. A higher score does not mean more personality, but another personality. The pooled estimation results (which are most precise) indicate for study programs that require the most developed personality traits (the reference PT programs) that students whose personality traits are conductive to success tend to dropout relatively more often. The programs that fall within this category are for example nursing and social work.

The estimated moderation effects (or interaction effects) tend to be positive, indicating that specifically those programs in which personality traits are less required students with relatively higher personality trait levels do dropout of the educational program.

Conclusion

This study empirically examined if, and to what extent, first-year dropout is associated with cognitive ability and personality traits as measured by a formative entry test just before enrollment in vocational education. The timing of the formative entry test ensures that the test outcomes gives an accurate reflection of the cognitive and personality state of the student when enrolling in the educational program, while a simultaneous consideration of personality traits and cognitive skills allows us to examine if and to what extent cognition and personality are separate-sphere drivers, in the sense that they both explain similar variation in first-year dropout. Until now, there has not been an empirical study on the potential drivers related to first-year dropout. Secondly, this study contributes by focusing on vocational education, because the examined relationship tends to be ignored for vocational education while a substantial amount of students enroll in vocational education. Thirdly, the focus on vocational education is interesting because of the diverse cognitive skills and personality traits within and between vocational tracks.

A principal component analysis suggests that personality traits and cognitive skills are independent factors. The cognitive skill constructs were highly correlated (as was already found over a century ago; Spearman 1904), as were the personality traits. The results indicate that the constructs stability, openness and agreeableness are related to first-year dropout at vocational level 1–2, and that conscientiousness and openness are related to first-year dropout at vocational level 3–4. However, it is not possible to pinpoint specifically which cognitive skill or personality trait would cause higher first-year dropout or improved study success. This means that it is not advisable to investigate the predictive value of individual cognitive skills (e.g. working memory) or individual character traits (e.g. conscientiousness) in isolation without measuring other constructs that are correlated with it.

Even with that caveat there was no robust evidence that points to a structural and significant association between the formative entry test outcomes and first-year dropout in vocational education. The inclusion of interaction effects, furthermore, did not support the claim of Heckman and Kautz (2012) that dropout is simultaneously influenced by cognitive skills and personality traits and that one skill/trait may reinforce the effect of another skill/trait. This indicates that the validated formative entry test is not informative with respect to how first-year dropout can be reduced.

Three possible explanations can be given for these findings. Firstly, it is possible that the entry test is not a good instrument to measure cognitive skills and personality traits that are predictive of dropout. We consider this to be an invalid argument, because the AMN is a certified and validated instrument that bears close resemblance to other cognitive and personality inventories. The cognitive subtests of the AMN closely resemble those commonly used in intelligence test batteries such as the WAIS (Wechsler 2009), while the personality questions are typical of those used in other big five-based personality inventories.

A second possible explanation is that cognitive skills and personality traits are less important for dropout in vocational education than for general secondary and higher education. This could be related to the focus on vocations that require profession-oriented actions rather than general cognitive skills or specific personality traits. Obviously, more research is needed to validate this general conclusion.

A third explanation departures from the idea that dropout may not be a purely negative choice for many students in vocational education, reflecting failure. Students may, for example, leave education for a better alternative, notably paid employment or another possibly better fitting program. The observation that dropout is higher for students older than eighteen, for whom there is no mandatory schooling, might be substantial—i.e., the students for whom dropping out does not mean starting in a different program, but can mean dropping out of education altogether.

We note that none of these explanations excludes the possibility that test outcomes are informative with respect to other student achievement measures (i.e. study duration or variation in grades). Potentially, for example, information on personality traits and cognitive skills can help to provide students with more optimal guidance. Also, ROC TOP as a vocational institute attracts relatively poor performing students. This may result in an outflow of students who possess the appropriate level of personality traits to be successful in the program towards institutes with a higher proportion of fellow students who also possess the appropriate level of personality traits. More research is needed to investigate this explanation.

As a final note, we would like to reiterate that the formative entry test outcomes are used during an intake interview with the objective to evaluate with students the match between their own skills and required skills. For students the outcomes should give information on what skills and (if possible) personality traits should be improved in order to be successful in the educational program. For the study program the outcomes should identify students who are potentially at risk (i.e. required and possessed skills differ much) at an early stage such that study counselling and guidance can be more effectively targeted. The empirical results show that the formative entry test outcomes do not accurately identify students at risk, because the association between first-year dropout and, respectively, cognitive skills and personality traits is generally not significant. Consequently, and of huge importance for both students and vocational education institutes, students cannot be properly informed about whether their skills and personality traits match the required skills and personality traits of the program.

Appendices

Appendix A

Table 10 importantly shows that not all students who are enrolling in a certain educational program take the formative entry test. This could potentially bias the results since we are considering a selective sample of students. We also show that the take-up increases over time, which indicates that the formative tests is relatively more frequently used in later academic years. In the empirical analysis where we evaluate how the formative entry test is associated with drop out in the first year we therefore control for the potential bias by, first of all, including the student controls and by including program fixed-effects empirical analysis.

Table 10 Distribution of students and AMN tests in parenthesis per educational program

Appendix B

See Table 11.

Table 11 The top 31 of the educational programs, ranked by six experts in vocational education