Development of mathematical competency in different German pre-vocational training programmes of the transition system

Abstract

Background

Mathematical competency is central to life in modern society, and it is particularly important for many occupations and professions. In Germany, young people with insufficient mathematical skills experience significant difficulties securing a training position within the dual system, and subsequently, they often enrol in pre-vocational programmes of the transition system. Thus, the various one-year pre-vocational training programmes aim to provide support for enhancing mathematical skills. Currently, there is a lack of information regarding whether fundamental competencies are effectively developed within the context of these pre-vocational training.

Methods

Therefore, this paper examines how competencies develop and are enhanced over the course of 1 year, based on data (N = 1.258) from three different 1-year pre-vocational programmes. Growth was based on a multidimensional mathematical competency construct measured at two distinct points: at the beginning and at the end of the pre-vocational training.

Results and discussion

Incorporating selected background variables, the results of the stable and valid measurement indicate that, on average, mathematical competencies did not change over the course of 1 year. However, when development was considered in greater depth, a second dimension became visible. Specifically, the mathematical competencies of one group of young people were lower after completing the prevocational programme than they were before, whereas another group achieved recognizable improvements in their competencies.

Background

Brief reflections on the importance of mathematical competency

Mathematical competency rank highly among the cultural fundamentals required for independent life in society, namely for full economic, political, social, and cultural participation. Some basic understanding of mathematical structures is a necessary precondition for everyday life as well as for virtually all vocations, and thus, it is an indispensable criterion for the successful pursuit of personal goals. In the public debate, the significance of mathematical competency is recognized from a number of different perspectives. For instance, mathematical skills are considered a “fundamental cultural competency for understanding the world” (Tenorth et al. 2010) and a “formal language […] that in many different forms has become a self-evident mean of communication in many professions and scientific disciplines” (Gschwendtner 2012). Accordingly, mathematics is considered to occupy a prominent position in the practice of economic processes and in the fulfilment of professional duties (OECD 2013), in particular in the contexts of technical and technological change and a democratic shortage of human resources in the job market (Seeber 2013a). Regarding long-term outcome prospects, the International Adult Literacy Study (OECD and Statistics Canada 2000) performed in the mid-nineties and the subsequent study Adult Literacy and Lifeskills (ALL) Survey provided early evidence of the effective link between fundamental mathematical competencies and criteria of success in later life, such as individual income and employment prospects.

Despite an improved labour market situation for vocational education and training in the high valued so-called “Dual System” a substantial proportion of school graduates and early school leaver still ends up in various pre-vocational training programmes within the transition system. The situation has hardly improved for young people without a qualification higher than a general school leaving certificate [Hauptschulabschluss] and for foreigners (Authoring Group Educational Reporting 2012; Autorengruppe Bildungsberichterstattung 2016). Early school leavers and young adults who have left school with a lower secondary qualification (general school diploma) have little chances of a proper vocational training as well as few options to receive an apprentice position according to their interests and aptitudes (ibid, p. 109ff.).

The transition system consists of a number of pre-vocational training programmes in different occupational fields. These pre-vocational programmes vary considerably with respect to the specific entry requirements for trainees and with regard to their curricula emphases. In particular, low-skilled young people with no or with a minimal educational certificate usually attend at least one (or more) vocational preparation programme before beginning an apprenticeship. The objective of all pre-vocational education programmes is to provide vocational orientation, to promote basic competencies of all attendees and thereby improve the likelihood of them obtaining a proper school leaving certificate (general school diploma) [Hauptschulabschluss] or to achieve a higher school leaving certificate, in particular an intermediate school diploma [Mittlerer Schulabschluss]. The last one can be considered as usual entry requirement for an apprenticeship (Baethge et al. 2007; Greinert and Braun 2005; Autorengruppe Bildungsberichterstattung 2016). Moreover, the programmes are designed to support the intersection between occupational orientation and training to prepare students for an apprenticeship, which is in Germany a combination of in-firm based and school-based learning (Dual System).

Research findings on the transition from school to company-based vocational education and training suggest that mathematical competencies play a central role in securing a vocational training position (Lehmann et al. 2005; Seeber 2009). For companies offering training, mathematical competency ranks high among expected prerequisites of trainability together with other basic competencies and working attitudes, and it also represents an important criterion when selecting candidates (van Buer and Fehring 2013). A curricular analysis of business professions confirms that mathematical skills are critical with respect to business and decision-making contexts, both of which require an understanding of numbers, the relationship between parameters and the ability to apply mathematical operations and analytical models (Wittmann 2013). The same is true of industrial-technical occupations. Empirical studies reveal that mathematical skills play an important role in developing job specific competencies in a number of occupations that require intermediate qualifications, such as business and administrative occupations, occupations in the information technology, in skilled crafts and industry (Wittmann 2013; Seeber and Lehmann 2013; Nickolaus et al. 2008; Nickolaus and Norwig 2009; Rosendahl and Straka 2011). The links between cognitive and motivational dispositions of young adults at the beginning of training and their professional performance at the end of training have been examined by the ULME-III study (Lehmann and Seeber 2007) with respect to a range of different occupations. Significant correlations were found between competencies in mathematics and reading on the one hand and vocational competencies on the other, in particular in the area of business and administration and in selected technical specializations, although the strength of these correlations varied significantly between occupations (Seeber and Lehmann 2011; 2013).

A small number of studies examined mathematical competencies and their development/enhancement within the vocational transition system. In the context of specific courses, these studies indicate that improvements in competency can, in fact, be achieved, but that this progress takes different forms depending on the choice of pre-vocational specialization (occupational area), e.g., business and administration, metal technology, wood technology, electronical engineering, health or social work (Lehmann et al. 2006; Behrendt et al. 2016). In Hamburg, the development of mathematical skills in young people was investigated in the context of a 2-year pre-vocational programmes offered specifically as a transitional scheme to candidates with who had achieved only low level educational certificates [Hauptschulabschluss], and therefore, they could not apply successfully for an apprenticeship due to their limited compulsory schooling. Substantial improvements in mathematical competency were observed in the fields of electrical engineering, metalworking, business and administration, while improvements in the fields of public health and social work were significantly lower. It was also found that the work was primarily performed in the spirit of compensation, as in particular the lowest quartile achieved improvements in performance. A significant difference in mathematical skills was also observed between genders at the end of the programme, with male participants exhibiting higher performance, and differences were also observed as a function of migrant status, ascertained by means of the language used at home, with migrant participants exhibiting lower performance than non-migrant participants (ibid).

Definition of mathematical competency

With respect to mathematical skills, the most common differences pertain to the function ascribed to mathematics. A distinction is usually drawn between scientific-propaedeutic and application-oriented perspectives (Tenorth et al. 2010). From the scientific-propaedeutic perspective, mathematical skills are differentiated according to subjects in school mathematics, which consist of the domains of arithmetic, algebra, analysis, geometry and stochastics (e.g., Bloemeke et al. 2008). The measurement of mathematical skills according to this perspective usually follows the approach of traditional mathematics lessons, in which completing exercises and tasks first and foremost requires formal knowledge. These tasks are correspondingly formulated without any form of context (Ulfig 2013). A functional perspective of mathematical skills requires “forging links between phenomena and concepts” (Freudenthal 1986) and describes mathematical skills in the context of application-oriented exercises. According to this perspective, mathematical content is organized into overarching ideas according to phenomenological origin. Examples of such overarching ideas are given by quantity (the use of numbers to describe structures and situations), change and relationships (relational and functional relationships), space and shapes (planes and spatial patterns) and uncertainty (statistical data or randomness) (Frey et al. 2010; also Blum et al. 2004). Although these ideas do not fully correspond to the mathematical domains mentioned above, substantial parallels can no doubt be drawn. With respect to the concept of application-oriented mathematics or mathematical literacy, the functional application of mathematics in extra-mathematical situations is emphasized, in which physical situations are translated into mathematical language and mathematics topics are then applied to these situations (Blum et al. 2004).

A more functional conception of mathematics is also applicable in the setting of prevocational and professional training, although the discussion is rife with controversy in this regard (for an overview, Wittmann 2013). In this context, working from a largely functional view of mathematics, a concept of literacy was developed as was a method for measuring mathematical skills in relation to everyday phenomena and real-life contexts. This approach differentiates the four subjects of (1) quantity (2) change and relationships (3) space and shape, and (4) uncertainty and data, which is analogous to PISA (Programme for International Student Assessment) (Frey et al. 2010). The test concept was also designed in such a way that mathematical concepts, procedures and operations could be applied in predominantly situation-specific exercises.

It has generally been found that the young adults enrolling in the various prevocational programmes primarily exhibit low performance levels, although there is considerable variation in the performance among the different domains (Gschwendtner 2012). Given the significance of mathematical skills in securing a training position (Lehmann et al. 2005; Harms et al. 2013) and the contributions in and their role in developing vocational competencies within a wide spectrum of trained occupations (Seeber 2013a; Geißel et al. 2013), we must determine whether the transition system succeeds in improving the typically low performance levels in mathematics.

Research questions

Until now, less was known about the development of mathematical skills for those individuals who could not find a training place and who had left general school with low basic competencies, e.g., in mathematics. Therefore, the central aim of this paper is to shed light on the development and growth of mathematical skills after 1 year in a specific programme provided by the vocational transition system. To do so, we examine the developments of mathematical competency of young people from the beginning to the end of the pre-vocational training in three different training schemes. Furthermore, we seek to determine the effects that a specific pre-vocational programme, prior education, gender, migrant status and professional orientation have on the development of and growth in mathematical skills.

We address the following research question:

What development of the mathematical competencies can be observed in German pre-vocational training programmes?

Important aspects of this general question relate, on the one hand, to psychometric properties of test scores and the association of assessment behaviour of low achieving students (see Pohl et al. 2016) and, on the other hand, to the development of mathematical competencies between different groups:

• Which test model is appropriate for the longitudinal modelling of the data of these specific groups?

• What changes can be observed between different groups, e.g., by type of pre-vocational programme, occupational areas, school leaving certificate, gender, and migrant background?

Methods

Test instruments and methodical approach

To address the research questions, analyses of data obtained from the longitudinal project IBIS (Individual educational trajectories in the transition system: about the interaction of individual and social characteristics and institutional conditions), funded by the German Ministry of Education and Research (BMBF), are conducted.

The test instrument consisted of 39 items in a multiple-choice format, with one correct choice out of four in each case. To respond correctly to the items, different facets of mathematical competency were clearly required, and the difficulty levels of the items varied within these facets. Initially, the test consisted of items attributable to the following four mathematical key concepts:

• 1. Quantity, i.e., all approaches involving the use of numbers to describe and organize situations, to understand magnitude, and to recognize numerical patterns.

• 2. Change and relations, i.e., mathematical representations of change over time as well as different types of relational and functional dependencies between mathematical objects.

• 3. Space and shape, i.e., all types of two- and three-dimensional configurations, forms, and patterns.

• 4. Uncertainty, i.e., mathematical phenomena and situations involving statistical data and chance.

The distribution of the items for each of the key mathematical concepts is presented in Table 1.

Table 1 Key mathematical concepts

To construct the test, tried-and-tested items from the ULME-I Study (Lehmann et al. 2005) and the BELLA-Study (Lehmann and Hoffmann 2009) were used. Figure 1 provides an illustrative example of one of the items.

Fig. 1

Item example

In addition, the initial data collection included the administration of the (revised) Culture Fair Intelligence Test (CFT-20 R), which measures reasoning (Weiss 1998). The CFT-20-R is used to ascertain the invariance of the mathematical test.

The sample: participants and pre-vocational programmes of the transition system

In the IBIS-study, the mathematical competencies of two cohorts of young adults were assessed according to the purposes previously discussed. The first cohort was assessed at the beginning of the 2012/2013 school year, and the second cohort was tested during the 2013/2014 school year in the context of one of three transitional schemes within the transition system. The transitional schemes, which include the vocational preparatory year [Berufsvorbereitungsjahr or BVJ], the vocational initiation grade [Berufseinstiegsklasse or BEK], and a 1-year course in pre-vocational schools [einjährige Berufsfachschule or BFS], are parts of the transition system and are further investigated in this study.

The vocational preparatory year (BVJ) is geared towards those who have graduated from secondary school or a special education programme after completing grade 8 or 9 without having received an appropriate school certificate. In many cases, these young adults have individual special education needs (e.g. learning difficulties, behavioural disorders and/or social deprivation). The goal of the BVJ is to provide the participants with some occupational orientation that will support in-firm practical training, improve work habits and social abilities, and strengthen the ability of the individual to make an appropriate occupational choice.

The population of the BEK (vocational initiation grade) consists of students who have left school without a proper certificate or with one that indicates a low level of achievement. Therefore, the aim is to help students attain a level that indicates readiness for vocational education so they may have access to it. Alternatively, the successful completion of the BEK facilitates access to the 1-year vocational programme, which in turn opens possibilities to obtain an intermediate school certificate.

The 1-year BFS programme is meant for graduates of the basic-level lower secondary school with a general school diploma [Hauptschulabschuss] or from the intermediate-level school with an intermediate school diploma [Mittlerer Schulabschluss] who have not succeeded in obtaining an in-firm apprenticeship contract. It is the aim of this programme to confer vocational knowledge in addition to some general education. Under certain conditions, the completion of the BFS may be recognized by firms as the 1st year of an apprenticeship.

A total of N = 1549 young persons were tested, each at two different points in time. Due to missing data, the final longitudinal sample consisted of N = 1258 young persons, of which 583 were young women (46.3 %). Young adults with immigrant backgrounds represented 29.4 % of the sample. The participants were distributed across the three programmes, with N = 387 (30.8 %) attending the vocational initiation school [Berufseinstiegsschule, BEK]; N = 248 (19.7 %) attending the vocational preparatory year [Berufsvorbereitungsjahr, BVJ] and N = 623 (49.5 %) attending the 1 year BFS programme [einjährige Berufsfachschule, BFS].

The distribution of participants across professional sectors was as follows: 330 participants (26.2 %) specialized in home economics, 370 (29.4 %) specialized in the industrial-technical sector and 558 students (44.4 %) specialized in economics and administration.

Furthermore, 219 (17.4 %) participants did not have a diploma, 147 (11.7 %) had a special needs diploma ([Förderschulabschluss], 340 (27.0 %) a general diploma [Hauptschulabschluss], and 543 (43.2 %) an intermediate school diploma [Mittlerer Schulabschluss], and 9 students had another diplomas (.7 %). As expected, the BVJ was predominantly attended by students without any diploma, N = 121 (48.8 %) or with a special needs diploma [Förderschulabschluss], N = 95 (38.3 %). By contrast, the young adults attending the BEK had the highest proportion of general diplomas [Hauptschulabschluss], with N = 212 (54.8 %), whereas more than half of the BFS (53.1 %) already possessed an intermediate diploma [Mittlerer Schulabschluss] upon enrolment in the programme. Thus, the young adults with a general diploma were roughly divided half and half between the BEK and the BFS groups. The distribution of gender and migrant status within each course was similar to the overall distribution within the sample (see Table 2).

Table 2 Sample

Longitudinal scaling and qualitity of the scales

To assess the development of mathematical competencies across two measurement points, scaling was performed according to item response theory. As the two test applications can be conceived as repeat measurements, a generalized Rasch model known as the Andersen model (Andersen 1985) was used. It was assumed that the two test instances functioned as a single dimension, each in a two-dimensional model. The developmental component could then be ascertained from the difference between proficiency estimates in the two dimensions. To achieve this, the item difficulty parameters of the second measurement were constrained by the values of the first. On the basis of this fixed item parameter linking (von Davier et al. 2008), the two measurement instances can then be projected as two dimensions onto a common scale (main dimension). If the model assumptions hold, the differences are then solely determined by the development of competency. Alternative item response models that define developments on the basis of variable difficulty parameters are discussed by Glück and Spiel (2007).

To model longitudinally the development of competencies, the second (final) test included items from the first test exercise as link items. Before linking the two tests by way of a two-dimensional model, the tests were scaled separately. Both tests were required to be compatible with the assumptions of the Rasch model. To verify this for each included item, several tests of model fit were conducted. First, the item discrimination was examined using the weighted mean squares index. Next, the corrected item-total correlations and the correlations of the distractors with the total score were inspected. Because of the relatively small number of surviving items, it was deemed superfluous to conduct a test of dimensionality. Differential item functioning (DIF) tests were applied, however, to determine whether concurrent calibration across all transition programmes was justified. Additionally, DIF analyses were conducted to test for the potential construct irrelevance of gender, migrant status and reasoning test score. Finally, the internal consistencies (reliabilities) as well as the variances were ascertained. Thus, it was ensured that the resulting dispersion was sufficient for further analysis (Table 2). Once Rasch homogeneity of the test had been established separately for both measurement points, the invariance of the remaining items across measurement points was examined. For this purpose, a DIF test was applied to the link items in a dataset that combined the data from both measurement points into a single body. Items that were not found to be invariant across measurement points and any remaining items were estimated freely in both dimensions (Carstensen 2007). The Andersen model was specified by constraining the difficulty parameters and setting the mean difficulty to zero in both dimensions. Finally, the goodness of fit of the common scale of the Andersen model was determined using the weighted mean squares index.

Results and discussion

Psychometric quality of separate tests, invariance across measurement points and model fit of the Andersen model

Of the original 39 items on the test at measurement point one, 15 had to be removed because of unsatisfactory t-statistics and one because of low correlations with the sum of remaining items. The reduction according to mathematical sub-competency was performed similarly. The remaining items had t-values between 2.0 and −2.6 and item-total correlations ranging from 0.32 to 0.51. The point-biserial correlations were inconspicuous. The variance of person ability estimates was 0.932 logits; thus, discrimination between persons appears to have been satisfactory, although their range was only −1.53 to 1.388. As a whole, the test is somewhat difficult for adolescents in the beginning of their transition programmes. The test reliability was 0.72 (WLE), and thus may be considered fair. The DIF analyses did not produce any compelling reason for the exclusion of further items. Measurement invariance and concurrent calibration were therefore deemed justified.

Measurement point two revealed similar results. The variance of person ability estimates [1.22 logits] and the test reliability of 0.75 were satisfactory. These results demonstrate that for the remaining item pool, Rasch homogeneity may be assumed for both testing sessions and all investigated groups. Thus, it is fair to assume that in all of these instances the measured notions of mathematical competency were equivalent.

On the basis of the remaining 19 link items, a DIF test was conducted to detect any violations of the assumed invariance across measurement points. However, no item was unfit for the projected linkage, and there was no difficulty gap of 0.3 logits between the two measurement points. Therefore, the Andersen model was implemented with 24 items in the first dimension and 21 items in the second, each with 19 constrained items (Table 3). The software used was ConQuest 4.25 (Adams et al. 2015).

Table 3 Overview of test items at t1 and t2

The t-values for the weighted mean squares in the first dimension indicate that one of the constrained items discriminated too strongly in the first dimension and that two others did not discriminate sufficiently (t = 2.6). Among the unconstrained items, two failed to discriminate sufficiently. In the second dimension, one constrained item had low discrimination, whereas for another, the discrimination exceeded the assumed common value. The reliability estimates were satisfactory for both dimensions. For the first measurement point, the variance was satisfactory and reached an acceptable level at the second measurement point with a value of 1.26. In summary, a satisfactory level of fit may be assumed for the longitudinal model (Table 4).

Table 4 Development of mathematical competency

Development of mathematical competency

When considering the development of competencies, one half of a standard deviation was considered to represent a moderate change, and one full standard deviation was considered to represent a substantial change. The analysis according to the longitudinal model resulted in an estimated development of 0.04 standard deviations between t1 and t2, which also provides confirmation of the latent correlation of r = .89. This indicates that in terms of competencies, very little changed over time. The slight increase in the standard deviation of the second measurement also suggests that the competency level barely shifted. However, a closer examination of these developments reveals that there are two groups of comparable size, one with a substantial learning progression (14.7 %) and another with a substantial learning regression (12.2 %). The groups with moderate development are also of mutually similar size as 33.9 % of participants experienced a moderate progression and 22.6 % experienced a moderate regression after 1 year. This suggests that the development of competency occurred at similar rates in two distinct directions, which explains the observation of zero net overall growth.

The development and the distribution of competency require a more in-depth analysis. However, because the results of the analysis of selected background variables is not based on a random sample, any generalisations of the data in any form are not possible.

Development of mathematical competency by pre-vocational programme

A closer examination of this overall finding reveals differences in the growth achieved by each programme. The students taking the BFS courses revealed the largest gains, though these gains are without significance (d = 0.14). Attending the BVJ courses did not appear to result in any improvement (d = 0.07), and in comparison, there was a slightly negative null-development for the participants enrolled in the BEK (d = −0.08). At the end of the programmes, the variances in achievement with respect to the courses with lower-achieving students was somewhat larger than it had been in the beginning. As expected, the young adults enrolled in the one-year vocational school presented the highest average competencies (0.40 logits) in the entry assessment, followed by the young adults enrolled in the BEK (−0.44) and the BVJ (−0.92). Figure 2 clearly indicates the strongly heterogeneous competency spectrum of this vocational school. The BVJ and the BEK, conversely, consisted of groups that were largely homogeneous in competency.

Fig. 2

Mathematical competency by pre-vocational programme and measurement point

As evidenced in Fig. 2, the distribution is narrow. Each of the courses included both young adults whose performance decreased and young adults whose competencies improved over the course of the year. The evolution of competency is most visible in the border regions, and the distribution suggests that performance decreases occurred most strongly in regions with weaker participants. This is confirmed by the analysis of growth in the BEK in which 30.7 % regressed by more than one-half of a standard deviation over the course of 1 year, and 22.7 % exhibited moderate regression. This indicates that the students in the BEK experienced a comparatively high level of regression. The picture is reversed for the BFS, in which approximately 33.9 % experienced higher than moderate growth and 22.6 % experienced moderate growth. In the BVJ, however, there is less dispersion in the development of competencies. This demonstrates that the significant developments occurred primarily within the BEK and the BVJ.

Development of mathematical competency by highest school leaving certificate

Comparable results are obtained when the development of competencies is considered as a function of the school leaving certificate (school diploma) attained prior to enrolling in a transitional programme. The results indicate that students’ competencies and growth differed according to their programmes and their stages in the transition system. Considering general education courses hierarchically according to their entry requirements, students with higher formal qualifications recorded higher achievement levels at the beginning of the transition programme. Noticeable development, however, could only be observed among students with intermediate levels of formal education [Mittlerer Schulabschluss] (d = 0.24). For students with other prior diplomas or no diplomas, no development in mathematical skills was observed. The lowest performers were the students with special needs diplomas [Förderschulabschluss]. These individuals scored an average competency of −1.08 logits. The next category, by a considerable margin, was that of students without diplomas (−0.52) and those with low leaving school certificate diplomas [Hauptschulabschluss] (−0.19). As expected, the highest performers by another large margin were the young adults with an intermediate school (leaving) certificate (.69). This hierarchy is also presented in Fig. 3. Notably, moderate and substantial growth were observed particularly for students who had an intermediate school education when entering the transition system. For example, 36.2 % of these improved by at least one-half of a standard deviation, of which 20.6 % improved by more than one standard deviation. Nonetheless, even in this category, 10.9 % of young adults regressed in competency by at least one standard deviation. A similar pictures exists in the other groups, with 25–30 % either progressing or regressing by one-half of a standard deviation, indicating that developments only occurred in the higher competency region and that the dispersion of the lower-performing groups, though lower, is homogeneous.

Fig. 3

Mathematical competency by occupational area and measurement point

Development of mathematical competency by occupational area

If the differences between groups are considered not from the perspective of previous education or transitional programmes but rather from the perspective of occupational area, it may be observed that participants who chose in their pre-vocational programme the area of business and administration constituted the highest performing group (0.31). With respect to home economics, the initial mean competency score was the lowest (−.64), while students with a technical focus exhibited some degree of competency (−.31). Furthermore, the competency distribution (Fig. 4) indicates that the highest performers were found in the group that chose business and administration, although this group was composed of young adults from all competency levels. From a longitudinal perspective, the general mathematical competencies did not improve overall, despite the different levels of support provided in each of the occupational areas, though slight development (d = .14) was observed among those who opted for programmes with economic emphasis. The distribution indicates that the strongest variations were experienced by the lowest performers, that is, those specializing in domestic economic areas for whom mathematical skills moderately regressed in 31.5 % of the group. However, the picture is reversed for the highest performers specializing in business and administration (commercial areas), for whom 31.5 % progressed by one-half of a standard deviation. In the groups with technical/natural specializations, variations were lower, and the results indicate that despite different levels of support and varied levels of correspondence with professional competencies there was no subject-specific development. However, competency developments were again observed in both directions.

Fig. 4

Mathematical competency by migrant status and measurement point

Development of mathematical competency by migrant status

On average, young adults with an immigrant background experienced zero growth, i.e., d = −0.01. The mean competency regarding entry of young adults with an immigrant background was −0.21 compared to −0.08 logits for those without an immigrant background. In other words, both groups were homogeneous with respect to their basic competencies (Fig. 4). Because the growth of the group with an immigrant background did not increase (d = 0.10), it can be concluded that the growth was unspecific to migrant status. Similar proportions of young adults both with and without immigrant backgrounds exhibited moderate and substantial growth. When past education and gender were considered, the results became slightly more differentiated. In this context, the groups of male participants, and preponderantly female participants with immigrant backgrounds, experienced stagnation. However, due to the very low sample size, a more in-depth analysis was not possible.

Development of mathematical competency by gender

An overall stagnant trend is visible when the development of mathematical skills is broken down by gender. For males, the development index is given by an effect size of d = 0.01, whereas for females, the development index is denoted by an effect size where d = 0.11. On average, female participants exhibited a lower competency level than male participants by 0.4 logits. If the gender-specific differences are considered as a function of past education, it appears that young women exhibited a lower performance than young men. The greatest difference in competency on entry was in the BVJ, with a difference of 0.67 logits. Growth was, however, not gender-specific. The mean progression of female students with respect to the BVJ and the BFS was zero, which is considered low. Gender, with a progression of at least one-half of a standard deviation, did not seem to play a role in the group of students. However, 27.7 % of female students progressed by at least one standard deviation, whereas the corresponding proportion of male students was 25.5 %. The groups that either progressed or regressed by at least one standard deviation were equal in proportion, at approximately 12.5 %. A similar picture is given when considering entry level by gender and by prior educational achievement. Young men displayed higher entry levels in each case, though the results for growth are slightly different. For example, young women with a special needs education diploma (Förderschule) exhibited a slight improvement (d = 0.19), whereas young men with a special needs education diploma (Förderschule) demonstrated a regression (d = −0.10). Thus, based on the findings of this study, in terms of development over time, no gender-specificity was detected.

Discussion

This paper investigated how competencies evolved during one-year programmes conducted as part of a transition system. A test of mathematical competency in several subjects was administered at two distinct moments in time within the BEK, BFJ and BFS programmes of the transition system. The test items measure skills based on a concept of mathematical competency assumed to be common across all courses. To assess the development of mathematical skills, the two tests were related via link items. The test quality at both measurement points was inspected separately using the Rasch model. The development was modelled using the longitudinal Andersen model from item response theory. By adjusting the difficulty of the time-independent link items, the development of a competency parameter was determined, and the development of competencies was analysed based on differences in the competency values between both assessments (dimensions).

Both assessments were found to be Rasch homogeneous, and the number of clearly discriminating items was determined to be sufficient. The measurement invariance of the competency test can also be established by means of a number of different DIF tests. Furthermore, a concurrent calibration of the difficulty levels was also possible using the chosen mathematical competency test. As the variances and reliabilities of the individual tests were sufficiently large, they adequately represented the individual differences in competency among the young adults. As the difficulty of the remaining link items was retained, the longitudinal multidimensional model used the fixed-difficulty link items as a basis. Accordingly, the choice of the longitudinal Andersen model is justified.

Using the mathematical test from the IBIS study, it was possible to establish the growth of mathematical competency due to the programmes of the transition system. The results of analysis are, however, sobering, as the mean mathematical competencies do not change (d = .04) over the course of 1 year. This growth is less than that observed in a one-year longitudinal study of TIMSS students as they transitioned from grade 7 to grade 8, where the mathematical development was estimated to be 0.25 standard deviations (Beaton et al. 1996), and it was less than that in a 1-year longitudinal study of PISA students who transitioned from grade 9 to grade 10 and demonstrated a growth of 0.33 standard deviations (excluding lower-achieving students from the hauptschule) (Ehmke et al. 2006). Nonetheless, with respect to the specific target group of students in prevocational programmes with learning and behaviour difficulties, this result should be perceived as positive. Furthermore, this result is well in accordance with the findings of other studies that observed low development rates in mathematics among students in the transition system (Nickolaus and Norwig 2009).

Even when these results are considered given the chosen co-variables, it can be proven that, on average, no development or growth occurred. In summary, the analysis indicates that growth, over time, in each sub-group was relatively similar and that there was no substantial increase in mathematical competency. There are some slight indications that students who successfully completed an intermediate level of secondary education attained the largest improvements in this domain. However, even in this case, the observed growth was minimal, and the presence of a Matthew effect in students with an intermediate diploma [Mittler Schulabschluss] cannot be concluded. Moreover, the divergence in learning opportunities across the various occupational branches does not appear to be related to the differential development of mathematical competency, nor was there any specific growth associated with gender or migrant status. Therefore, the results are consistent with expectations with respect to differences in competency as a function of prior diplomas, choice of programmes within the transition system, gender, and migrant status, as well as the choice of specialization.

One of the points raised by the analysis is that the observation of mean zero growth represents an insufficient description of the development of competency when examined more closely. Though a large proportion of young adults demonstrated improvement, many also regressed in their grasp of mathematical concepts, strategies, procedures and operations. A more in-depth analysis reveals that the evolution was not homogeneous, and therefore, the mean value of the evolution is insufficient in terms of diagnostic information. There were groups of young adults who achieved significant learning, and groups that “unlearned” what they knew. Though the proportional size of these groups varies, the variance is minimal. The proportion of young adults who experienced either an increase or a decrease in competency by one-half of a standard deviation was approximately 25 % in each case. Thus, half of the young adults experienced a moderate change in some direction. The tendencies of the proportion of participants who experienced a regression of one-half of a standard deviation can be interpreted with respect to the background variables. This group was composed of the lowest performers, e.g., those specializing in home economics or young adults with no diplomas. One factor that was not considered in the analysis was whether the changes could be the result of differences between content dimensions. Due to the non-demonstrable multidimensionality of the test, this question remains unresolved. Therefore, no statements or conclusions can be offered as to whether mathematical weaknesses or strengths existed in the context of individual mathematical topics. However, a series of studies suggests that mathematical topics correlate highly with the different phenomenological regions when viewed from a functional perspective (Seeber 2013b), and thus, a strong divergence in the development between topics is not expected to exist. This assumption is somewhat different for professional training as mathematical topics are strongly linked to the professional subject matter and mathematical learning opportunities specifically arise from the requirements associated with the profession. In the courses offered by the transition system, similarly distinct effects are not necessarily expected, as the specific mathematics lessons offered are geared towards the curricular requirements of those with the lowest educational certificate (Hauptschulabschluss), or in the case of the BFS, towards those with an intermediate diploma (Mittlerer Schulabschluss).

The underlying reasons for the differences in development between the transitional programmes are likely diverse in nature. The timing of the test, for example, was not at the end of an intervention but at the end of the entire course. By this time, the competencies acquired by many of the participants could already have begun to diminish. Moreover, developmental-psychological factors could also affect the ability of the participants to perform adequately on the assessments. Other reasons could possibly be traced back to the Rasch-based approach, which assumes that competencies develop homogeneously. Accordingly, to better integrate heterogeneity into the model, the measurement model approach of the general diagnostic model (von Davier 2008) could potentially be used. By replacing the continuous variable with discrete linear combinations in this approach, multiple latent competency group-specific developments could be modelled (von Davier et al. 2011).

One point that requires discussion is why many of the test items that fulfil the psychometric quality criteria do not fit the model. This paper adopts the view that the most precise person estimator was achieved using the remaining discriminating items according to the item information function of the Rasch model. With the items fitting the model, the highest (model-based) variance of mathematical competency parameters was determined, and thus, the objective of measuring individual differences was achieved. The elimination of specific items is justified by the invariance postulate of the Rasch model (specific objectivity) (Fischer 1987). According to this postulate, only items that permit a person-invariant specific objective comparison of the difficulties of the test instruments (items) should be considered. The discriminative quality of these items should not vary between groups of persons. Nonetheless, the excluded items suggest that the competency test within this type of population is problematic. In other studies, it has been found that there can be a variety of reasons for the exclusion of items (Pohl et al. 2016). It is a challenge to assess the competencies and skills of students with low levels of education or with special education needs using standard learning assessments. Reasons for ensuring test quality for these specific groups are evidenced by their test-taking behaviours, which include the unsystematic selection of response options throughout the test; problems related to motivation; the systematic failure to respond to tasks with complex response formats, as well as aspects of the test itself. Because the observed group of young adults in different training schemes of the transition system is composed of a substantial number of very low achieving students and students with special educational needs, one possible approach for identifying the scaling problem is to compare the results according to different measurement model classes. This type of analysis could provide insight into the heterogeneity of the development and into the reasons behind item exclusion.