Being an economic-civic competent citizen: A technology-based assessment of commercial apprentices in Germany and Switzerland

Abstract

Background

Citizens in modern democratic societies are confronted with complex demands, challenges, and economic problems, and they typically act as consumers, savers, voters, and employees. It is highly desirable for citizens to be able to competently manage these various demands. This ability we call economic-civic competence. Given the lack of empirical findings and adequate instruments on this issue, we analyze the economic-civic competence of commercial apprentices in Germany and Switzerland. In Swiss commercial apprenticeships, the fostering of this competence is explicitly anchored in the curricula of vocational schools. In Germany, the curricula coverage is not as broad as in Switzerland, which leads to an assumption of country differences in civic-economic competence.

Methods

A total of 1255 apprentices in the commercial occupations/branches of industrial clerk and logistic clerk were assessed at the end of their second or third training year. Our newly developed, reliable and valid technology-based assessment on economic-civic competence includes issues such as the Euro crisis and public debt.

Results and conclusions

An advantage among the male trainees, which was expected from previous research, was confirmed only among the German logistic clerks. The expected gender effect was not observed among the German industrial clerks or the Swiss clerks. Furthermore, the results confirm the hypothesized higher scores among German trainees with a university entrance degree. In both countries, the industrial clerks scored significantly higher than the logistic clerks did, and the Swiss trainees scored higher than did their German counterparts in a comparable occupation. This difference becomes markedly stronger if only learners without a university entrance degree are considered. One possible explanation for the country difference may be that the promotion of economic-civic competence has a more prominent role in Swiss vocational schools during commercial apprenticeship. Considering the methodological restrictions, the findings should be interpreted primarily as explorative depictions of the characteristics and group differences of economic-civic competence. However, our study can also be seen in the broader context of contemporary interest and challenges in developing technology-based assessment within the field of education. Thus, our study contributes to further research on the characteristics and genesis of economic-civic competences as well as on technology-based assessments.

Background

Citizens in modern democratic societies are confronted with complex economic demands, challenges, and problems. They typically act as consumers, savers, voters, and employees. The growing complexity of economic processes in private life, the workplace, and the modern internationalized world requires knowledge to understand basic economic principles, concepts and mechanisms. For instance, in direct democracies and in representative democracies, people have the right to vote on voting proposals or partisan programs that are closely related to economic objectives. To justify their voting decisions, people should understand the causes and effects of suggested solutions and the conflicting interests of the stakeholders involved. Thus, it is highly desirable that citizens have the ability to handle economic-related demands in various contexts. We call this economic-civic competence.¹

Fostering economic-civic competence has long been an explicit goal of the Swiss commercial apprenticeship program. In addition to professional competence, Swiss commercial trainees are required to achieve economic-civic competence, which is largely acquired in the subject “Economics and Society” (E&S) in vocational schools. In German commercial apprenticeships, the promotion of these types of competencies also plays a significant role (Winther 2010). To date, however, no studies have investigated economic-civic competence in an adequate empirical manner.

Considering the substantial lack of empirical findings, the aim of this study was to analyze in detail the economic-civic competence of Swiss and German commercial apprentices in the occupations and branches of “Industrial clerk” and “Logistic clerk”, respectively. In particular, we examine the effects of gender, prior education, occupation, and national affiliation. We introduce our concept of economic-civic competence and outline the related state of the research. Subsequently, we highlight the country-specific systems, the commercial apprenticeship and the curricular support for economic-civic competence in detail.

Concept of economic-civic competence

The concept of economic-civic competence combines the concepts of economic competence and civic competence (see Eberle 2015). There are research traditions in both economic competence/education (e.g., Wuttke et al. 2016) and civic competence/education (e.g., Print and Lange 2013). Given that civic demands (e.g., referenda) are often related to economic problems on the individual or social level, the weak connection between these two research areas is surprising. Our definition of economic-civic competence is outlined in the next section and considers both research approaches. A more detailed description of our model of economic-civic competence is outlined in Eberle et al. (2016).

Economic-civic competence is defined as the ability to understand private-economic, politico-economic and business issues in state and non-state social systems, to evaluate proposed solutions, and to develop solutions to simple problems (Eberle et al. 2016). According to existing definitions of competencies (e.g., Beck 1989; Weinert 2001), economic-civic competence is based on knowledge, skills and non-cognitive factors (e.g., an interest in economic problems, a motivation to solve economic problems, and attitudes and values that allow economic problems to be discussed and responsibly resolved). In the present study, we focus on the cognitive domain, although we emphasize the need to include non-cognitive facets in future research, as mentioned in our research outline.

Economic-civic competence is significant for all citizens of a society and not solely for specific occupations. Two examples are provided. (1) All western countries must address demographic changes and their effects on the retirement provision system. Considering the recurrent suggestions of politicians, parties, think tanks and experts regarding the “appropriate” solutions to this challenge, it is necessary to have a solid understanding of the basic economic principles of the existing system, the economic and social effects of the suggested transformations and the intentions of the involved stakeholders. (2) The European (Monetary) Union constitutes an important pillar of the economic integration of the member states of the European Union (EU). Near or executed shocks, such as Greece’s struggle (including the massive Greek public debt) since 2010 or BREXIT in 2016, clarify the importance of a basic understanding of the economic and societal causes and consequences of such incidents. Broader comprehension within society may have resulted in a different voting decision in the UK, for example. Of course, economic-civic competence is no panacea. However, with a higher competence, we argue that citizens may be less prone to populism and “simple” suggested solutions.

Because of this understanding, we use these economic and social issues as origins to assess the economic-civic competence of commercial learners in Switzerland and Germany. To illustrate these types of issues and their handling within our assessment, please see the example on the Euro crisis in the appendix (Fig. 1).

Fig. 1

Example of a text-based introduction to the Euro crisis issue

State of research

Explicit empirical research on economic-civic competence is lacking. Thus, the research on civic and economic education and competence exist relatively independently. At most, there is a connection with the related concept of financial literacy in democratic contexts (Davies 2015). In the following paragraphs, we focus on selected empirical findings on economic competence and their correlations because, along with our understanding of economic-civic competence, we assume a stronger interrelation with economic than with civic competence. For civic competence, we refer to recent studies (e.g., Print and Lange 2013; Schulz et al. 2016).

Historically, the origin of modeling and measuring economic competencies can be observed in the US in the report “Economic Education in School” (Committee for Economic Development [CED] 1961). Numerous definitions and test instruments to explain and measure economic competences were subsequently developed (see Kotte and Witt 1995). The test of economic literacy (TEL; Soper 1979; Walstad et al. 2013) is the most frequently applied instrument to assess economic competencies in upper secondary education. Many studies have used the TEL or a translation of it (e.g., the German translation “Wirtschaftskundlicher Bildungstest” by Beck and Krumm 1998). An overview of the results shows considerable differences among countries (Walstad 1994). Students in Australia, the UK and South Korea performed well, whereas the mean scores of students in the US, Germany, Austria, Switzerland and Greece were lower. Based on these findings, significant international differences can be assumed. However, comparative international studies of economic competencies are still lacking. Despite the widespread application of the TEL, three related drawbacks must be mentioned. First, the TEL focuses on only economics, whereas topics of business administration and individual budgeting are ignored (Tramm and Seeber 2006). Second, the data of several empirical studies cannot validate the hypothesized complexity of the items with Bloom`s taxonomy of knowledge. Third, due to the TEL’s use of forced-choice items only, an assessment of higher-order abilities is not possible. To reveal higher-order abilities, open-question formats should supplement the forced-choice items.

Several studies have examined learners` economic competencies in the vocational track in upper secondary education in German-speaking countries. Some of these studies used the German version of the TEL or parts of it (e.g., Beck and Krumm 1998; Lehmann and Seeber 2007). Furthermore, several studies investigated economic competencies in commercial apprenticeship programs (Klotz and Winther 2015; Seeber 2008; Winther 2010; Winther and Achtenhagen 2008) and revealed substantial differences among different programs. However, because of the use of cross-sectional designs, possible causes could not be identified. The main question is whether the program differences are affected mainly by program influences or by learners’ prerequisites. In general, one finding seems to be evident in most studies: when students complete more economic coursework, they score higher on related tests (for an overview, see Seeber et al. 2015).

Another question in the present study focuses on gender differences. The gender gap in terms of economic competence is well documented. Male students score significantly higher than their female peers do at several educational levels (e.g., Brückner et al. 2015; National Center for Education Statistics 2013; Schumann and Eberle 2014; Walstad 1994). However, the reason for this gap has not been explained (Schmidt et al. 2015). Furthermore, it is unclear whether this gap can be replicated for economic-civic competence.

There are robust findings regarding the correlations and predictors of economic competencies. Prior education (including course participation in economics), basic competencies in a native language and mathematics, and general cognitive abilities in terms of intelligence are some of the main factors that have been shown to correlate with students’ economic knowledge (Brückner et al. 2015; Schumann and Eberle 2014; Seeber et al. 2015). In addition, learners with higher prior knowledge, basic competencies and general cognitive abilities typically score better (Lehmann and Seeber 2007; Schumann and Eberle 2014; Seeber et al. 2015; Spoden et al. 2015; Winther 2010). Finally, the effects of socio-cultural background are unclear and have been reported inconsistently in the literature (Jüttler and Schumann 2016; Kotte and Lietz 1998).

The research overview reveals that although different educational levels have been considered, no study has focused on economic-civic competence at the commercial trainee level. This gap also leads to missing studies on trainee characteristics such as gender, prior education, and branch/affiliation as well as country comparisons at this educational level. These gaps are addressed by the present empirical study.

International studies tend to change their measurement procedures to use new technologies to measure students’ competences. The main goal of this shift in measurement procedure is a more precise measurement (e.g., improved accuracy in data collection, saving time, permitting more interactive question types, adaptive testing and multimedia support). Due to the technology evolution leading to an easy application of an automatic coding approach to open question formats in the future, the today dominant forced-choice items then will be rarely used and open questions will be preferred. Open question formats represent a more valid testing situation. Moreover, students’ writing in an open question format may represent a deeper knowledge structure than forced-choice items (see also our critics of the TEL, introduced before). Technology-based assessments also provide new opportunities for data collection and data saving by using data more efficiently for future analysis, such as meta-analysis that includes replication checks (see Kaufmann et al. 2016). Finally, technology-based assessments may increase students’ motivation for testing.

However, technology-based assessments for economic competencies have not previously been used. Hence, we see a need to use the advantages of technology-based assessment for future applications to measure civic-economic competence. For the first time in German-speaking countries, this study uses a technology-based approach to assess economic-civic competence.

Educational system and vocational education in Germany and Switzerland

At first glance, the German and Swiss educational systems appear to be similar. Historically, both systems have been characterized by a highly stratified educational system in lower secondary education and a somewhat impermeable divide between an academic track and a vocational track following compulsory school (Deißinger 2010). The smaller academic track usually leads to universities and typically requires a university entrance degree (UED) that is received from a general baccalaureate school (BS). In contrast, the larger vocational track is geared toward professions within employment systems and is characterized mainly by so-called dual apprenticeships. Dual apprenticeships are distinguished by a combination of company- and school-based training (Maurer and Gonon 2013; SERI 2016). Apprenticeships last from three to four years and qualify trainees for numerous specific occupations. There is no formal allocation process for the selection of trainees (e.g., through a general school certificate). Companies are free to choose their trainees (Deißinger 2010).

In a general comparison with the contemporary German system, the Swiss system can be described as more “traditional”. Approximately 20% of young adults pursue a “pure” academic track. This rate has been somewhat stable over the past 20 years (Eberle and Brüggenbock 2013). More than 90% of Swiss baccalaureate holders enter a university or a university of applied sciences (UAS) after leaving a BS. In contrast, approximately 70% of young adults in Switzerland pursue vocational education and training (VET) after their compulsory schooling (SERI 2016; Wettstein et al. 2014).

In contrast with Switzerland, a much stronger shift toward longer school attendance, higher general school certificates, and, especially, higher education can be observed in Germany, triggered by comprehensive school reform debates in the 1960s and 1970s (Deißinger and Gonon 2016). In most German federal states, the BS system currently has the largest enrollment in the general educational system. Because of the creation of an opportunity to acquire general school certificates within the VET system, more than 40% of youths earn a university entrance degree (UED), and an additional 15% earn a degree for entrance to a UAS (Authoring Group Educational Reporting 2016). In 2011, for the first time, there were as many people entering higher education as people enrolling in dual VET programs (Authoring Group Educational Reporting 2014). In contrast to Switzerland and its relatively low permeability between the academic and vocational tracks, much higher permeability can be currently observed in Germany. Many young adults enter the higher educational system through programs in the VET system. In contrast, a substantial proportion of learners with a UED subsequently start an apprenticeship. The most favored commercial training programs are attended by trainees with a UED (approximately 70% in the industrial clerk occupation and almost 60% of logistic clerks; see DESTATIS 2015).

Commercial apprenticeship

In both countries, commercial apprenticeships play an important role within the VET system. In Switzerland, with approximately 30,000 trainees, commercial apprenticeships are by far the most popular of the VET programs among male and female youths (Swiss Conference of Commercial Training and Examination Branches 2011a). Commercial training is allocated to the three following learning locations (SERI 2011): (1) training at the host company; (2) classroom instruction at vocational schools; and (3) training in branch training centers. In the branch courses, trainees spend 8–16 days for 3 years at branch training centers, where they are provided with essential practical skills for one of the 21 branches. There is only one commercial apprenticeship that leads to the same VET certificate for all learners. However, the program is differentiated with respect to the 21 commercial branches (e.g., banking, industry, and logistics). The school-based general education is identical and standardized across all branches, whereas the company-based and industry-based vocational training is more flexible and varies among the branches because of their specific business structures and processes (Swiss Conference of Commercial Training and Examination Branches 2011b).

Despite their many similarities, there are significant differences between German and Swiss commercial apprenticeships. First, in Germany, there are more than 50 types of commercial apprenticeships. The most prominent are the commercial apprenticeships to become office clerks (approximately 70,000), retail clerks (approximately 60,000), industrial clerks (approximately 50,000), and sales clerks (approximately 40,000) (Federal Institute for Vocational Education and Training 2016). Thus, the occupations and curricula are more varied in Germany than in Switzerland. Second, the occupational associations do not provide specific courses in Germany, as they do in Switzerland. The conceptual professional competencies and more general competencies must be learned in vocational schools. Third, unlike Switzerland and its subject-driven curricula, the German vocational school curricula are organized mainly into so-called learning fields. In general, learning fields are oriented toward typical demands within a particular occupation. Fourth, the vocational school curricula in Switzerland are standardized nationwide, whereas the school curricula in Germany vary slightly among the federal states. However, the curriculum should be closely oriented to the framework curriculum of the Standing Conference of the Ministers of Education and Cultural Affairs.

Fostering economic-civic competence in vocational schools

In both Switzerland and Germany, the systematic fostering of economic-civic competence is performed particularly in the vocational schools during commercial apprenticeships.

In Switzerland, the teaching of this competence largely occurs in the subject “Economics and Society” (E&S). The educational goals and learning content in E&S are formulated in the federal curriculum (Swiss Conference of Commercial Training and Examination Branches 2011b). The overarching educational goals in E&S are the following: commercial learners should understand the relations, challenges and problems in companies, the economy and society; commercial learners should be able to suggest, implement and evaluate solutions within their area of influence; and commercial learners should be aware of their responsibility and opportunities as economic and social citizens (Swiss Conference of Commercial Training and Examination Branches 2011b). E&S is divided into the four subfields of finance, management and business, economics, and civics and law. The learning material in the subfield of economics addresses a wide range of economic topics and policy issues (e.g., economic growth, unemployment, inflation, fiscal and monetary policy, and social security). Trainees must complete 520 h in E&S during their apprenticeship: 5 h/week in their first and second training years and 3 h/week in their final year.

As described above, the vocational school curricula of occupations in Germany are oriented to the framework curricula of the Standing Conference of the Ministers of Education and Cultural Affairs. Regardless of the differences among the several federal states, the curricula-driven promotion of economic-civic competence is not as broad and deep in Germany as in Switzerland. In the following, we discuss the trainees as “Industrial clerk” and “Logistic clerk”, which are the scope of our study.

“Industrial clerk” trainees must be taught in 12 learning fields for a total of 880 h during the apprenticeship (KMK 2002). Because of the system of the learning field, an accurate identification of subject-specific content is not possible. However, the business and entrepreneurial aspects of specific commercial occupations have received considerable attention. In Germany, general education in economics and politico-economics is not covered in depth, unlike in Switzerland. German industrial clerk trainees receive approximately 150–200 h of this type of instruction during their three-year apprenticeship (see also Ministry of School, Youth and Sport Baden-Wurttemberg 2016a, b, “Logistic clerk” trainees receive vocational school instruction in 15 learning fields for 880 h (KMK 2004). Most of the learning fields are oriented toward logistic material. More general economic content is covered by a total of 60–80 h during the entire apprenticeship, which is more than half of Germany’s industrial clerk trainee requirements.

Research questions

Because of the outlined research gap on economic-civic competence and the support during commercial apprenticeship, we focus on the following four research questions that consider the economic-civic competence of Swiss and German commercial trainees at the end of their vocational training. The following questions address the differences in the scores of economic-civic competence among several subgroups of trainees.

• 1. Gender differences According to the largely robust finding of males` higher scores in terms of economic competences, we expect to see an advantage for male trainees. Do male trainees actually score higher on economic-civic competence than female trainees across countries and in the different occupations?

• 2. Prior education Particularly among German commercial apprentices, a substantial proportion of the trainees hold a UED. Typically, these trainees are educated longer and better than trainees without a UED and show higher levels of academic achievement. Moreover, these trainees show higher general cognitive abilities on average. Because of the differences due to a UED, we expect that there are substantial advantages for trainees with a UED.

• 3. Occupation/branch affiliation Are there actual differences in the scoring of economic-civic competence between the two commercial branches/occupations in Germany and Switzerland?

• 4. Countries Considering the stronger curricular consideration of economic-civic competence that is promoted in Switzerland, higher scores for Swiss commercial trainees are plausible. However, until recently, there were no empirical findings on this matter. Therefore, we examine the differences between German and Swiss trainees within equivalent branches/occupations.

Methods

Development of the technology-based assessment

The development of the required economic-civic test is implemented in a technology-based assessment because our goal is to validly measure economic-civic competence and to increase students’ motivation. New technology provides new opportunities for the development of tests for international comparison. In addition to technology-based test development, consistent with Hambleton (2001) and Hambleton et al. (2005), we considered subject-, language- and culture-specific characteristics and adequately adapted them.

To measure the cognitive dimension of economic-civic competences (knowledge and skills), we developed a test with 69 items. Regarding the development of the achievement test, our work was based partially on our own previous study concerning economic competences (Schumann and Eberle 2014). Furthermore, we used 12 items from the German version of the TEL (Wirtschaftskundlicher Bildungstest from Beck and Krumm 1998) and 14 items from the OEKOMA test (see Schumann and Eberle 2014). Given our focus on the economic-civic aspect of the competence concept, we had to develop a new instrument. In the first step, we identified the typical economic issues to which people are exposed in their daily life. An analysis of Swiss referenda in the past 10–15 years led to a dozen possible issues. The second step contained the proof of applicability to the German context through an independent application assessment by experts from the area of German business and economics education. Finally, six issues were selected: the Euro crisis, youth debt, manager salaries, public debt, energy supply and retirement provision. Each issue was introduced by a technical text. The paragraphs of the introductory text were collected from various nationwide Swiss and German newspapers. The introductory text functioned as an anchor and ensured the complexity, topicality and authenticity of the issue. The introductory text was followed by 7–12 forced-choice and open items (for examples, see the Appendix, Figs. 1, 2 and 3). The items included economic basics (facts, key terms, and concepts) and economic problem solving (various solutions to a problem, advantages/disadvantages of a solution, and effects of a solution). The items varied in their theoretical difficulty with regard to Bloom’s taxonomy.

Fig. 2

Example of a forced-choice item on the Euro demand crisis

Fig. 3

Example of an open item on the Euro crisis issue including a final decision made by the test-taker

As noted, with the development of technology-based assessments, test developers face new challenges, which we considered carefully. After the item construction and technology-based implementation,² a comprehensive technology check was conducted. Sophisticated technologies (e.g., Javascript and multimedia elements) were excluded from our assessment to provide an optimal assessment condition. Moreover, because of different test screen sizes, the assessment screen was fixed. In our technology check, for example, the newly developed assessment was conducted using several browsers and different versions to exclude browser-specific differences in the layout. In a pilot study, a drop-out analysis was conducted to reveal any technical difficulties. Finally, in the pilot study, the automatically recorded assessment time of each participant was considered as an additional indicator of the function of the technology-based assessment. After the development of the technology-based assessment, a technology check was conducted by an online test to ensure an error-free assessment. This online pre-test revealed technology-based problems. Moreover, the pre-test checked, for example, whether sound was available and whether the Internet capacity was suitable.

Because we used a technology-based approach, we had the opportunity to easily implement a complex booklet design (see Frey et al. 2009) that was used with the German sample. Six different booklets that each included a combination of two of the six topics were used. Each participant was randomly assigned a booklet. Because of the small sample size in Switzerland, the use of the booklet design was not possible. Therefore, two topics—national debt and energy policy—were selected for use with the Swiss sample.

In addition to the advantages of a technology-based assessment compared to a traditional paper-and-pencil assessment approach, in the development of an instrument that includes open-response items, a technology-based approach is preferred because it improves the precision of data analysis and interpretation. In classical paper-and-pencil assessments, the reading of handwritten student data is much more time consuming and interpretations are more error prone compared to machine-aided student answers. This improvement clearly enhances the objectivity of the data analysis and interpretation.

Psychometric properties

In addition to the technology-based check, the psychometric quality of the assessment was proofed by pilot studies and within the current study.

These psychometric evaluations included different psychometric indices. We checked the psychometric assessment properties of each country. For the assessment evaluation based on the Swiss sample, the weighted mean squares and t-values were mostly acceptable. However, for the assessment evaluation based on the German sample, the weighted mean squares and t-values fit less well. A careful check revealed that two item types were critical. An in-depth item-based analysis revealed a satisfying distractor analysis. Therefore, despite the heterogeneity of the different psychometric assessment properties, the quality of our assessment instrument was acceptable (for details see Eberle et al. 2016).

For the analyses of the current study, the one-dimensional model solution of economic-civic competence was used. The reliability score was sufficient (weighted likelihood estimators (WLE) reliability 0.83).

Differential item functioning (DIF) analyses were conducted for occupations/branches within the countries and then for the countries within the occupations/branches (for detailed information, see "Appendix"). Moreover, there was gender DIF within the countries and within each respondent occupation branch within the countries.

DIF can suggest that the measured latent variable is not the same for different groups and that the answers cannot be scaled together and used for valid group comparison. However, the question concerning how to handle and interpret statistical DIF is not trivial. No common agreement exists regarding acceptable tolerance limits for the strength of DIF (Liedtke and Seeber 2015; Osterlind and Everson 2009). Furthermore, it is necessary to distinguish between construct-relevant and construct-irrelevant DIF (see Zumbo 1999). Thus, although statistical DIF is a necessary condition for item bias, it is not a sufficient one. To determine whether DIF is construct (ir-)relevant would require deeper content-related analyses (e.g., Liedtke and Seeber 2015; Spoden et al. 2015) that do not reflect the subject matter of this article. Therefore, it is not sensible to eliminate items for further analyses only because statistical DIF exists (Spoden et al. 2015; Zumbo 1999). In our paper, DIF can also be seen as a diagnostic instrument for the specific strengths and weaknesses of different groups regarding the same underlying construct that result from different learning opportunities (for details, see Scheuneman and Gerritz 1990). Although statistical DIF among the branch and country affiliations was evident, we assume curricula-driven differences rather than construct-related DIF (i.e., no item bias). Additionally, the DIF items were equally distributed among the groups. Therefore, DIF was not considered, and all items were used to calculate the WLE.

The test scores for economic-civic competence were developed and analyzed based on item response theory (IRT, see Masters 1982; Rasch 1960) using the software program “ConQuest” version 2.0 (Wu et al. 2007). For the current analyses, the WLE (see Warm 1989) of the specific personal abilities was used. Furthermore, the test scores were standardized to yield a mean value of 500 and a standard deviation of 100.

Design and procedure

The analyses are based on the data from the cross-sectional Study Modellierung und Messung wirtschaftsbürgerlicher Kompetenz (for more information, see Eberle et al. 2016).³ Data were gathered from the German samples between March and October 2014 and from the Swiss samples in the spring of 2015. The technology-based assessment consisted of a 45-min online speed test. Additionally, the trainees completed an online questionnaire that contained biographical, educational and social items (35 min). The test was administered by test administrators to groups of approximately 20 trainees in branch courses in Switzerland⁴ and in vocational schools in Germany.

In Switzerland, the research team together with the two commercial branch associations⁵ organized communications with the training firms and trainees, the arrangement of the test locations and the testing logistics. The assessments in Germany were managed by the members of the project teams of the German research cooperation partners in Network CoBALIT.

Samples

The sampling procedures differed depending on the country. The Swiss sample was representative of trainees in both surveyed occupational branches in the German-speaking part of Switzerland. In contrast, the German sample was a convenience sample of trainees in the two commercial occupations of industrial and logistic clerk.

The total sample in both countries consisted of 1255 apprentices. Table 1 shows the characteristics of the Swiss sample (n = 417). All Swiss apprentices were attending their final training year. The majority of the commercial trainees were from the industrial sector (n = 292), and there were 125 trainees from the logistics branch. As is typical of the commercial apprentice population, female trainees predominated (65%), although their majority in the industrial branch is much larger than their majority in the logistics branch (70% compared with 54%). The trainees’ mean age was 18.9 years, although the trainees in the logistics branch were approximately 1 year older, and the age variance in the logistics branch was slightly higher than the age variance among the industrial sector trainees. The distribution of the trainees’ ages indicates that most of them entered commercial apprenticeships immediately after finishing compulsory school. It is therefore not surprising that only 3% held a UED.

Table 1 Characteristics of the Swiss sample

Table 2 provides an overview of the German sample (n = 838). In contrast to their Swiss peers, approximately one-quarter (n = 229) were in their second training year, and three-quarters were in their last training year (n = 609). Similar to the Swiss sample, the majority were female trainees, although this proportion was not as high as in Switzerland. In the logistics occupation, there were more male than female students. Compared with the Swiss sample, the German trainees were clearly older, with a mean age of 21.6 years. Moreover, the age variance was much larger. The reason for this age difference was the large number of German trainees who held a UED (61%). As described in the theoretical section, in Germany, training companies prefer candidates with a UED for commercial apprenticeships.

Table 2 Characteristics of the German sample

Results

Gender differences

A well-known finding in the literature is the advantage of males with regard to economic competencies. The question was whether this finding could be replicated using the data on economic-civic competence in the present study (see Table 3).

Table 3 Country-specific competence scores by commercial occupation and gender

The male trainees scored significantly higher than their female peers did only in the German sample. No significant gender effect was evident among the Swiss trainees. In addition, the effect in Germany was small (Cohen’s d = 0.19). By specifically examining the differences within both occupations, a stronger effect can be identified among the German logistic clerks (Cohen’s d = 0.38). In contrast, no significant difference is observed among the industrial clerks.

Prior education

As described in the theoretical section, a substantial number of German commercial apprentices received a UED before entering their apprenticeships. This was the case for more than 60% of the current sample (see Table 2). It is well known that trainees with UEDs display higher levels of academic achievement than do trainees without a UED. Moreover, these young adults receive more general education than apprentices without a UED do. Thus, we expect to see a substantial advantage among learners with a UED. Table 4 shows the relevant scores from the German sample.⁶ It is obvious that the German UED holders scored significantly higher than their peers without a UED. The effect was stronger among the industrial clerks (Cohen’s d = 0.46; logistic clerks: Cohen’s d = 0.36).

Table 4 Competence scores of German trainees with and without a UED by occupation

Differences by occupation and country affiliation

Little is known about the correlations between occupation and country affiliation. Tables 5 and 6 present the relevant statistics. First, both occupations/branches within the countries were compared (see Table 5). In both countries, industrial clerks scored significantly higher than logistic clerks did, and this effect was slightly higher in Switzerland (Cohen’s d = 0.28 compared with 0.21).

Table 5 Country-specific competence scores by commercial occupation/branch

Table 6 Occupation/branch-specific competence scores by country affiliation

Second, we compared the country-specific competence scores within the corresponding occupations/branches. A slight advantage of the Swiss trainees was revealed, although these differences were not statistically significant (see Table 6, upper section). As reported above, apprentices with a UED score substantially higher. Given the high proportion of UED-holding trainees in the German sample and their small proportion in the Swiss sample, we analyzed the country differences using only the data from trainees without a UED (see Table 6, lower section). Among this subgroup, the Swiss trainees clearly scored significantly higher. The expected effect was slightly stronger among industrial clerks (Cohen’s d = 0.38 compared with 0.28).

Discussion

Citizens in modern democratic societies are confronted with complex challenges. Many of these demands are related to economic issues, which can occur on an individual level and on a societal level. Citizens typically act as consumers, savers, credit users, voters, and employees. From our perspective, it is highly desirable for citizens to be able to competently manage these demands in various contexts. We call this ability economic-civic competence.

Given the substantial lack of research, our study sheds new light on economic-civic competencies. This study used a new technology-based assessment to analyze in detail the economic-civic competence of commercial apprentices in Germany and Switzerland. Until now, there have been almost no empirical findings regarding this topic. In both Swiss and German commercial apprenticeship programs, the fostering of this competence is promoted by vocational schools. However, the extent of the curricula and lecture hours are more intense in Switzerland.

Summary and conclusions

Our main findings are summarized as follows.

• 1. Gender differences The advantage among male trainees, which was expected based on previous studies concerning economic competence (e.g., Brückner et al. 2015; National Center for Education Statistics 2013; Schmidt et al. 2015; Schumann and Eberle 2014; Walstad 1994), was confirmed only among German logistic clerks. The expected gender effect was not found among German industrial clerks or Swiss trainees.

• 2. Prior education In contrast to the Swiss trainees, a considerable number of German apprentices had acquired a UED before attending commercial training. Typically, these trainees display higher academic achievement than do trainees without a UED (Lehmann and Seeber 2007; Schumann and Eberle 2014; Seeber et al. 2015; Spoden et al. 2015; Winther 2010). Our results confirm the assumed higher scores among the German trainees with a UED. The effects are small among logistic clerks and moderate among industrial clerks.

• 3. Occupation/branch affiliation In both countries, industrial clerks scored significantly higher than logistic clerks did. This effect was slightly stronger in Switzerland. As stated in the theoretical background, the main question is whether the differences between the two commercial occupations/branches are affected mainly by program influences or by learners’ prerequisites. Considering the robust findings from several studies that students who experience more economic learning opportunities score higher on related tests, the advantage for industrial clerks in Germany can be explained by the approximately double number of curriculum-based hours on related content. However, the greater difference between Swiss industrial and logistic clerks cannot be explained by different learning opportunities because they are taught in vocational schools with the same curriculum in the subject of E&S. Therefore, learners’ preconditions must be scrutinized. One explanation may be the greater popularity of industrial clerk apprenticeships than of logistic clerk apprenticeships. Trainees with better cognitive and motivational preparedness are deciding to apprentice as industrial clerks, and the training companies of the industrial branch have the opportunity to choose among a larger group of better educated and motivated apprentices (the cascade effect). This assumption can be confirmed by the significantly higher math and language scores of industrial clerks than of logistic clerks in Switzerland. However, these findings are not reported here in detail because the data on these abilities are available only for a small subsample.

• 4. Countries In general, the Swiss trainees scored higher than their German counterparts did within a comparable occupation/branch. This expected advantage was significant and notably stronger among the trainees without a UED. A plausible explanation is the broader and more explicit promotion of economic-civic competence in Swiss vocational schools in commercial apprenticeships. Commercial trainees in Switzerland receive significantly more instruction in related content than their German colleagues do.

Limitations and outlook

Due to several methodological restrictions, the findings should be viewed with caution. Significant limitations regarding the design, sample selection (in Germany) and few control variables must be noted. First, our study design does not allow us to identify causal relations among the score differences. We do not know enough regarding the interplay of individual, institutional and contextual variables and their effects on economic-civic competence within the observed population. In times of increasing interest in large-scale international comparisons of educational outputs, international differences should be viewed with caution. This is particularly true with regard to the use of the convenience sample in Germany. In contrast, the Swiss sample was selected more systematically. Moreover, additional control variables were not used to check the robustness of the findings. Again, we emphasize that only the cognitive aspect is considered in this study. Future research should also include a non-cognitive component, which is implied in the competence concept (see also Klieme et al. 2007; Weinert 2001), especially economic-civic competence (see Eberle 2015).

Given that our study introduces a technology-based assessment comparison of civic-economic competence as well as the study’s results and methodological restrictions, several conclusions can be drawn. The findings should be interpreted primarily as explorative depictions of the characteristics and group differences that control economic-civic competence. Thus, the study can be used as a basis for further research on this topic. Open questions, such as the unexpected weakness of the gender effect or the identified branch and country differences, should be studied with more precise study designs and sampling. Sampling should be performed with a more systematic description of the basic population. In future efforts that aim to identify causal effects (i.e., selection and socialization effects), multi-perspective and multilevel longitudinal studies could be appropriate ways to obtain more knowledge about the genesis of economic-civic competence. This knowledge is clearly needed to understand the complexity of economic-civic competence in an effort to measure it with assessments other than technology-based assessments.

Our study can be seen in the broader context of the contemporary interest and challenges in developing technology-based assessment within the educational field. The focus of technology-based assessment reveals many open questions that require additional research, such as comparisons of the data quality of technology-based assessment and traditional assessment approaches. Our contribution highlights the advantages of a technology-based approach with regard to design development and data analysis. In our technology-based assessment, open questions were coded by experts. However, future applications should consider the automatic coding approach as an alternative. Coding open questions using an automatic coding approach would support the application of our technology-based instrument and increase knowledge of the complexity of economic-civic competence.

Appendices

Appendix

See Figs. 1, 2, 3, 4 and 5

Fig. 4

DIF item with an advantage for industrial clerks in Germany (0.96 logits less difficult)

Fig. 5

DIF item with an advantage for Swiss learners within the industrial clerks (0.94 Logits less difficult)

Detailed information on DIF

(1) Occupations/branches within countries

Sixteen items in Germany and 11 items in Switzerland were identified with DIF. In Germany, 8 of the 16 items provided an advantage for the industrial clerks, and 8 provided an advantage for the logistic clerks. In Switzerland, DIF was equally distributed across the branches: 5 items provided an advantage for the industrial clerks, and 6 provided an advantage for the logistic clerks. An advantage means that the item difficulty is lower for this group than for the other group. The differences in item difficulty between the two groups varied between 0.26 and 1.26 logist, which is approximately 0.5 to 2.3 times the standard deviation of the latent variable (Germany_SD = 0.84, Switzerland_SD = 0.54). The share of items that provided an advantage for the reference respondents of the focal group is approximately 50/50. The following are examples of the DIF items (the items with the strongest DIF) that were given one time with an advantage for the industrial clerks and one time for the logistic clerks for each country separately (Fig. 4).

(2) Countries within the occupations/branches

For the industrial clerks, 18 items with DIF were identified, and for the logistic clerks, 13 items with DIF were identified. For the industrial clerks, 7 items provided an advantage for the German learners, and 11 items provided an advantage for the Swiss learners. For the logistic clerks, it was more equally distributed: 6 items provided an advantage for the German learners, and 7 items provided an advantage for the Swiss learners. The differences in item difficulty varied between 0.19 and 1.35 logits, which is between 0.23 and 1.60 times the standard deviation of the latent variable (industrial clerks_SD = 0.82, logistic clerks_SD = 0.84). Within the groups of industrial clerks, there were more items that provided an advantage for the Swiss learners than for the German learners. The following section presents examples of the DIF items (the items with the strongest DIF) that were given one time with an advantage for German learners and one time with an advantage for Swiss learners for each occupation/branch separately (Fig. 5).