What makes an online learning journal powerful for VET? Distinguishing productive usage patterns and effective learning strategies

Abstract

Background

The current and future workforce is challenged to adapt to changing environments and become lifelong, self-regulated learners. Learning journals can regulate learning processes through scaffolding reflection on rich experiences. However, learning journals are not effective per se. This study investigates the effects of an online learning journal platform, called ‘LearnDoc’, that provides scaffolds for learners in their initial vocational education and training. The specific research questions addressed in this paper are what usage patterns emerge from learners’ use of the LearnDoc platform, when do learners create their journal entries, and how do usage patterns and procrastination relate to learners’ final exam performance.

Methods

The LearnDoc platform has been used throughout a 3-year training program by a cohort of Swiss vocational students (N = 132). Different usage patterns (intensity and pacing) and learning strategies (metacognitive and non-metacognitive) were distinguished through a mixed methods approach.

Results

Results indicate a significant link between learning journal pacing patterns and learners’ final assessment performance. Learners who finished with higher grades tended to start generating their learning journals earlier and made more use of the learning strategies “planning, monitoring, debugging” while weaker students focused more on “evaluation”. Findings suggest that students’ perceptions of the purpose of learning journals influence their pacing strategies and selection of learning strategies.

Conclusions

These results are valuable for the design of effective scaffolds for a better and a more reflective usage of learning journals in vocational education and training.

Background

Vocational education and training (VET) systems are challenged to prepare current and future workers not only to excel at routine work but also to be able to adapt to complex and changing work environments (Dall’Alba 2009). There is an increasing demand from employers for workers with more integrated knowledge that allows them to understand the whole labor process and to deal with new and unpredictable situations (Ertl and Sloane 2004). To adapt to changing workplace demands, learners must become self-regulated lifelong learners that can maintain and expand their competences. Lifelong learning requires the capacity to regulate one’s own learning through self-reflective metacognitive processes (Van den Boom et al. 2007). During the apprenticeship period, reflective practice can be modeled and facilitated through learning journals (LJs) that prompt reflection on experiences made in the workplace or at school (Nückles et al. 2009). Learning journals are used by learners to track their progress and support their reflections of workplace experiences (Mauroux et al. 2015). Different advantages of learning journals have been reported. Among others, they contribute to help learner connecting theory and practice (O’Connell and Dyment 2011), to foster deep or strategic learning (Entwistle 2000), and to promote self-regulated learning (Chang et al. 2015; Tanner et al. 2000). However, learning journals need scaffolds to be efficient for learning (Berthold et al. 2009; Eraut 2007; Hübner et al. 2010; Kicken et al. 2009). Thus, the open question is not anymore if using learning journals can be beneficial but if specific ways of using learning journals have a larger impact on student learning than others. In particular, building on the above-mentioned large body of studies supporting the importance of self-reflection for a better learning, we expect to confirm a positive correlation between learners who make more extensive use of their learning journals to reflect on their level of mastery of professional procedures and their final exam grades. This study also explores the related question if learners differ in their pace of creating their learning journal entries throughout their entire training. For self-reflection to be effective, it cannot be completed at the end of the training but must beginn much earlier to serve as a guide towards a well-paced progression in one’s learning. Therefore, learning journal entries should be created early in the training and not only at the end of one’s training. The literature on learning journals and portfolios (Moon 2013) reports a tension between the summative function (helping learners to show how good they are) (Zubizarreta 2009) and the formative function (helping them to keep track of and reflect on their learning needs) (Boud 2001). While the former function can still be achieved when starting late in the learning process, the latter function requires an early start in the process. Drawing on the literature on procrastination (Steel 2007; Tice and Baumeister 1997; Wäschle et al. 2014), we expect less successful learners to delay the generation of their learning journal entries while stronger students understand the benefits of setting up a learning journal from the beginning—or at least early in their training.

Research questions

This paper reports the effects of an online platform called ‘LearnDoc’, which was developed to provide scaffolds for learners to generate reflective learning documentations throughout their vocational education and training during 3 years. LearnDoc has been developed in collaboration with the Swiss professional association for bakery and confectionery (SBC) and deployed to all vocational schools in the country. LearnDoc aims to make generating a recipe book and a learning journal easier. Building on a pilot study (Mauroux et al. 2015), this paper extends this line of research by including a larger sample over a longer period of time. The current study merges quantitative analyses of logging data extracted at different times during the training program, content analysis of learning journal entries, and final exam grades.

The research questions addressed in this paper are (1) ‘What kind of usage patterns emerge from learners’ use of the LearnDoc platform?’, (2) ‘When in their training and at what pace do learners complete their recipe book and their learning journals?’, (3) ‘How do different usage patterns relate to learners’ summative assessment performance?’, and (4) ‘Is procrastination to complete their recipe book and/or learning journals related to lower performance at the final examination?’.

Theoretical background

Experiences and reflection

In the so-called “dual vocational education programs”, learners (apprentices) spend the major part of their training in real workplaces, where they make rich, situated experiences. However, experiencing per se is not enough to learn—one needs to reflect on experiences (Dewey 1938; Engeström et al. 1995; Kolb 2014; Walker et al. 1985). The necessity to reflect on experiences is present in numerous views, such as learning by doing (Dewey 1938), experiential learning (Kolb 2014), or workplace learning (Billett 2004; Tynjälä 2008). The capacity to reflect on work experiences is fundamental for professionals to maintain, update and extend their competences (Van den Boom et al. 2007). However, there is often only limited time or opportunities for reflection during the activity (reflection-in-action) or immediately after it (reflection-on-action). Capturing experiences in artefacts (such as texts or photos) can allow reflection at a later stage (Challis 2005; Schön 1987). This analysis of past actions and their consequences informs future actions and is a fundamental workplace learning process (Driessen et al. 2008).

Learning strategies and self-regulated learning

Writing learning journals can foster the development of learning strategies (McCrindle and Christensen 1995), which is essential to becoming a reflective practitioner (Schön 1987). Learning strategies can be described as “behaviors and thoughts that a learner engages in during learning and that are intended to influence the encoding process” (Weinstein and Mayer 1986, p 315).

The ability to mobilize adequate learning strategies contributes to the development of self-regulated learning. The link between self-regulated learning and metacognition is also present in the work of Azevedo and Boekaerts. Azevedo (2005) defines self-regulated learning as an active, constructive process whereby learners set learning goals and then attempt to monitor, regulate, and control their cognitive and metacognitive processes in the service of those goals and Boekaerts (1999) describes self-regulated learning as a composite that includes (among others) the use of metacognitive knowledge and skills to direct one’s learning. The definition of self-regulated learning by Hadwin et al. (2011) highlights the connection between learning strategies and self-regulated learning as “a learner’s deliberate planning, monitoring and regulating of cognitive, behavioral and motivational/emotional process toward completion of an academic task/goal” (p 68).

Learning journals as reflection tools

One widely used tool to capture experiences and facilitate reflection is the learning journal. Use of learning journals has been reported as instruments to support self-directed learning in a wide variety of professional educational settings, including nursing (Epp 2008; Könings and Gijselaers 2015), physiotherapy (Wessel and Larin 2006), teacher education (Darling 2001; Dyment and O’Connell 2014; Mansvelder-Longayroux et al. 2007), music education (Baker 2007), physical education (Tsangaridou 2005), design (Betrabet Gulwadi 2009), information and technology education (Gleaves et al. 2007), and medicine (Boenink et al. 2004; Driessen et al. 2005). Different benefits of LJ writing have been identified (Dyment and O’Connell 2010; Hubbs and Brand 2005; O’Connell and Dyment 2011). Learning journals can track learners’ progress and learning path over time (Wade and Yarbrough 1996); take on the responsibility for selecting traces, making connections and interpreting their own learning (Lamont 2007), and to consider how experiences can be applied to future situations (Ghaye 2010; Minott 2008). LJs can positively affect competence development (Brown 2009) and self-regulated learning (Chang et al. 2015; Tanner et al. 2000) through metacognitive activities (Lew and Schmidt 2011; McCrindle and Christensen 1995) in which the learner critically analyzes knowledge, skills, and dispositions in different contexts (Connor-Greene 2000). Over time, a collection of LJs can document progressive learning gains as well as competences and experiences. As LJs are designed to focus learners’ attention on their understanding of professional procedures and to encourage reflection on action, LJs could encourage a deep or a strategic learning approach (Entwistle 2000). Such approaches are known to be linked to the quality of learning.

Learning journals are being increasingly used in vocational education and training (VET). In VET, learning journals not only support reflection on workplace experiences but also help tracking learners’ signs of progress. Mauroux et al. (2015) also suggest that LJs can assist in connecting theoretical and practical knowledge acquired in different contexts. Learners who generated LJs more frequently achieved higher results in their final examination of their professional skills, especially when prompted to elaborate and reflect on demanding tasks. In Switzerland, most sectors of VET ask learners’ supervisors to regularly discuss and evaluate their apprentices’ learning journals, a recommendation well in line with the research literature on the subject (Driessen et al. 2005; Kicken et al. 2009; Van den Boom et al. 2007). Mentoring is considered an important condition of effective reflective LJs writing (Lamont 2007). Könings and Gijselaers (2015) found that reflecting on workplace experiences in LJs is useful for the professional development of medical doctors, but that well-designed support is needed to raise awareness of valuable moments of learning and to reflect on them afterward. These initial, promising studies indicate that further research on LJs as reflection tools in vocational training are still needed.

Challenges of learning journals in VET

Using learning journals in VET contexts can be challenging for several reasons. First, the production-oriented rationale of workplaces (Illeris 2011) definitely favors what Schön (1987) called reflection-in-action over reflection-on-action. Obviously, workplaces leave often little time or limited opportunities to discuss with colleagues or supervisors directly after completing a task (reflection-on-action). Second, learners often do not perform all steps required to produce a complete product but only execute isolated segments. Their experiences are often scattered across time and contexts (Mauroux et al. 2015). To understand a whole procedure or production sequence, learners need to integrate those fragmented workplace experiences. Third, learners are commonly asked to document only idealized professional procedures (standardized routines), for example a chef’s recipe, but hardly reflect on critical incidents, including when something went wrong or happened unexpectedly. As the production-oriented rational at the workplace discourages critical incidents or mistakes, it might, therefore, seem uncommon to document errors in learning documentations. Fourth, writing is a cognitively demanding process, especially for novices (Galbraith 2009). Particularly in VET, where learners might lack proficiency or show reluctance for reflective writing, it becomes important to investigate effective scaffolds (Schneider 1993).

Scaffolding for reflection

Reflecting on workplace experiences is a difficult task for learners without explicit guidance (Taylor and Freeman 2011) and does (mostly) not occur spontaneously (Jong et al. 2006; Könings and Gijselaers 2015). Many learners find reflective journal writing difficult as they often offer little guidance in structure or content (O’Connell amd Dyment 2013). Learners tend to use journals to report descriptive accounts of events with no deeper reflection of underlying meaning (Dyment and O’Connell 2011; Hargreaves 2004; Minott 2008; O’Connell and Dyment 2004; Richardson and Maltby 1995; Ryan 2013; Wessel and Larin 2006). Learners need tools and scaffolds to report and reflect on their experiences (Berthold et al. 2009; Hübner et al. 2010; Kicken et al. 2009). Prompts have been studied as scaffolds that aim to structure learners’ reflections (Nückles et al. 2009). Prompting learners to write for reflection can be a creative process contributing to change the writer’s understanding of the topic (Bereiter and Scardamalia 2013). Prompts can refer to successful or unsuccessful experiences.

Learning journals: from paper-and-pencil to online formats

Currently, many people still use pen and paper to capture and annotate their experiences (Creme 2005; Hoff et al. 2009). Online learning journals promise several advantages over paper-based journals (Chang and Lin 2014). Dehler et al. (2011) compared paper-based LJs with online LJs and found that online learning journals can facilitate capturing, editing, and storing artefacts. Learners indicated that they considered online LJs generally easy-to-use and useful. Gleaves et al. (2007, 2008) found that learners using online learning tools generated more entries and were more willing to engage in reflection than participants who used paper-based journals (Caruso et al. 2016). Online LJs can be accessed location-independent, stored securely, easily accessed by multiple people, include different media (for example photos (Mauroux et al. 2015) or videos (Motta et al. 2013), and offer reflection prompts to guide learning processes (Kicken et al. 2009).

Methods

Design of the online platform LearnDoc

LearnDoc is a web-based platform allowing learners to complete a recipe book (collection of recipes) and to associate each of these recipes with a learning journal. The collection of recipes constituting the learner’s recipe book is summarized in an overview page, which sorts the recipes by type and by the current degree of mastery, as assessed by the learner and by the supervisor. The recipe book serves as a record of the recipes practiced in the company in which the learner accomplishes the practical training (generally 4 days a week for 3 years). Learners are motivated to complete it not only because it forms a personal collection useful for future work but also because it can be used as a resource for the final qualification exams. Learners enter their recipes in a preformatted document describing an idealized procedure. This document (see the left side of Fig. 1) contains specified fields for entering the name and type of the product, the name of the company, ingredients, baking temperature and time needed as well as the step-by-step procedure. Each recipe can be enriched with a so-called “ideal picture”, a picture selected by the learner that shows the ideal final product (see the top left corner of Fig. 1). Each recipe is linked to a learning journal (see right side of Fig. 1), which prompts learners to reflect on their experiences and level of mastery. This can be done in various ways—by inserting pictures of intermediary stages or by answering prompts. The prompts presented in the learning journal were inspired by Kicken et al. (2009) and Mauroux et al. (2014). The four prompts are: “I already master…” (assessment of own competence), “I still have to learn…” (identify learning needs), and “In order to improve, I will…” (select an appropriate learning task), as well as a more open procedural prompt ‘In this recipe, I have to pay special attention to…’. Our prompts were not designed to elicit one particular type of learning strategy. “I already master…” is a way to allude to the fact that the learner does not need help and is performing at the expected level (resource management); “I still have to learn…” might prompt a cognitive strategy (see definition in Table 1, “repetition”) while “In order to improve, I will…“could prompt a metacognitive strategy.

Fig. 1

An example of a recipe (left), including an “ideal picture, and of a learning journal (right) including pictures presenting intermediate states as well as answers to prompts

Table 1 Coding scheme for learner’s learning journal entries (final version after intercoder agreement)

Participants

Keeping a recipe book and a learning journal is a requirement for all bakers, chocolate makers, and pastry chef apprentices in Switzerland (Professional Ordinance of Bakery and Pastry Cook Training 2018). The LearnDoc platform has been made available to all learners in this profession, but they could freely decide to use LearnDoc or not. In LearnDoc, learners could use only the recipe book, only the learning journal, both, or neither. The first cohort of learners who could use it throughout their training program (3 years of training) passed their final qualification exam in 2014. The cohort consisted of 567 learners. In the present study, only learners who created complete entries (by meeting the triple condition of having generated their complete recipe book with LearnDoc, having inserted at least one picture in their learning journal, and having given at least one answer to one of the prompts provided) were included in the analysis (N = 132). The final sample includes 47 men and 85 women; they had completed their training in more than 100 different workplaces and come from all linguistic regions of Switzerland.

Data collection

Contents of LearnDoc

The full content of the LearnDoc database was extracted for analysis twice a year throughout the 3-year program. Each dataset includes data concerning the number of recipes posted on LearnDoc by each learner, the number of photos attached to a given recipe, the number of recipes with an ideal picture (see Fig. 1) as well as whether or not a learning journal has been linked to a given recipe (at the time of data extraction). The values of these indicators at the end of the training program will be used to indicate the learners’ intensity of use of LearnDoc.

By comparing these values to those observed earlier in the training, we will be able to assess what we call the usage pacing of LearnDoc.

To assess how early or how late each learner completed his or her learning documentation, it was decided to compare the content of the database at the end of the training to that of the database extracted midway in the training.

Additionally, a qualitative analysis of the content of all learners’ answers to the prompts proposed by the learning journals was conducted at the end of the training, to identify specific learning strategies. These strategies were coded based on the categories proposed by Schraw and Dennison (1994), Weinstein and Mayer (1986) and McKeachie (1987). The examples were taken from our own data.

Six researchers conducted the coding. After an initial phase of training, coders were divided into pairs; each pair of coders evaluated an initial random sample of 100 entries. An agreement test between the pairs was then conducted, which showed a fair inter-coder agreement (average Cohen’s kappa coefficient .88). Potential disagreements were resolved through discussion within the group of coders and by revising the initial coding scheme. After this agreement phase, coders conducted their coding independently. Overall, 3234 entries were coded. The vast majority of the entries were very short and only referred to one learning strategy. In the rare cases where more than one learning strategy was evoked in the same entry, two or more codes were attributed to it.

Learners’ final exam grades

At the end of their training, learners have to pass a final exam developed by the professional association and conducted by a body of expert examiners and teachers. It consists of a practical exam performed at the learner’s workplace, a professional knowledge exam as well as a general knowledge exam (related to such subjects as first and second languages, economics, law, ethics, and technology). Prior to the practical exam, examiners selected a certain number of recipes from the candidate’s recipe book to be executed in front of them during the exam. The overall grade was computed on the following, official scheme: practical exam (weight 5), professional knowledge exam (weight 2), general knowledge exam (weight 2), and a so-called “experience grade” (average grade obtained at school the year before) (weight 1). Our analysis will use this overall, weighted grade as it determined pass or fail of learner’s final assessment.

Procedure

Our analysis will be conducted in two phases. In the first phase, we will identify the different usages of the platform, based on the intensity and pacing of use of its various features and on the learning strategies recognized in learners’ learning journals. In the second phase of our analysis, we will match LearnDoc usage patterns with learners’ final exam results to explore whether specific LearnDoc usage patterns correlate with better results in the final exam. This study applies a mixed methods research design (Smith 2006), combining quantitative and qualitative content analyses. Repeated measures analysis of variance will be conducted in order to establish the pace at which learners completed their dossier throughout their training. Correlational analysis will be performed to look for two-way relationships between the intensity of use of LearnDoc (numbers of recipes presented in their recipe books, number of pictures inserted, etc.), the various learning strategies detected in the learners learning journals, both halfway through training and at its end, as well as the overall grade obtained by each learner for the final qualification exams. Finally, a hierarchical regression analysis will be performed to analyze whether learners’ overall grades at their final exams can be predicted by the way they had used LearnDoc throughout their training and the learning strategies they had mentioned in their learning journals.

Results

To address our research questions, the results will be presented in two sections. We will first present the content analysis of the LearnDoc database to distinguish usage patterns (intensity and pacing) from the automatically generated codes as well as to distinguish learning strategies through the qualitative analysis of the content of the learning journals. In the second section, we will correlate these data to learners’ summative assessment performance to explore which usage of LearnDoc and which learning strategies best predict the grades learners received at the end of their training.

Distinguishing usage patterns of the LearnDoc platform

What is the intensity of use of the specific elements of the platform?

Although large differences could be seen at the end of the training in the log files between learners who used LearnDoc minimally and those who used it extensively, learners tended to use the possibility to simply describe recipes most (M = 113.83 at the end of the training), inserted a fair number of pictures (M = 91.77) and only answered prompts in the learning journals in approximately two-thirds of their recipes (M = 77.97). The least frequently used feature was the option to insert a so-called “ideal picture” of the product (M = 57.11). Only approximately half of the recipes displayed in the learners’ recipe books at the end of the training included with such an `’ideal picture”.

Interestingly, the standard deviations remain small for the number of recipes completed (SD = 3.66), with or without an ideal picture (SD = 3.16). They are much larger for the indicators related to the use of the learning journal—respectively for the total number of pictures inserted (SD = 7.78) and the number of reflection prompts answered (SD = 31.58). This is, in part, due to the fact that the professional association has given clear instructions regarding how many recipes of each type a typical recipe book should contain at the end of the training. On the contrary, no specific indication had been given regarding how the learning journal should be used. Another possible explanation for this difference could be that the recipe book had to be presented to the experts in the final exams, while the learning journals could remain private and used as a self-help during the practical exam.

Which learning strategies are evoked the most?

The vast majority of the learning strategies mentioned in answer to a prompt were metacognitive (2689; 83.2%), followed by cognitive ones (409; 12.6%); entries coded as resource management were comparatively very rare (136; 4.2%). This is an interesting finding as our prompts were not designed to elicit mostly metacognitive learning strategies. Only one of the four prompts was metacognitive. However, results show that 83% of all used learning strategies were metacognitive (see Table 2).

Table 2 Percentages of each learning strategy detected in the entries of a learner’s learning journals (mean percentages) and correlations between individual learning strategies and final performance

In total, evaluation is, by far, the most frequently mentioned learning strategy, followed by monitoring and planning (three metacognitive learning strategies). Repetition is the most observed cognitive strategy, actually not so much less frequent as planning and twice as frequent as debugging.

When is the platform used during training?

As already mentioned, our second research question deals with the pacing of the learners’ use of LearnDoc. Since we extracted the content of the log files every semester,¹ we are in a position to track the progression of each learner’s recipe book and its attached learning journals in certain intervals. As the purpose of this study is to contrast early starters with those who started documenting their learning later, we have decided to compare the use of LearnDoc during the first half of the training to that of the second half by contrasting the numbers of recipes, pictures, ideal pictures and answers to prompts after three semesters (column 2 of Table 3) to those observed at the end of the training (i.e. after 6 semesters, column 4 of Table 3).

Table 3 Mean numbers of recipes developed, pictures included, ideal picture inserted, and recipes with an answer to at least one of the prompts presented in the learning journal at the end of the training and half-way through it

Clearly, all the ratios computed between the corresponding numbers displayed in column 2 and column 4 of Table 3 stay below .5, attesting that, on average, learners mostly tended to complete LearnDoc in the second part of their training. A repeated measure analysis of variance contrasting the production of the first half with that of the second half of the training (end of training—first half of the training) shows very large and significant increases in all the indicators of use (respectively F_(1.183)= 134.275 for number of recipes, F_(1.183)= 261.344 for number of pictures, F_(1.183)= 251.15 for number of recipes with ideal-pictures and F_(1.183)= 307.993 for numbers of recipes with at least one prompt answered in the learning journal). This leads to the conclusion that the summative function of LearnDoc—i.e. displaying how good you are and what you master—appears more crucial than its formative function—using one’s recipe book and learning journals to record what you still need to improve. At the same time, it confirms our hypothesis that towards this activity, learners actually present a high tendency to procrastinate. Nevertheless, important differences exist between learners in the proportions of recipes, pictures, recipes with ideal-pictures, and learning journal entries they had already produced in the first half of their training. While some learners did not generate any recipes during the first half of their training, others had already completed all their recipes during that time. Similarly, some learners had no pictures in their database at midterm while others had already entered ¾ of them halfway in their training. In the next section, we will explore whether these differences in the pacing of their usage of LearnDoc relate with overall grade differences at the final exams.

Another important observation can be made when looking at Table 3 vertically. Clearly, learners made during the second half of their training a proportionally more intensive use of pictures and ideal pictures than what they had been doing in the first half of their training. While the number of pictures learners inserted reached 80% (i.e. 91.77/113.83) of that of the recipes described in the book at the end of the training, this ratio remained low (approx. 35%) in the first half of the training. The same is true with respect to the ideal picture accompanying the presentation of a recipe. While ideal pictures ornate approximately 80% of the recipes at the end of the training, only 17% of the recipes included such a picture mid-way through the training. This is probably due to the fact that the products, which they produced early in their training were not considered “worth a picture”, while those towards the end of the training were of higher quality. Interestingly, the number of recipes with answers to a prompt displays quite a different picture: the recipes completed in the first half of the training did show answer(s) to prompt(s) in the attached learning journals almost as often as those inserted in the second half of the training (62 vs 68.5%).

Usage of LearnDoc and of learning strategies in relation with final grades

The third research question addressed in this paper focuses on the relationship between the ways learners used LearnDoc throughout their training and their final exam grades. This section extends the data reported so far by correlating the various patterns of use of LearnDoc to the performances demonstrated by the learners at their final qualification exam. Again, these observations should not be taken as an indication of which use of LearnDoc led to better results—since no pre-test of abilities were collected before the start of the training. The aim of these analyses is to identify differences in the use of LearnDoc and of learning strategies between ‘stronger learners’ (learners with higher overall grades) and ‘weaker learners’ (learners with lower overall grades). This will be done in two steps. First, using a correlational approach, we will relate learners’ overall grades with specific indicators of the intensity of their use of LearnDoc, both at mid-point and at the end of the training, as well as with the types of learning strategies mostly seen in their learning journals. Second, a hierarchical regression will be conducted in order to see how much each of the three aspects examined here—the intensity of use of LearnDoc, the pacing of its use throughout training as well as the learning strategies evoked—contributes to the prediction of learners’ overall grades at their final exam.

Globally, positive correlations can be observed between all indicators of the intensity of use of LearnDoc and final grades obtained at the end of the training (right half of Table 4). This indicates that stronger learners tended to invest more effort in completing their recipe book and their learning journals. Again, the critical elements are not so much the number of recipes inserted in the recipe book or the number of answers to prompts in the learning journal but the number of pictures and ideal pictures added to these recipes.

Table 4 Pearson correlations between various indicators of the intensity of use of LearnDoc, both half-way through training and at its end, and learners’ overall grade at the final exams

Stronger learners paid indeed special attention to inserting pictures and adding ideal pictures to their recipes. More than merely the intensity of use of LearnDoc, it is what we could call the “richness of its use” that seems to be the relevant characteristic of a strong learner’s use of LearnDoc.

The fourth and last research question addressed in this paper concerns the possible relation between the pacing of completion of one’s recipe book and learning journal and the grades obtained at the final exams. In other words, do early starters also have better results than those procrastinating in starting to complete their recipe book and their learning journals. An examination of the correlations between the indicators of use of the platform half-way through the training and the grades obtained at the final exams also allows for an interesting observation. While learners’ final grades were not significantly correlated with the number of recipes at the end of the training, the corresponding correlation mid-way into the training was significant (r = .266**). This indicates that stronger learners tended to write down recipes in their recipe book earlier than weaker learners. But as the correlations with the other indicators of use are not significant yet (see columns 3, 4 and 5 of Table 4), this also suggests that stronger learners although they started earlier than weaker students to deal with their recipe book, they only completed their learning journals at a later stage, i.e. mainly in the second half of their training. It seems that stronger learners do complete their entries in successive steps, writing down the recipes first and illustrating them with pictures and ideal pictures only later on, while weaker learners tend to complete their recipes in one shot, towards the end of the training only.

Pearson correlations indicate that learning strategies are all but one (self-management) positively related to the overall grades received by the learners at their final exams. While some of the correlations are very weak (see Table 2), significant correlations exist for the sum of all strategies (“Total strategies”) as well as for the sum of the metacognitive learning strategies (“Total metacognition”). Interestingly, not all of the strategies of that type are significantly correlated with the learners’ grades; only, planning, monitoring and debugging do present significant correlations with the learners’ overall grades, while evaluation does not. Organization of work environment also significantly correlates with better grades at the final exams but this observation has to be taken carefully since the mention of such a learning strategy was seen in less than 1% of answers the learners gave to the prompts they were presented with.

So far, we could show that the intensity of use of LearnDoc, the precocity of its use during training as well as the metacognitive learning strategies evoked in a learner’s learning journals are all related to better grades at the final exams. We will now investigate how these different dimensions actually contribute to a better prediction of a learner’s overall grade. This will be done by conducting a Hierarchical Linear Modeling analysis (Osborne 2000; Woltman et al. 2012) on the data, with overall grades as the dependent variable and the others as independent. As an indicator for the precocity of use of LearnDoc or, put in another way for the learner’s capacity to avoid procrastination in completing his or her recipe book, we selected the ratio between the number of recipes already inserted halfway into the training to that of the total number of recipes contained in a learner’s recipe book when he or she finished the training. As the indicator of a recipe’s completeness or richness of a learner’s recipe book, we chose the number of recipes displaying an ideal picture on its upper left corner at the end of the training. Finally, as the indicator of the quality of a learner’s reflexion on his or her recipes, we selected the number of prompts they had answered evoking any metacognitive learning strategy. Results of this analysis will be presented in Table 5.

Table 5 Contributions to the learners’ overall grades at the final exams by various groups of variables extracted from LearnDoc

The first model shows that precocity of use of LearnDoc alone accounts for 5% (p < .01) of the variability of the learners’ overall grades at the final exams. The combination of the precocity of use of LearnDoc with the richness of use explains 8.8% (p < .01) of the variation of the final scores (Model 2). Finally, the addition of the variable “reflection on mastery” to the other two variables again improves the model (Model 3). Together, these three independent variables explain 11.2% (p < .01) of the variability of the learners’ overall grades at the final exams. All three groups of variables—the precocity of use, the richness of use and the level of reflection learners made on their mastery of the recipes presented—actually contribute, although mildly, to the prediction of a learner’s final grade. The inclusion of an additional variable does not generally lead to a reduction of the impact of the other variables on the learners’ results at their final exams.

Discussion and conclusion

Keeping a learning journal is a popular practice wherever professional competences must be acquired, from nursing education and teacher training to vocational education (Kicken et al. 2009; Könings and Gijselaers 2015). Learning journals have been shown to lead to better learning (Brown 2009; McCrindle and Christensen 1995), by engaging learners in a continuous reflection about their learning (Driessen et al. 2005; Mansvelder-Longayroux et al. 2007), and triggering the adoption of metacognitive learning strategies (Azevedo et al. 2010). As learners’ reflections are stimulated by the use of prompts (Berthold et al. 2007; Mauroux et al. 2015), more and more researchers integrate prompts in learning journals to promote reflection and trigger the use of learning strategies (Roelle et al. 2017), both cognitive and metacognitive (Berthold et al. 2007; Roelle et al. 2017).

In a previous study, we have already shown how prompts can stimulate the evocation of metacognitive learning strategies and how the regular mention of such learning strategies can be related to better learning outcomes and performances at the final exams (Mauroux et al. 2015). The present study extends that pilot study both in terms of the number of learners concerned as well as with respect to its duration. This confirms that asking learners to reflect on their level of mastery of a given process (“I already master ….”), their current difficulties (“I still have to learn…”), and to think how they could overcome them (“In order to improve, I will…”) through prompts stimulates the evocation of learning strategies, a practice found to be related to better grades in a previous study (Mauroux et al. 2015). However, our results indicate that not all learning strategies are equally effective; evoking metacognitive learning strategies such as “planning”, “monitoring”, or “debugging” did indeed relate to better grades in the final examination while focusing on other learning strategies did not. Additionally, our data show that the choice of metacognitive learning strategies aligns with the pacing of learning journal generation. Reflections conducted at the beginning of the training contained significantly more planning, monitoring and debugging while procrastinating reflections made towards the end of the training tended to be limited to mere global evaluations of one’s mastery of the whole recipe. The main contribution of this study is the importance it gives to appropriate pacing in completing one’s learning journal. Although a general tendency to procrastinate in completing one’s recipe book and learning journals was clearly visible in our data, we could also observe that more successful learners, as assessed by their overall grades in the final exam, tended to use the platform earlier in their training than their less successful peers. In addition, our data clearly show that starting earlier also allowed learners to complete their training with more thoroughly documented recipes; in particular, we found in the learning journals of learners, who started to complete their recipe books earlier, more pictures documenting intermediate work-steps as well as more pictures showing the final product (ideal picture) than in documents of later starting learners. Contrasting their journal entries half-way in the training with that observed at the end of the training, findings suggest that “early starters” tended to complete their journals in successive steps, while “late starters” proceeded more in a “one-shot” mode. This confirms the superiority of building what Goda et al. (2015) called “learning habits” over adopting learning behaviors mainly affected by a procrastination tendency.

Taken altogether, our findings suggest that generating a learning journal is not effective per se. It depends on the way the actors involved perceive the purpose of such a tool, the learners of course but certainly also their supervisors, and their professional organizations, as the supervisors are supposed by their professional organizations to control the learners’ recipe books and learning journals at least once a semester. Where learners were allowed, possibly even encouraged, to generate their recipe book and their learning journals towards the end of their training only, it is likely that these documents are conceived rather as reports of competences acquired (Zubizarreta 2009). Wherever learners are encouraged to start generating their learning journals early and to keep enriching them throughout their training program the potential of learning journals as scaffolds for the development of a reflective practice is better recognized (Boud 2001). Globally, our observations suggest that the tendency to view a learning journal as a final report is common, both among learners as well as among their supervisors. Changing participants’ conceptions of learning journals could be as useful as changing their practices. Deciding to start using a learning journal early in one’s training requires accepting that it could include mistakes, errors, and proofs of temporary incompetence—a potentially difficult commitment for learners who want to present their learning journal as evidence attesting their competences and skills. Changing learners’ use of learning journals (staggered throughout the program) requires changing their conceptions of learning journals and of what they can be useful for.

Limitations and perspectives

Although generating a recipe book is common practice in the training of bakers or chefs, the requirement to keep a reflective learning journal is quite recent. At the time the LearnDoc platform was introduced, the vast majority of vocational supervisors had never asked their apprentices to keep a learning journal, despite a newly adopted regulation imposing learning journals as part of all new baker apprentices’ requirements. When LearnDoc was first introduced to apprentices and supervisors, a majority of them quickly adopted the recipe book functions of the platform but only few supervisors also encouraged their apprentices to keep reflective learning journals. The limited use of the learning journals we observed, compared to that of the recipe book, might well be in part due to the unfamiliarity of most professionals with learning journals rather than to a general resistance towards verbalizing one’s reflections and evaluating one’s mastery.

The low numbers of pictures included on the platform until mid-training (see Table 3) also indicate that most learners—and their supervisors—were initially unaccustomed to the operation of documenting one’s learning process. Repeating this study in the future, once the habits of keeping a learning journal and documenting one’s learning have been more firmly established in the profession, might further explain the nature of the learners’ difficulty to engage right away in the elaboration of such a learning support, even as technology has made it easy to use.

Another limitation of the present study is the absence of pretest data, which would allow assessing the level of the participants before or very early in their training. This would have made the search for explanation patterns between behaviors and learning gains possible. In the absence of such entry values, we could only establish the presence of significant correlations between specific behaviors (early start of use, larger numbers of ideal pictures associated with recipes, or more frequent use of metacognitive strategies) and final performances (final exam results). We do not know whether adopting these behaviors led to better learning or whether better learners used them more than weaker learners.

Another weakness of the present study is the absence of a time stamp for each answer given to our prompts. Only the presence or the absence of an answer was recorded, but not the nature of these answers. This, unfortunately, does not allow knowing if the learning strategies evoked early during one’s training are the same as those mentioned towards its end. One can reasonably hypothesize that strategies such as planning or debugging would be more frequent in the first part of one’s training and mere evaluations more often in the last months of training when most recipes should be mastered. This hypothesis, however, remains to be explored in future research.

Finally, the present study did not investigate how learners could be further encouraged to generate learning journals more frequently, to answer prompts more systematically, and become more reflexive. Good results in such direction have been obtained by researches making the answering of prompts mandatory or by sequencing the prompts in a specific order (Nückles et al. 2009; Roelle et al. 2017). Given the difficulties apprentices do experiment with writing, allowing learners to answer prompts orally also could possibly lead to more systematic reflections; finally modifying learners’ conceptions of learning journals, or trying to convince supervisors to be more explicit in stimulating learners’ systematic reflections on their level of mastery also represent some possible solutions to explore in future work.

Currently, the LearnDoc platform is only used in the workplace context. To stimulate reflective activities further, LearnDoc could also be made accessible to vocational school teachers. Schools might be more used to reflection-on-action than workplaces (Eraut 1995). We have proposed elsewhere a pedagogical model called ‘Erfahrraum’ featuring an iterative loop between workplace and school contexts to foster systematic reflection (Dillenbourg 2009; Dillenbourg and Jermann 2010; Schwendimann et al. 2015). The model highlights that (unprocessed) experiences alone do not lead to knowledge (Herzog and von Felten 2001). Knowledge cannot be directly experienced but needs to be constructed through reflection processes. It is now well established that digital artefacts and hypermedia can support reflective processes (Azevedo 2005), thus facilitating the integration of context-bound social practices and theoretical knowledge (Brown 2001). As work environments continue to develop and change rapidly, it is imperative that learners develop reflective skills necessary to become autonomous lifelong learners.