A mixed-methods study of physiological reactivity to domain-specific problem solving: methodological perspectives for process-accompanying research in VET

Abstract

Background

The study aims to investigate stress-inducing potentials of problem-solving activities (e.g., goal elaboration, decision making, and information seeking) within an authentic problem-solving task from the business administration domain. Furthermore, the study aims to investigate stress-reducing potentials of personal characteristics (e.g., self-efficacy beliefs, vocational experience).

Methods

A mixed-methods design was chosen to investigate in-depth processes during domain-specific problem solving, using a computer-based office simulation. Personal characteristics were assessed by questionnaires and tests before the task. Cardiovascular and electrodermal reactivity were measured continuously during the task. Problem-solving activities were coded on the basis of screencasts and think-aloud recordings. Changes in physiological reactivity were estimated on the basis of problem-solving activities and personal characteristics via multilevel regression analyses.

Results

The problem-solving task in general was associated with stress reactions. There were no significant main effects of self-efficacy beliefs, vocational experience, and general intelligence. However, changes in heart rate depended on an interaction between vocational experience and activities including goal elaboration and definition. Furthermore, problem-solving activities including decision making were significantly associated with an increase of amplitudes of detected skin conductance responses. A negative correlation found between the problem-solving score and the LF/HF ratio indicates that higher physiological arousal during the problem-solving task was accompanied by lower problem-solving performance.

Conclusion

It seems to be worthwhile to integrate physiological methods in domain-specific research practice to a greater extent. An essential advantage of such methods can be seen in the measurements’ relative independence from self-reported biases that seems to be especially important for high-frequency measurements within the scope of process-accompanying surveys and/or when investigating implicit aspects of action processes: from this, some new methodological perspectives for empirical research in VET could be developed. However, one has to consider that physiological measures alone are not objective or meaningful in this context, but rather have to be interpreted in their interplay with psychological parameters (e.g., experiences, behaviors) or with particular situational stimuli.

Background

Contemporary research on domain-specific problem solving in vocational education and training (VET) uses sophisticated statistical approaches for the assessment of structure models of domain-specific problem-solving competence (e.g., Achtenhagen and Winther 2009; Nickolaus et al. 2012). Furthermore, some of these measurement approaches also focus on emotional states, which occur in response to problem confrontation (Rausch et al. 2016). A complex problem is characterized by a barrier between a given state and an intended goal state that a person would like to achieve but does not immediately know how to achieve. Effort has to be put into cognitive and behavioral problem-solving activities in order to reduce a given barrier (Funke 2012; Newell and Simon 1972). To overcome such barriers, action-regulation processes are necessary. Such processes require different tasks and activities including goal elaboration and definition, information seeking and recording, forecasting and planning, decision making, performing goal-oriented actions and operations, and action control and reflection (Dörner and Wearing 1995).

Complex problems are a potential source of stress and arousal because they are characterized by informational and analytical complexity, uncertainty, a lack of transparency, a high degree of connectivity of variables, time pressure, and unclearly specified problem-solving operators (Dörner and Wearing 1995; Funke 1991, 2012; Mayer and Wittrock 2006). In the vocational context, for instance, everyday problem solving in back-office departments has been identified as a main source of everyday work-related stress as vocational problems provoke states of high psychological arousal and negative emotional states (Rausch et al. 2015). Whether a vocational task is perceived as an easy job or as a complex challenge mainly depends on the individual’s competence as determined by, among other things, prior experience, domain-specific knowledge, skills, and self-efficacy beliefs (Bandura 1994; Dörner 1996; Dörner and Wearing 1995; Goode and Beckmann 2010; Haerem and Rau 2007; Mayer 1994).

There is a gap in the previous literature concerning empirical work on the association between vocational problem-solving activities and stress responses. Thus, this article aims to investigate how and to what extent specific problem-solving activities are associated with physiological responses if multiple problem-solving activities are considered simultaneously. Furthermore, it addresses the question how and to what extent physiological reactions to problem-solving activities are affected by personal characteristics (such as general intelligence, domain-specific expertise, and self-efficacy beliefs). In an effort to deepen understanding of problem-solving processes, an in-depth analysis of vocational problem solving will be provided.

Stress-inducing potentials of problem-solving activities

Dörner and Wearing (1995) emphasize that the process of action regulation within problem solving can be subdivided into the following partial activities: goal elaboration and definition; information seeking and recording; forecasting, and planning; decision-making; performing goal-oriented operations; and monitoring and reflection of outcomes. The authors state that these different phases do not always show up in subjects’ behavior, nor does the sequence always strictly occur. In the remaining portion of this section, stress-inducing potentials of these partial problem-solving activities will be described on the basis of a literature review.

First, the problem solver has to elaborate on what should be the preferred outcome of his/her problem-solving activities. Often, the main goal is not clearly defined and global goals (e.g., “choose the optimal supplier”) must be decomposed into subgoals (e.g., “do a supplier value analysis,” “elaborate different weighting criteria,” etc.) which in turn need further specification. As a consequence of the decomposition of global goals into many different subgoals and of the normally limited time frame, it is necessary to order the priority of goals and subgoals (Dörner and Wearing 1995). The elaboration and definition of (possibly conflicting) goals and subgoals within the problem-solving process can be associated with both positive and negative emotions. Since complex problem situations are characterized by a low degree of transparency and a lack of perceived control, they are likely to trigger feelings of uncertainty and anxiety (Dörner and Wearing 1995; Reither and Stäudel 1985; Schwarz and Bless 1991). If goals can be self-defined by an individual, then goal disengagement and goal reengagement behaviors can compensate for the experience of distress which is associated with the continued pursuit of a goal that cannot be attained (Wrosch et al. 2003). If the realization of a goal seems to be realistic, or if a distressing goal-incongruent strategy can be changed for a better one, one could feel positive emotions such as happiness or relief (Lazarus 1991, 1999). Such “adaptive goal processes” (Folkman 2008) seem to be realistic in problem situations where the individual has the autonomy for self-defining goals, but not to that extent in the case of non-self-defining goals (cf. Gollwitzer and Wicklund 1985).

After goal definition, information has to be collected and evaluated in order to build hypotheses about the system (Dörner and Wearing 1995). Therefore, in analytic problem solving, relevant information has to be derived by deductive reasoning from different (possibly ambiguous) information sources (Brand-Gruwel et al. 2009; Leutner et al. 2005). Furthermore, the problem solver usually does not know a priori which information is most important to reach a goal or subgoal. That is because he or she is not able to perceive and process the full range of information presented, but just a small part which he/she considers to be important (Dörner and Wearing 1995). Thus, stress-inducing potentials of information-processing requirements are discussed within the scope of “information overload” (Schneider 1987). Eppler and Mengis (2004) reviewed a wide range of existing literature on management-related overload situations and found that managerial decisions in general and strategic analysis activities in particular act as sources of information overload. Fischer et al. (2012) state that information reduction is a most important aspect of coping with complexity within problem solving as great amounts of information may overstrain human processing capacity. In that regard, cognitive load theory recognizes two main sources of stress-inducing potentials: the intrinsic cognitive load of a problem, referring to the internal structure of the information that is given to the problem solver and that he or she needs to acquire to reach the intended goal, and the extraneous cognitive load, associated not with the inherent structure of the problem-related information, but with the manner of its presentation (Sweller 1988; Sweller et al. 2011; see Bley et al. (2015) for an application of cognitive load theory in the context of VET).

If the problem solver has collected enough information about a task’s structure and its actual state, it is necessary to predict what will happen after specific actions and operations have been performed. Thus, one has to anticipate and forecast possible effects and consequences of actions and operations under conditions of uncertainty. Within problem solving, the planning of sequences of consecutive actions and operations will usually be necessary (Fischer et al. 2012). The complexity of action planning depends on the structure of the domain and content one has to cope with, and on the number of potential actions available to influence the system’s development (Dörner and Wearing 1995).

After anticipating pros and cons of planned actions and operations, one has to make decisions in order to choose those specific actions and operations which should actually be performed. In complex problem-solving scenarios, decision making usually occurs under uncertain conditions. Because of time pressure and the amount of information that has to be processed, one can very seldom make exhaustive evaluations of all options and their possible consequences. Thus, decisions are usually based on strategies (so called “heuristics”) using less information and only a limited amount of information processing capacity (Fischer et al. 2012). Furthermore, in complex problem-solving tasks, it is never quite clear whether a specific decision really has the intended effects (Dörner and Wearing 1995). Acting under conditions of uncertainty (i.e., if few or ambiguous pieces of information are available) bears the risk of possible negative consequences. Within problem situations, such possible negative or even undesired outcomes have to be anticipated and evaluated (Dawson et al. 2011). Sembill et al. (2013) emphasize the importance of such unconscious evaluative reactions to internal and external stimuli (so called “affects”). The authors state that affects remain unconscious, as they emerge from neurophysiological systems (in particular the limbic system). If unconscious evaluations exceed a critical arousal threshold, distinct emotions (which are conscious, intensive, and directed toward the evaluated contents) may arise. Contradictory affective evaluations may lead to states of high arousal, which are likely to trigger stress emotions. Sembill et al. (2013, p. 202) state that “affects and emotions thus serve as a continual feedback on success and failure in need satisfaction and goal achievement and, therefore, are vital in any processes of problem solving.”

After coming to a decision, the problem-solver has to perform intended and goal-oriented actions and operations. As usual in daily office work, one has to use a notepad and/or a spreadsheet application and apply calculating procedures. Mental operations can be accompanied by perceived stress and physiological stress responses (e.g., Hjortskov et al. 2004; Zapf 1993) as such operations strain working memory capacity (cf. Sweller 1988). On the other hand, well-planned and analytic problem-solving activities are associated with achieving important goals at work (Folkman et al. 1986a, b), with positive reappraisal, and with a significant reduction of psychological symptomatology (Folkman et al. 1986a, b).

Right after performing operations, one has to control and reflect on the effects of the performed actions and evaluate success with reference to the defined goals. As mentioned above, adaptive goal definition and redefinition processes are associated with a reduction of psychological distress (Folkman 2008; Wrosch et al. 2003), and reaching a defined goal is associated with positive emotional states (Lazarus 1991, 1999). If an action is evaluated as unrewarding or unsuccessful, that appraisal can be associated with stress emotions like worry and anxiety (cf. Conroy et al. 2002).

Electrodermal and cardiovascular reactivity to mental stress

Selye (1973, p. 692) defines stress as “the nonspecific response of the body to any demand made upon it.” The concept of physiological arousal is a key component of the stress response (Winsky-Sommerer et al. 2005) and it is closely linked to states and processes of the autonomic nervous system (ANS) (Cacioppo et al. 1996).

The most commonly assessed indices of states related to ANS activation are based on cardiovascular [e.g., heart rate (HR), heart rate variability (HRV)] and electrodermal (e.g., skin conductance responses) reactivity, as they are sensitive to valence and arousal (Mauss and Robinson 2009). Electrodermal components are the skin conductance level (SCL) and the skin conductance response (SCR), as they are used as common indicators of sympathetic neuronal activity. The SCL is defined as the tonic level of electrical conductivity of the skin, and the SCR is defined as the phasic change in the skin’s electrical conductivity (Braithwaite et al. 2013). Skin conductance is affected by cortical (for example, the orbitofrontal cortex) and subcortical (the amygdala, for instance, responds to uncertainty and ambiguity) brain areas. These areas subserve processes of attention, anticipation, evaluation of stimulus significance, and decision making (Dawson et al. 2011; Mauss and Robinson 2009).

HRV as another indicator of the reactivity of the ANS reflects the activity of the parasympathetic (which controls physiological resting functions) and sympathetic (primarily activated under stress) nervous systems (Kemeny 2003). In this conjunction, the ratio of low-frequency components [low-frequency domain (LF), 0.04–0.15 Hz] to high-frequency components [high-frequency domain (HF), 0.15–0.4 Hz] is used as an indicator for the relation of sympathetic (reflected by low-frequency components) and parasympathetic (reflected by high-frequency components) activity. The LF/HF ratio is used as a frequency domain measure. The root mean square of successive differences (RMSSD) is another HRV indicator. RMSSD is used as a time domain measure and primarily reflects the reactivity of the parasympathetic nervous system (Malik et al. 1996; Mauss and Robinson 2009). Heart rate (variability) and skin conductance responses vary in terms of whether they primarily reflect sympathetic activity, parasympathetic activity, or both: SCL predominantly reflects activity of the sympathetic nervous system; HR reflects both sympathetic and parasympathetic activity; HRV is closely associated with parasympathetic activity (Larsen et al. 2008; Mauss and Robinson 2009).

The reactivity of the ANS is a major component of affective and emotional responses. For instance, several studies report anxiety- and fear-related increases in HR, SCL, and SCR and decreases in HRV (indicated by decreases of parasympathetic activity related to increases of sympathetic activity) (see Kreibig 2010 for a broad literature review). Concerning specific psychophysiological biomarkers, Chandola et al. (2010) conducted a systematic literature review. The authors state that the findings for specific biomarkers in the various studies should be interpreted with caution because of several limitations, such as publication aspects (e.g. publication bias), the inconsistent use of psychophysiological biomarkers between studies, and the inconsistent use of stimulus materials. Even though there are several limitations, the authors conclude that there is empirical evidence that work stressors are related to elevated stress responses including sympatho-adrenal responses. However, taken by themselves, electrodermal and cardiovascular measures are non-specific indicators of physiological arousal (Kohlisch and Schaefer 1996). Such indicators are not meaningful per se but have to be interpreted in their interplay with psychological parameters (e.g., traits, experiences, behaviors) and/or with particular situational stimuli.

Existing studies show that both HR (e.g., Clays et al. 2011; Hjortskov et al. 2004; Taelman et al. 2011) and skin conductance (e.g., Reinhardt et al. 2012) increase in response to mental stress. The LF/HF ratio shows an increase under stress, which indicates a predominant state of the sympathetic nervous system under stress-inducing conditions (e.g., Hjortskov et al. 2004; Isowa et al. 2006). The RMSSD shows a decrease under stress (e.g., Baert et al. 2012). Furthermore, there are also associations between electrodermal and cardiovascular measures. For instance, Lazarus et al. (1963) report an intraindividual correlation between heart rate and skin conductance of r = 0.5 under stress-inducing conditions. From a methodological point of view, electrodermal responses can be used as involuntary physiological measures, as they reflect evaluative processes (Dawson et al. 2011; Starcke and Brand 2012). Dawson et al. (2011) conclude that such anticipatory SCRs may reflect the conscious expectancy of a stimulus with a possible significant (negative) consequence or outcome. The SCR may also reflect unconscious affective processes and/or serve as a somatic marker that guides future decision making and helps the individual to act and get oriented in risky and/or uncertain situations [the anticipatory effects of SCR were intensively investigated and discussed within the “somatic marker hypothesis” by Damasio (1994); see also Bechara et al. (1997)]. Besides the SCR, the HRV also responds to situational threat and uncertainty. In that regard, Thayer et al. (2012) conducted a meta-analysis and found that a number of cerebral regions, including the amygdala and ventromedial prefrontal cortex, are associated with HRV regulation. With regard to action-regulation processes, Botvinick and Rosen (2009) state that adaptive and goal-oriented action requires that the prospective rewards of an action be weighed against its attendant costs associated with mental effort. The authors found an anticipatory SCR prior to actions, resulting in a high level of cognitive demand. They conclude that requirements for effort-intensive cognitive control are anticipated during the selection of specific actions and thus, that action selection is guided by the anticipation of action outcomes. Concerning physiological reactivity to mental load, Blitz et al. (1970) conducted an experimental study where they systematically varied mental loads in order to investigate effects on HR and HRV (indicated by sinus arrhythmia). The authors found that HR was positive affected by increasing levels of mental load, whereas HRV was negatively affected. Kohlisch and Schaefer (1996) found that the HR reflects the attentional aspects of mental load. The skin conductance is assumed to be a sensitive indicator of the emotional consequences of mental load, as SCRs reflect an affective arousal caused by information processing demands.

Associations between stress and performance

To perform well in a problem-solving task depends not only on adequate knowledge application or on executing the right mental operations, but also on appropriately coping with stress and arousal while working on a task (cf. Heller et al. 2010). Existing research shows that physiological stress affects working memory and long-term memory performance by the impairment of memory retrieval (e.g., Cornelisse et al. 2011; Smeets 2011). Furthermore, research on test anxiety shows that working under stress impairs attention and produces cognitive interference and task-irrelevant thoughts that result in reduced levels of task performance (Deffenbacher 1978; Eysenck et al. 2007). Vogel and Schwabe (2016) state that moderate or high stress levels may hinder memory retrieval and the integration of new information into existing knowledge structures, resulting in an inhibition of deep understanding of concepts. For instance, Dörner and Pfeifer (1992) found in an experimental study on stress and problem solving that those participants who worked under stress conditions more frequently decided in favor of ineffective operations compared to the participants in the control condition. Such “actionism” behavior is not goal-directed but just directed toward a demonstration of competence [summarized in Dörner and Wearing (1995)].

Stress-reducing potentials of general intelligence, domain-specific expertise, and self-efficacy beliefs

The construct of action competence is a complex system including, among other things, intellectual abilities, domain-specific knowledge and strategies, motivational tendencies, and volitional control systems, all of which components are required to fulfill vocational demands (Weinert 2001). Dörner and Wearing (1995) emphasize the capacity of conscious thinking and the feeling of being able to act effectively as important factors for triggering information processing when coping with complex problem situations. According to Rausch and Wuttke’s (2016) model of domain-specific problem-solving competence (see also Rausch et al. 2016; Sembill et al. 2013), cognitive, emotional, and motivational components of competence have to be taken into consideration. The cognitive comprehension of the problem’s core and the application of domain-specific knowledge include the identification of needs for action and information gaps, information processing, coming to well-founded decisions, and appropriate communicating of decisions. Furthermore, expectations about one’s efficacy related to solving a specific problem are important.

Gottfredson (1997a) defines intelligence as “a very general mental capability that, among other things, involves the ability to reason, plan, solve problems, think abstractly, comprehend complex ideas, learn quickly and learn from experience.” Intelligence (especially the g factor) predicts performance at work settings because it is an important ability to deal with complex information in the workplace (Gottfredson 1997b). According to Sternberg (1998), intelligence is composed of analytical (e.g., evaluating, comparing), creative (e.g., discovering, imagining), and practical (e.g., applying, implementing) abilities that are crucial within problem-solving processes. Cederblad et al. (1995) found that high intellectual functioning increases resilience. Werner (2000) considers average/above-average intelligence as a protective factor, but she also notes that there is little evidence that high intellectual abilities alone promote effective coping.

For successful problem solving, not only intellectual abilities but also domain-specific knowledge is essential (e.g., Achtenhagen and Winther 2009; Sembill et al. 2013). Tenenbaum et al. (2008) emphasize domain-specific expertise as an important coping resource in arousal-provoking performance situations. The authors conclude that high pressure prevents novices from performing complex skills well and competently because attention is shifted to task-irrelevant concerns, and the lack of automaticity results in an increase of procedural errors. Experts, on the other hand, have acquired coping strategies that enable them to minimize stress in arousal-provoking situations and remain focused on the task. According to Nitsch (1982; summarizing Tenenbaum et al. 2008), expertise may be determined through the extent to which a problem-solver has assimilated and accommodated the arousal-coping strategies, which are linked through exposure to problem-solving practice. Continuous and repeated exposure to arousal-provoking problem situations shifts the operational mode from an intentional mode to an automated mode. Such adaptive processes reduce vulnerability, anxiety, and uncertainty (Ibid.).

Bandura (1994, p. 71) defines perceived self-efficacy “as people’s beliefs about their capabilities to produce designated levels of performance that exercise influence over events that affect their lives.” There is broad evidence in existing literature that self-efficacy beliefs can have buffering effects on perceived stress (e.g., Ebstrup et al. 2011; Hahn et al. 2011) and on somatic stress responses (e.g., Bandura et al. 1982, 1988; Sanz and Villamarin 2001). Bandura (1994) states that rapid technological development at the workplace puts high requirements on higher problem-solving skills and self-efficacy to cope effectively with demanding vocational activities. For instance, Blair et al. (1999) used a computer-based problem-solving task and found a significantly negative correlation between stated self-efficacy and stated worry during problem solving. Rausch et al. (2016) used problem-solving tasks within authentic office simulations and found that situational confidence in one’s competence was significantly positively associated with tolerance for ambiguity and uncertainty and with maintaining positive and active emotional states.

Research questions and hypotheses

Three research questions (RQ) including corresponding hypotheses (H) arise from the review of theoretical foundations and existing findings.

RQ1: Is working on a problem-solving task in general associated with physiological stress reactions and perceived stress, and to what extent are specific problem-solving activities associated with physiological stress responses if multiple activities are considered simultaneously?

First, the study aims to investigate stress-inducing potentials of problem-solving activities within an authentic problem-solving task from the business administration domain. Existing literature provides evidence that certain problem-solving activities are accompanied by an increase of stress responses. These activities include: goal elaboration and definition (e.g., Dörner and Wearing 1995; Reither and Stäudel 1985), information seeking and recording (e.g., Eppler and Mengis 2004; Sweller 1988), forecasting and planning (e.g., Dörner and Wearing 1995; Fischer et al. 2012), decision-making (e.g., Botvinick and Rosen 2009; Dawson et al. 2011), performing goal-oriented actions and operations (e.g., Hjortskov et al. 2004; Zapf 1993), and action control and reflection (in the case of not reaching a defined goal or subgoal) (e.g., Conroy et al. 2002; Folkman 2008; Wrosch et al. 2003). It will be assumed that the problem-solving task in general is associated with stress (H1a). Furthermore, it will be assumed that problem-solving activities including goal elaboration and definition (H1b), information seeking (H1c), information recording (H1d), forecasting and planning (H1e), decision-making (H1f), performing goal-oriented actions and operations (H1g), and action control and reflection (H1h) are associated with stress responses.

RQ2: How, and to what extent, are physiological stress reactions during domain-specific problem-solving activities affected by personal characteristics?

Second, the study aims to investigate stress-reducing potentials of personal characteristics. Existing literature provides evidence that general intelligence increases resilience (Cederblad et al. 1995). Domain-specific expertise and knowledge are assumed to be an important coping resource in arousal-provoking performance situations (e.g., Nitsch 1982; Tenenbaum et al. 2008). Self-efficacy beliefs are assumed to buffer effects of demanding conditions and to be essential to cope effectively with demanding vocational activities (e.g., Bandura 1994; Ebstrup et al. 2011). It will be assumed that general intelligence (H2a), domain-specific expertise and knowledge (H2b), and self-efficacy beliefs (H2c) buffer stress-responses to problem-solving activities.

RQ3: To what extent is the problem-solving performance associated with stress?

Third, the study aims to investigate associations between problem-solving performance and stress. Existing research shows that stress affects performance by the impairment of memory retrieval and by cognitive interference and task-irrelevant thoughts (e.g., Cornelisse et al. 2011; Eysenck et al. 2007; Vogel and Schwabe 2016). It will be assumed that stress is negatively associated with problem-solving performance (H3).

Methods

Participants

Eighteen volunteers were recruited through advertisements at the University of Bamberg (Germany).¹ All participants were graduate students in a university course related to business administration. This was a necessary condition of working in the domain-specific problem scenario. Participants with serious medical conditions (e.g., cardiac arrhythmias) were excluded from the study because of possible confounding effects with regard to the physiological measurement. The study was conducted in accordance with the Declaration of Helsinki, and all participants provided written, informed consent.

Procedure, sequence of events, and timeline

The participants were tested individually and the procedure, sequence of events, and timeline were standardized for all participants. First, the participants were welcomed and told they would have to solve a problem from the domain of business administration using a computer-based office simulation. Thereafter the participants took a general intelligence test and filled in a questionnaire assessing personal characteristics. After this they watched a video tutorial on using the technical features of the simulated office environment. Figure 1 graphs the sequence of events and timeline.

Fig. 1

Procedure, sequence of events, and timeline

After the video tutorial, a rest period of 5 min was scheduled and the baselines of the electrodermal and cardiovascular parameters were measured. The baseline measurements were aggregated per person via arithmetic means. The baseline values were used for calculating the baseline-corrected measures during the task (see below).

The participants then started to work on the computer-based office simulation for 30 min. The participants were confronted with an authentic and complex domain-specific problem-solving task that is common in everyday office work in back-office business administration. The scenario required the participants to select a supplier by calculating prices, balancing pros and cons of different weightings of various quality criteria, and doing supplier value analysis. The computer-based office simulation provided the typical features of an authentic office environment such as a file system with hierarchical folder structure, an email client, a calculator, a notepad, and a spreadsheet application. The problem can be classed as a complex analytic problem because it features many interrelated variables, multiple conflicting goals, non-transparent information, and time pressure (Rausch et al. 2016). The office simulation was used as stimulus material because it offers a test format that has been extensively validated for large-scale measurement of the problem-solving competence of industrial business management assistants (Rausch et al. 2016; Seifried et al. 2016). Therefore, the stimulus material offers an authentic test format that has ecological validity, but that can be used under controlled and standardized conditions. After finishing the problem-solving task, the participants were given a 5-min rest period.

Perceived stress was assessed before and after working on the problem scenario. Cardiovascular and electrodermal parameters were measured continuously during the task and during the rest periods (before and after the task). Participants’ problem-solving behaviors were recorded via screencasts and think-aloud recordings using the software Screencast-O-Matic^®, and they were coded subsequently via video-based analysis. Figure 2 graphs the technical realization of screencasting, think-aloud recording, and time-sampling analysis.

Fig. 2

Technical realization of screencast recording, think-aloud recording, and time-sampling analysis

Measures and operationalization

Participant characteristics

Via questionnaire, information was collected on sex, age, and weight and height (both used for calculating the body mass index, BMI). Sex, age, and BMI are variables to control possible confounding effects on the physiological measures. In that regard, Steptoe et al. (1996) found that HR responses to mental stress were larger in younger than in older individuals (independent from sex). Steptoe and Wardle (2005) found that BMI was associated with impaired post-stress recovery of systolic pressure and diastolic pressure independently of age, gender, and baseline cardiovascular activity. Doberenz et al. (2011) found that sex affected non-specific skin conductance fluctuation amplitudes and the coefficient of skin conductance variation: men had significantly lower values compared to women. Furthermore, BMI significantly lowered the number of non-specific skin conductance fluctuations and was negatively but not significantly associated with SCL and SCL standard deviation with small to medium effect sizes.

Vocational experience was operationalized by the reported work experience that is assumed to be an indicator for domain-specific expertise and knowledge. Nine participants (50% of the sample) have a VET certificate, and all VET certificates refer to a vocation from the domain of business and administration: 1 trained retail salesman, 1 management assistant in office communication, 2 qualified bank clerks, 1 commercial assistant, 2 legal assistants, 1 management assistant in freight forwarding, and 1 industrial management assistant.

General intelligence was measured using the German version of the Culture-Fair Intelligence Test (CFT 20-R) (Weiß 2006), assessing fluid intelligence via four subtests: “series completion” (15 items), “classifications” (15 items), “matrices” (15 items), and “topologies” (11 items).

Participants’ self-efficacy beliefs were measured using an adapted scale from Schwarzer and Jerusalem (1999): 7 items (α = .67), e.g., “I can always manage to solve difficult problems if I try hard enough”; 4-point Likert-type scale, from 1 = “Not at all true” to 4 = “Exactly true.”

Problem-solving activities

In reference to Dörner (1976, 1996), Dörner and Wearing (1995), and Bransford and Stein (1993), six main types of activities within the problem-solving process were coded on the basis of the screencast videos and think-aloud recordings of participants’ problem-solving behaviors. Time-frames at 1-s intervals were coded via video-based analysis using the Videograph^® software (Rimmele 2013).² Consensual validation was used, and critical aspects were discussed by two professional coders in order to find common coding solutions. The 1-s coding intervals were aggregated per person at 10-s intervals via sum scores, to synchronize the assessed problem-solving activities with the physiological measurements. The following problem-solving activities were coded:

• 1. Goal elaboration and definition The participants had to identify the problem’s core and elaborate and define goals and subgoals; e.g., “I have to do a supplier value analysis and give more weight to acquisition prices.”

• 2. Information seeking and recording Business documents of various types (invoices, letters, notes, etc.) were provided in the simulation, and no single document provided a complete solution to the problem. The participants had to examine and read different types of information (e.g., invoices from different suppliers) with regard to their relevance for solving the problem (information seeking). Furthermore, the participants noted relevant information for later processing on the notepad (information recording).

• 3. Forecasting and planning The participants anticipated possible effects of their planned actions and forecast possible consequences; e.g., “If I change the weighting of the quality criteria in that way, than the supplier from the Czech Republic may be will be the best choice.”

• 4. Decision making After balancing pros and cons of a planned action, participants made a decision and chose specific actions and operations; e.g., “Then I’ll give more weight to the purchase price.”

• 5. Performing goal-oriented actions and operations After coming to a decision, the participants performed specific actions and operations; e.g., calculating the supplier benchmark on the basis of the chosen weighting by using the spreadsheet application.

• 6. Action control and reflection The participants controlled the effects of their performed actions and evaluated their success with reference to the defined goals or subgoals. Where necessary (e.g., if the action was unrewarding), participants had to return to a previous phase of the problem-solving process and try to reach the goal or subgoal again. In the analyses of the screencasts and think-aloud recordings, only cases where participants were not able to reach a goal/subgoal were identified and coded as such. In such cases the participants had to return and try again; e.g., “Now I get an odd solution and I have to check it up again.”

The above-mentioned problem-solving activities are assumed to refer primarily to the internal structure of the task that was given to the problem solver (cf. intrinsic cognitive load, Sweller 1988). However, aspects of the presentational characteristics and possible problems with the handling of the computer-based office simulation were also coded (cf. extraneous cognitive load, Ibid.). In that regard, for example, participants had problems regarding copy-and-paste functions, spreadsheet application use, open files, or software hanging problems.

Problem-solving performance

Problem-solving performance ratings are based on categories developed and tested in a large-scale study of problem-solving competence in the business domain (Rausch et al. 2016). The coding was performed using Rating Suite software to display the participants’ solutions. The coding guide provided definitions, coding rules, and coding examples for the following categories (maximum points in parentheses): “identifying needs for action and information gaps” (13); “processing information” (21); “coming to well-founded decisions” (5); “communicating decisions appropriately” (7) [for further information to the coding guide see Rausch et al. (2016)]. Participants’ problem-solving performance was operationalized by the sum of ratios of achieved scores to maximum scores (in terms of relative scores for each category). Overall, participants achieved a score of 1.92 (±.50 SD, Min. = 1.24, Max. = 2.96, within a possible range from 0 to 4).

Physiological measures

Electrodermal reactivity

When assessing electrodermal activity via equipment from movisens^® (edaMove; movisens 2015), the skin conductance level and the mean amplitudes of detected skin conductance responses were used as indicators. The electrodermal activity data were low-pass filtered, and the filter frequency was 0.1 Hz. Measuring-time intervals of 10 s were chosen and, for each output interval, the mean values of the internally calculated values were used for calculation. SCR amplitudes were automatically detected, and specification of event markers was neither necessary in the current analysis nor possible with the technical equipment used. The default minimal rise time for the detection of SCR was 0.05 µS/s, the default minimal amplitude for the detection of SCR was 0.1 µS, and the default maximal rise time for the detection of SCR was 0.9 s (cf. movisens 2015).

Cardiovascular reactivity

Participants’ cardiovascular reactivity was measured continuously during the task via a chest belt and storage devices that recorded the data wirelessly on an integrated memory chip. The internal sample frequency was 1 kHz, and the HR raw data were transmitted to the software used (Medeia Ltd.^®) in order to calculate the HRV indicators. The following HRV parameters were taken into account: the LF/HF ratio was used as the frequency domain measure, as it mirrors the sympathovagal balance (cf. Malik et al. 1996; Sleight and Bernardi 1998); the root mean square of successive differences (RMSSD) was used as the time domain measure of HRV (cf. Malik et al. 1996). The measures were aggregated per person at 10-s measurement intervals via arithmetic means in order to synchronize the cardiovascular and the electrodermal measurements.

Taking into account individual changes from baselines, the baseline values (SCL, SCR, HR, LF/HF ratio, RMSSD) were subtracted from the successive 10-s interval measures as recommended by Roberts et al. (2004) to obtain baseline-corrected values.

Perceived stress

Perceived stress (PS) was assessed using a visual analogue scale (cf. Luria 1975). The participants were asked to rate how stressed they felt by placing a cross on a 100 mm horizontal line labeled “Not at all stressed” at the left end (0 mm) and “Extremely stressed” at the right end (100 mm). The visual analogue scale was also used to assess the internal validity of study conditions, assessing whether working on the problem was even perceived as stressful.

Statistical analyses

Changes in perceived and physiological stress over the general periods (before, during, and after the task; see Fig. 1) were analyzed via repeated-measures analysis of variance (ANOVA), with time of measurement as the repeated within-subjects factor (aggregated measures for the physiological measures: before, during, and after the task). Greenhouse-Geisser-adjusted p values for the analysis of multiple measures and Bonferroni-corrected p values for the multiple comparisons to control for alpha error inflation are reported.

Pearson product-moment correlations were calculated in order to identify significant associations between the physiological measures and the independent variables.

Multilevel analyses were used to estimate the changes in physiological measures during the task based on personal characteristics and problem-solving activities. Standardized values were used. The analyses were conducted using the SPSS^® Mixed Procedure™. The restricted maximum likelihood method was chosen, as it is adequate for small data sets (Heck et al. 2010). A first-order autoregressive covariance structure with homogenous variances was modeled, because the successive measures (nested within participants) were assumed to be auto-correlated to each other (cf. Hox 2002; Littell et al. 2000). The following hierarchical data structure (measurement intervals nested within participants) was modeled:

• On Level 2 (18 participants), participant characteristics (e.g., BMI, vocational experience, general intelligence) and the individuals’ baseline values at perceived stress were modeled;

• On Level 1 (overall 1792–3226 person-related 10-s measurement intervals), problem-solving activities including (1) goal elaboration and definition, (2) information seeking, (3) information recording, (4) forecasting and planning, (5) decision making, (6) performing goal-oriented actions and operations, and (7) action control and reflection were modeled. In addition, the time from starting with the task was modeled on Level 1. Thus, each 10-s measurement interval within the problem-solving process was characterized by different problem-solving activities and by the time-point within working on the task.

The statistical analyses were performed using SPSS 23^® (IBM^®, Chicago, USA).

Results

Descriptive data and Pearson product-moment correlations

Descriptive data and Pearson product-moment correlations (r _{10-s measurementintervals}/r _persons) are reported in Table 1. HR is significantly negatively correlated with RMSSD, and positively correlated with SCL and the LF/HF ratio; the LF/HF ratio is positively associated with SCL and SCR. The baseline of perceived stress is significantly positively associated with HR, the LF/HF ratio, and SCL. Furthermore, the baseline of perceived stress is significantly negatively associated with RMSSD and SCR (each on the basis of 10-s measurement intervals). There are also certain significant correlations between the physiological measures, personal characteristics, and problem-solving activities. In the multilevel analysis, only those variables will be considered as predictors that are significantly correlated to the physiological measures on the basis of 10-s measurement intervals. Because of the high correlations (not reported in Table 1) between sex and BMI (r = −.66, p = .003; male = 0, female = 1), age and BMI (r = .55, p = .019), and vocational experience and age (r = .53, p = .022), the variables sex and age will not be included in the multilevel analysis in order to exclude effects of multicollinearity.³

Table 1 Descriptive data and Pearson product-moment correlations

Time spent on the problem-solving task (starting time to end time, counted in 10-s time intervals) is significantly correlated with problem-solving activities including goal elaboration and definition (r = −.11**), information seeking (r = −.28**), information recording (r = −.25**), decision making (r = .16**), performing actions and operations (r = .52**), and action control and reflection (r = .04*) (r _{10-s measurement intervals} are reported). That indicates that specific activities occur earlier or later than other activities.

Stress before, during, and after the task

Repeated-measures ANOVAs showed significant effects of TIME (before, during, and after the task; cf. Fig. 1) for perceived stress (before < after; F(1, 17) = 34.230, p < .001, η ² = .668) and SCL (before < during < after; F(1.10, 18.76) = 37.847, p < .001, η ² = .690). No significant effects were found for HR [F(1.32, 21.18) = .179, p = .745, η ² = .011], RMSSD [F(1.42, 19.83) = 1.644, p = .219, η ² = .105], or LF/HF ratio [F(1.42, 22.73) = 2.362, p = .130, η ² = .129]. SCR average amplitudes are higher after the task compared to the SCR amplitudes before the task [F(1.30, 22.13) = 4.136, p = .045, η ² = .196, Bonferroni-corrected p = .064], but after Bonferroni correction these differences just failed to reach the 5% level of significance (Table 2).

Table 2 Stress before, during, and after the task

Multilevel analysis

Null models

The analysis of the null models for the physiological measures nested within participants shows significant random intercept variances for HR (σ ²_u  = .501), RMSSD (σ ²_u  = .799), LF/HF ratio (σ ²_u  = .238), SCL (σ ²_u  = .861), and SCR amplitudes (σ ²_u  = .200), indicating remarkable variability between participants (Table 3). Though the current study has a relatively small sample size, random intercept variances and intraclass correlation coefficients (ICC = σ ²_u /[σ ²_u  + σ ²_e ]) indicate adequate variability across the nesting units (participants) at Level 2 (cf. Heck and Thomas 2015).

Table 3 Null models

Mixed-effects models

Models 1, 2, 3, and 4 show non-significant effects for personal characteristics. Only the SCR (Model 5) is significantly negatively associated with the baseline of perceived stress (β = −.231, p = .032). On Level 1, the time from starting is significantly associated with HR and SCL. Problem-solving activities including goal elaboration and definition are significantly associated with changes in heart rate during the task (β = .108, p < .001) (limitations mentioned below). Problem-solving activities including decision making significantly affect changes in SCR mean amplitudes (β = .057, p = .004).

When checking possible moderation effects, in Model 1 a significant interaction effect between vocational experience and problem-solving activities including goal elaboration and definition was found (β = −.168, p < .001). Figure 3 illustrates the group-specific linear growth trajectories of the two groups (classification via median split): participants below the median (0–3 years of vocational experience; n = 10) show a stronger increase in HR, with increasing values of goal elaboration and definition, than participants above the median (≥4 years of vocational experience; n = 8).

Fig. 3

Interaction effect: vocational experience × goal elaboration and definition

Because of the significant interaction effect, the characteristics of main effects and interaction effects were checked graphically with reference to the classification of Leigh and Kinnear (1980): the rank order of levels of the factor “goal elaboration and definition” does not remain constant over all levels of the factor “vocational experience.” This describes a non-monotonic relation, and the main effect of “goal elaboration and definition” cannot be sensibly interpreted.

The repeated-measures effects reveal significant auto-correlations for HR (ρ = .619), RMSSD (ρ = .844), LF/HF (ρ = .830), and SCL (ρ = .842) (all p < .001), but not for SCR (ρ = .029, p = .348).

The comparison of the information criteria (−2LL, AIC, BIC) shows that all the mixed-effects models (predictors included at Levels 1 and 2) fit better to the data than the corresponding null models (Table 4).

Table 4 Mixed-effects models

Associations between problem-solving performance, stress, and problem-solving activities

The problem-solving score shows a significantly negative association with the baseline-corrected and person-aggregated LF/HF ratio (r = −.52, p = .028). The negative association indicates that higher physiological stress during the problem-solving task is accompanied by lower problem-solving performance (Fig. 4). There are neither significant linear nor non-linear (quadratic polynomial) associations between problem-solving performance and the other stress variables.

Fig. 4

Association between LF/HF ratio and problem-solving performance

With regard to the coded problem-solving activities, the problem-solving score correlates significantly positively with activities including “performing goal-oriented actions and operations” (r = .62, p = .006), and positively but not significantly with activities including “information seeking” (r = .17, p = .501), “forecasting and planning” (r = .23, p = .359), “decision making” (r = .22, p = .389), and “action control and reflection” (r = .07, p = .796). Participants who spend a lot of time on activities including “goal elaboration and definition” (r = −.31, p = .212) and “information recording” (r = −.47, p = .051) tend to show lower problem-solving scores (each correlation on the basis of person-aggregated values).

Discussion of findings and limitations

Eighteen participants were confronted with a computer-based office simulation, and they had to work on a problem from the domain of business administration. Electrodermal and cardiovascular responses were measured continuously during the task. Problem-solving behaviors were recorded via screencast and think-aloud recordings (Fig. 2). In this section, the empirical findings will be discussed with reference to the research questions and hypotheses.

Stress-inducing potentials of problem-solving activities (RQ1)

With reference to the first research question, hypothesis H1a can be confirmed. The comparison of the general study phases (before, during, and after the task; see Fig. 1) shows that the problem-solving task in general was associated with physiological stress reactions (SCL at p < .001; SCR at p < .1) and with perceived stress (Table 2).The mixed-effects analysis revealed that the time from starting was significantly associated with changes in HR and SCL, indicating effects of increasing time pressure during the task (Table 4).

With reference to hypotheses H1b to H1h, only the effect of problem-solving activities including decision making (H1f) conforms to the assumptions. The mixed-effects model (Model 5: SCR) reveals a significant association between the mean amplitudes of detected skin conductance responses and decision making (β = .057, p = .004; see Table 4). The participants had to make decisions under uncertainty because they did not immediately know which possible consequences their decisions would have. As existing literature shows, SCRs reflect unconscious evaluative processes that are related to the anticipation of forthcoming events (Damasio 1994; Dawson et al. 2011; Mauss and Robinson 2009). Even though changes in HR are significantly associated with problem-solving activities including goal elaboration and definition (see Model 1: HR, Table 4), the main effect found cannot be sensibly interpreted because of the significant interaction effect between participants’ vocational experience and problem-solving activities including goal elaboration and definition (cf. Leigh and Kinnear 1980).

For problem-solving activities including information seeking, information recording, forecasting and planning, performing goal-oriented actions and operations, and action control and reflection, no significant effects were found. There are some possible explanations for the absent effects. When assessing participants’ problem-solving activities in the current study via codings based on screencast videos and think-aloud recordings, there is the problem that some participants told us much while other participants provided very few usable verbal data. Furthermore, the video-based analyses of problem-solving activities showed that the main activities observed were information seeking (M = 13.83 min.), information recording (M = 3.66 min.), and performing goal-oriented operations (M = 7.38 min.). This finding could mean either that some activities occurred more frequently than other activities or that some activities were just more observable than others. Therefore, in further research, the use of log file analyses seems to be worthwhile because it is expected to provide a very detailed analysis of problem-solving behaviors, heuristics, and algorithms. Furthermore, in the current study the predefinition of strictly defined event markers was not possible with the technical equipment that was used. Therefore, further research should emphasize the specification of predefined event markers within experimental studies: e.g., variations of problem-solving conditions such as decision making under uncertain vs. transparent conditions, or self-defining vs. non-self-defining goals, and variations of domain-specific contents. In that regard, a combination of field studies (to enhance external/ecological validity) and experimental studies (to enhance internal validity) seems to be worthwhile with regard to an evidence-based validation of findings regarding problem-solving situations.

Besides methodological issues, the absent effects of the problem-solving activities mentioned above could also result from contentual issues. Possibly, the considered activities are not stress-inducing. Furthermore, it is also possible that some activities even have stress-reducing potentials, as they may act as coping behaviors within the problem-solving process (cf. Folkman 2008; Lazarus 1991, 1999; Wrosch et al. 2003). However, there remain some open questions, which have to be considered in further research.

Stress-reducing potentials of personal characteristics (RQ2)

The assumptions that general intelligence (H2a) and self-efficacy beliefs (H2c) have stress-reducing potentials cannot be confirmed unambiguously (aside from the finding that none of the named variables was significantly associated with changes in physiological reactivity in the multilevel analyses). On the one hand, general intelligence is significantly negatively correlated with LF/HF ratio (r = −.05*), and self-efficacy beliefs are significantly negatively associated with SCR (r = −.08**); both of those findings support the assumption of a stress-reducing function. On the other hand, general intelligence and self-efficacy beliefs are significantly positively correlated with HR (r = .19** and r = .16**, respectively), and both variables are significantly negatively associated with RMSSD (r = −.14** and r = −.21**). Furthermore, self-efficacy beliefs are significantly positively correlated with the LF/HF ratio (r = .15**). These findings in turn indicate a stress-inducing function. Although existing studies report positive associations between self-efficacy beliefs and physiological stress (e.g., Schwerdtfeger et al. 2008), more research about the (possibly non-linear) relation among intelligence, self-efficacy, and physiological stress seems to be appropriate.

One reason for the associations found which do not conform to the assumptions could be operationalization issues. In the current study, participants’ self-efficacy beliefs were measured as a trait variable and not as a state variable. Blair et al. (1999) used a state self-efficacy variable that was assumed to vary in intensity and change over time and be very situationally specific (e.g., “I understand the XYZ Product support job very well”; Ibid., p. 523). Also, Rausch et al. (2016) measured confidence in one’s competence as a very situationally specific state variable closely related to the perception of concrete situations within authentic office simulations. General intelligence is assumed to be a relatively stable personal disposition. Concerning vocational problem solving, Rausch (2016) found evidence that domain-general cognitive abilities (such as general intelligence) play a minor role, whereas domain-specific problem situations require domain-specific expertise and knowledge to a greater extent.

Concerning the control variables, the BMI does not significantly affect physiological reactivity (see also Note 3). The baseline of perceived stress is significantly negatively associated with SCR changes (β = −.231, p = .032). This finding is against the assumption, as one would have expected a positive association between perceived stress and SCR. The negative effect found could be interpreted with reference to the complex interactions between the central nervous system together with the autonomic nervous system and the hypothalamus–pituitary–adrenal axis, all included in stress responses (Andrews et al. 2013). Because of such complex physiological interactions, as well as assessment and methodological issues and possible mediating factors, interindividual differences, and contextual factors, associations between physiological and psychological stress responses may vary (Campbell and Ehlert 2012; Michaud et al. 2008).

Concerning hypothesis H2b, the correlation analysis revealed significantly negative relations between vocational experience and HR (r = −.17**), LF/HF ratio (r = −.22**) and SCL (r = −.46**/−.51**), and a slight but significant positive association with RMSSD (r = .04*) (Table 1), indicating stress-reducing potentials of expertise in domain-specific problem solving. The main effects of vocational experience do not remain significant in the multilevel analysis. However, a significant interaction effect between vocational experience and problem-solving activities including goal elaboration and definition was found. With regard to the existing literature, the stress-buffering effect of domain-specific expertise can be interpreted as follows. Experts (compared to novices) may have acquired arousal-coping strategies which are linked through repeated exposure to problem-solving practice. Such strategies enable them to minimize stress in arousal-provoking situations due to high degrees of automaticity in action-regulation processes (Nitsch 1982; Tenenbaum et al. 2008). Participants with less or even no vocational experience (cf. the under-median group, 0–3 years) may have perceived the start of the problem-solving task as uncertain or anxiety-provoking because of the low degree of transparency and the lack of perceived control (cf. Dörner and Wearing 1995; Reither and Stäudel 1985; Schwarz and Bless 1991). Participants with more vocational experience (cf. the above-median group, ≥4 years) have by this point acquired adequate coping strategies and may consider successful problem solving realistic (cf. Lazarus 1991, 1999; Tenenbaum et al. 2008). Taking into consideration that the HR indicator primarily reflects attentional aspects of information processing demands (Kohlisch and Schaefer 1996), the stronger increase in HR with increasing degrees of activities including goal elaboration and definition could be interpreted as follows: the “novices” have to concentrate their attention on goal elaboration to a greater extent than the “experts” (who already have developed routines) may have to do. Taking into account that individuals’ states (such as physiological reactivity to demanding situations) are affected not only by situational conditions but also by personal characteristics (cf. Kärner and Kögler 2016; Kärner et al. 2017), the interaction effect found between vocational experience and goal elaboration provides evidence for person-situation interactions, also within narrowly defined vocational problem-solving situations.

The absent fixed effects for the personal characteristics could also be attributable to methodological/statistical issues because of the small sample size at the second level and its possible effects on the accuracy of the estimation. A number of references emphasize that the restricted-maximum-likelihood method for estimation of regression weights works well in small sample settings, and there are a number of exemplary multilevel applications that refer to small sample sizes at Level 2 (N = 15–20) (e.g., Kenward and Roger 1997; McNeish and Stapleton 2016; Skene and Kenward 2010; Stegmueller 2013). However, the current study contains a relatively small sample size (N = 18; participants at Level 2) in relation to a large sample of repeated measurements nested within persons (the medians of frequencies of 10-s measurement intervals range from 101 to 184 measures per person; Level 1). Snijders (2005) states that the statistical power in multilevel modeling depends primarily on the total sample sizes for each level and that the sample size at the highest level (in the current analysis the second level) is the main limiting characteristic of the statistical design. Also, Scherbaum and Ferreter (2009, p. 352) note, “as a general rule of thumb, increasing the sample size at the highest level (i.e., sampling more groups) will do more to increase power than increasing the number of individuals in the groups.” Maas and Hox (2005) found that a small sample size at Level 2 (≤50) leads to biased estimates of the second-level standard errors and parameters.

Stress and problem-solving performance (RQ3)

Concerning the assumed relation between stress and problem-solving performance, hypothesis H3 can be confirmed for the significantly negative association between the LF/HF ratio and the problem-solving score. In that regard, existing research shows that stress affects performance by the impairment of memory retrieval and by cognitive interference (cf. Eysenck et al. 2007; Vogel and Schwabe 2016). However, the existing literature also provides evidence for non-linear relationships between stress and performance. With reference to the findings of Yerkes and Dodson (1908) and Hebb (1955), it has to be taken into account that the relation between performance and stress can be described as an inverted U-shaped curve. Both overly low and overly high degrees of activation may lead to low degrees of performance. The optimum of performance is reached at a moderate level of activation (Frankenhaeuser and Lundberg 1977). In the current study no significant non-linear (quadratic polynomial) relations between problem-solving performance and the stress variables were found. However, it seems to be worthwhile to consider non-linear relations as well as interindividual differences in performance-activation relationships to a greater extent in further research (cf. VaezMousavi et al. 2009).

Study design

The study design which was used has both advantages and limitations. The computer-based office simulation offers an authentic test format that has ecological validity (cf. Rausch et al. 2016; Seifried et al. 2016) and that can be used as stimulus material under controlled and standardized conditions. However, the problem scenario covers only a part of the wide range of tasks in the business administration domain, and so further studies should look at other problem topics and domains.

A possible reason for the absent main effects of personal characteristics could also be attributable to the study design. Maybe the participants were not seriously stressed in the laboratory context, as compared to a real-life-situation at the workplace. I found that participants reported above-average values of perceived stress after the problem-solving task (M = 57.06, SD = 21.97; within a possible range from 0 to 100; see Table 2). However, the relatively high standard deviation indicates that some participants were more stressed than others. Furthermore, the relatively small changes in the physiological values in response to the task indicate that working on the problem indeed affected the autonomic nervous system, but that it did not massively stress them as real-life problem-solving situations at the workplace might. Therefore, additional investigation of everyday business problem-solving activities in naturalistic field contexts will be necessary to increase the internal validity of research results.

The study aimed to investigate stress responses to problem-solving activities in a domain-specific context. Thus, the stress responses should be primarily attributed to the internal structure of the task (cf. intrinsic cognitive load, Sweller 1988) and not to problems with the handling of the computer-based office simulation (cf. extraneous cognitive load, Ibid.). To control for possible effects of presentational characteristics, problems regarding copy-and-paste functions, spreadsheet application use, open files, and software hanging problems were also coded. In that regard, no significant correlations between “handling software problems” and the stress measures were found (Table 1).

Conclusions and further research

The reported methodological approach is not new in its form. In previous empirical research in VET, comparable physiological methods (e.g., Abele et al. 2017; Beck and Sczesny 1993; Kärner et al. 2017; Santjer-Schnabel 2002) and systematic observations of in-depth action processes related to problem-solving tasks (e.g., Bley et al. 2015) have already been applied. Moreover, theoretical and conceptual frameworks have already been developed (e.g., Beck 1994; Sembill 1992 et passim). Sembill (2015a), for instance, discusses a theoretical conceptual approach that integrates different levels of analysis—from the “Nano-level” (e.g., neurophysiological states and processes) to the “Macro-level” (e.g., socio-economic circumstances). Sembill’s approach assumes balancing processes and evaluation processes as pivotal characteristics at each level. However, research to date has lacked systematic efforts to integrate physiological methods into VET research practice. Besides the reported responses of the autonomic nervous system, in further research, the application, for example, of functional neuroimaging methods such as functional magnetic resonance imaging, positron emission tomography, and electroencephalography, or even eye-tracking technology, would seem to be worthwhile to investigate controlled (explicit, conscious) and automatic (implicit, unconscious) processes in the course of domain-specific actions. An essential advantage of physiological methods can be located in the objectivity of measurement that is relatively independent from self-reporting biases (cf. Dawson et al. 2011). That seems to be important, especially when investigating implicit aspects of action processes that cannot be directly verbalized by the individual. However, one has to consider that physiological measures are not meaningful per se in the VET context but have to be interpreted in their interplay with psychological parameters (e.g., traits, experiences, behaviors) or with particular situational stimuli.

Due to the more explorative character of the study and because of the relatively small sample size at Level 2, the empirical findings are only specific for the sample and cannot be generalized. However, the mixed-methods study could make a contribution to further research practice in VET as the study design combines (1) high-frequency measurements of physiological data, (2) in-depth video-based observation of problem-solving activities within an authentic test format that has ecological validity, and (3) assessment of personal characteristics.