Skip to main content

Table 5 Overview of research questions/hypotheses and main findings

From: Analysing domain-specific problem-solving processes within authentic computer-based learning and training environments by using eye-tracking: a scoping review

References

Research questions/hypotheses

Main findings

Abele et al. 2017

Successful subjects show a higher total critical fixation duration than less successful subjects

- Higher performance longer total fixation time

- Lower performance shorter total fixation time or substantial longer fixation time

- More prolonged fixation might indicate confusion at some point

Cloude et al. 2020

Time fixating and interacting with scientific reasoning-related game elements predicts post-test scores (RQ1)

Time fixating on scientific reasoning-related game elements predicts time interacting with scientific reasoning-related game elements (RQ2)

Time fixating on non-scientific reasoning-related game elements predicts the time interacting with non-scientific reasoning-related game elements (RQ3)

- Lower performance higher proportion of time gathering information and less time generating hypotheses

- Lower prior knowledge positively moderates the relation between interaction and fixation on gathering information in the GBLE while a negative relation for higher prior knowledge was found

- No relation between interaction and fixation for non-scientific reasoning was found

- Eye-tracking data helps to supplement and contextualize log-files

- Multichannel data may be needed for individualized learning analytics approaches

Dubovi 2022

Students’ cognitive and emotional engagement can be measured by their self-reports and psycho-physiological real-time measurements and the synergistic effect of cognitive and emotional engagement on learning

- Emotional and cognitive engagement via multimodal metrics explained 51% of post-test learning achievement

- No significant impact of joy expression on post-test

- Frequent anger expressions were associated with lower post-test scores

- No significant change in self-reported affective state over three times

- Blink rate is negatively associated with post-test scores and shows significantly lower rates during the actual problem

- More visual attention is spent on similar medicine, indicating processing difficulties through fixation counts and dwells

- Significant correlation between EDA peaks and blinks but not with emotional engagement

- Positive emotions were related to inducing blinks

- A higher level of presence was related to more visual attention to relevant medicine

Emerson et al. 2020

Student gameplay behavior traces, facial expressions of emotions and eye gaze classify low, medium, and high performing groups. (RQ1)

Student gameplay behavior traces, facial expressions of emotions and eye gaze classify low, medium, and high-interest groups. (RQ2)

- Gaze as a feature (unimodal) or gameplay + face as a multimodal feature approach yields in the accuracy of 0.67 for prediction among three performance groups

- Gameplay + face (multimodal) yields in 0.59 accuracy for prediction among three interest level groups

- Adding more modalities comes at the cost of noise, so feature selection must be done carefully to avoid overfitting

Gomes et al. 2013

Differences in eye-tracking patterns exist between students with low and high performance in the three engineering-related computer games

- Shorter time to first fixation, fewer clicks, more unique fixations, and a longer duration per fixation for high-performance cluster

- Longer time for the first fixation, a higher number of clicks and short fixation durations might indicate a lack of focus on strategy or reasoning before action (“trial-and-error”)

- Shorter durations for first fixations might indicate higher attentional readiness and indicates more time spent on reasoning before action

Kang and Landry 2014

The performance will be different for novices exposed to the expert scan path compared to the control group or novices without treatment

- Treatment (expert scan path) group showed significantly fewer false alarms than the verbal instruction group or control group

- Treated novices tend to follow a professional expert scan pattern after treatment (circular)

- Treatment group perceived expert scan paths as helpful, and a scan path could improve the training of novices

Lee et al. 2019

Participants with high domain-specific prior knowledge (DSPK, i.e., experts) show higher systematicity in approach than participants with low DSPK (i.e., novices). (H1)

Experts show higher accuracy in visual selection by allocating more visual attention to critical diagnosis areas (H2a) and in motor reactions by completing more interventions (H2b) than novices

Experts show higher speed in performance by completing interventions faster than novices. (H3)

Experts experience lower cognitive load than novices. (H4)

- Experts-Novice comparison shows for experts:

More systematicity (indicated by HMM score)

Higher proportions of dwell time to total time (large effect), a higher ratio of fixation count to total fixation counts (medium effect), and longer fixation duration (large effect) on critically relevant information

No difference for other AOIs except for the intervention area; a lower proportion of total fixation counts (medium effect)

No difference for average fixation duration and fixation count, but cognitive load and transition rate correlate negatively with self-reported NASA-TLX score

Lee et al. 2020

Cognitive load in the pause-available condition (PA) would be higher than in the pause-unavailable condition (PU), at the overall level. (H1a)

Performance in the PA would be higher than in the PU, at the overall level. (H1b)

Within PA, cognitive load in the pause-taking group (PAn) would be lower than in the no-pause-taking group (PA0), at the overall level. (H2a)

Within the PA, performance in the PAn would be higher than in the PA0, at the overall level. (H2b)

In the absence of intense events, the cognitive load would increase during pauses. (H3a)

In an intense situation, the cognitive load would decrease during pauses. (H3b)

- Overall, allowing pauses increases performance and cognitive load, regardless of whether pauses were taken or not

- When pauses were available, taking those pauses did not further benefit cognitive load or performance

- During pauses cognitive load was lower compared to simulation

- Pupillometry might be a valid measure of the cognitive load next to self-reports

Sohn et al. 2005

Participants learn to pay more attention to task-relevant regions and less attention to task-irrelevant regions with practice over time

- Information-seeking behaviour changed over time; reduction in time on relevant and irrelevant regions

Taub et al. 2017

The more books’ participants read, and the more often they read each book, the fewer concept matrix submission attempts they made, resulting in better performance. (H1)

The longer fixation durations on the book content and concept matrices, the fewer concept matrix attempts, resulting in better performance. (H2)

There will be a significant interaction, such that log file data (number of books and frequency of reading each book) and eye-tracking data (proportions of fixations on book content and book concept matrices) will jointly impact concept matrix submission attempts, with higher levels of all variables resulting in fewer attempts, and thus greater performance. (H3)

- Negative effect between number of books and performance as well as for frequency of books and performance

- But the best performance was associated with fewer books and higher frequencies per book

- Reading more books (quantity) might not improve performance while reading books several times (quality) might do

- No unique association between proportions of fixations on book content or book concept matrix with submission attempts were found, but a significant interaction effect. Low proportions of fixations on book content and concept matrices were related to high performance

- Significant associations between performance and the multimodal predictors as well as for the interaction term. The highest performance was related to a higher frequency of books, fewer books, and lower proportions of fixations on book content or concept matrix

Tsai et al. 2016

Do players with different conceptual comprehension in GBL:

- have different visual attention distributions while playing games? If yes, what are the patterns for high and low-achievement players? (RQ1)

- have different patterns of visual attention transactions (representing the players' control strategies of multi-tasking coordination applied in the game)? (RQ2)

- experience different levels of game flow? (RQ3)

Low comprehension group:

- Higher PFD and PCD in the components area

- Higher mental effort (heatmap)

- Viewed graphical information more frequently than the high comprehension group

- Paid more attention to graphic information according to heatmap analyses (while the high comprehension group spent less attention on graphical and more attention on textual information)

- Low comprehension group tended to get stuck in the message (cues) and out-of-screen gaze while the successful group tend to transfer knowledge and might use out-of-screen gaze as a pausing/reasoning strategy

high comprehension group showed

- A higher sense of control and concentration,

flow experience and visual attention association:

higher flow time distortion fixations on the main task

lower flow time distortion fixations on the message prompt

(van Gog et al. 2005a)

Higher expertise participants spend more time on problem orientation, problem formulation, deciding on actions and evaluating them, while lower expertise participants are more likely to test out the functioning of the circuit to try to generate new hypotheses. (1)

Higher expertise participants’ orientation and evaluation phase will be less cognitively demanding than reasoning, and all these processes are more demanding for lower than higher expertise participants. In the ‘problem orientation’ phase, higher expertise participants will have a higher proportion of fixations on components related to major faults. (2)

Eye movement and concurrent verbal protocol data together show how eye movement data may make to the investigation of cognitive processes. (3)

- High expertise participants spent more time in the ‘problem orientation’ and ‘action evaluate & next action decision’ phase (but not for the ‘problem formulation’ phase)

- Higher expertise groups only differ for mean fixation duration over all phases, but show more fixations on fault-related components, they show:

Shorter mean fixation duration in the ‘orientation’ phase

Longer mean fixation duration in the ‘problem formulation’ phase

Verbal data that reveals predictive behaviour

- Low expertise participants' verbal data show no orientation and an unstructured initial testing approach