Table 5 Instruction and scoring rules of the CCER test items

1

Please derive the two main research questions from the abstract of the study by Koch (2006)

0 = no research question is derived correctly

1 = one of the research question is derived correctly

2 = both research questions are derived correctly

You can find some selected sentences of Koch’s study below. To which part of the introduction do these sentences refer?

Parts of an introduction:

 A) Definition/defining the research topic

 B) State of the art

 C) Relevance of the problem

 D) Research gap

2–5

Sentence 1: ‘A long duration of study, subject changes and dropouts are quite characteristic for university studies in Germany [...]. ’

0 = identified aspect is wrong

Sentence 2: ‘In psychology, responsibility is closely related to perceived control and self-efficacy [...]. These two constructs then influence study success [...].’

1 = identified aspect is correct

Sentence 3: ‘A person is self-determined if he takes responsibility for his own actions. We assume that self-determined students are more likely to demonstrate intrinsic motivation for their studies. This in turn seems to be desirable in pedagogical contexts [...].’

Sentence 4: ‘Schlenker et al. [...] introduce a social-psychological model of personal responsibility and examine its application to study success.’

6

What is the difficulty in operationalizing a latent construct? Please explain

0 = no aspect of difficulty is explained correctly

1 = one aspect of difficulty is explained correctly

2 = two aspects of difficulty are explained correctly

7, 9, 11, 13

Table 1: factor-analyzes “personal responsibility” (main component analysis, varimax rotation, N = 133)

0 = information is wrong, insufficient or not denoted

Which information is given to the reader by the marked numbers: “1”/“2”/“3”/“4”?

1 = information is denoted correctly

8, 10, 12, 14

Table 1: factor-analyzes “personal responsibility” (main component analysis, varimax rotation, N = 133)

0 = information is wrong, insufficient or not assessed

Please assess the information given by the marked numbers “1”/“2”/“3”/“4”?

1 = information is assessed correctly

15

Factor analysis provides us with various quality measures to assess latent constructs. Please name two quality measures and explain under which conditions the measure is assessed as “well fulfilled” (an approximate value is sufficient if you want to specify numbers)!

0 = no measure is outlined correctly AND no condition is explained

OR

One measure is outlined correctly BUT associated condition is wrong (explained)

1 = one measure is outlined correctly AND the associated condition is correctly explained

OR

Two measures are outlined correctly BUT for neither of them the associated condition(s) are (explained) correctly

2 = two measures are outlined correctly AND for both of them the associated condition(s) are (explained) correctly

OR

Two measures are outlined correctly BUT only for one of them the associated condition(s) are (explained) correctly

16

The quality of the presented factor analysis cannot be assessed, if the assessment is solely based on Table 1. Which aspects are missing for a complete assessment of the quality? Please outline two aspects!

0 = no aspect is outlined (correctly)

1 = one aspect is outlined correctly

2 = two aspects are outlined correctly

17

Data collection is carried out in Koch’s (2006) study by means of a written survey with a closed answer format. Which other data collection methods do you know? Please name four additional methods!

0 = one or no method is outlined correctly

1 = three or two methods are outlined correctly

2 = four methods are outlined correctly

18

How do you assess the decision in this study to use a written survey with a closed answer format? Please justify your assessment with two arguments!

0 = no assessment and/or no argument (on the grounds)

1 = assessment and one correct argument (on the grounds)

2 = assessment and two correct arguments (on the grounds)

19

The summary (Koch 2006, p. 1) states: “It is claimed, that personal responsibility for study success […] has a positive effect on study management and performance.” Would you prefer a correlation or a regression analysis to examine this statement? Please justify your decision with two arguments!

0 = no decision and/or no argument (on the grounds)

1 = decision and one correct argument (on the grounds)

2 = decision and two correct arguments (on the grounds)

20–23

Based on the presented correlation matrix included in Koch’s (2006) study, a researcher interpreted the following statement:

(1) “Final university examination grade and commitment to the studies are two independent criteria for success.”

0 = no or incorrectly marked value AND no or incorrect explanation

(2) “Personal responsibility for your own studies and study success are positively correlated.”

1 = correctly marked value BUT no or incorrect explanation

(3) “The two sub dimensions “clarity of purpose” and “significance” are dependent factors.”

OR

(4) “Measuring accuracy of the sub dimension “significance” can be assessed as good.”

Incorrectly marked value BUT correct explanation

Please mark the value that resulted in the given statement (1)—(4) within the presented output (e.g. by circling) and explain in one sentence why the argument is derived correctly!

2 = correctly marked value AND correct explanation

24

Correlation analyses were supplemented by regression analyses. These analyses also show a positive effect of personal responsibility and study success. Nevertheless, the author argue in the conclusion part, that the results do not have any predictive character. That means variance differences in study success cannot traced back casually to variances in taking personal responsibility for their own study

0 = wrong or no aspect is denoted and justified

Why could the authors come to this conclusion? Please denote a central aspect and justify your explanation!

1 = correct aspect is denoted BUT wrong or not justified

2 = aspect is denoted and justified correctly

25

In a following research project, Koch and colleagues developed a training to strengthen personal responsibility of students for their studies

0 = no or one column is sketched correctly

Please sketch a suitable experimental plan in the usual matrix format for evaluating the effectiveness of the new developed training!

1 = two or three columns are sketched correctly

2 = all four columns are sketched correctly

26

Please describe each component of a (quasi-)experimental design for evaluating the effectiveness of the newly developed training!

0 = no or one component is described correctly

1 = two or three components are described correctly

2 = four or five components are described correctly