Occupation-specific social competences in vocational education and training (VET): the example of a technology-based assessment

Abstract

Background

The difficulties regarding the definition and measurement of social competences, crucial to the research in vocational education and training, are well known. In this paper, we show, based on the example of medical assistants in Germany, that these skills have to be assessed in an occupation-specific way, and develop an appropriate simulation-based Situational Judgment Test. Competence modeling, the translation into the test format and results of a data collection are discussed in detail for the dimension of communicative strategies.

Methods

Based on empirical and theoretical analysis, we derived a normative competence model underlying the development of a video-based situational judgment test. The final assessment contained 2 × 6 prototypical interactional types of situations, presented in short videos. At the end of every scene, questions of multiple choice format or open questions were asked for each of the social competence dimensions (emotion regulation, perspective coordination, communication strategies). In regard to the evaluation, for the dimension of communication strategies, for example, open answers were rated by using a weighted checklist giving a final score for each situation or all over the test. After two pre-evaluations on N = 236 and N = 260 pupils at the end of their apprenticeship towards becoming medical assistants, validity aspects due to e.g. format were analyzed and edited. A final inquiry on N = 405 pupils confirmed the expected reliability and dimensionality aspects.

Results and discussion

Validity measures could additionally be confirmed by a structural equation model (N = 405, Χ² = 594.319, df = 518, CFI = .96, RMSEA = .02 and SRMR = .07) showing unexpected but meaningful underlying factorial structures for the dimensions of emotion regulation and communication strategies. Results of the test show that emotional dissonance is present in every situation. Regarding test-scores, high levels are rarely achieved for all dimensions.

Conclusion

By using an occupation-specific approach, we show that social competences can be defined and performance-relatedly assessed. However, results show that the competence levels are marginally satisfactory for all dimensions, considering the stressful environment of medical assistants. Hence, contents of this nature should be integrated more concretely in their educational pathway.

Background

In recent years, competence research and diagnostics in vocational education and training (VET) have made great progress in providing an empirical foundation for occupation-related competence models and their assessment. Empirically verified competence models have been introduced for many occupations in the industrial-technical and commercial sector (Nickolaus and Seeber 2013). These models can be converted into technically high standard assessment procedures. This makes the evaluation of detailed competence levels possible on a big scale, validating the underlying structural models empirically. Still, the occupations’ dimensions of social competences are not, just partially or only superficially treated. This is due to the nature of the concept: Although the amount of research on social competences has increased considerably in the past decades (Kanning 2003), a congruent understanding is still lacking, which makes objective comparisons between different measurements of these essential aptitudes virtually impossible (Monnier 2015). Already in Seyfried (1995) described “stumbling blocks” causing the fundamental difficulties on the way to measuring social competences that still apply to today’s research on the matter. One fundamental difficulty lies in the vagueness of the terminological construct of “social competence”. In the field of psychological research for example, Kanning (2005) points out that definitions and systematizations are frequently based on assumptions of plausibility, which furthermore are not sufficiently supported by empirical data. In addition, the attempts at definitions vary, depending on the theoretical approach: In the context of developmental psychological theories, for example, emphasis is placed on adjusting social competences to social standards and rules, while the assertion of one’s own interests is more strongly stressed within the framework of clinical psychological approaches (Kanning 2005). This disagreement is further enforced by the existence of various related concepts such as “emotional intelligence”, “social skills” or “social intelligence” which make a sensible and content-based differentiation from the concept of social competence nearly impossible (Monnier 2015). Bayer, Ditton and Wohlkinger (2012) for example point out that the definition problem also exists for the concepts of social skills, interactional skills, being assertive, self-esteem, social- or interpersonal competence, where no clear delimitation of the concepts is possible (Döpfner et al. 1981; Rose-Krasnor 1997; Arnold et al. 2012). These difficulties are also found in VET research (Euler and Bauer-Klebl 2009; Euler 2012; Nickolaus and Seeber 2013). Different dimensions and aspects with reference to social competences have been addressed, for example in studies dealing with moral judgment in commercial occupations (Beck et al. 1996, 1998), communicative abilities (van Buer and Matthäus 1994; Wittmann 2001) and teamwork skills (Goméz 2009). In comparison to these individual dimensions, studies on the relationship between these facets working together as the composite factor of social competences, or overlapping and differing contents in different occupations are missing so far (Monnier 2015).

The aim of this paper is to identify how definition problems can be solved by restricting the framework to a controllable set of requirements given by the occupation and by allowing a clear scope of validity of the findings and characterizations. We will show how these occupational requirements are identified and how competence dimensions can be derived from them by using the example of medical assistants in Germany. We will then present how the competence model can be translated into an adequate test format. Due to the limited space available, this will only be done in detail for the dimension of communication. Finally results and their meaning for practice and the scientific community will we presented.

From competences to social competences

The discussion on competences has become more important since the intent to make them a foundation of educational qualification frameworks has arisen in order to enhance success in “work, citizenship, and self-actualization” (Dede 2007). A widespread definition of competences by Weinert (2001) says that competences are “cognitive abilities and skills which are available or trainable for individuals to solve specific problems as well as associated motivational, volitional and social willingness and skills to use problem solving strategies successfully and responsibly in variable situations”. The first part of Weinert’s definition implies that competences are a dynamic (trainable) cognitive concept, representing an essential prerequisite to competent behavior. Transferring this presumption onto social competences, modeling and measuring them could only be possible if the focus is set on the underlying “social- and emotional-cognitive dispositions’’, and not the final (social) behavior (Tschöpe et al. 2016). The awareness of social competences’ nature is important for the following discussion on the actual state of research.

Stumbling block: definition and dimensionality of social competences

Beyond all intra-disciplinary and trans-disciplinary definitions there is agreement within scholarly analysis that social competences are determined through an interplay of several competences and thus, are a multi-dimensional construct (Schuler and Barthelme 1995; Kanning 2005; Euler and Bauer-Klebl 2009). Which competences these are specifically and what terminology should be used to differentiate them from each other is still an open question (Kanning 2005). The available lists and catalogues of terms like communicative ability, assertive ability, ability to accept criticism etc. are still just hypothetical constructs. Also, because of their dependence on the values and attitudes of the individuals involved and on the rules, standards and professional behavior expectations shaping the setting in question, their influence on social interactions can only be empirically determined in specific situations and contexts (Kanning 2005). Although social competences usually have a positive association, such an assessment can vary greatly against the background of different and possibly incompatible values and standards (Kanning 2005). As a consequence, one has to consider that social competent behavior can mean very different things depending on context and situation: What is considered as demonstrating assertive ability in one field may appear as uncooperative or even overstepping the boundaries in another area. At the same time, some facets of social competence may be negligible in certain contexts or occupations but central in others. Descriptive and evaluative statements on social competences can therefore only be made and empirically substantiated in defined areas and situations with clear reference to values and standards, which differentiate socially adequate from socially inadequate behavior.

Stumbling block: measuring social competences

Considering the foregoing paragraph, the fact that the majority of social competence measurements aspire to enlighten general social competences makes their assessment very difficult, if not impossible. They often overlook the specific requirements on social competences in particular situations or occupations, so they seem hardly suitable for use in occupational diagnostics (see more details on this in Dietzen et al. 2012; Dietzen et al. in press). Also, established methods, such as Kanning’s (2009) Social Skills Inventory (ISK), are mostly based on self-assessment questionnaires. Usually, they have only relatively low correlations with external assessments and test-based data (Nickolaus and Seeber 2013). Because of their possibilities of corruption, they are rated as problematic in terms of a valid diagnosis in competence measurement (Bühner 2011). There are only few studies that relate to social competences in a particular occupational field, one example being the work of Gartmeier et al. (2011) on teachers and physicians and, there is clearly a need for a systematic approach to the problem.

Situational judgment tests

As said before, established methods are mostly based on self-assessment questionnaires. They can be combined with standardized role-plays as done by Gartmeier et al. (2011) to gain information about the actual performance level. But in the long-term and large-scale perspective this would represent an enormous time and work investment, making it uneconomic or even impossible to implement. Also, having in mind the competence definition by Weinert (2001), we recall that competences are “cognitive abilities and skills which are available or trainable for individuals to solve specific problems”. They do not necessarily arise in performance in every situation. The difficulty in identifying these “underlying” cognitive dimensions becomes clear.

In aptitude diagnostic practice, simulation-based testing is used, in which people are supposed to describe their fictional behavior in certain typical situations, so-called Situational Judgment Tests (SJT) (Ployhart and MacKenzie 2011). In comparison to role-plays, questions can be focused in detail on different competence dimensions, allowing the identification of structures and levels, while the observation of the act itself would only show the resulting conglomerate of the dimensions working together (Lane and Rollnick 2007). SJTs are mostly used in the selection of executives and designed to fit specific professional positions, which is why specific procedures need to be developed for each context of requirements. One such context is the occupation itself. Based on a requirements analysis to determine the work tasks, interactions and tools composing the occupation one could imagine defining tasks that can display the participants social competences in the specific context of their work. Tschöpe (unpublished manuscript 2012) for example has developed an SJT for the measurement of counseling skills among bankers, containing social competences as a major pluri-factoral component. In this paper we want to present an SJT related to the work of Tschöpe (unpublished manuscript 2012) in the medical field, more specifically for medical assistants in Germany. The project, called Competence Measurement based on Simulations and adaptive Testing in Medical Settings (CoSMed), was funded through the ASCOT (technology orientated competence measurement in vocational education and training) research initiative by the Federal Ministry of Education and Research (BMBF). The overall CoSMed project develops computer-based test procedures on a collaborative basis for measuring the occupational competences of medical assistants. In this paper, only the aspect of social competences will be treated but compared with the other findings in the end.

Methods

Creating a SJT for medical assistants’ social competences

As described above, past research on the matter of social competences has shown that they are strongly context-dependent, making it quasi impossible to find an all embracing definition (Monnier 2015) and thereby precluding the transition to objective competence assessments. Reducing the context to a manageable amount of influencing factors, i.e. those resulting from a well-defined professional role, opens the road to a quantitative analysis.

Medical assistants

The vocational education program to become a medical assistant in Germany is a three-year dual education. They must quickly appraise a sick person’s situation and emotional state, calm down agitated patients and stabilize them in difficult situations, and carry out certain procedures that patients find unpleasant. At the same time, the occupation calls for good coordination within the practice team. So alongside healthcare and business competences, social competences are fundamental to the occupational profile of the Medical Assistant. Germany’s training regulation for the recognized occupation, dating from April 26, 2006, enumerates various tasks in patient care and consultation, which especially underscore the importance of social competences when working in this occupation.

Key steps toward the SJT

SJTs start with a variety of situations, which are typical for the occupation. The subject group then assesses how they would behave in reality or evaluate behavioral options in terms of their quality. The situations can be presented using various methods (e.g. in writing, audio recording, video). They should be selected to represent the occupational field for which the test was devised as closely as possible. Some important requirements must form the basis of this testing approach to the modeling and measurement of occupation-specific and social competences. The following therefore explains the key steps in the generation of the competence model and its implementation into a test format by means of the project CoSMed.

Occupational domains and requirements analysis

A requirements analysis to determine the work tasks, interactions and tools, etc. of the environment for which the test is designed should always form the basis of an SJT. The Critical Incident Technique (Flanagan 1954) is often applied in this case. This method is particularly suited to determining success-critical situations as this involves questioning experts about challenging situations in which major differences between competent and less competent employees become apparent. Questions might also be asked about which types of behavior might be shown by competent and less competent employees in the respective situations. The situations and behaviors determined in this way serve as the basis for the situation descriptions and the alternative answers in the SJT. This was the approach we adopted in the CosMed project. A qualitative study was initially completed in order to analyze the occupational requirements. Besides a comprehensive analysis of documentation covering training regulations, the framework curriculum, examination regulations, training material, etc., the study was based primarily on group discussion with practicing experts and medical assistant trainees in the third year of training as well as on semi-structured interviews with doctors, medical assistants with vocational experience and trainees (N = 13). In the interviews we used the Critical Incident Technique (Flanagan 1954) to identify challenging situations. These situations were used for the test construction later on. A dual approach was selected for the job and requirements analysis (Krumm et al. 2012): Following a “bottom up” approach, respondents were asked to describe specific favorable and less favorable behavior in certain situations and, in the following evaluation, we concluded on the underlying competences on which these behaviors were based. Respondents were also asked which social competences are important for their work and to then relate these competences to situations and behaviors. Since the understanding of social competences remains controversial, answers of different nature were given and, once again, we concluded on the underlying competences, based on the understanding that they are a dynamic (trainable) cognitive concept, representing an essential prerequisite to competent behavior (Weinert 2001). The combination of these approaches resulted in a comprehensive practice based picture of work-related requirements for medical assistants’ social competences.

In addition to this study an explorative analysis was conducted of approx. 1800 job adverts for medical assistants. By word counts we looked at the number of mentioned demands related to social interactions. This clarified the requirements which exist in terms of social and communicative competences of medical assistants from the perspective of personnel managers in hospitals and practices in different specialized medical fields. The most frequent demands were abilities like “being friendly”, that are strongly composite and of unclear expectations. Finally, behavioral observations in physicians’ offices would also have been helpful; however it was not possible to implement these due to the very strict regulations regarding data protection in the area of health.

Deriving the competence model from empirical and theoretical analysis

Systematization of authentic situations

The data and results of the empirical exploration were evaluated in order to derive prototypical situations with distinctions made for interaction partners, situational references and types of conflict. This was approached as follows. A range of situations which presented particular challenges in terms of social capabilities were compiled from the empirical preparatory work and these were summarized by type (for more information see Dietzen et al. 2016). In order to guarantee authenticity, it was ensured that these represented prototypical interactional situations in the occupational fields of medical assistants by using an online survey in which 28 practitioners were asked if they thought that the scenes were in step with the actual practice, authentic and frequent. In these situations three types of distinctions could be made: The first distinction was made between internal and external types of conflict (Theuerkauf 2005). External conflicts describe those situations, which involve an actual dispute with another person, e.g. complaints from a patient. Internal conflicts were also represented. These describe situations in which the medical assistant is in conflict with “himself or herself” while being confronted to another person, e.g. if she/he has to deliberate between several urgent jobs and becomes insecure as a result. Since we used the Critical Incident Technique (Flanagan 1954) during the interviews and workshops, situations could easily be attributed to the two groups of conflicts. The second distinction was made between the different locations in a prototypical medical practice. The scenarios could be attributed to three main areas, which proved to be key for social situations in the domain analysis: the reception area, the treatment room and the practice team’s break room. Finally, with respect to interaction partners, a distinction was made between patients and their relatives and team members. The reason for this is that patients and their relatives must be confronted by adopting a much more professional role, while relationships with team members are in most cases more casual, but may involve potential conflict due to hierarchical differences. Many participants reported explicitly that they were feeling and behaving differently with their colleagues in comparison to their patients [for more details see Srbeny et al. (2015)].

In the end, six types of situations could be defined:

• Reception area with patients or their relatives and either an internal or an external conflict

• Treatment room with patients or their relatives and either an internal or an external conflict

• Practice team’s break room with colleagues and either an internal or an external conflict.

Systematization of social and communicative competence dimensions

The social and communicative behaviors cited by the respondents as being necessary to successfully pass the above described critical situations were systematized, standardized and “translated” into theory based competence terms or concepts which could be distinguished from one another. This proved to be particularly difficult because personality characteristics, capabilities, skills or socialization requirements were generally subsumed under the concept of social competences without any distinctions being made by the respondents. An example and its handling is given in the next paragraph. In the end, the categorized information was analyzed once more; both “top down” and “bottom up”. This was then processed and supplemented, and the overall outcome for the key competence areas in the occupation of medical assistant was determined using a rating procedure by psychologists and sociologists.

In the next step that was partly done in parallel, psychological, pedagogic, work-based sociological and communication theories and models were used for the final implementation of the occupation-specific models of social and communicative competences of medical assistants. These models and theories were integrated into the empirically developed competence dimensions in order to clearly distinguish between the individual sub-dimensions of social competence. Also, missing elements from a scientific point of view that were not usual in the field of practice were added by using theoretical approaches for the requirements of interactive activities (Greene and Burleson 2003; Hacker 2009; Nerdinger 2011). The following example shows how the translation into theoretical competence facets and the addition of theoretically meaningful but empirically not named dimensions were made: many of the practical experts mentioned “being empathic and compassionate” as a “competence” that medical assistants should have in social interactions. This means that the person would have to feel what the patient is feeling too. But, a strong emotional state could have major impact on the parallel execution of “non-interactional” occupational tasks. Therefore, instead of integrating empathy or compassion as an element of our model, we used the theory of perspective-coordination by Selman (2003). It represents a cognitive analysis of all present perspectives (the medical assistant’s, the patient’s but also the super-ordinate medical practice’s one) and detecting their connections and dependencies.

This combination of practical-empirical and scientific-theoretical elements was the next step towards the competence model. The reference to theoretically established concepts was also used to derive the criteria for the grading of the final test responses later on. In the end, normative aspects contained in the theories for distinguishing between favorable and less favorable behaviors were extracted for this purpose and were compared to and supplemented by the results of the dialogue with experts in practice and diagnostic experts. On this basis, it was finally possible to derive a competence model of social and communicative competences for medical assistants, which is presented graphically in Fig. 1. The visible division of competences in the model into a light red and dark red area indicates the distinction between the above-described internal and external conflict in the situations. Each competence is also shown as half light-blue and half dark-blue, differentiating between the interactional partners (patients and their relatives and colleagues). The initial core competence of a medical assistant is emotion regulation. This involves controlling one’s own emotions internally, even in difficult social situations and, in doing so, being able to respond appropriately externally. For this dimension we used theories on emotion regulation strategies (Gross 2009) containing for example strategies as “acting out” (meaning that one does not regulate emotions but display them “rawly” instead), “inhibition of emotion expression” (meaning that one does not display the feeling that is on the “inside”), “distraction” (meaning that one tries to change the attention focus away from the source of emotion, to change one’s inner emotional state), etc. Also, we looked at the differentiation between “deep acting’’ and “surface acting’’ (Hochschild 1983; Giardini and Frese 2006) meaning that there is a difference if I just display adequate feelings while the felt emotion is different or if I also try to change my inner state.

Fig. 1

Medical assistants’ social competences model in CoSMed

The second core competence is described as perspective coordination based on the developmental theory by Selman (2003). For this to be implemented adequately, the medical assistant must be able to understand her/his own and the conversation partner’s perspective, emotions and needs, even in difficult situations, and, at the same time, to maintain an overview of the practice requirements and also to relate all aspects to one another as constructively as possible.

Communication strategies come into the model as the third core competence. Competent communication strategies are evident in the way the medical assistant guides the communication towards a good solution for both parties, even in difficult situations, and can make the conversation partner feel they are being taken seriously and are understood. We will use this competence dimension to describe the implementation process of practical-empirical and scientific-theoretical into a normative model in more detail:

Empirically we repetitively found “being nice” and “friendly” as an important behavior for practice experts. During interviews we were told that some practical applications of being “friendly”, usual in the field, could be to always start an answer by saying “I understand you”. From a theoretical point of view, we looked at traditional communication models as for example Schulz von Thun’s communication square (2004) or Rosenberg’s Nonviolent Communication (2006). These theories say that there are diverse ways of understanding and sending a message and that for example establishing a clear common ground (Clark and Schaefer 1989) by repeating what happened and which possibilities exist (sharing one’s knowledge) can clarify potential misunderstandings. Finally we integrated these various concrete elements into a normative category system of communicative strategies that will be described in the next chapter.

The two sub dimensions of listening and clear speech were, in fact, also identified as important social competences of a medical assistant. However, due to a lack of operational options in a written format, these were not included in the test.

Development of the measurement instrument

Due to the criteria of economy and time, we used a video-simulation based SJT-format: Participants are confronted to situations in a first-person perspective giving a more realistic impression, lessening identification effects (Folkes 1982) and increasing the level of emotional involvement, as they automatically become one of the interactional partners. For the identification with the workspace they were shown a filmed introduction of a fictitious practice in which the following video scenarios took place.

Precisely, two prototypical scenarios for each of the above-described six situational types were transformed into a video of 15–56 s. Each of the video blocks starts with a short text-based introduction. The introductory text in block 1, for example, reads as follows (translated from German): “An emergency patient has come into the practice. He has been given priority over the other patients.” The participants then see a video in which the respective interaction partners talk directly to the camera. In the example of block 1, the patient steps towards the reception desk and shouts that he has been waiting for over an hour, that he has a meeting and has to go to the doctor’s room immediately. After each video, participants are asked questions for each one of the core competence dimensions. In order to cater for the requirements of each of the dimensions, different answer formats were tried in the test pilots. Text-based questions with open answer categories (e.g. “What would you answer Mister Fischer?” for the dimension of communication strategies) and with responses in multiple-choice format (e.g. “How would you deal with your emotions in this situation?” and then selecting one of four possible answers for the dimension of emotion regulation) were selected for the final version. The evaluation of the open questions was done by standardized criteria derived from our normative theoretical-empirical model. For the dimension of communication strategies for example, we constructed a checklist, where important elements named in the empirical study were integrated with communication-theoretical elements. In many validation loops, the elements were processed. For example theoretical elements were tested for their practicability, the identification of individual elements was tested by repeated interrater-reliabilities for three raters on n = 80 (with over 98 % of agreement, varying in both dimensions with open question format between r = .7 and r = .9 and, even in the case of external individuals without rating-training between r = .6 and r = .9) and insecurities were reported back for a group-rating. Also, the importance of the elements was identified by comparing their impact on the general impression of the quality of the answer (Sust et al. 2009) and then operationalized. Finally an individual checklist for each of the twelve scenes was defined. For block 1, the scene with the angry patient presented above, the checklist contains two criteria of exclusion that will automatically set the participants points to zero. The first one concerns the “tone”, for example giving a sarcastic answer and the second one is in respect to the occupational possibilities, in this example “giving in”: The medical assistant cannot give into the patient’s demand, since there is an emergency in the practice; she has to find another solution. Furthermore, the checklist contains positive elements as for example giving an explanation of the situation, in this case explaining that there was an emergency and, by that creating a common ground (Clark and Schaefer 1989). But also negatively rated elements as for example taking it personally (e.g. “It’s not my fault!”) are found in the checklist. The final score for communication strategies is calculated by taking the sum of points for each block. No differentiation of negative sums is made and hence, all set to zero. For a cross-scene standardization, the sum is then divided by the scene’s maximum and the quotient’s mean over the 12 blocks is calculated as final score.

Validity aspects

Due to the fact that the approach of measuring social competence as a “performance test” is new, verifications were made at various points in order to ensure the validity. The authenticity of the test was assured through direct feedback of the participants during the pilot phase as well as in an online-survey with practitioners and experts in the field described above. The results were very satisfactory (e.g. for authenticity N = 28, M = 5.1, Std.Dev. = 0.23 on a scale from 1 to 6). The content validity in CosMed was developed over several steps. As already described before, at the start of the project a comprehensive requirements analysis was conducted with various interdisciplinary sources from academia and practice. The competence model was derived based on this by integrating first-hand analyses and general models of social competence (e.g. Kanning 2005). This occurred in two directions; bottom-up and top-down (Krumm et al. 2012). At the end of this stage of ensuring content validity, psychologists and sociologists with diagnostic training carried out an expert rating and selected competence dimensions. As a following validity check, we looked if the video scenarios really did trigger the emotions required, as this contributes significantly to the evaluation of the results. Particular attention was paid here to whether the scenarios which cover the area of internal conflict create a feeling of insecurity and whether those scenarios covering external conflict cause anger. This was determined by asking openly about the emotions experienced, and the strength of this emotion. The emotions proved to be those, which were intended. The strength of the emotion had a quasi-normal distribution. The individual answer formats were tested in a third step. Different multiple-choice variants were compared to start with. These proved to be unsuitable for the dimensions of perspective-coordination and communication strategies (emotion regulation: Cronbach’s α = .66–.87, perspective-coordination: Cronbach’s α = .26–.28 and communication strategies: Cronbach’s α = .31–.46) partly, due to the strong effect of the school-leaving qualification or native language, unsatisfactory reliability measures and a low level of acceptance among participants (e.g. communication strategies: Cronbach’s α = .71–.85 if German is one’s mother tongue). All these measures turned out to be at least satisfactory to good after changing to an open answer format (perspective-coordination: Cronbach’s α = .75 and communication strategies: Cronbach’s α = .68).

Results and discussion

Data collection and sampling

The test module was used in three data collection phases with a total of N = 901 medical assistant trainees in the third year of training. All data collections took place in vocational school classes in Germany using the schools’ own computers. The nature and aims of the data collections were explained to the trainees beforehand using an information letter and participation was voluntary and anonymous. On top of the competence measurements, participants completed a questionnaire containing information about their social and educational context, as well as a self-reported estimation of their general level of social competences on a scale from one to ten, composed by Baethge et al. (2015) during all three phases. This allowed identifying critical factors that could be considered and if necessary corrected for the final test (third phase). The first two data collections were used to pilot the instrument. The focus was much less on the level of performance of the participants and much more on checking the quality of the instrument in accordance with criteria relating to test theories and content. The initial pilot was completed in early 2013 with N = 236 trainees, the second pilot then followed in winter 2013/2014 with N = 260 trainees. We have omitted detailed descriptions of these two samples here. However, their distributions are strongly comparable to the sample for the main survey described in the following.

The main survey took place from April to June 2014 and was conducted with a further N = 405 trainees in the third year of training. The average age of the cohort, 21.3 years, was slightly lower than the value of approx. 22.7 years for the cohort in the Germany-wide vocational education and training statistics for the occupation (BIBB 2014). The proportion of female trainees at 98.7 % was exactly the same as for the 2013 cohort (BIBB 2013). School leaving qualifications were distributed similarly in both the sample and the population, although more higher-level-school leaving qualifications occur in the sample. This may be linked to the fact that schools tend to provide more highly performing classes for such data collections.

A question was asked regarding the country of origin in order to capture the migration background of the participants (Baethge et al. 2015). This was also similarly distributed in both, the sample (Germany = 87.8 %) and the 2013 cohort (90.7 % cf. BIBB 2014). German was specified as the native language by 57.3 % in the sample, a further 39.4 % spoke both German and another language in their family of origin and 3.3 % spoke only another language. This shows that a relatively high proportion of trainees have grown up bilingual.

The data collection was based on a random sample and was only conducted in three German states for reasons of cost and efficiency. Taking these limitations into account, however, the transferability of these results to other medical assistant trainees in the third year of training may be assumed to be good on the basis of the distribution of the demographic data.

Results

All analyses were made using the R software (R Core Team 2015) with the psych (Revelle 2016) and the lavaan (Rosseel 2012) packages. The statistical analysis of the quality of the test, based on mean values and sum scores, delivers adequate to good results. The individual competence dimensions which are measured using an open answer format exhibit satisfactory to high inter-rater correlations (r = .6 to r = .9). Below, we describe the results for our three dimensions, of which only the one on communication strategies will be discussed in full detail. Confirmatory factor analyses (CFA) for the competence dimension of perspective coordination with N = 228 very clearly exhibit a single factor structure (Χ² = 62.017, df = 54, CFI = .97, RMSEA = .03 and SRMR = .05). By contrast, the dimension of emotion regulation at this point in time indicates a theoretically and statistically acceptable four-factor model (N = 405, Χ² = 93.189, df = 49, CFI = .91, RMSEA = .05 und SRMR = .03). For the dimension of communication strategies we found a one-, a four- and a five-factor model after an exploratory factor analysis. Since the five-factor model makes sense with regard to the content and explains a total of 44 % of the variance (factor 1: 10 %, factor 2: 10 %, factor 3: 11 %, factor 4: 6 %, factor 5: 7 %) we conducted a CFA by using the maximum-likelihood procedure with a promax-rotation given that the components correlate. The model (see Fig. 2) shows how the five components were composed and the individual loadings of the items on the latent factor. Also, the model fits (N = 405, Χ² = 32.904, df = 37, p = .662, CFI = 1, RMSEA = .0 und SRMR = .047) are very satisfactory. Still, the composite reliabilities for each one of the factors (“confronted with strong sadness”-CR = 1, “confronted with bad diagnosis”-CR = 1, “confronted with distributional conflict”-CR = .61, “confronted with anger”-CR = .44 and “confronted with mobbing”-CR = .63) show that there are some further analyses to do. Also, three of the five factors consist of one single item and for some items the loadings are low.

Fig. 2

Factor model of the dimension of communication strategies found in CoSMed

At this point we integrated the three competence dimensions in one model (see Fig. 3). The model shown in Fig. 3 with N = 405 shows the following values: Χ² = 594.319, df = 518, CFI = .96, RMSEA = .02 and SRMR = .07. A significant correlation is evident here between the dimensions of communication strategy and perspective coordination (r = .67, p = 05). The current measures of emotion regulation also exhibit a correlation of r = .39 (p = .03) with the communication strategies, although these show no significant relationships with perspective coordination.

Fig. 3

Structural model of the social competence dimensions in CoSMed

After completing basic analyses for the functionality and dimensionality of the test, we found interesting test-outcomes. For example, the results of the competence dimension emotion regulation show that there is a significant difference between the strength of the felt and shown emotion, so-called emotional dissonance in each one of the 12 scenes (Table 1).

Table 1 Paired samples T test between felt and shown emotions for the 12 situations

The distributions of the final score of perspective coordination show that over a third of the trainees manage to achieve level four of seven, meaning that they are mostly able to perceive the perspective not only of an individual person involved (e.g. one’s own) but also the perspective of all people involved in the interaction and when doing so to analyze at least one of the perspectives in depth.

With regard to the competence dimension of communication strategies, Fig. 4 shows how the final scores are distributed. 99 % of participants have an acceptable tone and mostly do not use one of the other criteria of exclusion; still the distribution shows that the mean (M = 0.37 on a scale from 0 to 1) is very low. For example, participants are rewarded with more points if they communicate “authentically“(in their own words) in order to comfort somebody as compared to the use of the usual “empty phrases”. But, distributions show that if comfort is given at all, 87 % use “empty phrases”. Trainees also differ strongly in the amount of action relevant information given to the patients, their relatives or colleagues. While this is counted in 10 scenes, information is given M = 5.8 times with a large standard deviation of 1.7.

Fig. 4

Distribution of final score for the dimension of communication strategies in CoSMed

Finally, the self-assessed level of general social competences on a scale from one to ten exhibited no correlation with the test values (Spearman’s-rho with emotion regulation r = .012, p = .861, with perspective-coordination r = .009, p = .861, with communication strategies r = .024, p = .543) in CoSMed and did not bring further information to the structural equation model.

Discussion

We found evidence of strong dissonance between the participants experienced emotion and shown emotion, meaning that the emotional expression is adapted. Emotion regulation strategies that belong to the family of surface acting (in comparison to deep acting or “no acting”) can have long-term consequences for the well-being of the medical assistants and, for example, may lead to apathy and insensitivity and even to depression (Zapf 2002). It has to be assumed that a strategy, which reduces the emotional dissonance by a conscious reevaluation and reinterpretation of the situation, represents a healthier way to deal with your own emotions in the long term. Furthermore, the latter strategy of “deep acting” (Hochschild 1983) also results in an expression of feelings, which will be experienced as authentic by the conversation partner. However, this strategy was selected around five times less frequently in the sample. This result is significant for the very stressful day-to-day work of the medical assistant and calls for information to be provided about potential ways to deal with your own emotional state in the course of the job-training.

Concerning the perspective coordination level, it seems good to see that the perception of both parties, while being able to analyze at least one of them deeply is achieved by a third of the participants. This occurs despite having to constantly coordinate practice interests with, in some cases, the opposing interest of the patient under pressure of time. It is apparent that the strength of the own feeling may slightly reduce this performance, which once again demonstrates the importance of handling one’s own emotionality well. Also, this means that about two-third do not achieve this level, making praxis coordination more difficult, since it depends on the perception and organization of all represented parties.

Concerning communication strategies, the found factors make sense regarding the situation types. It seems that communication is different whether one is confronted with strong sadness that one has no time to take care of, with bad diagnosis, with distributional conflicts, with anger or even with mobbing. This fits assumptions that are used in popular trainings on social competences, as for example from Hinsch and Pfingsten (2007), where three prototypical types of situations are used (to assert one’s right, to maintain relationships, to bid for sympathy). Still, for the composed score we see that the scores are still very low. One could imagine concrete trainings in formulating authentically or making participants attentive to the important aspects in the five types of situations for example, to enhance communication strategies.

Finally, an interesting finding is that the self-estimated general social competence level does not correlate with the social competence test dimensions.

Conclusion

The general deficit in terms of research on social competences has been known about for a long time while the requirements for these competences as a basis for vocational and specialist professionalism are growing. In this article we have shown how the definition and measurement difficulties of these competences can be handled, i.e., by using the reduced context of a specific occupation, and the concomitant set of social rules and values, an unambiguous definition becomes possible. The translation into Situational Judgment Tests can then be done by detecting work tasks, interactions and tools, etc. of the specific work environments by using a requirement analysis. This represents the basis for the competence model containing all necessary competence dimensions for achieving the occupational demands found in the previous investigation. In a final step Situational Judgment Tests based on an authentic environment can be devised, allowing an individual statement on the social competence dimension’s levels. First results in CoSMed show, that the distribution of competence levels is not satisfactory for all dimensions considering the stressful environment of medical assistants. It becomes visible that information and support in the area of social competences appear to be urgently needed during training for occupations with a great amount of social interactions. That is why future research should emphasize the development of models of learning for the underlying competence dimensions. This would support professional conduct when dealing with patients, clients, colleagues, etc. and facilitate a personally satisfactory and sustainable way of dealing with critical situations in a socially demanding occupation. Thus, a detailed analysis on the situational dependencies and factorial distributions has to be made to clearly define the model’s scope of application. Also, in regard to the newly emerging field of research, more comparable studies should be done for validation purposes. Finally, one could argue that the three dimensions can also be found in other occupations and they most probably would. However, their peculiarity and structure can only be holistically described in a clearly defined framework of for example a specific occupation. Yet, this does not exclude an inter-contextual influence, meaning that a medical assistant achieving a high level in one of the social-communication dimensions of their occupation could probably achieve a high level in the same social-communication dimension in another occupation more easily than a medical assistant with a low level. Assumptions like these should also be content of future research.