Skip to main content
Empirical Research in Vocational Education and Training
Download PDF
  • Published:

On the measurement of competency

Empirical Research in Vocational Education and Training volume 2, pages 41–63 (2010)Cite this article

Abstract

Across multiple societal sectors, demand is growing to measure individual and group competencies. This paper unpacks Hartig et al.’s (2008) competency definition as a complex ability construct closely related to real-life-situation performance to make it amenable to measurement. Unpacked following the assessment triangle (construct, observation, inference), competency measurement is exemplified by research from business, military and education sectors. Generalizability theory, a statistical theory for modeling and evaluating the dependability of competency scores, is applied to several of these examples. The paper then pulls together the threads into a general competency measurement model.

References

  • Baxter, G.P. & Shavelson, R.J. (1994.) Science performance assessments: Benchmarks and surrogates. International Journal of Educational Research, 21, 279–298.

    Article  Google Scholar 

  • Brennan, R. L. (2001). Generalizability theory. New York: Springer-Verlag.

    Book  Google Scholar 

  • Cardinet, J., Johnson, S. & Pini, G. (2009). Applying generalizability theory using EduG. New York: Routledge/Psychology Press.

    Google Scholar 

  • Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies. Cambridge, MA: Cambridge University Press.

    Book  Google Scholar 

  • Cronbach, L.J., Gleser, G.C., Nanda, H. & Rajaratnam, N. (1972). The dependability of behavioral measurements. New York: Wiley.

    Google Scholar 

  • Fitzpatrick, R. & Morrison, E. J. (1971). Performance and product evaluation. In R. L. Thorndike (Ed.) Educational measurement. Washington, DC: American Council on Education.

    Google Scholar 

  • Green, B. & Shavelson, R.J. (1987). Distinguished panel discussion on issues in the joint-service JPM program. In H.G. Baker and G.J. Laabs (Eds.) Proceedings of the Department of Defense/Educational Testing Service Conference on Job Performance Measurement Technologies., Washington, D.C.: Office of the Assistant Secretary of Defense (Force Management and Personnel).

    Google Scholar 

  • Hartig, J., Klieme, E. & Leutner, D. (Eds.) (2008). Assessment of Competencies in Educational Contexts: State of the Art and Future Prospects. Göttingen: Hogrefe & Huber.

    Google Scholar 

  • Kane, M. T. (1982). A sampling model for validity. Applied Psychological Measurement, 6, 125–160.

    Article  Google Scholar 

  • Klein, S. (2007). Characteristics of hand and machine-assigned scores to college students’ answers to open-ended tasks. In D. Nolan & T. Speed (Eds) Probability and Statistics: Essays in Honor of David A. Freedman. IMS Collections, Volume 2. Beachwood, OH: Institute for Mathematical Statistics.

    Google Scholar 

  • McClelland, D.C. (1973). Testing for competence rather than testing for “intelligence”. American Psychologist, 28(1), 1–14.

    Article  Google Scholar 

  • National Research Council (2001). Knowing what students know: The science and design of Educational Assessment. Washington, DC: National Academy Press.

    Google Scholar 

  • Ruiz-Primo, M.A. & Shavelson, R.J. (1996). Rhetoric and reality in science performance assessments: An update. Journal of Research in Science Teaching, 33(10), 1045–1063.

    Article  Google Scholar 

  • Shavelson, R.J. (1967). Man’s performance of basic maintenance tasks in lunar gravity. Unpublished master’s thesis, San Jose State College.

  • Shavelson, R.J. (1991). Generalizability of military performance measurements: I. Individual performance. In A.K. Wigdor & B.F. Green, Jr., (Eds.) Performance assessment for the workplace (Vol. II): Technical issues. Washington, D.C.: National Academy Press.

    Google Scholar 

  • Shavelson, R.J. (2010). Measuring college learning responsibly: Accountability in a new era. Stanford, CA: Stanford University Press.

    Google Scholar 

  • Shavelson, R.J., Baxter, G.P. & Gao, X. (1993). Sampling variability of performance assessments. Journal of Educational Measurement, 30(3), 215–232.

    Article  Google Scholar 

  • Shavelson, R.J., Klein, S. & Benjamin, R. (2009). The limitations of portfolios. Inside Higher Education. Retrieved August 10, 2009 from on http://www.insidehighered.com/views/2009/10/16/shavelson.

  • Shavelson, R.J., Roeser, R.W., Kupermintz, H., Lau, S., Ayala, C., Haydel, A., Schultz, S., Quihuis, G. & Gallagher, L. (2002). Richard E. Snow’s remaking of the concept of aptitude and multidimensional test validity: Introduction to the special issue. Educational Assessment, 8(2), 77–100.

    Article  Google Scholar 

  • Shavelson, R.J., Ruiz-Primo, M.A. & Wiley, E.W. (1999). Note on sources of sampling variability in science performance assessments. Journal of Educational Measurement, 36 (1), 61–71.

    Article  Google Scholar 

  • Shavelson, R.J. & Seminara, J. (1968). Effect of lunar gravity on man’s performance of basic maintenance tasks. Journal of Applied Psychology, 52(3), 177–183.

    Article  Google Scholar 

  • Shavelson, R.J. & Webb, N.M. (1981). Generalizability theory: 1973–1980. British Journal of Mathematical and Statistical Psychology, 34, 133–166.

    Article  Google Scholar 

  • Webb, N.M., Shavelson, R.J., Kim, K-S & Chen, Z. (1989). Reliability (generalizability) of job performance measurements: Navy machinists mates. Military Psychology, 1(2), 91–110.

    Article  Google Scholar 

  • Wigdor, A.K. & Green, B.F. Jr., (Eds.) (1991). Performance assessment for the workplace (Vol. I). Washington, D.C.: National Academy Press.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. 308 Cubberley-School of Education, Stanford University, 485 Lasuen Mall, Stanford, CA, 94305-3096, USA

    Richard J. Shavelson

Authors
  1. Richard J. Shavelson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Richard J. Shavelson.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shavelson, R.J. On the measurement of competency. Empirical Res Voc Ed Train 2, 41–63 (2010). https://doi.org/10.1007/BF03546488

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF03546488

Keywords