Competency Research in Higher Education: Conceptual and Methodological Challenges and Perspectives for Future Interdisciplinary Research

In recent years, an increasing number of research projects have been dedicated to competency assessment not only in school and vocational education, but also in higher education (Blomeke, Zlatkin-T...

Valid Competency Assessment in Higher Education

The aim of the 15 collaborative projects conducted during the new funding phase of the German research program Modeling and Measuring Competencies in Higher Education—Validation and Methodological Innovations (KoKoHs) is to make a significant contribution to advancing the field of modeling and valid measurement of competencies acquired in higher education. The KoKoHs research teams assess generic competencies and domain-specific competencies in teacher education, social and economic sciences, and medicine based on findings from and using competency models and assessment instruments developed during the first KoKoHs funding phase. Further, they enhance, validate, and test measurement approa…

Validation and Applications of KoKoHs Assessments

Modeling and Measuring Competencies in Germany

In order to meet the challenges facing higher education policy and practice, and initiate the necessary fundamental assessment-research, the German Federal Ministry of Education and Research (BMBF) established the Modeling and Measuring Competencies in Higher Education (KoKoHs) research program in 2011. The program was part of the BMBF funding priority Research on Science and Higher Education aimed at building a capacity for empirical research in higher education.

Diagnostik von studentischen Kompetenzen im Hochschulsektor

An Overview of Assessment Practices

On a global scale, attempts to specify conceptually academic learning outcomes have been made mostly in OECD countries. Definitions of learning outcomes can be used not only as an orientation for accrediting degree courses and universities; in some countries, they are used also as a basis for the development of assessment methods. The following will provide an overview of various key approaches and projects in this field.

Überblick zu nationalen und internationalen Assessments

Zur begrifflichen und konzeptionellen Prazisierung dessen, was unter akademischen Lernergebnissen zu verstehen sei, finden sich – weltweit betrachtet – die meisten Initiativen in OECD-Landern. Solche Definitionen werden nicht nur als Orientierungsrahmen bei Akkreditierungen von Studiengangen und Hochschulen genutzt, sondern bilden in einigen Landern auch eine Basis bei der Entwicklung von Assessmentverfahren. Im vorliegenden Kontext sollen einige zentrale Entwicklungen und Projekte aus diesem Bereich skizziert werden.

Adapting and Validating the Collegiate Learning Assessment to Measure Generic Academic Skills of Students in Germany: Implications for International Assessment Studies in Higher Education

Starting in 2015, a German research team from the program Modeling and Measuring Competencies in Higher Education (KoKoHs), in collaboration with the US Council for Aid to Education (CAE), adapted and validated the Collegiate Learning Assessment (CLA+) for the German language and cultural context to measure generic higher-order cognitive skills of university students and graduates in Germany. In this chapter, the conceptual and methodological background, the framework of the adaptation and validation study, as well as preliminary results are presented. Finally, findings are discussed critically, and future challenges and perspectives are explored.

Correction to: Measuring Scientific Reasoning Competencies

The Impact of Entry Preconditions on Student Dropout and Subject Change in Business and Economics

The proportion of students dropping out of higher education economics is remarkably high at approximately 25 % and has been constant for years. The results on the entry preconditions of dropping out and changing subjects in Bachelor’s degree programs in business and economics presented in this paper are based on a representative longitudinal study in Germany. In two survey waves, the cognitive entry preconditions and characteristics of business and economics students related to their study and learning processes were tested. Dropouts have lower levels of previous knowledge than students changing subjects and students who continue to study business and economics. The students’ characteristic…

An Overview of KoKoHs Projects, Models, and Instruments

Demands in Higher Education

The field testing of the models and instruments at higher education institutions throughout Germany and the analyses of the data gathered on students’ competencies and their determinants pointed to a number of common challenges that institutions, policy-makers, practitioners, and students face with regard to different study phases of higher education today.

Measurement Advances and Challenges in Competency Assessment in Higher Education

Domain-specific and generic competencies have become target outcomes of students' learning in higher education, and competency assessment plays an increasingly important role in higher education on an international scale. Recent reviews of the literature show that there is still a substantial lack of research on assessment practices in higher education, especially on measurement models and valid instruments for competency assessment. This special issue provides an overview of current international research and innovative developments in this research area, covering a broad range of assessment perspectives, addressing prevalent shortcomings of educational assessment practices, and presenting…

Modeling and Measuring Competencies in Higher Education

Measuring competencies acquired in higher education has to be regarded as a widely neglected research field. The progress made in empirical research on the school system since the 1990s – for example, through large-scale assessments such as the Trends in International Mathematics and Science Study (TIMSS) and the Programme for International Student Assessment (PISA) and through a massive expansion of instructional research in general – has revealed that nothing comparable exists at the higher education level.

Konzeptioneller und methodischer Rahmen zur Analyse des Forschungsstandes

Die vorliegende Forschungssynthese dokumentiert den Stand der nationalen und internationalen Forschung zur Erfassung von akademisch vermittelten Kompetenzen bei Studierenden und Hochschulabsolventen fur die Jahre 2010 bis 2014. Zu seiner Erstellung wurde im Sommer/Herbst 2014 – Winter 2015 eine breit angelegte Dokumentenanalyse durch Literatur- und Datenbankrecherchen (inkl. qualitativen Inhaltsanalysen) durchgefuhrt. Um eine systematische Vorgehensweise zu gewahrleisten, wurde der Recherche eine Auswahl von national und international relevanten hochschul- und bildungsforschungsspezifischen sowie disziplinubergreifenden Literatur- und Projektdatenbanken zugrunde gelegt; auf nationaler Ebene…

Development Perspectives for Higher Education Research and Practice

In the 24 collaborative projects of the KoKoHs program, comprising more than 70 individual research projects, objective test instruments were developed for assessing the competencies of students and graduates and tested in higher education practice (see table in section 8). The KoKoHs assessments provide empirically substantiated information on students’ competency levels and insight into the effectiveness of teaching-and-learning opportunities in higher education. KoKoHs has offered means to address major challenges of higher education in a more effective manner and, in the long term, contributed to improving structural, organizational, and individual conditions for teaching and learning a…

Publication of KoKoHs Projects

Competency Orientation in Higher Education

In recent years, higher education, particularly in Germany, has been marked by fundamental structural and programmatic changes after the implementation of the Bologna reform and the orientation towards competencies in the national standards for education, for instance, set forth by the Standing Conference of the Ministers of Education and Cultural Affairs in Germany. The quality of academic education and its social returns have received increasing attention, and endeavors to develop and implement fair assessments of students’ learning outcomes have revealed diverse challenges. In Germany and internationally, it is becoming increasingly important to define and assess in an objective, reliabl…

Validity and fairness of a new entry diagnostics test in higher education economics

Abstract Research has been focusing increasingly on measuring students’ prior economic knowledge in higher education. However, in German-speaking countries, valid instruments are rare. A diagnostic study investigated the validity and fairness of an internationally established economic knowledge test (adapted German version of TUCE IV and TEL IV) with a representative sample of 7664 students from 46 universities across Germany. Previous findings with the dataset ( Zlatkin-Troitschanskaia et al., 2019a ) suggest that native language significantly predicts missing answers in the knowledge test. Similar effects can be found for gender. After coding missing answers as incorrect, students with a …

Implikationen für die weitere Forschung

Der hier vorgelegte Uberblick zum nationalen und internationalen Forschungsstand im Bereich der Messung akademisch vermittelter Kompetenzen von Studierenden und Hochschulabsolventen belegt, dass Bedarf und Bedeutung dieses Forschungsfeldes vielerorts inzwischen erkannt wurden. Die bisher in Deutschland vor allem im Rahmen des KoKoHs-Programms (s. Tabelle 2c) entwickelten und erprobten Kompetenzmodelle und Assessmentdesigns sowie die zugehorigen Messinstrumente liefern eine Grundlage, um dieses Feld auf nationaler Ebene systematisch weiter auszubauen. Es wird zukunftig darum gehen, insbesondere langsschnittlich angelegte, mehrere Ebenen umfassende Ansatze zu verfolgen, die durch (feld-)exper…

Assessing student learning outcomes in higher education: challenges and international perspectives

Profound structural and programmatic changes in higher education, due in part to globalisation and the harmonisation of educational structures and study models (Lucena et al. 2008; Brennan, Patel, ...

Standards für pädagogisches Testen

Die „Standards for Educational and Psychological Testing“ bieten eine umfangreiche Darstellung von uber 240 Standards, die zur Entwicklung, Durchfuhrung und Evaluation padagogischer und psychologischer Tests praktische Handlungsempfehlungen geben. Zur Berucksichtigung der Besonderheiten der Zielsetzung und Entwicklung padagogischer Tests konnen die Standards insbesondere zu Fragen der Validitat einen Beitrag leisten. Weitere Implikationen aus den Standards fur Anforderungen padagogischen Testens lassen sich u. a. zum Standardsetting, zur Fairness, zur Transparenz des Untersuchungsgegenstands und Interpretation, zu Formen der Diagnostik, zum Feedback sowie zum Datenmanagement finden. Zusatzl…

Kompetenzen von Studierenden – Ergebnisse des „KoKoHs“-Programms im Kontext der nationalen und internationalen Assessmentpraxis

Ausgehend von aktuellen Entwicklungen im Hochschulbereich und daraus resultierenden Anforderungen wird in diesem Beitrag die nationale und internationale Assessmentpraxis skizziert und vor diesem Hintergrund zentrale Ergebnisse des „KoKoHs“-Programms dargestellt. Abschliesend werden weitere theoretische, methodologische und empirische Herausforderungen sowie Entwicklungsperspektiven fur Hochschulpolitik und -praxis aufgezeigt.