Related papers: A baseline for multiple-choice testing in the univ…
This paper shows how bibliometric assessment can be implemented at individual level. This has been successfully done at the University of Vienna carried out by the Bibliometrics and Publication Strategies Department of the Vienna University…
Unified multimodal models aim to jointly enable visual understanding and generation, yet current benchmarks rarely examine their true integration. Existing evaluations either treat the two abilities in isolation or overlook tasks that…
Estimating the difficulty of multiple-choice questions would be great help for educators who must spend substantial time creating and piloting stimuli for their tests, and for learners who want to practice. Supervised approaches to…
The promotion of the national education digitalization strategy has facilitated the development of teaching quality evaluation towards all-round, process-oriented, precise, and intelligent directions, inspiring explorations into new methods…
There is a worldwide trend towards application of bibliometric research evaluation, in support of the needs of policy makers and research administrators. However the assumptions and limitations of bibliometric measurements suggest a…
We present a question bank consisting of over 250 multiple-choice and true-false questions covering a broad range of material typically taught in an introductory undergraduate course in numerical analysis or scientific computing. The…
Tabletop exercises are used to train personnel in the efficient mitigation and resolution of incidents. They are applied in practice to support the preparedness of organizations and to highlight inefficient processes. Since tabletop…
The introduction of computing education into curricula worldwide requires multi-year assessments to evaluate the long-term impact on learning. However, no single Computational Thinking (CT) assessment spans primary school, and no group of…
In the past, several works have investigated ways for combining quantitative and qualitative methods in research assessment exercises. In this work, we aim at introducing a methodology to explore whether citation-based metrics, calculated…
Randomised control trials are currently the definitive gold standard approach for formal educational experiments. Although conclusions from these experiments are highly credible, their relatively slow experimentation rate, high expense and…
This article deals with peer-assessment in the context of higher education teaching in mathematics, and examines the nature of student activity when assessing work produced by peers. After an overview of research on peer assessment, we…
As the capabilities of large language models (LLMs) continue to advance, evaluating their performance becomes increasingly crucial and challenging. This paper aims to bridge this gap by introducing CMMLU, a comprehensive Chinese benchmark…
Research funding and reputation in the UK have, for over two decades, been increasingly dependent on a regular peer-review of all UK departments. This is to move to a system more based on bibliometrics. Assessment exercises of this kind…
We review motivations, definition, approaches, and methodology for unsupervised cross-lingual learning and call for a more rigorous position in each of them. An existing rationale for such research is based on the lack of parallel data for…
This paper introduces a novel revealed-preference approach to ranking colleges and professional schools based on applicants' choices and standardized test scores. Unlike traditional rankings that rely on data supplied by institutions or…
Context: Over the past decade Software Engineering research has seen a steady increase in survey-based studies, and there are several guidelines providing support for those willing to carry out surveys. The need for auditing survey research…
In this paper we present a method of modeling and analysis that permits the extraction and quantitative display of detailed information about the effects of instruction on a class's knowledge. The method relies on a congitive model that…
Acquiring the mathematical, conceptual, and problem-solving skills required in university-level physics courses is hard work, and the average student often lacks the knowledge and study skills they need to succeed in the introductory…
Various processes in academic organizations include the decision points where selecting people through their assessment and ranking is performed, and the impact of wrong or right choices can be very high. How do we simultaneously ensure…
As large language models continue to advance, their application in educational contexts remains underexplored and under-optimized. In this paper, we address this gap by introducing the first diverse benchmark tailored for educational…