Related papers: Analyzing-Evaluating-Creating: Assessing Computati…
We developed a self-assessment tool for computing students in higher education to measure their Creative Problem Solving skills. Our survey encompasses 7 dimensions of creativity, based on existing validated scales and conducted focus…
The introduction of computing education into curricula worldwide requires multi-year assessments to evaluate the long-term impact on learning. However, no single Computational Thinking (CT) assessment spans primary school, and no group of…
With the increasing importance of Computational Thinking (CT) at all levels of education, it is essential to have valid and reliable assessments. Currently, there is a lack of such assessments in upper primary school. That is why we present…
The ability to invent novel and interesting problems is a remarkable feature of human intelligence that drives innovation, art, and science. We propose a method that aims to automate this process by harnessing the power of state-of-the-art…
This mixed-methods study examined computational thinking (CT) development among 93 pre-high school students in a five-day AI agent creation workshop using CocoFlow, a no-code platform. Integrating pre-post assessments, behavioral logs, and…
Computational Thinking (CT) has emerged as a critical component in modern education, essential to equip students with the skills necessary to thrive in a technology-driven world. This survey provides a comprehensive analysis of the presence…
Generative models have demonstrated human-level proficiency in various benchmarks across domains like programming, natural sciences, and general knowledge. Despite these promising results on competitive benchmarks, they still struggle with…
This paper addresses the incorporation of problem decomposition skills as an important component of computational thinking (CT) in K-12 computer science (CS) education. Despite the growing integration of CS in schools, there is a lack of…
In recent years, the emphasis on computational thinking (CT) has intensified as an effect of accelerated digitalisation. While most researchers are concentrating on defining CT and developing tools for its instruction and assessment, we…
Background: As traditional coding tasks in education become increasingly vulnerable to the use of Generative AI, there is a critical need for authentic, project-based assessments that evaluate students' scientific inquiry. To address this…
Current evaluation frameworks for foundation models rely heavily on static, manually curated benchmarks, limiting their ability to capture the full breadth of model capabilities. This paper introduces Active learning for Capability…
Well-developed problem-solving skills are essential for any student enrolled in a science, technology, engineering or mathematics (STEM) course as well as for graduates in the workforce. One of the most essential skills is the ability to…
A key component of creativity is associative reasoning: the ability to draw novel yet meaningful connections between concepts. We introduce CREATE, a benchmark designed to evaluate models' capacity for creative associative reasoning. CREATE…
In this world of the digital era, in which we are living, one of the fundamental competences that students must acquire is the competence in Computational Thinking (CT). Although there is no general consensus on a formal definition, there…
Many students in introductory programming courses fare poorly in the code writing tasks of the final summative assessment. Such tasks are designed to assess whether novices have developed the analytical skills to translate from the given…
We introduce the first version of the AI Consumer Index (ACE), a benchmark for assessing whether frontier AI models can perform everyday consumer tasks. ACE contains a hidden heldout set of 400 test cases, split across four consumer…
As reasoning models scale rapidly, the essential role of multimodality in human cognition has come into sharp relief, driving a growing need to probe vision-centric cognitive behaviors. Yet, existing multimodal benchmarks either…
Computing ethics education aims to develop students' critical reflection and agency. We need validated ways to measure whether our efforts succeed. Through two survey administrations (N=474, N=464) with computing students and professionals,…
Developing competency in artificial intelligence is becoming increasingly crucial for computer science (CS) students at all levels of the CS curriculum. However, most previous research focuses on advanced CS courses, as traditional…
Recent advancements in the field of natural language generation have facilitated the use of large language models to assess the quality of generated text. Although these models have shown promising results in tasks such as machine…