Human-in-the-Loop and AI: Crowdsourcing Metadata Vocabulary for Materials Science
Abstract
Metadata vocabularies are essential for advancing FAIR and FARR data principles, but their development constrained by limited human resources and inconsistent standardization practices. This paper introduces MatSci-YAMZ, a platform that integrates artificial intelligence (AI) and human-in-the-loop (HILT), including crowdsourcing, to support metadata vocabulary development. The paper reports on a proof-of-concept use case evaluating the AI-HILT model in materials science, a highly interdisciplinary domain Six (6) participants affiliated with the NSF Institute for Data-Driven Dynamical Design (ID4) engaged with the MatSci-YAMZ plaform over several weeks, contributing term definitions and providing examples to prompt the AI-definitions refinement. Nineteen (19) AI-generated definitions were successfully created, with iterative feedback loops demonstrating the feasibility of AI-HILT refinement. Findings confirm the feasibility AI-HILT model highlighting 1) a successful proof of concept, 2) alignment with FAIR and open-science principles, 3) a research protocol to guide future studies, and 4) the potential for scalability across domains. Overall, MatSci-YAMZ's underlying model has the capacity to enhance semantic transparency and reduce time required for consensus building and metadata vocabulary development.
Cite
@article{arxiv.2512.09895,
title = {Human-in-the-Loop and AI: Crowdsourcing Metadata Vocabulary for Materials Science},
author = {Jane Greenberg and Scott McClellan and Addy Ireland and Robert Sammarco and Colton Gerber and Christopher B. Rauch and Mat Kelly and John Kunze and Yuan An and Eric Toberer},
journal= {arXiv preprint arXiv:2512.09895},
year = {2025}
}
Comments
Metadata and Semantics Research Conference 2025, 14 pages, 7 figures