Croissant：一种面向机器学习就绪数据集的元数据格式
机器学习
2024-12-10 v3 人工智能
数据库
信息检索
摘要
数据是机器学习（ML）的关键资源，然而处理数据仍是一个主要的摩擦点。本文介绍了 Croissant，一种用于数据集的元数据格式，可在 ML 工具、框架和平台之间创建共享表示。Croissant 使数据集更具可发现性、可移植性和互操作性，从而解决 ML 数据管理中的重大挑战。Croissant 已被多个流行的数据集存储库支持，涵盖数十万个数据集，无论数据存储在何处，都能轻松加载到最常用的 ML 框架中。我们由人类评估者进行的初步评估表明，Croissant 元数据可读、易懂、完整且简洁。
引用
@article{arxiv.2403.19546,
title = {Croissant: A Metadata Format for ML-Ready Datasets},
author = {Mubashara Akhtar and Omar Benjelloun and Costanza Conforti and Luca Foschini and Joan Giner-Miguelez and Pieter Gijsbers and Sujata Goswami and Nitisha Jain and Michalis Karamousadakis and Michael Kuchnik and Satyapriya Krishna and Sylvain Lesage and Quentin Lhoest and Pierre Marcenac and Manil Maskey and Peter Mattson and Luis Oala and Hamidah Oderinwale and Pierre Ruyssen and Tim Santos and Rajat Shinde and Elena Simperl and Arjun Suresh and Goeffry Thomas and Slava Tykhonov and Joaquin Vanschoren and Susheel Varma and Jos van der Velde and Steffen Vogler and Carole-Jean Wu and Luyao Zhang},
journal= {arXiv preprint arXiv:2403.19546},
year = {2024}
}
备注
Published at the NeurIPS 2024 Datasets and Benchmark Track. A shorter version appeared earlier in Proceedings of ACM SIGMOD/PODS'24 Data Management for End-to-End Machine Learning (DEEM) Workshop https://dl.acm.org/doi/10.1145/3650203.3663326