English

NeurIPS 2020 NLC2CMD Competition: Translating Natural Language to Bash Commands

Computation and Language 2021-08-10 v2
Authors: Mayank Agarwal , Tathagata Chakraborti , Quchen Fu , David Gros , Xi Victoria Lin , Jaron Maene , Kartik Talamadupula , Zhongwei Teng , Jules White
View on arXiv ↗ PDF ↗

Abstract

The NLC2CMD Competition hosted at NeurIPS 2020 aimed to bring the power of natural language processing to the command line. Participants were tasked with building models that can transform descriptions of command line tasks in English to their Bash syntax. This is a report on the competition with details of the task, metrics, data, attempted solutions, and lessons learned.

Cite

@article{arxiv.2103.02523,
  title  = {NeurIPS 2020 NLC2CMD Competition: Translating Natural Language to Bash Commands},
  author = {Mayank Agarwal and Tathagata Chakraborti and Quchen Fu and David Gros and Xi Victoria Lin and Jaron Maene and Kartik Talamadupula and Zhongwei Teng and Jules White},
  journal= {arXiv preprint arXiv:2103.02523},
  year   = {2021}
}

Comments

Appears in PMLR Volume 133: NeurIPS 2020 Competition and Demonstration Track. Competition URL: http://ibm.biz/nlc2cmd

Related papers

View all related →
Computation and Language · Computer Science
NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating System
Xi Victoria Lin, Chenglong Wang, Luke Zettlemoyer, Michael D. Ernst
2018-03-05
Computation and Language · Computer Science
NL2CMD: An Updated Workflow for Natural Language to Bash Commands Translation
Quchen Fu, Zhongwei Teng, Marco Georgaklis, Jules White +1
2023-06-21
Computation and Language · Computer Science
LLM-Supported Natural Language to Bash Translation
Finnian Westenfelder, Erik Hemberg, Miguel Tulla, Stephen Moskal +2
2025-02-12
Computation and Language · Computer Science
NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptions
Rindranirina Ramamonjison, Timothy T. Yu, Raymond Li, Haley Li +7
2023-03-28
Computers and Society · Computer Science
Motivando el uso y aprendizaje de Bash a trav\'es de concursos de programaci\'on
Luis Costero, Jorge Villarrubia, Francisco D. Igual
2025-06-03
Computation and Language · Computer Science
NeurIPS 2023 LLM Efficiency Fine-tuning Competition
Mark Saroufim, Yotam Perlitz, Leshem Choshen, Luca Antiga +10
2025-03-19
Machine Learning · Computer Science
Shell Language Processing: Unix command parsing for Machine Learning
Dmitrijs Trizna
2022-07-08
Computation and Language · Computer Science
A Novel Approach for Auto-Formulation of Optimization Problems
Yuting Ning, Jiayu Liu, Longhu Qin, Tong Xiao +5
2023-02-10
Computation and Language · Computer Science
IGLU 2022: Interactive Grounded Language Understanding in a Collaborative Environment at NeurIPS 2022
Julia Kiseleva, Alexey Skrynnik, Artem Zholus, Shrestha Mohanty +13
2022-05-30
Artificial Intelligence · Computer Science
NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environment
Julia Kiseleva, Ziming Li, Mohammad Aliannejadi, Shrestha Mohanty +11
2021-10-18
Machine Learning · Computer Science
Analysis of Systems' Performance in Natural Language Processing Competitions
Sergio Nava-Muñoz, Mario Graff, Hugo Jair Escalante
2024-08-22
Computation and Language · Computer Science
Sebis at SemEval-2023 Task 7: A Joint System for Natural Language Inference and Evidence Retrieval from Clinical Trial Reports
Juraj Vladika, Florian Matthes
2023-05-03
Artificial Intelligence · Computer Science
The NeurIPS 2022 Neural MMO Challenge: A Massively Multiagent Competition with Specialization and Trade
Enhong Liu, Joseph Suarez, Chenhui You, Bo Wu +19
2023-11-08
Artificial Intelligence · Computer Science
Towards Solving Fuzzy Tasks with Human Feedback: A Retrospective of the MineRL BASALT 2022 Competition
Stephanie Milani, Anssi Kanervisto, Karolis Ramanauskas, Sander Schulhoff +26
2023-03-24
Artificial Intelligence · Computer Science
The Second Conversational Intelligence Challenge (ConvAI2)
Emily Dinan, Varvara Logacheva, Valentin Malykh, Alexander Miller +13
2019-02-04
Computation and Language · Computer Science
Structural Transfer Learning in NL-to-Bash Semantic Parsers
Kyle Duffy, Satwik Bhattamishra, Phil Blunsom
2023-08-01
Machine Learning · Computer Science
Measuring Sample Efficiency and Generalization in Reinforcement Learning Benchmarks: NeurIPS 2020 Procgen Benchmark
Sharada Mohanty, Jyotish Poonganam, Adrien Gaidon, Andrey Kolobov +19
2021-03-30
Logic in Computer Science · Computer Science
Proceedings 38th International Conference on Logic Programming
Yuliya Lierler, Jose F. Morales, Carmine Dodaro, Veronica Dahl +2
2022-08-05
Information Retrieval · Computer Science
Some Practice for Improving the Search Results of E-commerce
Fanyou Wu, Yang Liu, Rado Gazo, Benes Bedrich +1
2022-08-02
Artificial Intelligence · Computer Science
NeurIPS 2025 E2LM Competition : Early Training Evaluation of Language Models
Mouadh Yagoubi, Yasser Dahou, Billel Mokeddem, Younes Belkada +11
2025-06-10
Logic in Computer Science · Computer Science
The Second International Verification of Neural Networks Competition (VNN-COMP 2021): Summary and Results
Stanley Bak, Changliu Liu, Taylor Johnson
2021-09-08
Robotics · Computer Science
Natural Language as Policies: Reasoning for Coordinate-Level Embodied Control with LLMs
Yusuke Mikami, Andrew Melnik, Jun Miura, Ville Hautamäki
2024-04-09
Hardware Architecture · Computer Science
NL2GDS: LLM-aided interface for Open Source Chip Design
Max Eland, Jeyan Thiyagalingam, Dinesh Pamunuwa, Roshan Weerasekera
2026-03-06
Machine Learning · Computer Science
Proceedings of the NeurIPS 2020 Workshop on Machine Learning for the Developing World: Improving Resilience
Tejumade Afonja, Konstantin Klemmer, Aya Salama, Paula Rodriguez Diaz +2
2021-01-13
Networking and Internet Architecture · Computer Science
Neural language models for network configuration: Opportunities and reality check
Zied Ben Houidi, Dario Rossi
2022-07-26
R2 v1 2026-06-23T23:43:08.416Z