Measuring Competency of Machine Learning Systems and Enforcing Reliability
Machine Learning
2022-12-06 v1
Abstract
We explore the impact of environmental conditions on the competency of machine learning agents and how real-time competency assessments improve the reliability of ML agents. We learn a representation of conditions which impact the strategies and performance of the ML agent enabling determination of actions the agent can make to maintain operator expectations in the case of a convolutional neural network that leverages visual imagery to aid in the obstacle avoidance task of a simulated self-driving vehicle.
Cite
@article{arxiv.2212.01415,
title = {Measuring Competency of Machine Learning Systems and Enforcing Reliability},
author = {M. Planer and J. M. Sierchio and for BAE Systems},
journal= {arXiv preprint arXiv:2212.01415},
year = {2022}
}
Comments
2 pages with 1 figure, Appears in Proceedings of AAAI FSS-22 Symposium Lessons Learned for Autonomous Assessment of Machine Abilities (LLAAMA)