Keeping agents safe in deployment, from risky consequences of benign requests to human oversight.
Agents take actions, so their mistakes have consequences. My team works on measuring those risks and on qualifying agents before they are deployed.
BARC-Bench (Li et al., 2026) (Benign Asks, Risky Consequences) looks at requests that seem harmless but carry risky consequences. READY (Chatrath et al., 2026) qualifies agents for enterprise deployment by measuring the reliability of the combined human-AI system and selecting the cheapest human-oversight policy that meets a stated reliability target. See also Benchmarks and Evals and Enterprise Agents.
@article{li2026barcbench,title={{BARC}-Bench: Benign Asks, Risky Consequences},author={Li, Kangkang and Luu, Kelvin and Ma, Lily and Trinh, Tu and Liu, Ying and Elfeki, Mohamed},journal={arXiv preprint arXiv:2610.32024},year={2026},month=oct,}
@article{chatrath2026ready,title={{READY} or Not: Reliable Enterprise Agent Deployment},author={Chatrath, Veronica and Zhu, Bryan and Fan, Jingxuan and Pu, George and Tiwari, Soham Dinesh and Dan, Soham and Young, Ryan and Li, Yuan and Yao, Yuang and Shanker, Apaar and Yang, Minglai and Zhang, Daniel Yue and He, Yunzhong and Liu, Ying and Wang, Chenguang and Yin, Zhijun and Xue, Yuan},journal={arXiv preprint arXiv:2609.02095},year={2026},month=sep,}