ABSTRACT
With the growing capabilities of intelligent systems, the integration of robots in our everyday life is increasing. However, when interacting in such complex human environments, the occasional failure of robotic systems is inevitable. The field of explainable AI has sought to make complex-decision making systems more interpretable but most existing techniques target domain experts. On the contrary, in many failure cases, robots will require recovery assistance from non-expert users. In this work, we introduce a new type of explanation, εerr, that explains the cause of an unexpected failure during an agent's plan execution to non-experts. In order for error explanations to be meaningful, we investigate what types of information within a set of hand-scripted explanations are most helpful to non-experts for failure and solution identification. Additionally, we investigate how such explanations can be autonomously generated, extending an existing encoder-decoder model, and generalized across environments. We investigate such questions in the context of a robot performing a pick-and-place manipulation task in the home environment. Our results show that explanations capturing the context of a failure and history of past actions, are the most effective for failure and solution identification among non-experts. Furthermore, through a second user evaluation, we verify that our model-generated explanations can generalize to an unseen office environment, and are just as effective as the hand-scripted explanations.
- Boussad Abci, Maan El Badaoui El Najjar, Vincent Cocquempot, and Gérald Dherbomez. 2020. An informational approach for sensor and actuator fault diagnosis for autonomous mobile robots. Journal of Intelligent & Robotic Systems 99, 2 (2020), 387--406.Google ScholarDigital Library
- Amina Adadi and Mohammed Berrada. 2018. Peeking inside the black-box: A survey on Explainable Artificial Intelligence (XAI). IEEE Access 6 (2018), 52138--52160.Google ScholarCross Ref
- Dan Amir and Ofra Amir. 2018. Highlights: Summarizing agent behavior to people. In Proc. of the 17th International Conference on Autonomous Agents and MultiAgent Systems. 1168--1176.Google Scholar
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations, ICLR 2015.Google Scholar
- Siddhartha Banerjee, Angel Daruna, David Kent, Weiyu Liu, Jonathan Balloch, Abhinav Jain, Akshay Krishnan, Muhammad Asif Rana, Harish Ravichandar, Binit Shah, Nithin Shrivatsav, and Sonia Chernova. 2019. Taking Recoveries to Task: Recovery-Driven Development for Recipe-based Robot Tasks. ISRR (2019).Google Scholar
- Joost Bastings. 2018. The Annotated Encoder-Decoder with Attention.Google Scholar
- Andrea Bauer, DirkWollherr, and Martin Buss. 2008. Human--robot collaboration: a survey. International Journal of Humanoid Robotics 5, 01 (2008), 47--66.Google ScholarCross Ref
- Anders Billesø Beck, Anders Due Schwartz, Andreas Rune Fugl, Martin Naumann, and Björn Kahl. 2015. Skill-based Exception Handling and Error Recovery for Collaborative Industrial Robots.. In FinE-R@ IROS. 5--10.Google Scholar
- Richard Bloss. 2011. Mobile hospital robots cure numerous logistic needs. Industrial Robot: An International Journal (2011).Google ScholarCross Ref
- Adrian Boteanu, David Kent, Anahita Mohseni-Kabir, Charles Rich, and Sonia Chernova. 2015. Towards robot adaptability in new situations. In 2015 AAAI Fall Symposium Series.Google Scholar
- Tathagata Chakraborti, Anagha Kulkarni, Sarath Sreedharan, David E Smith, and Subbarao Kambhampati. 2019. Explicability? legibility? predictability? transparency? privacy? security? the emerging landscape of interpretable agent behavior. In Proc. of the international conference on automated planning and scheduling, Vol. 29. 86--96.Google Scholar
- Tathagata Chakraborti, Sarath Sreedharan, and Subbarao Kambhampati. 2020. The Emerging Landscape of Explainable AI Planning and Decision Making. arXiv preprint arXiv:2002.11697 (2020).Google Scholar
- Tathagata Chakraborti, Sarath Sreedharan, Yu Zhang, and Subbarao Kambhampati. 2017. Plan explanations as model reconciliation: Moving beyond explanation as soliloquy. arXiv preprint arXiv:1701.08317 (2017).Google ScholarDigital Library
- Kai-Hsiung Chang, Hyungoo Han, and William B Day. 1993. A comparison of failure-handling approaches for planning systems-Replanning vs. recovery. Applied Intelligence 3, 4 (1993), 275--300.Google ScholarCross Ref
- Chao Chen, Rui Xu, Shengying Zhu, Zhaoyu Li, and Huiping Jiang. 2020. RPRS: A reactive Plan repair strategy for rapid response to Plan failures of deep space missions. Acta Astronautica (2020).Google Scholar
- Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).Google Scholar
- D. Crestani, K. Godary-Dejean, and L. Lapierre. 2015. Enhancing fault tolerance of autonomous mobile robots. Robotics and Autonomous Systems 68 (jun 2015), 140--155. https://doi.org/10.1016/j.robot.2014.12.015Google Scholar
- Devleena Das and Sonia Chernova. 2020. Leveraging rationales to improve human task performance. In Proc. of the 25th International Conference on Intelligent User Interfaces. 510--518.Google ScholarDigital Library
- Upol Ehsan, Brent Harrison, Larry Chan, and Mark O Riedl. 2018. Rationalization: A neural machine translation approach to generating natural language explanations. In Proc. of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. 81--87.Google ScholarDigital Library
- Upol Ehsan, Pradyumna Tambwekar, Larry Chan, Brent Harrison, and Mark O Riedl. 2019. Automated rationale generation: a technique for explainable AI and its effects on human perceptions. In Proc. of the 24th International Conference on Intelligent User Interfaces. 263--274.Google ScholarDigital Library
- David Gunning and David W Aha. 2019. DARPA's explainable artificial intelligence program. AI Magazine 40, 2 (2019), 44--58.Google ScholarDigital Library
- Martin Hägele, Klas Nilsson, J Norberto Pires, and Rainer Bischoff. 2016. Industrial robotics. In Springer handbook of robotics. Springer, 1385--1422.Google Scholar
- Kristian J Hammond. 1990. Explaining and repairing plans that fail. Artificial intelligence 45, 1--2 (1990), 173--228.Google Scholar
- Jörg Hoffmann and Daniele Magazzeni. 2019. Explainable AI Planning (XAIP): Overview and the Case of Contrastive Explanation. In ReasoningWeb. Explainable Artificial Intelligence. Springer, 277--282.Google Scholar
- John Hu, Aaron Edsinger, Yi-Je Lim, Nick Donaldson, Mario Solano, Aaron Solochek, and Ronald Marchessault. 2011. An advanced medical robotic system augmenting healthcare capabilities-robotic nursing assistant. In 2011 IEEE international conference on robotics and automation. IEEE, 6264--6269.Google ScholarCross Ref
- Subbarao Kambhampati. 2019. Synthesizing explainable behavior for human-AI collaboration. In Proc. of the 18th International Conference on Autonomous Agents and Multi-Agent Systems. 1--2.Google ScholarDigital Library
- Gayane Kazhoyan, Arthur Niedzwiecki, and Michael Beetz. 2020. Towards Plan Transformations for Real-World Mobile Fetch and Place. In 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 11011--11017.Google Scholar
- Eliahu Khalastchi and Meir Kalech. 2018. A sensor-based approach for fault detection and diagnosis for robotic systems. Autonomous Robots 42, 6 (aug 2018), 1231--1248. https://doi.org/10.1007/s10514-017-9688-zGoogle ScholarDigital Library
- Eliahu Khalastchi and Meir Kalech. 2018. On fault detection and diagnosis in robotic systems. ACM Computing Surveys (CSUR) 51, 1 (2018), 1--24.Google ScholarDigital Library
- Dominik Kirchner, Stefan Niemczyk, and Kurt Geihs. 2014. RoSHA: A Multi-robot Self-healing Architecture. In RoboCup 2013: Robot World Cup XVII (lecture no ed.). Springer, Berlin, Heidelberg, 304--315.Google Scholar
- Ross A Knepper, Stefanie Tellex, Adrian Li, Nicholas Roy, and Daniela Rus. 2015. Recovering from failure by asking for help. Autonomous Robots 39, 3 (2015), 347--362.Google ScholarDigital Library
- Benjamin Krarup, Michael Cashmore, Daniele Magazzeni, and Tim Miller. 2019. Model-based contrastive explanations for explainable planning. (2019).Google Scholar
- Jim Lawton. 2016. Collaborative robots. International Society of Automation (2016), 12--14.Google Scholar
- Lakshmi Nair and Sonia Chernova. 2020. Feature Guided Search for Creative Problem Solving Through Tool Construction. arXiv preprint arXiv:2008.10685 (2020).Google Scholar
- Daehyung Park, Hokeun Kim, Yuuna Hoshi, Zackory Erickson, Ariel Kapusta, and Charles C. Kemp. 2017. A multimodal execution monitor with anomaly classification for robot-assisted feeding. In IROS. IEEE, 5406--5413.Google Scholar
- Lynne Parker and Balajee Kannan. 2006. Adaptive Causal Models for Fault Diagnosis and Recovery in Multi-Robot Teams. In IROS. IEEE, 2703--2710.Google Scholar
- Ola Pettersson, L. Karlsson, and A. Saffiotti. 2007. Model-Free Execution Monitoring in Behavior-Based Robotics. IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) 37, 4 (aug 2007), 890--901. https://doi.org/10. 1109/TSMCB.2007.895359Google ScholarDigital Library
- Arun Rai. 2020. Explainable AI: From black box to glass box. Journal of the Academy of Marketing Science 48, 1 (2020), 137--141.Google ScholarCross Ref
- Vasumathi Raman and Hadas Kress-Gazit. 2012. Explaining impossible high-level robot behaviors. IEEE Transactions on Robotics 29, 1 (2012), 94--104.Google ScholarDigital Library
- Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why should I trust you?" Explaining the predictions of any classifier. In Proc. of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 1135--1144.Google ScholarDigital Library
- Allison Sauppé and Bilge Mutlu. 2015. The social impact of a robot co-worker in industrial settings. In Proc. of the 33rd annual ACM conference on human factors in computing systems. 3613--3622.Google ScholarDigital Library
- Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proc. of the IEEE international conference on computer vision. 618--626.Google ScholarCross Ref
- Sarath Sreedharan, Siddharth Srivastava, David E Smith, and Subbarao Kambhampati. 2019. Why Can't You Do That HAL? Explaining Unsolvability of Planning Tasks. In IJCAI. 1422--1430.Google Scholar
- V. Verma, G. Gordon, R. Simmons, and S. Thrun. 2004. Real-time fault diagnosis. IEEE Robotics & Automation Magazine 11, 2 (jun 2004), 56--66. https://doi.org/10. 1109/MRA.2004.1310942Google ScholarCross Ref
- Xiting Wang, Yiru Chen, Jie Yang, Le Wu, Zhengtao Wu, and Xing Xie. 2018. A reinforcement learning framework for explainable recommendation. In 2018 IEEE International Conference on Data Mining (ICDM). IEEE, 587--596.Google ScholarCross Ref
- Melonee Wise, Michael Ferguson, Derek King, Eric Diehr, and David Dymesich. 2016. Fetch and freight: Standard platforms for service robot applications. In Workshop on autonomous mobile service robots.Google Scholar
- Hongmin Wu, Shuangqi Luo, Longxin Chen, Shuangda Duan, Sakmongkon Chumkamon, Dong Liu, Yisheng Guan, and Juan Rojas. 2018. Endowing Robots with Longer-term Autonomy by Recovering from External Disturbances in Manipulation through Grounded Anomaly Classification and Recovery Policies. (sep 2018). arXiv:1809.03979Google Scholar
- Mike Wu, Michael C Hughes, Sonali Parbhoo, Maurizio Zazzi, Volker Roth, and Finale Doshi-Velez. 2017. Beyond sparsity: Tree regularization of deep models for interpretability. arXiv preprint arXiv:1711.06178 (2017).Google Scholar
- Pin-Chu Yang, Kazuma Sasaki, Kanata Suzuki, Kei Kase, Shigeki Sugano, and Tetsuya Ogata. 2016. Repeatable folding task by humanoid robot worker using deep learning. IEEE Robotics and Automation Letters 2, 2 (2016), 397--403.Google ScholarCross Ref
- Safdar Zaman, Gerald Steinbauer, Johannes Maurer, Peter Lepej, and Suzana Uran. 2013. An integrated model-based diagnosis and repair architecture for ROS-based robot systems. In 2013 IEEE International Conference on Robotics and Automation. IEEE, 482--489. https://doi.org/10.1109/ICRA.2013.6630618Google ScholarCross Ref
- Quanshi Zhang, Yu Yang, Haotian Ma, and Ying Nian Wu. 2019. Interpreting cnns via decision trees. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition. 6261--6270.Google ScholarCross Ref
- Yu Zhang, Sarath Sreedharan, Anagha Kulkarni, Tathagata Chakraborti, Hankz Hankui Zhuo, and Subbarao Kambhampati. 2017. Plan explicability and predictability for robot task planning. In 2017 IEEE international conference on robotics and automation (ICRA). IEEE, 1313--1320.Google ScholarCross Ref
- Zhengjiang Zhang and Junghui Chen. 2019. Fault detection and diagnosis based on particle filters combined with interactive multiple-model estimation in dynamic process systems. ISA transactions 85 (2019), 247--261.Google Scholar
Index Terms
- Explainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recovery
Recommendations
Counterfactual Explainable Recommendation
CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge ManagementBy providing explanations for users and system designers to facilitate better understanding and decision making, explainable recommendation has been an important research problem. In this paper, we propose Counterfactual Explainable Recommendation (...
Counterfactuals and causability in explainable artificial intelligence: Theory, algorithms, and applications
AbstractDeep learning models have achieved high performance across different domains, such as medical decision-making, autonomous vehicles, decision support systems, among many others. However, despite this success, the inner mechanisms of these models ...
Highlights- A survey on model-agnostic counterfactual approaches for XAI.
- A novel taxonomy for model-agnostic counterfactual approaches for XAI.
- A set of properties for causability systems for XAI.
- Opportunities and challenges for ...
Explainable Argumentation for Wellness Consultation
Explainable, Transparent Autonomous Agents and Multi-Agent SystemsAbstractThere has been a recent resurgence in the area of explainable artificial intelligence as researchers and practitioners seek to provide more transparency to their algorithms. Much of this research is focused on explicitly explaining decisions or ...
Comments