Publicerad: 1997-09-10
ISBN:
ISSN: 1650-3686 (tryckt), 1650-3740 (online)
Currently; few tools are available for assisting developers with debugging intelligent systems. Because these systems rely heavily on context dependent knowledge and sometimes stochastic decision making; replicating problematic performance may be difficult. Consequently; we adopt a statistical approach to modeling behavior as the basis for identifying potential causes of failure. This paper describes an algorithm for constructing state transition models of system behavior from execution traces. The algorithm is the latest in a family of statistics based algorithms for modelling system execution called Dependency Detection. We present preliminary accuracy results for the algorithm on synthetically generated data and an example of its use in debugging a neural network controller for a race car simulator.
[1] Marc Abrams, Alan Batongbacal, Randy Ribler, and Devendra Vazirani. CHITRA94 A tool to dynamically characterize ensembles of traces for input data modeling and output analysis. Computer Science Dept. 94-21, Virginia Polytechnical Institute, June 1994.
[2] Horacio T. Cadiz. The development of a CHAIDbased model for CHITRA. Computer Science dept., Virginia Polytechnic Institute, February 1994.
[3] S. Chien, H. Mortensen, C. Ying, and S. Hsiao. Integrated planning for auto mated image processing. In Working Notes of the AAAI Spring Symposium on Integrated Planning Applications, March 1995.
[4] Steve A. Chien. Static and completion analysis for planning knowledge base development and vericiation. In Proceedings of the Third International Con. ference on Articial Intelligence Planning Systems (AIPS96), pages 53-61, May 1996.
[5] David Christianson and Chung Kwok. PDB Reference Manual. University of Washington, 4.0 edition.
[6] Kristian J. Hammond. Explaining and repairing plans that fail. In Proceedings of the Tenth International Joint Conference on Articial Intelligence, pages 109-114, Milan, Italy, 1987.
[7] Adele E. Howe. Improving the reliability of AI planning systems by analyzing their failure recovery. IEEE Transactions on Knowledge and Data Engineering, 7(1):14-25, 1995.
[8] Adele E. Howe, editor. Working Notes of the AAAI Spring Symposium on Integrated Planning Systems, Palo Alto, CA, March 1995. AAAI.
[9] Adele E. Howe. Detecting imperfect patterns in event streams using local search. In D. Fisher and H. Lenz, editors, Learning from Data: Artificial Intelligence and Statistics V. Springer-Verlag, 1996.
[10] Adele E. Howe and Paul R. Cohen. Detecting and explaining dependencies in execution traces. In P. Cheeseman and R.W. Oldford, editors, Selecting Mod. els from Data Artificial Intelligence and Statistics IV, pages 71-78. Springer-Verlag, NY, NY, 1994.
[11] Adele E. Howe and Paul R. Cohen. Understanding planner behavior. Artificial Intelligence, 76(1.2):125-166, 1995.
[12] Adele E. Howe and Larry D. Pyeatt. Constructing transition models of AI planner behavior. In Proceedings of the th Knowledge. Based Software Engineering Conference, September 1996.
[13] Eleftherios Koutsoos and Stephen C. North. Drawing graphs with dot. AT T Bell Laboratories, Murray Hill, NJ, October 1993.
[14] Larry D. Pyeatt, Adele E. Howe, and Charles W. Anderson. Learning coordinated behaviors for control of a simulated robot. Technical report, Computer Science Dept, Colorado State University, 1996.
[15] Reid G. Simmons. A theory of debugging plans and interpretations. In Pro. ceedings of the Seventh National Conference on Articial Intelligence, pages 94-99, Minneapolis, MN, 1988.
[16] Robert R. Sokal and F. James Rohlf. Biometry: The Principles and Practice of Statistics in Biological Research. W.H. Freeman and Co., New York, second edition, 1981.
[17] Raghavan Srinivasan and Adele E. Howe. Comparison of methods for improving search efficiency in a partial-order planner. In Proceedings of the th International Joint Conference on Artificial Intelligence, pages 1620-1626, Montreal, CA, August 1995.
[18] David E. Wilkins. SIPE-2: A system for interactive planning and execution. http://www.ai.sri.om /sipe/, April 1995.
