Logo image
Back
AdaAX: Explaining Recurrent Neural Networks by Learning Automata with Adaptive States
Conference proceeding   Open access

AdaAX: Explaining Recurrent Neural Networks by Learning Automata with Adaptive States

Dat Hong, Alberto M Segre and Tong Wang
KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.574-584
KDD '22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Washington DC, USA, 08/14/2022–08/18/2022)
08/14/2022
DOI: 10.1145/3534678.3539356
Appears in  UI Libraries Support Open Access
url
https://doi.org/10.1145/3534678.3539356View
Published (Version of record) Open Access

Abstract

Recurrent neural networks (RNN) are widely used for handling sequence data. However, their black-box nature makes it difficult for users to interpret the decision-making process. We propose a new method to construct deterministic finite automata to explain RNN. In an automaton, states are abstracted from hidden states produced by the RNN, and the transitions represent input symbols. Thus, users can follow the paths of transitions, called patterns, to understand how a prediction is produced. Existing methods for extracting automata partition the hidden state space at the beginning of the extraction, which often leads to solutions that are either inaccurate or too large in size to comprehend. Unlike previous methods, our approach allows the automata states to be formed adaptively during the extraction. Instead of defining patterns on pre-determined clusters, our proposed model, AdaAX, identifies small sets of hidden states determined by patterns with finer granularity in data. Then these small sets are gradually merged to form states, allowing users to trade fidelity for lower complexity. Experiments show that our automata can achieve higher fidelity while being significantly smaller in size than baseline methods on synthetic and complex real datasets.
 Automata model diagnosis recurrent neural networks post-hoc explanations UIOWA OA Agreement

Details

Metrics

39 Record Views
Logo image