Ajou University repository

Improving explainability of integrated gradients with guided non-linearity
Citations

SCOPUS

0

Citation Export

Publication Year
2020-01-01
Journal
Proceedings - International Conference on Pattern Recognition
Publisher
Institute of Electrical and Electronics Engineers Inc.
Citation
Proceedings - International Conference on Pattern Recognition, pp.385-391
Mesh Keyword
Action potentialsChain reactionInhibitory postsynaptic potentialsInput perturbationMechanism of actionNeural network modelPost-synaptic neuronsResearch topics
All Science Classification Codes (ASJC)
Computer Vision and Pattern Recognition
Abstract
Along with the performance improvements of neural network models, developing methods that enable the explanation of their behavior is a significant research topic. For convolutional neural networks, the explainability is usually achieved with attribution (heatmap) that visualizes pixel-level importance or contribution of input to its corresponding result. This attribution should reflect the relation (dependency) between inputs and outputs, which has been studied with a variety of methods, e.g., derivative of an output with respect to an input pixel value, a weighted sum of gradients, amount of output changes to input perturbations, and so on. In this paper, we present a new method that improves the measure of attribution, and incorporates it into the integrated gradients method. To be precise, rather than using the conventional chain-rule, we propose a method called guided non-linearity that propagates gradients more effectively through non-linear units (e.g., ReLU and max-pool) so that only positive gradients backpropagate through nonlinear units. Our method is inspired by the mechanism of action potential generation in postsynaptic neurons, where the firing of action potentials depends on the sum of excitatory (EPSP) and inhibitory postsynaptic potentials (IPSP). We believe that paths consisting of EPSP-giving-neurons faithfully reflect the contribution of inputs to the output, and we make gradients flow only along those paths (i.e., paths of positive chain reactions). Experiments with 5 deep neural networks have shown that the proposed method outperforms others in terms of the deletion metrics, and yields fine-grained and more human-interpretable attribution.
ISSN
1051-4651
Language
eng
URI
https://aurora.ajou.ac.kr/handle/2018.oak/36581
https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85110468375&origin=inward
DOI
https://doi.org/10.1109/icpr48806.2021.9412936
Journal URL
http://ieeexplore.ieee.org/xpl/conhome.jsp?punumber=1000545
Type
Conference
Funding
This research was supported by Samsung Electronics Co., Ltd.
Show full item record

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

Related Researcher

 KOO, HYUNG IL Image
KOO, HYUNG IL구형일
Department of Electrical and Computer Engineering
Read More

Total Views & Downloads

File Download

  • There are no files associated with this item.