XAI Methods Categorization
1. Taxonomy
1) XAI 관련 표준이라 할 수 있는 taxonomy는 존재하지 않음.
2) 개인적으로 Reference 3의 Mind-map이 전체적인 그림을 잘 보여주는 버전이라고 생각함.
2. Transparent Model
1) 모델 자체로 (Intrinsic/Ante-hoc) Explainability 제공.
2) Three Levels of Transparency
- Algorithmic Transparency, Decomposability, and Simulatability –> 각각의 상세 설명은 Reference 1의 2.5.1.에서 확인.
- Simulatability나 Decomposability가 어떤 의미가 있는지 모르겠음.
- Transparent Model이라고 하면 보통 Algorithmic Transparency를 이야기하는 것으로 생각됨. Algorithmic Transparency를 가지는 예 –> 선형 모델
3) Linear/Logistic Regression
- “Logistic Regression is a classification model to predict a dependent variable(category) that is dichotomous(binary). Howerver, when the dependent variable is continuous, linear regression would be its homonym.”
- Explanation Using Statistics, Visualization
4) Decision Trees
- Resgression, Classification
5) K-Nearest Neighbors(KNN)
- Classification, Regression
6) Rlue-based Learning
7) General Additive Models
8) Bayesian Models
3. Post-hoc Explainability
1) 여러 방법들이 가능함: Text Explanation, Visual Explanation, Local Explanation, Explanation by Example, Explanation by Simplification, Feature Relevance Explanation –> 사람들이 어떤 것을 설명하는 방식들임.
2) Model-agnostic and Model-specific Methods
3) Model-agnositic Post-hoc Method 1: Explanation by Simplification
- Almost all techniques taking this path for model simplification are based on rule extraction techniques.
- LIME(Local Interpretable Model-agnostic Explanations) builds locally linear models around the predictions of an opaque model to explain it. LIME is a local surrogate method.
- G-REX, etc
4) Model-agnositic Post-hoc Method 2: Feature Relevance Explanation
- aims to describe the functioning of an opaque model by ranking or measuring the influence, relevance or importance each feature has in the prediction output by the model to be explained.
- SHAP(SHapley Additive exPlanations)
- Etc
5) Model-agnositic Post-hoc Method 3: Visual Explanation
6) Post-hoc Method for Shallow ML Model 1: Tree Ensembles, Random Forests
- Tree Ensemble: To circumvent the issue of single tree’s overfitting
- Methods Using “Explanation by Simplification” and/or “Feature Relevance Exaplanation”
7) Post-hoc Method for Shallow ML Model 2: Support Vector Machine
- Post-hoc explainability applied to SVMs covers explanation by simplification, local explanations, visualizations and explanations by example.
8) Post-hoc Method for DL 1: Multi-layer NN(MLP)
- Model Simplification Method: DeepRED, etc
- Feature Revelavnce Method: DeepLIFT, etc
9) Post-hoc Method for DL 2: CNN
- Two major approaches: 1) Those that try to understand the decision process by mapping back the output in the input space to see which parts of the input were discriminative for the output(Saliency/Salience), 2) Those that try to delve inside the network and interpret how the intermediate layers see the external world, not necessarily related to any specific input, but in general.
- Approach 1: CAM(Class Activation Map), Heatmap Usinig LRP, Grad-CAM(Gradient-weighted CAM), Attribution, etc
- Approach 2: A Method Reconstructing the Visual Information Contained Inside the CNN
- Others: Text Explanation(e.g. combining a CNN feature exatractor with an RNN attention model), Using LIME
- Visualization mixed with feature relevance methods are the most adopted approach to explainability in CNNs
10) Post-hoc Method for DL 3: RNN
- Two major approaches: 1) explainability by understanding what a RNN model has learned (mainly via feature relevance methods, 2) explainability by modifying RNN architectures to provide insights about the decisions they make (local explanations)
- Approach 1: Using LRP, Etc
- Approach 2: RETAIN, Etc
References
