Interpretable AI (Explainable AI / XAI)
Interpretable AI refers to artificial intelligence models where humans can understand the reasoning behind the output.
Techniques: 1. Feature Importance. 2. Partial Dependence Plots. 3. Saliency Maps. 4. LIME/SHAP. 5. Attention Visualization.
graph LR
Center["Interpretable AI (Explainable AI / XAI)"]:::main
Rel_token_standard["token-standard"]:::related -.-> Center
click Rel_token_standard "/terms/token-standard"
Rel_token_security["token-security"]:::related -.-> Center
click Rel_token_security "/terms/token-security"
classDef main fill:#7c3aed,stroke:#8b5cf6,stroke-width:2px,color:white,font-weight:bold,rx:5,ry:5;
classDef pre fill:#0f172a,stroke:#3b82f6,color:#94a3b8,rx:5,ry:5;
classDef child fill:#0f172a,stroke:#10b981,color:#94a3b8,rx:5,ry:5;
classDef related fill:#0f172a,stroke:#8b5cf6,stroke-dasharray: 5 5,color:#94a3b8,rx:5,ry:5;
linkStyle default stroke:#4b5563,stroke-width:2px;
🧒 Explain Like I'm 5
Imagine two math students. Student A gives you the right answer but won't show you how they did it. Student B gives you the answer and draws a map showing every step they took. Interpretable AI is Student B: it doesn't just give you an answer; it explains its [logic](/en/terms/logic) so you can trust it.
🤓 Expert Deep Dive
Technically, interpretability is divided into 'Glass-box' models (inherently simple, like Decision Trees or GLMs) and 'Post-hoc' explanations for 'Black-box' models. Tools like 'SHAP' use game theory to assign a value to each input feature, showing how much it contributed to the final prediction. Another technique is 'Saliency Mapping' in computer vision, which highlights the pixels in an image that the AI was looking at when it identified a 'cat'. The 'Interpretability-Accuracy Trade-off' remains a core challenge: usually, the more accurate a model becomes (by adding more layers and parameters), the harder it is for humans to interpret its internal state.