Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| public:t_720_atai:atai-18:lecture_notes_autonomous-x [2018/10/29 12:52] – thorisson | public:t_720_atai:atai-18:lecture_notes_autonomous-x [2024/04/29 13:33] (current) – external edit 127.0.0.1 |
|---|
| ==== Explainability ==== | ==== Explainability ==== |
| |
| | | | | | What It Is | The ability of a controller to explain, after the fact or before, why it did or intends to do something. | |
| | | | | | Why It Is Important | If a controller does something we don't want it to repeat - e.g. crash an airplane full of people - it needs to be able to explain why it did what it did. If it can't it means we can never be sure of why this autonomous system did what it did, or even whether it had any other choice. | |
| | | Human-Level AI | Even more importantly, to grow and learn and self-inspect the AI system must be able to sort out causal chains. If it can't it will not only be incapable of explaining to others why it is like it is, it will be incapable of explaining to itself why things are the way they are, and thus, it will be incapable of sorting out whether something it did is better for its own growth than something else. Explanation is the big black hole of ANNs: In principle ANNs are black boxes, and thus they are in principle unexplainable - whether to themselves or others. \\ AERA tries to address this by encapsulating knowledge as hierarchical models that are built up over time, and can be de-constructed at any time. | |
| |
| \\ | \\ |
| \\ | \\ |
| | \\ |
| | \\ |
| ------------ | ------------ |
| |
