[[http://cadia.ru.is/wiki/public:t-720-atai:atai-18:main|T-720-ATAI-2018 Main]] \\
[[http://cadia.ru.is/wiki/public:t-720-atai:atai-18:Lecture_Notes|Links to Lecture Notes]]



=====T-720-ATAI-2018=====
==== Lecture Notes, W9: Autonomous-X: Predictability, Reliability, Explainability  ====


\\
\\
\\
\\

====Autonomous-X====

|  Autonomy  | Implies that the machine "does it alone".    |
|  Predictability  | Predictability is a desired feature of any useful AI. \\ An autonomous machine that is not predictable has severely limited utility.     |
|  Reliability  | Reliability is another desired feature of any useful AI. \\ An autonomous machine with low reliability has severely compromised utility.     |
|  Explainability  | Explainability is a third desired feature of any useful AI. \\ An autonomous machine whose actions cannot be explained also cannot be predicted.   |

\\
\\

==== Autonomous-X ====

|  What It Is  | Autonomy is a key feature of intelligence - the ability of a system to "act on its own". Autonomous-X is anything that "autonomy" is relevant for or applies to in a system's operation.   |
|  Why It Is Important  | This table exists to highlight some really key features of autonomy that any human-level intelligence probably must have. We say "probably" because, since we don't have any yet, and because there is no proper theory of intelligence, we cannot be sure.   |
|  Take Action  | Any autonomy requires some primitive actions - some action or inaction must be an option for an autonomous system, otherwise no other features are relevant. Robots are a typical example of what is meant by "taking action": an arm or a hand moves, or stands still, as a result of a computation - a decision made by a controller whose autonomy we are about to inspect.   |
|  Learning  | We already have machines that learn autonomously, although most of the available methods are limited in that they (a) rely heavily on quality selection of learning material/environments, (b) require careful setup of training, and (c ) careful and detailed specifications of how progress is evaluated.  |
|  Selection of Variables  | Very few if any existing learning methods can decide for themselves whether, from a set of variables with potential relevance for its learning, any one of them (a) is relevant, (b) and if so how much and (c ) in what way it is relevant.   |
|  Goal-Generation  | Very few if any existing learning methods can generate their own (sub-) goals. Of those that might be said to be able to, none can do so freely for any topic or domain.     |
|  Control of Resources  | By "resources" we mean computing power (think time), time, and energy, at the very least. Few if any existing learning methods are any good at (a) controlling their resource use, (b) planning for it, (c ) assessing it, or (d) explaining it.   |
|  Self-Inspection  | Virtually no systems exist as of yet that has been demonstrated to be able to inspect (measure, quantify, compare, track, make use of) their own development for use in its continued growth - whether learning, goal-generation, selection of variables, resource usage, or other self-X.   |
|  Self-Growth  | No System as of yet has been demonstrated to be able to autonomously manage its own self-growth. Self-Growth is necessary for autonomous learning in task-environments with complexities far higher than the controller operating in it. It is even more important where certain bootstrapping thresholds are necessary before safe transition into more powerful/different learning schemes. \\ For instance, if only a few bits of knowledge can be programmed into a controller's seed ("DNA"), because we want it to have maximal flexibility in what it can learn, then we want to put something there that is essential to protect the controller while it develops more sophisticated learning. An example is that nature programmed human babies with an innate fear of heights.    |
\\
\\
==== Predictability ====

|  What It Is  | The ability of an outsider to predict the behavior of a controller based on some information.   |
|  Why It Is Important  | Predicting the behavior of (semi-) autonomous machines is important if we want to ensure their safe operation, or be sure that they do what we want them to do.    |
|  How To Do It  | Predicting the future behavior of ANNs (of any kind) is easier if we switch off their learning after they have been trained, because there exists no method for predicting where their development will lead them if they continue to learn after the leave the lab. Predicting ANN behavior on novel input can be done statistically, but there is no way to be sure that novel input will not completely reverse their behavior. There are very few if any methods for giving ANNs the ability to judge the "novelty" of any input, which might to some extent possibly help with this issue. Reinforcement learning addresses this by only scaling to a handful of variables with known max and min.  |
\\
\\
====Reliability====

|  What It Is  | The ability of a machine to always return the same - or similar - answer to the same input.   |
|  Why It Is Important  | Simple machine learning algorithms are very good in this respect, delivering high reliability. Human-level AI, on the other hand, may have the same limitations as humans in this respect, i.e. not being able to give any guarantees.   |
|  Human-Level AI  | To make human-level AI reliable is important because a human-level AI without reliability cannot be trusted, and hence would defeat most of the purpose for creating it in the first place. AERA proposes a method for this - through continuous pee-wee model generation and refinement.   |
\\
\\
==== Explainability ====

|  What It Is  | The ability of a controller to explain, after the fact or before, why it did or intends to do something.   |
|  Why It Is Important  | If a controller does something we don't want it to repeat - e.g. crash an airplane full of people - it needs to be able to explain why it did what it did. If it can't it means we can never be sure of why this autonomous system did what it did, or even whether it had any other choice.     |
|  Human-Level AI  | Even more importantly, to grow and learn and self-inspect the AI system must be able to sort out causal chains. If it can't it will not only be incapable of explaining to others why it is like it is, it will be incapable of explaining to itself why things are the way they are, and thus, it will be incapable of sorting out whether something it did is better for its own growth than something else. Explanation is the big black hole of ANNs: In principle ANNs are black boxes, and thus they are in principle unexplainable - whether to themselves or others. \\ AERA tries to address this by encapsulating knowledge as hierarchical models that are built up over time, and can be de-constructed at any time.   |

\\
\\
\\
\\
------------

2018(c)K. R. Thórisson  

//EOF//