[[/public:t-713-mers:mers-24:main|DCS-T-713-MERS-2024 Main]] \\
[[/public:t-713-mers:mers-24:lecture_notes|Lecture Notes]]



\\
\\
======Empirical Reasoning (III)======
\\
\\
\\


=====Worlds & Regularity=====

|  Noise  | A world with no regularity is a completely unpredictable world. \\ In such worlds, learning is impossible.   |
|  Complete Regularity  | Worlds with complete regularity are deterministic -- completely predictable. \\ In such worlds, knowledge can be recorded in DNA.   |
|  Partial Regularity  | Worlds with regularity support learning. \\ Partial regularity means that learning must always be ON - lifelong.   |
|  Causation  | Causation is a Platonic model for the relationship between two or more variables, whereby changes in one always **precede** changes in the other - i.e. they are temporally ordered.    | 
|  Why It Is Important  | Causation is the foundation of empirical science. Without knowledge about causal relations it is impossible to get anything systematically done.     |
|  \\ Why It Is Important \\ in AI  | The main purpose of intelligence is to //figure out how to get new stuff done, given limited time and energy (LTE)//, i.e. to get stuff done cheaply but well. \\ To get stuff done means knowing how to produce effects. \\ In this case //reliable// methods for getting stuff done are worth more to an intelligence than unreliable ones. \\ A relationship that approximates a Platonic cause-effect is worth more than one that does not.     |
|  Regularity Means Logic  | Regularity - even partial - implies rules. Learning in such worlds involves extracting the rules.   |
|  The Role of Reasoning  | In such worlds, the role of reasoning is to extract the rules in a **compact and convenient form**, so that it may be used for **simulating the world** in cognition.    |
|  **Causal Models**   | The direction of the "causal arrow" is critically necessary for guiding action of an intelligent autonomous agent. \\ Knowing which way the arrows point in any large set of correlated variables can be found out through empirical experimentation.   |
|  What Causal Models Enable  | Causal models enable the creation of a coherent set of rules, where each new rule supports or contradicts others. The more coherent that a picture of the world painted this way looks, the more likely it is to be a **useful model of the world**.    |
\\
\\

=====Causation=====

|  Deduction = Prediction  | Since logical correlation is sufficient to produce a prediction.    |
|  Correlation Supports Prediction  | Correlation is sufficient for simple prediction (if **A** and **B**  correlate highly, then it does not matter if we see an **A** //OR// a **B**, we can predict that the other is likely on the scene).    |
|  Abduction = Planning  | Abduction cannot proceed without knowledge of causal relations.    |
|  Knowledge of Causation Supports Action  | We may know that **A** and **B** correlate, but if we don't know whether **B** is a result of **A** or vice versa, and we want **B** to disappear, we don't know whether it will suffice to modify **A**. \\ //Example: The position of the light switch and the state of the light bulb correlate. Only by knowing that the light switch controls the bulb can we go directly to the switch if we want the light to turn on.//  \\ Causation allows producing chains of events (if **A** causes **B**, **A** can be used to produce **B**). If we know how to affect **A**, we can achieve **B**. \\ Causal knowledge subsumes correlational knowledge: Correlation can be produced from cause-effect information, but not vice versa.   |
|  Knowledge of Causation Supports Understanding  | If we know a collection of cause-effect relations related to a phenomenon, we can both predict, explain, achieve goals, and even re-create the phenomenon.   |
|  **Causal Models:** \\ Necessary To Guide Action  | While correlation gives us indication of causation, the direction of the "causal arrow" is critically necessary for guiding action. \\ Luckily, knowing which way the arrows point in any large set of correlated variables is usually not too hard to find out, by empirical experimentation.   |

\\
\\


===== Self-Programming =====
|  \\ What it is  | //Self-programming// here means, with respect to some virtual machine **M**, the production of one or more programs created by **M** itself, whose //principles// for creation were provided to **M** at design time, but whose details were //decided by// **M** at runtime, based on its //experience//.  |
|  Self-Generated Program  | \\ Determined by some factors in the interaction between the system and its environment.   |
|  Historical note  | Concept of self-programming is old (J. von Neumann one of the first to talk about self-replication in machines). However, few if any proposals for how to achieve this has been fielded.  [[https://en.wikipedia.org/wiki/Von_Neumann_universal_constructor|Von Neumann's universal constructor on Wikipedia]]   |
|  No guarantee  | The fact that a system has the ability to program itself is not a guarantee that it is in a better position than a traditional system. In fact, it is in a worse situation because in this case there are more ways in which its performance can go wrong.    |
|  Why needed  | The inherent limitations of hand-coding methods make traditional manual programming approaches unlikely to reach a level of a human-grade generally intelligent system, simply because to be able to adapt to a wide range of tasks, situations, and domains, a system must be able to modify itself in more fundamental ways than a traditional software system is capable of.   |
|  Remedy  | Sufficiently powerful principles are needed to insure against the system going rogue.    |
|  \\ The //Self// of a machine  | **C1:** The processes that act on the world and the self (via senctors) evaluate the structure and execution of code in the system and, respectively, synthesize new code. \\  **C2:** The models that describe the processes in C1, entities and phenomena in the world -- including the self in the world -- and processes in the self. Goals contextualize models and they also belong to C2. \\ **C3:** The states of the self and of the world -- past, present and anticipated -- including the inputs/outputs of the machine.  |
|  Bootstrap code  | A.k.a. the "seed". Bootstrap code may consist of ontologies, states, models, internal drives, exemplary behaviors and programming skills.   |

\\
\\

===== Programming for Self-Programming =====

|  \\ Why Self-Programming?  | Building a machine that can write (sensible, meaningful!) programs means that that machine is smart enough to **understand** (to a pragmatically meaningful level) the code it produces. If the purpose of its programming is to //become// smart, and the programming language we give to it //assumes it's smart already//, we have defeated the purpose of creating the self-programming machine that gets smarter over time, because its operation requires that its's already smart.    |
|  How Can We Program \\ for Self-Programming?   | \\ Self-programming involves automatic code writing. Code that is automatically written must be verifiable (non-axiomatically, i.e. no mathematical proofs!); therefore, only programming languages that allow reflection will work.     | 
|  Can we use Python?   | Any fully reflective language (like LISP, Haskel, Prolog, Python, etc.), i.e. which comes with the ability to inspect itself, turn data into code and code into data, should //in theory// be capable of sustaining a self-programming machine. (That is because no theory of intelligence exists that takes **time pressure** (limited time and energy - LTE) properly into their account of intelligence.)  |
|  \\ Theory vs. practice  | In computer science, seeing a potential solution to something "in theory" is, most of the time, //not good enough// if we want to see something tangible in the next decade or two, and this is the case here too. \\ In the case of self-programming, what may be great for a human programmer is not good enough for a system supposed to synthesize its own code in real-time - in a way that makes its behavior **temporally predictable**. \\ Why is that important? Because the world presents deadlines, and if the controller is not capable of temporally predictable behavior deadlines cannot be dealt with properly by that controller.  |
|  What can we do?  | We must create a programming language with //simple enough// semantics so that a simple machine (perhaps with some clever emergent properties) can use it to bootstrap itself in learning to write programs.  |
|  Does such a language exist?  | Yes. It's called [[http://alumni.media.mit.edu/~kris/ftp/nivel_thorisson_replicode_AGI13.pdf|Replicode]].   |

\\
\\

=====Levels of Self-Programming=====
|  Level 1  | Level one self-programming capability is the ability of a system to make programs that exclusively make use of its primitive actions from action set.  |
|  Level 2  | Subsumes Level 1; additionally generates new primitives.   |
|  \\ Level 3  | Subsumes Levels 1 and 2; adds the ability to \\ //change the principles by which Level 1 and Level 2 operate//, in other words, \\ Level-3 self-programming systems are capable of what we would here call **meta-programming**. This would involve //changing or replacing some or all of the programs provided to the system at design time//. \\ Of course, the generation of primitives, and the changes of principles, are also controlled by some programs.   |
|  Infinite regress?  | Though the process of self-programming can be carried out in more than one level, eventually the regress will stop at a certain level. The more levels are involved, the more flexible the system will be, though at the same time it will be less stable and more complicated to be analyzed.   |
|  Likely to be many ways?  | For AGI the set of relevant self-programming approaches is likely to be a much smaller set than that typically discussed in computer science, and in all likelihood much smaller than often implied in AGI.    |
|  \\ Architecture  | The possible solutions for effective and efficient self-programming are likely to be strongly linked to what we generally think of as the //architectural structure// of AI systems, since self-programming for AGI may fundamentally have to change, modify, or partly duplicate, some aspect of the architecture of the system itself, for the purpose of being better equipped to perform some task or set of tasks.   |
|  What is Needed  | What is NOT needed is a system that spews out rule after rule after rule, filling up a giant database of rules. That misses the point because what is needed, for any //particular// situation, is a //particular// reasoning chain -- in other words, we need **customized reasoning**  \\ What is neede d    |
|  Achieving **Customized Reasoning**   | What is called for is the equivalent of a just-in-time-compiler but for //reasoning//: A reasoner that produces exactly the kind of reasoning needed for the particular situation. This would be the most compact way for creating logically consistent results where trustworthiness is //part of the reasoning//.     |
|  Trustworthiness Requires Meta-Reasoning  | Trustworthiness of reasoning output can only be done with knowledge of the reliability of the rules used, in other words, //rules about the rules//. This means that meta-reasoning is an inseparable part of the reasoning.      |

\\
\\

=====Existing Systems Which Target Self-Programming=====
^  Label  ^  What  ^  Example  ^Description^
|  \\ [S]  |  \\ State-space search   |  \\ GPS (Newell et al. 1963)  | The atomic actions are state-changing operators, and a program is represented as a path from the initial state to a final state. Variants of this approach include program search (examples: Gödel Machine (Schmidhuber 2006)): Given the action set A, in principle all programs formed by it can be exhaustively listed and evaluated to find an optimal one according to certain criteria.   |
|  \\ [P]  |  Production system   |  SOAR (Laird 1987)  | Each production rule specifies the condition for a sequence of actions that correspond to a program. Mechanisms that produce new production rules, such as chunking, can be considered self-programming.   |
|  \\ [R]  |  Reinforcement learning  |  AIXI (Hutter 2007)  | When an action of an agent changes the state of the environment, and each state has a reward value associated, a program corresponds to a policy in reinforcement learning. When the state transition function is probabilistic, this becomes a Markov decision process.   |
|  \\ [G]  |  \\ Genetic programming  |  Koza’s Invention Machine (Koza et al. 2000)  | A program is formed from the system’s actions, initially randomly but subsequently via genetic operators over the best performers from prior solutions, possibly by using the output of some actions as input of some other actions. An evolution process provides a utility function that is used to select the best programs, and the process is repeated.   |
|  [I]  |  Inductive logic programming  |  (Muggleton 1994)  | A program is a statement with a procedural interpretation, which can be learned from given positive and negative examples, plus background knowledge.   |
|  [E]  |  Evidential reasoning  |  NARS (Wang 2006)   | A program is a statement with a procedural interpretation, and it can be learned using multi-strategy (ampliative) uncertain reasoning.  |
|  \\ \\ [A]  |  \\  Autocatalytic model-driven bi-directional search  |  \\ AERA (Nivel et al. 2014) \\ & \\ Ikon Flux (Nivel 2007)  | In this context the architecture is in large part comprised of a large collection of models, acting as hierarchically organized controllers, executed through a contextually-informed, continuous auto-catalytic process. New models are produced automatically, based on experience, their quality evaluated in light of this experience, and improvements produced as a result. Self-programming occurs at two levels: The lower one is concerned with performance in a set of domains, making models of how best to achieve goals in the external world at any point in time, the higher level is concerned with the operation of the lower one, implementing integrated cognitive control and meta-learning capabilities. Semantically closed auto-catalytic processes maintain the system’s growth after they are deployed.   |
|  Source | [[http://alumni.media.mit.edu/~kris/ftp/JAGI-Special-Self-Progr-Editorial-ThorissonEtAl-09.pdf|Thórisson, Nivel, Sanz & Wang 2012]]     |||

\\
\\

=====Design Assumptions in The Above Approaches=====
|  \\ How does the system represent a basic action?  | a) As an operator that transforms a state to another state, either deterministically or probably, and goal as state to be reached [R, S] \\ b) As a function that maps some input arguments to some output arguments [G] \\ c) As a realizable statement with preconditions and consequences [A, E, I, P] \\ Relevant assumptions: \\ Is the knowledge about an action complete and certain? \\ Is the action set discrete and finite?   |
|  \\ Can a program be used as an "action" in other programs?  | a) Yes, programs can be built recursively [A, E, G, I] \\ b) No, a program can only contain basic actions [R, S, P] \\ Relevant assumptions: \\  Do the programs and actions form a hierarchy? \\ Can these recursions have closed loops?  |
|  \\ How does the system represent goals?  | a) As states to be reached [S] \\ b) As values to be optimized [G, R] \\ c) As statements to be realized [E, P, A] \\ d) As functions to be approximated [I]  \\ Relevant assumptions: \\  Is the knowledge about goals complete? \\ Is the knowledge about goals certain? \\ Can all the goals be reached with a concrete action set?   |
|  \\ Are there derived goals?  | a) Yes, and they are logically dependent to the original goals [I, S, P] \\ b) Yes, and they may become logically independent to the original goals [A, E] \\ c) No, all goals are given or innate [G, R] \\ Relevant assumptions: \\  Are the goals constant or variable? \\ Are the goals externally imposed or internally generated?  |
|  \\ Can the system learn new knowledge about actions and goals?  | a) Yes, and the learning process normally converges [G, I, R] \\ b) Yes, and the learning process may not converge [A, E, P] \\ c) No, all the knowledge are given or innate [S] \\ Relevant assumptions: \\  Are the goals constant or variable? \\ Are the actions constant or variable?   |
|  \\ What is the extent of resources demanded?  | a) Unlimited time and/or space [I, R, S, P] \\ b) Limited time and space [A, E, G] \\ Relevant assumption: Are the resources used an attribute of the problem, or of the solution?   |
|  \\ When is the quality of a program evaluated?  | a) After execution, according to its actual contribution [G] \\ b) Before execution, according to its definition or historical record [I, S, P] \\ c) Both of the above [A, E, R] \\ Relevant assumption: \\  Are adaptation and prediction necessary?   |
|  source  | [[http://alumni.media.mit.edu/~kris/ftp/JAGI-Special-Self-Progr-Editorial-ThorissonEtAl-09.pdf|Thórisson et al. 2012]]   |

\\
\\


===== Predictability =====

|  What It Is  | The ability of an outsider to predict the behavior of a controller based on some information.   |
|  Why It Is Important  | Predicting the behavior of (semi-) autonomous machines is important if we want to ensure their safe operation, or be sure that they do what we want them to do.    |
|  \\ How To Do It  | Predicting the future behavior of ANNs (of any kind) is easier if we switch off their learning after they have been trained, because there exists no method for predicting where their development will lead them if they continue to learn after the leave the lab. Predicting ANN behavior on novel input can be done statistically, but there is no way to be sure that novel input will not completely reverse their behavior. There are very few if any methods for giving ANNs the ability to judge the "novelty" of any input, which might to some extent possibly help with this issue. Reinforcement learning addresses this by only scaling to a handful of variables with known max and min.  |
\\

=====Reliability=====

|  What It Is  | The ability of a machine to always return the same - or similar - answer to the same input.   |
|  Why It Is Important  | Simple machine learning algorithms are very good in this respect, delivering high reliability. Human-level AI, on the other hand, may have the same limitations as humans in this respect, i.e. not being able to give any guarantees.   |
|  Human-Level AI  | To make human-level AI reliable is important because a human-level AI without reliability cannot be trusted, and hence would defeat most of the purpose for creating it in the first place. (AERA proposes a method for this - through continuous pee-wee model generation and refinement.)   |
|  To Achieve Reliability  | Requires **predictability**. Predictability requires sorting out //causal relations// (without these we can never be sure what lead to what).   |
|  Predictability is Hard to Achieve  | In a growing, developing system that is adapting and learning (3 or 4 levels of dynamics!) achieving predictability can only be achieved by **abstraction**: Going to the next level of detail (e.g. I cannot be sure //what exactly// I will eat for dinner, but I can be pretty sure that I //will// eat dinner).    |
|  Achieving Abstraction  | Can be done through hierarchy (but it needs to be //dynamic// - i.e. tailored to its intended usage, as the circumstances call for - because the world has too complex combinatorics to store precomputed hierarchies for everything).   |

\\


\\

=====Trustworthiness=====

|  What It Is  | The ability of a machine's owner to trust that the machine will do what it is supposed to do.   |
|  Why It Is Important  | Any machine created by humans is created for a purpose. The more reliably it does its job (and nothing else) the more trustworthy it is. Trusting simple machines like thermostats involves mostly durability, since they have very few open variables (unbound variables at time of manufacture).   |
|  Human-Level AI  | To make human-level AI trustworthy is very different from creating simple machines because so many variables are unbound at manufacture time. What does trustworthiness mean in this context? We can look at human trustworthiness: Numerous methods exist for ensuring trustworthiness (license to drive, air traffic controller training, certification programs, etc.). We can have the same certification programs for all humans because their principles of operation are shared at multiple levels of detail (biology, sociology, psychology). For an AI this is different because the variability in the makeup of the machines is enormous. This makes trustworthiness of AI robots a complex issue.   |
|  To Achieve Trustworthiness  | Requires **reliability**, and **predictability** at multiple levels of operation. Trustworthiness can be ascertained through special certification programs geared directly at the **kind of robot/AI system in question** (kind of like certifying a particular horse as safe for a particular circumstance and purpose, e.g. horseback riding kids).   |
|  Trustworthiness Methods  | For AI are in their infancy.     |

\\
\\


\\
\\
\\
------------

2024(c)K. R. Thórisson