| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| public:t-713-mers:mers-24:causation-methodology-architecture [2024/10/29 14:33] – [Programming for Self-Programming] thorisson | public:t-713-mers:mers-24:causation-methodology-architecture [2024/11/05 11:35] (current) – thorisson |
|---|
| |
| |
| ====Worlds & Regularity==== | =====Worlds & Regularity===== |
| |
| | Noise | A world with no regularity is a completely unpredictable world. \\ In such worlds, learning is impossible. | | | Noise | A world with no regularity is a completely unpredictable world. \\ In such worlds, learning is impossible. | |
| \\ | \\ |
| |
| ====Causation==== | =====Causation===== |
| |
| | Deduction = Prediction | Since logical correlation is sufficient to produce a prediction. | | | Deduction = Prediction | Since logical correlation is sufficient to produce a prediction. | |
| |
| |
| ==== Self-Programming ==== | ===== Self-Programming ===== |
| | \\ What it is | //Self-programming// here means, with respect to some virtual machine **M**, the production of one or more programs created by <m>M</m> itself, whose //principles// for creation were provided to **M** at design time, but whose details were //decided by// **M** at runtime, based on its //experience//. | | | \\ What it is | //Self-programming// here means, with respect to some virtual machine **M**, the production of one or more programs created by **M** itself, whose //principles// for creation were provided to **M** at design time, but whose details were //decided by// **M** at runtime, based on its //experience//. | |
| | Self-Generated Program | \\ Determined by some factors in the interaction between the system and its environment. | | | Self-Generated Program | \\ Determined by some factors in the interaction between the system and its environment. | |
| | Historical note | Concept of self-programming is old (J. von Neumann one of the first to talk about self-replication in machines). However, few if any proposals for how to achieve this has been fielded. [[https://en.wikipedia.org/wiki/Von_Neumann_universal_constructor|Von Neumann's universal constructor on Wikipedia]] | | | Historical note | Concept of self-programming is old (J. von Neumann one of the first to talk about self-replication in machines). However, few if any proposals for how to achieve this has been fielded. [[https://en.wikipedia.org/wiki/Von_Neumann_universal_constructor|Von Neumann's universal constructor on Wikipedia]] | |
| | No guarantee | The fact that a system has the ability to program itself is not a guarantee that it is in a better position than a traditional system. In fact, it is in a worse situation because in this case there are more ways in which its performance can go wrong. | | | No guarantee | The fact that a system has the ability to program itself is not a guarantee that it is in a better position than a traditional system. In fact, it is in a worse situation because in this case there are more ways in which its performance can go wrong. | |
| | Why we need it | The inherent limitations of hand-coding methods make traditional manual programming approaches unlikely to reach a level of a human-grade generally intelligent system, simply because to be able to adapt to a wide range of tasks, situations, and domains, a system must be able to modify itself in more fundamental ways than a traditional software system is capable of. | | | Why needed | The inherent limitations of hand-coding methods make traditional manual programming approaches unlikely to reach a level of a human-grade generally intelligent system, simply because to be able to adapt to a wide range of tasks, situations, and domains, a system must be able to modify itself in more fundamental ways than a traditional software system is capable of. | |
| | Remedy | Sufficiently powerful principles are needed to insure against the system going rogue. | | | Remedy | Sufficiently powerful principles are needed to insure against the system going rogue. | |
| | \\ The //Self// of a machine | **C1:** The processes that act on the world and the self (via senctors) evaluate the structure and execution of code in the system and, respectively, synthesize new code. \\ **C2:** The models that describe the processes in C1, entities and phenomena in the world -- including the self in the world -- and processes in the self. Goals contextualize models and they also belong to C2. \\ **C3:** The states of the self and of the world -- past, present and anticipated -- including the inputs/outputs of the machine. | | | \\ The //Self// of a machine | **C1:** The processes that act on the world and the self (via senctors) evaluate the structure and execution of code in the system and, respectively, synthesize new code. \\ **C2:** The models that describe the processes in C1, entities and phenomena in the world -- including the self in the world -- and processes in the self. Goals contextualize models and they also belong to C2. \\ **C3:** The states of the self and of the world -- past, present and anticipated -- including the inputs/outputs of the machine. | |
| \\ | \\ |
| |
| ==== Programming for Self-Programming ==== | ===== Programming for Self-Programming ===== |
| |
| | \\ Why Self-Programming? | Building a machine that can write (sensible, meaningful!) programs means that that machine is smart enough to **understand** (to a pragmatically meaningful level) the code it produces. If the purpose of its programming is to //become// smart, and the programming language we give to it //assumes it's smart already//, we have defeated the purpose of creating the self-programming machine that gets smarter over time, because its operation requires that its's already smart. | | | \\ Why Self-Programming? | Building a machine that can write (sensible, meaningful!) programs means that that machine is smart enough to **understand** (to a pragmatically meaningful level) the code it produces. If the purpose of its programming is to //become// smart, and the programming language we give to it //assumes it's smart already//, we have defeated the purpose of creating the self-programming machine that gets smarter over time, because its operation requires that its's already smart. | |
| \\ | \\ |
| |
| ====Levels of Self-Programming==== | =====Levels of Self-Programming===== |
| | Level 1 | Level one self-programming capability is the ability of a system to make programs that exclusively make use of its primitive actions from action set. | | | Level 1 | Level one self-programming capability is the ability of a system to make programs that exclusively make use of its primitive actions from action set. | |
| | Level 2 | Subsumes Level 1; additionally generates new primitives. | | | Level 2 | Subsumes Level 1; additionally generates new primitives. | |
| | Infinite regress? | Though the process of self-programming can be carried out in more than one level, eventually the regress will stop at a certain level. The more levels are involved, the more flexible the system will be, though at the same time it will be less stable and more complicated to be analyzed. | | | Infinite regress? | Though the process of self-programming can be carried out in more than one level, eventually the regress will stop at a certain level. The more levels are involved, the more flexible the system will be, though at the same time it will be less stable and more complicated to be analyzed. | |
| | Likely to be many ways? | For AGI the set of relevant self-programming approaches is likely to be a much smaller set than that typically discussed in computer science, and in all likelihood much smaller than often implied in AGI. | | | Likely to be many ways? | For AGI the set of relevant self-programming approaches is likely to be a much smaller set than that typically discussed in computer science, and in all likelihood much smaller than often implied in AGI. | |
| | \\ Architecture | The possible solutions for effective and efficient self-programming are likely to be strongly linked to what we generally think of as the //architectural structure// of AI systems, since self-programming for AGI may fundamentally have to change, modify, or partly duplicate, some aspect of the architecture of the system, for the purpose of being better equipped to perform some task or set of tasks. | | | \\ Architecture | The possible solutions for effective and efficient self-programming are likely to be strongly linked to what we generally think of as the //architectural structure// of AI systems, since self-programming for AGI may fundamentally have to change, modify, or partly duplicate, some aspect of the architecture of the system itself, for the purpose of being better equipped to perform some task or set of tasks. | |
| | | What is Needed | What is NOT needed is a system that spews out rule after rule after rule, filling up a giant database of rules. That misses the point because what is needed, for any //particular// situation, is a //particular// reasoning chain -- in other words, we need **customized reasoning** \\ What is neede d | |
| | | Achieving **Customized Reasoning** | What is called for is the equivalent of a just-in-time-compiler but for //reasoning//: A reasoner that produces exactly the kind of reasoning needed for the particular situation. This would be the most compact way for creating logically consistent results where trustworthiness is //part of the reasoning//. | |
| | | Trustworthiness Requires Meta-Reasoning | Trustworthiness of reasoning output can only be done with knowledge of the reliability of the rules used, in other words, //rules about the rules//. This means that meta-reasoning is an inseparable part of the reasoning. | |
| |
| \\ | \\ |
| \\ | \\ |
| |
| ====Existing Systems Which Target Self-Programming==== | =====Existing Systems Which Target Self-Programming===== |
| ^ Label ^ What ^ Example ^Description^ | ^ Label ^ What ^ Example ^Description^ |
| | \\ [S] | \\ State-space search | \\ GPS (Newell et al. 1963) | The atomic actions are state-changing operators, and a program is represented as a path from the initial state to a final state. Variants of this approach include program search (examples: Gödel Machine (Schmidhuber 2006)): Given the action set A, in principle all programs formed by it can be exhaustively listed and evaluated to find an optimal one according to certain criteria. | | | \\ [S] | \\ State-space search | \\ GPS (Newell et al. 1963) | The atomic actions are state-changing operators, and a program is represented as a path from the initial state to a final state. Variants of this approach include program search (examples: Gödel Machine (Schmidhuber 2006)): Given the action set A, in principle all programs formed by it can be exhaustively listed and evaluated to find an optimal one according to certain criteria. | |
| \\ | \\ |
| |
| ====Design Assumptions in The Above Approaches==== | =====Design Assumptions in The Above Approaches===== |
| | \\ How does the system represent a basic action? | a) As an operator that transforms a state to another state, either deterministically or probably, and goal as state to be reached [R, S] \\ b) As a function that maps some input arguments to some output arguments [G] \\ c) As a realizable statement with preconditions and consequences [A, E, I, P] \\ Relevant assumptions: \\ Is the knowledge about an action complete and certain? \\ Is the action set discrete and finite? | | | \\ How does the system represent a basic action? | a) As an operator that transforms a state to another state, either deterministically or probably, and goal as state to be reached [R, S] \\ b) As a function that maps some input arguments to some output arguments [G] \\ c) As a realizable statement with preconditions and consequences [A, E, I, P] \\ Relevant assumptions: \\ Is the knowledge about an action complete and certain? \\ Is the action set discrete and finite? | |
| | \\ Can a program be used as an "action" in other programs? | a) Yes, programs can be built recursively [A, E, G, I] \\ b) No, a program can only contain basic actions [R, S, P] \\ Relevant assumptions: \\ Do the programs and actions form a hierarchy? \\ Can these recursions have closed loops? | | | \\ Can a program be used as an "action" in other programs? | a) Yes, programs can be built recursively [A, E, G, I] \\ b) No, a program can only contain basic actions [R, S, P] \\ Relevant assumptions: \\ Do the programs and actions form a hierarchy? \\ Can these recursions have closed loops? | |
| |
| |
| ==== Predictability ==== | ===== Predictability ===== |
| |
| | What It Is | The ability of an outsider to predict the behavior of a controller based on some information. | | | What It Is | The ability of an outsider to predict the behavior of a controller based on some information. | |
| \\ | \\ |
| |
| ====Reliability==== | =====Reliability===== |
| |
| | What It Is | The ability of a machine to always return the same - or similar - answer to the same input. | | | What It Is | The ability of a machine to always return the same - or similar - answer to the same input. | |
| \\ | \\ |
| |
| ====Trustworthiness==== | =====Trustworthiness===== |
| |
| | What It Is | The ability of a machine's owner to trust that the machine will do what it is supposed to do. | | | What It Is | The ability of a machine's owner to trust that the machine will do what it is supposed to do. | |
