Center for Analysis and Design of Intelligent Agents

This is an old revision of the document!

T-720-ATAI-2019

Lecture Notes: AI Architectures

System Architecture

What it is	In CS: the organization of the software that implements a system. In AI: The total system that has direct and independent control of the behavior of an Agent via its sensors and effectors.
Why it's important	The system architecture determines what kind of information processing can be done, and what the system as a whole is capable of in a particular Task-Environemnt.
Key concepts	process types; process initiation; information storage; information flow.
Graph representation	Common way to represent processes as nodes, information flow as edges.
Relation to AI	The term “system” not only includes the processing components, the functions these implement, their input and output, and relationships, but also temporal aspects of the system's behavior as a whole. This is important in AI because any controller of an agent is supposed to control it in such a way that its behavior can be classified as being “intelligent”. But what are the necessary and sufficient components of that behavior set?
Rationality	The “rationality hypothesis” models an intelligent agent as a “rational” agent: An agent that would always do the most “sensible” thing at any point in time. The problem with the rationality hypothesis is that given insufficient resources, including time, the concept of rationality doesn't hold up, because it assumes you have time to weigh all alternatives (or, if you have limited time, that you can choose to evaluate the most relevant options and choose among those). But since such decisions are always about the future, and we cannot predict the future perfectly, for most decisions that we get a choice in how to proceed there is no such thing as a rational choice.
Satisficing	Herbert Simon proposed the concept of “satisficing” to replace the concept of “optimizing” when talking about intelligent action in a complex task-environment. Actions that meet a particular minimum requirement in light of a particular goal 'satisfy' and 'suffice' for the purposes of that goal.
Intelligence is in part a systemic phenomenon	Thought experiment: Take any system we deem intelligent, e.g. a 10-year old human, and isolate any of his/her skills and features. A machine that implements any single one of these is unlikely to seem worthy of being called “intelligent” (viz chess programs), without further qualification (e.g. “a limited expert in a sub-field”). “The intelligence is the architecture.” - KRTh

Inferred AGI Architectural Features

Large architecture	An architecture that is considerably more complex than systems being built in most AI labs today is likely unavoidable. In a complex architecture the issue of concurrency of processes must be addressed, a problem that has not yet been sufficiently resolved in present software and hardware. This scaling problem cannot be addressed by the usual “we’ll wait for Moore’s law to catch up” because the issue does not primarily revolve around speed of execution but around the nature of the architectural principles of the system and their runtime operation.
Predictable Robustness in Novel Circumstances	The system must have a robustness in light of all kinds of task-environment and embodiment perturbations, otherwise no reliable plans can be made, and thus no reliable execution of tasks can ever be reached, no matter how powerful the learning capacity. This robustness must be predictable a-priori at some level of abstraction – for a wide range of novel circumstances it cannot be a complete surprise that the system “holds up”. (If this were the case then the system itself would not be able to predict its chances of success in face of novel circumstances, thus eliminating an important part of the “G” from its “AGI” label.)
Graceful Degradation	Part of the robustness requirement is that the system be constructed in a way as to minimize potential for catastrophic (and upredictable) failure. A programmer forgets to delimit a command in a compiled program and the whole application crashes; this kind of brittleness is not an option for cognitive systems operating in partially stochastic environments, where perturbations may come in any form at any time (and perfect prediction is impossible).
Transversal Functions	The system must have pan-architectural characteristics that enable it to operate consistently as a whole, to be highly adaptive (yet robust) in its own operation across the board, including metacognitive abilities. Some functions likely to be needed to achieve this include attention, learning, analogy-making capabilities, and self-inspection.
	Transversal Time	Ignoring (general) temporal constraints is not an option if we want AGI. Move over Turing! Time is a semantic property, and the system must be able to understand – and be able to learn to understand – time as a real-world phenomenon in relation to its own skills and architectural operation. Time is everywhere, and is different from other resources in that there is a global clock which cannot, for many task-environments, be turned backwards. Energy must also be addressed, but may not be as fundamentally detrimental to ignore as time while we are in the early stages of exploring methods for developing auto-catalytic knowledge acquisition and cognitive growth mechanisms. Time must be a tightly integrated phenomenon in any AGI architecture - managing and understanding time cannot be retrofitted to a complex architecture!
	Transversal Learning	The system should be able to learn anything and everything, which means learning probably not best located in a particular “module” or “modules” in the architecture. Learning must be a tightly integrated phenomenon in any AGI architecture, and must be part of the design from the beginning - implementing general learning into an existing architecture is out of the question: Learning cannot be retrofitted to a complex architecture!
	Transversal Resource Management	Resource management - attention - must be tightly integrated. Attention must be part of the system design from the beginning - retrofitting resource management into a architecture that didn't include this from the beginning is next to impossible!
	Transversal Analogies	Analogies must be included in the system design from the beginning - retrofitting the ability to make general analogies between anything and everything is impossible!
	Transversal Self-Inspection	Reflectivity, as it is known, is a fundamental property of knowledge representation. The fact that we humans can talk about the stuff that we think about, and can talk about the fact that we talk about the fact that we can talk about it, is a strong implication that reflectivity is a key property of AGI systems. Reflectivity must be part of the architecture from the beginning - retrofitting this ability into any architecture is virtually impossible!
	Transversal Integration	A general-purpose system must tightly and finely coordinate a host of skills, including their acquisition, transitions between skills at runtime, how to combine two or more skills, and transfer of learning between them over time at many levels of temporal and topical detail.

Self-Programming

[S] State-space search (example: GPS (Newell 1963). The atomic actions are state-changing operators, and a program is represented as a path from the initial state to a final state. Variants of this approach include program search (examples: Gödel Machine (Schmidhuber 2006)): Given the action set A, in principle all programs formed by it can be exhaustively listed and evaluated to find an optimal one according to certain criteria.

[P] Production system (example: SOAR (Laird 1987)). Each production rule specifies the condition for a sequence of actions that correspond to a program. Mechanisms that produce new production rules, such as chunking, can be considered self-programming. [R] Reinforcement learning (example: AIXI (Hutter 2007)). When an action of an agent changes the state of the environment, and each state has a reward value associated, a program corresponds to a policy in reinforcement learning. When the state transition function is probabilistic, this becomes a Markov decision process.

[G] Genetic programming (example: Koza’s Invention Machine (Koza et al. 2000). A program is formed from the system’s actions, initially randomly but subsequently via genetic operators over the best performers from prior solutions, possibly by using the output of some actions as input of some other actions. An evolution process provides a utility function that is used to select the best programs, and the process is repeated.

[I] Inductive logic programming (c.f. Muggleton 1994). A program is a statement with a procedural interpretation, which can be learned from given positive and negative examples, plus background knowledge. Besides the above fairly well-known AI approaches, we add here two less known ones that have recently become relevant in the context of self-programming: [E] Evidential reasoning (example: NARS (Wang 2006)). A program is a statement with a procedural interpretation, and it can be learned using multi-strategy (ampliative) uncertain reasoning. The details of this approach is described in the article by Wang in this issue.

[A] Autocatalysis (example: Ikon Flux (Nivel 2007)). In this context the architecture is in large part comprised of a large collection of models, acting as hierarchically organized controllers, executed through a contextually-informed, continuous auto-catalytic process. New models are produced automatically, based on experience, their quality evaluated in light of this experience, and improvements produced as a result. Self-programming occurs at two levels: The lower one is concerned with performance in a set of domains, making models of how best to achieve goals in the external world at any point in time, the higher level is concerned with the operation of the lower one, implementing integrated cognitive control and meta-learning capabilities. Semantically closed auto-catalytic processes maintain the system’s growth after they are deployed.

The SOAR Architecture

SOAR

What it is	One of the oldest cognitive architectures in history.
Why is it important	One of the oldest cognitive architectures in history.
How does it work	Reasoning engine does pattern-matching with hand-coded 'production' rules and 'operators' to solve problems, with an ability to “chunk” - create 'shortcuts' for long transitive reasoning chains. Upon 'impasse' (break in the flow of reasoning/problemsolving) a reasoning process tries to resolve it via successive application of relevant rules.
Recent Additionns	Reinforcement learning for steering reasoning. Sub-symbolic processing for low-level perception.
Missing in Action	Attention (resource control, self-control), symbolic learning (other than chunking).

SOAR is a relatively mature cognitive architecture that has been used by many researchers worldwide during its 20 year life span. During this time it has also been revised and extended in a number of ways. The architecture consists of heterogenous components that interact during each decision cycle. These are working memory and three types of long-term memory: semantic, procedural and episodic. Working memory is where information related to the present is stored with its contents being supplied by sensors or copied from other memory structures based on relevancy to the present situation. Working memory also contains an activation mechanism, used in conjunction with episodic memory, that indicates the relevancy and usefulness of working memory elements. Production rules are matched and fired on the contents of working memory during the decision cycle, implementing both an associative memory mechanism (as rules can bring data from long-term memory into working memory) and action selection (as rules propose, evaluate and apply operators). Operators are procedural data stored in procedural memory. The application of an operator is carried out by a production rule and either causes changes in the working memory or triggers an external action. In cases where operator selection fails due to insufficient knowledge, an impasse event occurs and a process to resolve the impasse is started. This process involves reasoning and inference upon existing knowledge using the same decision cycle in a recursive fashion, the results of this process are converted to production rules by a process termed chunking. Reinforcement learning is used for production rules relating to operator selection to maximize future rewards in similar situations. One of the most recent additions to the SOAR architecture is sub-symbolic processing used for visual capabilities, where the bridge between sub-symbolic to symbolic processing consists of feature detection. As the working memory can contain execution traces, introspective abilities are possible.

The SOAR architecture provides one of the largest collection of simultaneous running cognitive processes of any cognitive architecture so far. However, there is no explicit mechanism for control of attention and the architecture is not designed for real-time operation. The latter may be especially problematic as execution is in strict step-lock form and in particular, the duration (amount of computation) in each decision cycle can vary greatly due to impasse events that are raised occasionally. One might argue that the development of SOAR has been somewhat characterized by “adding boxes” (components) to the architecture when it might be better to follow a more unified approach putting integration at the forefront.

There are a few cognitive architectures that somewhat resemble SOAR and can be placed categorically on the same track. These include ICARUS, which has a strong emphasis on embodiment and has shown promise in terms of generality in a number of toy problems such as in-city driving, and LIDA which was developed for the US Navy to automatically organize and negotiate assignments with sailors but does not have embodiment as a design goal. As in SOAR, both of these implement different types of memory in specialized components and have a step-locked decision cycle.

2013©Helgi P. Helgason

High-Level View of AERA

AERA	The Auto-Catalytic Endogenous Reflective Architecture is an AGI-aspiring self-programming system that combines feedback and feed-forward control in a model-based and model-driven system that is programmed with a seed.

High-level view of the three main functions at work in a running AERA system and their interaction with its knowledge store.
Models	All models are stored in a central memory, and the three processes of planning, attention (resource management) and learning happen as a result of programs that operate on models by matching, activating, and scoring them. Models that predict correctly – not just “what happens next?” but also “what will happen if I do X?” – get a success point. Every time a model 'fires' like that it gets counted, so the ratio of success over counts gives you the “goodness” of a model. Models that have the lowest scores are deleted, models with a good score that suddenly fail result in the generation of new versions of itself (think of it as hypotheses for why it failed this time), and this process over time increases the quality and utility of the knowledge of the controller, in other words it learns.
Attention	Attention is nothing more than resource management, in the case of cognitive controllers it typically involves management of knowledge, time, energy, and computing power. Attention in AERA is the set of functions that decides how the controller uses its compute time, how long it “mulls things over”, and how far into the future it allows itself to “think”. It also involves which models the system works with at any point in time, how much it explores models outside of the obvious candidate set at any point in time.
Planning	Planning is the set of operations involved with looking at alternative ways of proceeding, based on predictions into the future and the quality of the solutions found so far, at any point in time. The plans produced by AERA are of a mixed opportunistic (short time horizon)/firm commitment (long time horizon) kind, and their stability (subject to change drastically over their course) depend solely on the dependability of the models involved – i.e. how well the models represent what is actually going on in the world (including the controllers “mind”).
Learning	Learning happens as a result of the accumulation of models; as they increasingly describe “reality” better (i.e. their target phenomenon) they get better for planning and attention, which in turn improves the learning.
Memory	AREA's “global knowledge base” is in some ways similar to the idea of blackboards: AERA stores all its knowledge in a “global workspace” or memory. Unlike (Selfridge's idea of) blackboards, the blackboard contains executive functions that manage the knowledge dynamically, in addition to “the experts”, which in AERA's case are very tiny and better thought of as “models with codelet helpers”.
Pervasive Use of Codelets	A codelet is a piece of code that is smaller than a typical self-contained program, typically a few lines long, and can only be executed in particular contexts. Programs are constructed on the fly by the operation of the whole system selecting which codelets to run when, based on the knowledge of the system, the active goals, and the state it finds itself in at any point in time.
No “Modules”	Note that the diagram above may imply the false impression that AERA consists of these four software “modules”, or “classes”, or the like. Nothing could be further from the truth: All of AERA's mechanism above are a set of functions that are “welded in with” the operation of the whole system, distributed in a myriad of mechanisms and actions. Does this mean that AERA is spaghetti code, or a mess of a design? On the contrary, the integration and overlap of various mechanisms to achieve the high-level functions depicted in the diagram are surprisingly clean, simple, and coherent in their implementation and operation. This does not mean, however, that AERA is easy to understand – mainly because it uses concepts and implements mechanisms and relies on concepts that are very different from most traditional software systems commonly recognized in computer science.

Autonomous Model Acquisition

What it is

The ability to create a model of some target phenomenon automatically.

Challenge

Unless we know beforehand which signals cause perturbations in <m>o</m> and can hard-wire these from the get-go in the controller, the controller must search for these signals.
In task-domains where the number of available signals is vastly greater than the controller's resources available to do such search, it may take an unacceptable time for the controller to find good predictive variables to create models with.
<m>V_te » V_mem</m>, where the former is the total number of potentially observable and manipulatable variables in the task-environment and the latter is the number of variables that the agent can hold in its memory at any point in time.

Model Acquisition Function


The agent has a model generation function <m>P_M</m> implemented in its controller. The role of the function is to take observed chains of events and produce models intended to capture the events' causal relationships.

A learning agent is situated so as to perceive the effects of the relationships between variables. The agent observes the interaction between the variables for a while, rendering some data about their relations (but not enough to be certain about it, and certainly not enough to create a complete model of it). This generates hypotheses about the relation between variables, in the form of candidate relational models of the observed events.

Model Generation & Evaluation

Based on prior observations, of the variables and their temporal execution in some context, the controller's model generation function <m>P_M</m> may have captured their causal relationship in three alternative models, <m>M_1, M_2, M_3</m>, each slightly but measurably different from the others. Each can be considered a hypothesis of the actual relationship between the included variables, when in the context provided by <m>V_5, V_6</m>.

The agent's model generation mechanisms allow it to produce models of events it sees. Here it creates models (a) <m>M_1</m> and (b) <m>M_2</m>. The usefulness / utility of these models can be tested by performing an operation on the world (c ) as prescribed by the models. (Ideally, when one wants to find on which one is best, the most efficient method is an (energy-preserving) intervention that can only leave one as the winner.)

The result of feedback (reinforcement) may result in the deletion, rewriting, or some other modification of the original model selected for prediction. Here the feedback has resulted in a modified model <m>M{prime}_2</m>.

Demo Of AERA In Action

Demos	The most complex demo of an AERA system was the S1 agent learning to do an interview (in the EU-funded HUMANOBS research project). Main HUMANOBS page
TV Interview	In the style of a TV interview, the agent S1 watched two humans engaged in a “TV-style” interview about the recycling of six everyday objects made out of various materials.
Data	S1 received realtime timestamped data from the 3D movement of the humans (digitized via appropriate tracking methods at 20 Hz), words generated by a speech recognizer, and prosody (fundamental pitch of voice at 60 Hz, along with timestamped starts and stops).
Seed	The seed consisted of a handful of top-level goals for each agent in the interview (interviewer and interviewee), and a small knowledge base about entities in the scene.
What Was Given	* actions: grab, release, point-at, look-at (defined as event types constrained by geometric relationships) * stopping the interview clock ends the session * objects: glass-bottle, plastic-bottle, cardboard-box, wodden-cube, newspaper, wooden-cube * objects have properties (e.g. made-of) * interviewee-role * interviewer-role * Model for interviewer * top-level goal of interviewer: prompt interviewee to communicate * in interruption case: an imposed interview duration time limit * Models for interviewee * top-level goal of interviewee: to communicate * never communicate unless prompted * communicate about properties of objects being asked about, for as long as there still are properties available * don’t communicate about properties that have already been mentioned
What Had To Be Learned	GENERAL INTERVIEW PRINCIPLES * word order in sentences (with no a-priori grammar) * disambiguation via co-verbal deictic references * role of interviewer and interviewee * interview involves serialization of joint actions (a series of Qs and As by each participant) MULTIMODAL COORDINATION & JOINT ACTION * take turns speaking * co-verbal deictic reference * manipulation as deictic reference * looking as deictic reference * pointing as deictic reference INTERVIEWER * to ask a series of questions, not repeating questions about objects already addressed * “thank you” stops the interview clock * interruption condition: using “hold on, let’s go to the next question” can be used to keep interview within time limits INTERVIEWEE * what to answer based on what is asked * an object property is not spoken of if it is not asked for * a silence from the interviewer means “go on” * a nod from the interviewer means “go on”
Result	After having observed two humans interact in a simulated TV interview for some time, the AERA agent S1 takes the role of interviewee, continuing the interview in precisely the same fasion as before, answering the questions of the human interviewer (see videos HH.no_interrupt.mp4 and HH.no_interrupt.mp4 for the human-human interaction that S1 observed; see HM.no_interrupt_mp4 and HM_interrupt_mp4 for other examples of the skills that S1 has acquired by observation). In the “interrupt” scenario S1 has learned to use interruption as a method to keep the interview from going over a pre-defined time limit. The results are recorded in a set of three videos: Human-human interaction (what S1 observes) Human-S1 interaction (S1 interviewing a human) S1-Human Interaction (S1 being interviewed by a human)

The X Architecture

TBD

Table of Contents