SOAR is a relatively mature cognitive architecture that has been used by many researchers worldwide during its 20 year life span. During this time it has also been revised and extended in a number of ways. The architecture consists of heterogenous components that interact during each decision cycle. These are working memory and three types of long-term memory: semantic, procedural and episodic. Working memory is where information related to the present is stored with its contents being supplied by sensors or copied from other memory structures based on relevancy to the present situation. Working memory also contains an activation mechanism, used in conjunction with episodic memory, that indicates the relevancy and usefulness of working memory elements. Production rules are matched and fired on the contents of working memory during the decision cycle, implementing both an associative memory mechanism (as rules can bring data from long-term memory into working memory) and action selection (as rules propose, evaluate and apply operators). Operators are procedural data stored in procedural memory. The application of an operator is carried out by a production rule and either causes changes in the working memory or triggers an external action. In cases where operator selection fails due to insufficient knowledge, an impasse event occurs and a process to resolve the impasse is started. This process involves reasoning and inference upon existing knowledge using the same decision cycle in a recursive fashion, the results of this process are converted to production rules by a process termed chunking. Reinforcement learning is used for production rules relating to operator selection to maximize future rewards in similar situations. One of the most recent additions to the SOAR architecture is sub-symbolic processing used for visual capabilities, where the bridge between sub-symbolic to symbolic processing consists of feature detection. As the working memory can contain execution traces, introspective abilities are possible.

The SOAR architecture provides one of the largest collection of simultaneous running cognitive processes of any cognitive architecture so far. However, there is no explicit mechanism for control of attention and the architecture is not designed for real-time operation. The latter may be especially problematic as execution is in strict step-lock form and in particular, the duration (amount of computation) in each decision cycle can vary greatly due to impasse events that are raised occasionally. One might argue that the development of SOAR has been somewhat characterized by “adding boxes” (components) to the architecture when it might be better to follow a more unified approach putting integration at the forefront.

There are a few cognitive architectures that somewhat resemble SOAR and can be placed categorically on the same track. These include ICARUS, which has a strong emphasis on embodiment and has shown promise in terms of generality in a number of toy problems such as in-city driving, and LIDA which was developed for the US Navy to automatically organize and negotiate assignments with sailors but does not have embodiment as a design goal. As in SOAR, both of these implement different types of memory in specialized components and have a step-locked decision cycle.

2013©Helgi P. Helgason

The Auto-catalytic Endogenous Reflective Architecture – AERA – is an AGI-aspiring architectural blueprint that was produced as part of the HUMANOBS FP7 project. It encompasses several fundamentally new ideas in the history of AI, including a new programming language specifically conceived to solve some major limitations of prior efforts in this respect, including self-inspection and self-representation, distributed representation of knowledge, and distributed reasoning. AERA systems are any-time, real-time, incremental/continuous learning, on-line learning systems.

AERA's knowledge is stored in models, which essentially encode transformations on input, to produce output. Models have a trigger side (left-hand side) and a result side (right-hand side). In a forward-chaining scenario, when a particular piece of data matches on the left hand of a model (it is only allowed to test the match if the data has high enough saliency and the program has sufficient activation) the model fires, producing the output specified by its left-hand side and injecting it into a global memory store. The semantics of the output is prediction, and the semantics of the input is either fact or prediction. Notice that a model in AERA is not a production rule; a model relating A to B does not mean “A entails B”, it means A predicts B, and it has an associated confidence value. Such models stem invariably (read: most of the time) from the system's experience, and in early stages of learning an AERA-based system's set of models may mostly consist fairly useless and bad models, all with relatively low confidence values (“not all models are created equal – some are in fact better than others”).

In backward-chaining – to implement the process of abduction – models act the other way around, namely, when some data match the right-hand side, a model produces new data patterned after its left side, whose semantics essentially state that “if you want a B (on the right-hand side) perhaps it would help to get an A (term on the left-hand side). The semantics of either (both the input and output) is “goal”.

A key principle of AERA operation is that of a distributed production process: Each program in AERA has a level of activation that determines if it is allowed to run or not. Every piece of data has a corresponding saliency level that determines how visible it is inside the system. In AERA there is a single memory, but it (typically) embeds groups that allow sets of data and programs to be addressed, e.g. changing their activation, saliency, or existence (via creation or deletion).

AERA	The Auto-Catalytic Endogenous Reflective Architecture is an AGI-aspiring self-programming system that combines feedback and feed-forward control in a model-based and model-driven system that is programmed with a seed.
High-level view of the three main functions at work in a running AERA system and their interaction with its knowledge store.
Models	All models are stored in a central memory, and the three processes of planning, attention (resource management) and learning happen as a result of programs that operate on models by matching, activating, and scoring them. Models that predict correctly – not just “what happens next?” but also “what will happen if I do X?” – get a success point. Every time a model 'fires' like that it gets counted, so the ratio of success over counts gives you the “goodness” of a model. Models that have the lowest scores are deleted, models with a good score that suddenly fail result in the generation of new versions of itself (think of it as hypotheses for why it failed this time), and this process over time increases the quality and utility of the knowledge of the controller, in other words it learns.
Attention	Attention is nothing more than resource management, in the case of cognitive controllers it typically involves management of knowledge, time, energy, and computing power. Attention in AERA is the set of functions that decides how the controller uses its compute time, how long it “mulls things over”, and how far into the future it allows itself to “think”. It also involves which models the system works with at any point in time, how much it explores models outside of the obvious candidate set at any point in time.
Planning	Planning is the set of operations involved with looking at alternative ways of proceeding, based on predictions into the future and the quality of the solutions found so far, at any point in time. The plans produced by AERA are of a mixed opportunistic (short time horizon)/firm commitment (long time horizon) kind, and their stability (subject to change drastically over their course) depend solely on the dependability of the models involved – i.e. how well the models represent what is actually going on in the world (including the controllers “mind”).
Learning	Learning happens as a result of the accumulation of models; as they increasingly describe “reality” better (i.e. their target phenomenon) they get better for planning and attention, which in turn improves the learning.
Memory	AREA's “global knowledge base” is in some ways similar to the idea of blackboards: AERA stores all its knowledge in a “global workspace” or memory. Unlike (Selfridge's idea of) blackboards, the blackboard contains executive functions that manage the knowledge dynamically, in addition to “the experts”, which in AERA's case are very tiny and better thought of as “models with codelet helpers”.
Pervasive Use of Codelets	A codelet is a piece of code that is smaller than a typical self-contained program, typically a few lines long, and can only be executed in particular contexts. Programs are constructed on the fly by the operation of the whole system selecting which codelets to run when, based on the knowledge of the system, the active goals, and the state it finds itself in at any point in time.
No “Modules”	Note that the diagram above may imply the false impression that AERA consists of these four software “modules”, or “classes”, or the like. Nothing could be further from the truth: All of AERA's mechanism above are a set of functions that are “welded in with” the operation of the whole system, distributed in a myriad of mechanisms and actions. Does this mean that AERA is spaghetti code, or a mess of a design? On the contrary, the integration and overlap of various mechanisms to achieve the high-level functions depicted in the diagram are surprisingly clean, simple, and coherent in their implementation and operation. This does not mean, however, that AERA is easy to understand – mainly because it uses concepts and implements mechanisms and relies on concepts that are very different from most traditional software systems commonly recognized in computer science.

2019©K. R. Thórisson

EOF