Center for Analysis and Design of Intelligent Agents

T-720-ATAI-2018

Lecture Notes, W2: Agents & Control

2020©Kristinn R. Thórisson

Controller

An abstraction of a controller: A controller has an input, <m>i_t</m>, selected from a task-environment, current state <m>S</m>, at least one goal <m>G</m> (implicit or explicit - see table below) and output <m>o_t</m> in the form of atomic actions (selected from a set of atomic possible outputs), and a set of processes <m>P</m>.
The internals of a controller for the complex, adaptive control of a situated agent is referred to as cognitive architecture.
Any practical controller is embodied, in that it interacts with its environment through interfaces whereby its internal computations are turned into physical actions: <m>i</m> enters via measuring devices (“sensors”) and <m>o</m> exits the controller via effectors.

Goal

What it is	A state of a (subset) of a world.
What are its components	A set of patterns, expressed as variables with error constraints, that refer to the world.
What we can do with it	Define a task: task := goal + timeframe + initial world state
Why it is important	Goals are needed for concrete tasks, and tasks are a key part of why we would want AI in the first place. For any complex tasks there will be identifiable sub-goals – talking about these in compressed manners (e.g. using natural language) is important for learning and for monitoring of task progress.
Historically speaking	Goals have been with the field of AI from the very beginning, but definitions vary.
What to be aware of	We can assign goals to an AI without the AI having an explicit data structure that we can say matches the goal directly (see Braitenberg Vehicles - below). These are called implicit goals. We may conjecture that if we want an AI to be able to talk about its goals they will have to be – in some sense – explicit, that is, having a discrete representation in the AI's “mind” that can be manipulated, inspected, compressed / decompressed, and related to other data structures for various purposes.

Early Implementation: Centrifugal Governor

What it is	A mechanical system for controlling power of a motor or engine. Centrifugal governors were used to regulate the distance and pressure between millstones in windmills in the 17th century. REF
Why it's important	Earliest example of automatic regulation with proportional feedback.
Modern equivalents	Servo motors (servos), PID control. Introduction to PID control.
Curious feature	The signal represented and the control (Action) output uses the same mechanical system, fusing the information represented with the control mechanism. This is the least flexible way of implementing control.

A model of Watt's Governor, invented by James Watts in 1788 and used to govern the steam engine of the first trains. As the vertical shaft spins faster, the weights get pulled outwards due to the centrifugal force, lifting up small threaded wheel at bottom, which in turn pulls up a shaft used to open a valve or other device. As that valve opens it reduces the power output of the engine which rotates the vertical shaft, so the shaft spins slower and the balls are lowered, turning the power back up. Thus this mechanism regulates fluctuations in the power source and keeps them around a set target.
Excellent explanation of Watt's Governor (albeit a little slow.)

A simple control pipeline consists of at least one sensor, at least one control process of some sort, and at least one end effector. The goal resides in the controller. Based on what the Sensor senses and sends to the Controller, the Controller produces (in some way, e.g. via computations) an action plan (if it's really simple it´s a bit counter intuitive to call it a “plan”, but it's technically a plan since it and its desired effect are already known before it has been performed), and sends it to an end-effector (Act) that executes it.
The Controller will keep a copy of what was sent to the end-effector (inner loop a := efferent copy) as well as monitor the effect of what the end-effector does to the outside workd (outer loop := afferent copy).

Generalization

Diagram
	c	Controller.
	Plant	That which is to be controlled.
	Delta	Comparison of r and feedback signal.
	r	Reference value or signal.
Method	Uses error signal to correct operation.
SISO	Single-input / single-output.
MIMO	Multiple-input / multiple-output.

Key Features of Feedback (Reactive) Control

Sensor	A kind of transducer that changes one type of energy to another type.
Decider	A Decision Process is a function whose output is a commitment to a particular action. The computation of a decision may take a long time, and the implementation of an action committed to may take a long time. Therefore, the exact moment of commitment may not be a single infinitely small moment in time. However, it is often treated as such.
Actuator	A physical or virtual mechanism that implements an action that has been committed to.
Principle	Predefined causal connection between a measured variable <m>v</m> and a controllable variable <m>v_c</m> where <m>v = f(v_c)</m>.
Mechanical controller	Fuses control mechanism with measurement mechanism via mechanical coupling. Adaptation would require mechanical structure to change. Makes adaptation very difficult to implement.
Digital controllers	Separates the stages of measurement, analysis, and control. Makes adaptive control feasible.
Feedback	For a variable <m>v</m>, information of its value at time <m>t_1</m> is transmitted back to the controller through a feedback mechanism as <m>v{prime}</m>, where <m>v{prime}(t) > v(t)</m> that is, there is a latency in the transmission, which is a function of the speed of transmission (encoding (measurement) time + transmission time + decoding (read-back) time).
Latency	A measure for the size of the difference between <m>v</m> and <m>v{prime}</m>.
Jitter	The change in Latency over time. Second-order latency.

Key Features of Feedforward (Predictive) Control

Feedforward	Using prediction, the change of a control signal <m>v</m> can be done before perturbations of <m>v</m> happens, so that the output of the plant <m>o</m> stays constant.
What it requires	This requires information about the entity controlled in the form of a predictive model, and a second set of signals <m>p</m> that are antecedents of <m>o</m> and can thus be used to predict the behavior of <m>o</m>.
Signal behavior	When predicting a time-varying signal <m>v</m> the frequency of change, the possible patterns of change, and the magnitude of change of <m>v</m> are of key importance, as are these factors for the information used to predict its behavior <m>p</m>.
Learning predictive control	By deploying a learner capable of learning predictive control a more robust behavior can be achieved in the controller, even with low sampling rates.
Challenge	Unless we know beforehand which signals cause perturbations in <m>o</m> and can hard-wire these from the get-go in the controller, the controller must search for these signals. In task-domains where the number of available signals is vastly greater than the controller's search resources, it may take an unacceptable time for the controller to find good predictive variables.

Intelligent Agents

An agent consists of a sensor, an effector, and a controller.

Minimal agent	single goal; inability to create sub-goals; sensory data → decision → action
Perception	Transducer that turns energy into information representation.
Decision	Computation that uses perceptual data; chooses one alternative over (potentially) many for implementation.
Action	Potential of the Agent to influence its task-environment, e.g. to move its body, grasp an object, utter some words, etc. Decisions turned into Actions produces Behavior.
Learning agent	Uses memory to enhance control.

Complexity of Agents

Agent complexity	Determined by <m>I X P X O</m>, not just <m>P, i,</m> or <m>o</m>.
Agent action complexity potential	Potential for <m>P</m> to control combinatorics of, or change, <m>o</m>, beyond initial <m>i</m> (at “birth”).
Agent input complexity potential	Potential for <m>P</m> to structure i in post-processing, and to extend <m>i</m>.
Agent <m>P</m> complexity potential	Potential for <m>P</m> to acquire and effectively and efficiently store and access past <m>i</m> (learning); potential for <m>P</m> to change <m>P</m>.
Agent intelligence potential	Potential for <m>P</m> to coherently coordinate all of the above to improve its own ability to use its resources, acquire more resources, in light of drives (top-level goals).

Reactive Agent Architecture

Architecture	Largely fixed for the entire lifetime of the agent.
Why “Reactive”?	Named “reactive” because there is no prediction - the agent simply reacts to stimuli (sensory data) when/after it happens.
super simple	Sensors connected directly to motors, e.g. Braitenberg Vehicles.
simple	Deterministic connections between components with small memory, e.g. chess engines, Roomba vacuum cleaner.
Complex	Grossly modular architecture (< 30 modules) with multiple relationships at more than one level of control detail (LoC), e.g. speech-controlled dialogue systems like Siri and Alexa.
Super complex	Large number of modules (> 30) at various sizes, each with multiple relationships to others, at more than one LoC, e.g. some robots using the subsumption architecture.

Example: Braitenberg Vehicles

Braitenberg vehicle example control scheme: “love”. Steers towards (and crashes into) that which its sensors sense.

Braitenberg vehicle example control scheme: “hate”. Avoids that which it senses.

Braitenberg vehicle example control scheme: “curious”. The thinner wires are weighted-down signals, changing the behavior of “love” by avoiding crashing into things.

Braitenberg Vehicles Online Code Example


Scratch @ MIT	https://scratch.mit.edu/projects/26099509/

EOF

Table of Contents