Table of Contents
T-720-ATAI-2019 Main
Links to Lecture Notes
T-720-ATAI-2019
Lecture Notes, W2: Agents & Controllers
Controller
 |
| An abstraction of a controller: A controller has an input, <m>i_t</m>, selected from a task-environment, current state <m>S</m>, at least one goal <m>G</m> (implicit or explicit - see table below) and output <m>o_t</m> in the form of atomic actions (selected from a set of atomic possible outputs), and a set of processes <m>P</m>. The internals of a controller for the complex, adaptive control of a situated agent is referred to as cognitive architecture. Any practical controller is embodied, in that it interacts with its environment through interfaces whereby its internal computations are turned into physical actions: <m>i</m> enters via measuring devices (“sensors”) and <m>o</m> exits the controller via effectors. |
Goal
| What it is | A state of a (subset) of a world. |
| What are its components | A set of patterns, expressed as variables with error constraints, that refer to the world. |
| What we can do with it | Define a task: task := goal + timeframe + initial world state |
| Why it is important | Goals are needed for concrete tasks, and tasks are a key part of why we would want AI in the first place. For any complex tasks there will be identifiable sub-goals – talking about these in compressed manners (e.g. using natural language) is important for learning and for monitoring of task progress. |
| Historically speaking | Goals have been with the field of AI from the very beginning, but definitions vary. |
| What to be aware of | We can assign goals to an AI without the AI having an explicit data structure that we can say matches the goal directly (see Braitenberg Vehicles - below). These are called implicit goals. We may conjecture that if we want an AI to be able to talk about its goals they will have to be – in some sense – explicit, that is, having a discrete representation in the AI's “mind” that can be manipulated, inspected, compressed / decompressed, and related to other data structures for various purposes. |
Early Implementation: Centrifugal Governor
| What it is | A mechanical system for controlling power of a motor or engine. Centrifugal governors were used to regulate the distance and pressure between millstones in windmills in the 17th century. REF |
| Why it's important | Earliest example of automatic regulation with proportional feedback. |
| Modern equivalents | Servo motors (servos), PID control. Introduction to PID control. |
| Curious feature | The signal represented and the control (Action) output uses the same mechanical system, fusing the information represented with the control mechanism. This is the least flexible way of implementing control. |
 |
| A model of Watt's Governor, invented by James Watts in 1788 and used to govern the steam engine of the first trains. As the vertical shaft spins faster, the weights get pulled outwards due to the centrifugal force, lifting up small threaded wheel at bottom, which in turn pulls up a shaft used to open a valve or other device. As that valve opens it reduces the power output of the engine which rotates the vertical shaft, so the shaft spins slower and the balls are lowered, turning the power back up. Thus this mechanism regulates fluctuations in the power source and keeps them around a set target. Excellent explanation of Watt's Governor (albeit a little slow.) |
 |
| A simple control pipeline consists of at least one sensor, at least one control process of some sort, and at least one end effector. The goal resides in the controller. Based on what the Sensor senses and sends to the Controller, the Controller produces (in some way, e.g. via computations) an action plan (if it's really simple it´s a bit counter intuitive to call it a “plan”, but it's technically a plan since it and its desired effect are already known before it has been performed), and sends it to an end-effector (Act) that executes it. The Controller will keep a copy of what was sent to the end-effector (inner loop a := efferent copy) as well as monitor the effect of what the end-effector does to the outside workd (outer loop := afferent copy). |
Generalization
| Diagram |  |
|
| c | Controller. | |
| Plant | That which is to be controlled. | |
| Delta | Comparison of r and feedback signal. | |
| r | Reference value or signal. | |
| Method | Uses error signal to correct operation. | |
| SISO | Single-input / single-output. | |
| MIMO | Multiple-input / multiple-output. | |
Key Features of Feedback (Reactive) Control
| Sensor | A kind of transducer that changes one type of energy to another type. |
| Decider | A Decision Process is a function whose output is a commitment to a particular action. The computation of a decision may take a long time, and the implementation of an action committed to may take a long time. Therefore, the exact moment of commitment may not be a single infinitely small moment in time. However, it is often treated as such. |
| Actuator | A physical or virtual mechanism that implements an action that has been committed to. |
| Principle | Predefined causal connection between a measured variable <m>v</m> and a controllable variable <m>v_c</m> where <m>v = f(v_c)</m>. |
| Mechanical controller | Fuses control mechanism with measurement mechanism via mechanical coupling. Adaptation would require mechanical structure to change. Makes adaptation very difficult to implement. |
| Digital controllers | Separates the stages of measurement, analysis, and control. Makes adaptive control feasible. |
| Feedback | For a variable <m>v</m>, information of its value at time <m>t_1</m> is transmitted back to the controller through a feedback mechanism as <m>v{prime}</m>, where <m>v{prime}(t) > v(t)</m> that is, there is a latency in the transmission, which is a function of the speed of transmission (encoding (measurement) time + transmission time + decoding (read-back) time). |
| Latency | A measure for the size of the difference between <m>v</m> and <m>v{prime}</m>. |
| Jitter | The change in Latency over time. Second-order latency. |
Key Features of Feedforward (Predictive) Control
| Feedforward | Using prediction, the change of a control signal <m>v</m> can be done before perturbations of <m>v</m> happens, so that the output of the plant <m>o</m> stays constant. |
| What it requires | This requires information about the entity controlled in the form of a predictive model, and a second set of signals <m>p</m> that are antecedents of <m>o</m> and can thus be used to predict the behavior of <m>o</m>. |
| Signal behavior | When predicting a time-varying signal <m>v</m> the frequency of change, the possible patterns of change, and the magnitude of change of <m>v</m> are of key importance, as are these factors for the information used to predict its behavior <m>p</m>. |
| Learning predictive control | By deploying a learner capable of learning predictive control a more robust behavior can be achieved in the controller, even with low sampling rates. |
| Challenge | Unless we know beforehand which signals cause perturbations in <m>o</m> and can hard-wire these from the get-go in the controller, the controller must search for these signals. In task-domains where the number of available signals is vastly greater than the controller's search resources, it may take an unacceptable time for the controller to find good predictive variables. |
Intelligent Agents
 |
| An agent consists of a sensor, an effector, and a controller. |
| Minimal agent | single goal; inability to create sub-goals; sensory data → decision → action |
| Perception | Transducer that turns energy into information representation. |
| Decision | Computation that uses perceptual data; chooses one alternative over (potentially) many for implementation. |
| Action | Potential of the Agent to influence its task-environment, e.g. to move its body, grasp an object, utter some words, etc. Decisions turned into Actions produces Behavior. |
| Learning agent | Uses memory to enhance control. |
Complexity of Agents
| Agent complexity | Determined by <m>I X P X O</m>, not just <m>P, i,</m> or <m>o</m>. |
| Agent action complexity potential | Potential for <m>P</m> to control combinatorics of, or change, <m>o</m>, beyond initial <m>i</m> (at “birth”). |
| Agent input complexity potential | Potential for <m>P</m> to structure i in post-processing, and to extend <m>i</m>. |
| Agent <m>P</m> complexity potential | Potential for <m>P</m> to acquire and effectively and efficiently store and access past <m>i</m> (learning); potential for <m>P</m> to change <m>P</m>. |
| Agent intelligence potential | Potential for <m>P</m> to coherently coordinate all of the above to improve its own ability to use its resources, acquire more resources, in light of drives (top-level goals). |
Reactive Agent Architecture
| Architecture | Largely fixed for the entire lifetime of the agent. |
| super simple | Sensors connected directly to motors, e.g. Braitenberg Vehicles. |
| simple | Deterministic connections between components with small memory, e.g. chess engines, Roomba vacuum cleaner. |
| Complex | Grossly modular architecture (< 30 modules) with multiple relationships at more than one level of control detail (LoC), e.g. speech-controlled dialogue systems like Siri. |
| Super complex | Large number of modules (> 30) at various sizes, each with multiple relationships to others, at more than one LoC, e.g. subsumption architecture. |
Example: Braitenberg Vehicles
 |
| Braitenberg vehicle example control scheme: “love”. Steers towards (and crashes into) that which its sensors sense. |
 |
| Braitenberg vehicle example control scheme: “hate”. Avoids that which it senses. |
 |
| Braitenberg vehicle example control scheme: “curious”. The thinner wires are weighted-down signals, changing the behavior of “love” by avoiding crashing into things. |
Braitenberg Vehicles Online Code Example
 |
|
| Scratch @ MIT | https://scratch.mit.edu/projects/26099509/ |
EOF