Table of Contents
T-720-ATAI-2016
Lecture Notes, F-9 09.02.2016
Requirements for Evaluation: Features That Evaluators Should Be Able To Control
| Determinism | Both full determinism and partial stochasticity (for realism regarding, e.g. noise, stochastic events, etc.) must be supported. |
| Ergodicity | The reachability of (aspects of) states from others determines the degree to which the agent can undo things and get second chances. |
| Continuity | For evaluation to be relevant to e.g.robotics, it is critical to allow continuous variables, to appropriately represent continu- ous spatial and temporal features. The degree to which continuity is approximated (discretization granularity) should be changeable for any variable. |
| Asynchronicity | Any action in the task-environment, including sensors and controls, may operate on arbitrary time scales and interact at any time, letting an agent respond when it can. |
| Dynamism | A static task-environment’s state only changes in response to the AI’s actions. The most simplistic ones are step-lock, where the agent makes one move and the environment responds with another (e.g. board games). More complex environments can be dynamic to various degrees in terms of speed and magnitude, and may be caused by interactions between environmental factors, or simply due to the passage of time. |
| Observability | Task-environments can be partially observable to varying degrees, depending on the type, range, refresh rate, and precision of available sensors, affecting the difficulty and general nature of the task-environment. |
| Controllability | The control that the agent can exercise over the environ- ment to achieve its goals can be partial or full, depending on the capability, type, range, inherent latency, and precision of available actuators. |
| Multiple Parallel Causal Chains | Any generally intelligent system in a complex environment is likely to be trying to meet multiple objectives, that can be co-dependent in various ways through any number of causal chains in the task-environment. Actions, observations, and tasks may occur sequentially or in parallel (at the same time). Needed to implement real- world clock environments. |
| Periodicity | Many structures and events in nature are repetitive to some extent, and therefore contain a (learnable) periodic cycle – e.g. the day-night cycle or blocks of identical houses. |
| Repeatability | Both fully deterministic and partially stochastic environ- ments must be fully repeatable, for traceable transparency. |
| REF | Thorisson, Bieger, Schiffel & Garrett |
Requirements for Evaluation: Settings That Must Be Obtainable
| Complexity | ENVIRONMENT IS COMPLEX WITH DIVERSE INTERACTING OBJECTS |
| Dynamicity | ENVIRONMENT IS DYNAMIC |
| Regularity | TASK-RELEVANT REGULARITIES EXIST AT MULTIPLE TIME SCALES |
| Task Diversity | TASKS CAN BE COMPLEX, DIVERSE, AND NOVEL |
| Interactions | AGENT/ENVIRONMENT/TASK INTERACTIONS ARE COMPLEX AND LIMITED |
| Computational limitations | AGENT COMPUTATIONAL RESOURCES ARE LIMITED |
| Persistence | AGENT EXISTENCE IS LONG-TERM AND CONTINUAL |
| REF | Laird et al. |
Controllers & Agents
Centrifugal Governor
| What it is | A mechanical system for controlling power of a motor or engine. Centrifugal governors were used to regulate the distance and pressure between millstones in windmills in the 17th century. REF |
| Why it's important | Earliest example of automatic regulation with proportional feedback. |
| Modern equivalents | Servo motors (servos), PID control. |
| Curious feature | The signal represented and the control (Action) output uses the same mechanical system, fusing the information represented with the control mechanism. This is the least flexible way of implementing control. |
 |
| A model of Watt's Governor, invented by James Watts in 1788 and used to govern the steam engine of the first trains. As the vertical shaft spins faster, the weights get pulled outwards due to the centrifugal force, lifting up horizontal wheel at bottom, which can be used to control a valve or other device that changes the power that turns the vertical shaft, thus regulating fluctuations in the power source that turns the shaft. |
Generalization
| Diagram |  |
|
| c | Controller. | |
| Plant | That which is to be controlled. | |
| Delta | Comparison of r and feedback signal. | |
| r | Reference value or signal. | |
| Method | Uses error signal to correct operation. | |
| SISO | Single-input / single-output. | |
| MIMO | Multiple-input / multiple-output. | |
Key Features of Feedback Control
| Sensor | A kind of transducer that changes one type of energy to another type. |
| Decider | A Decision Process is a function whose output is a commitment to a particular action. The computation of a decision may take a long time, and the implementation of an action committed to may take a long time. Therefore, the exact moment of commitment may not be a single infinitely small moment in time. However, it is often treated as such. |
| Actuator | A physical or virtual mechanism that implements an action that has been committed to. |
| Principle | Predefined causal connection between a measured variable <m>v</m> and a controllable variable <m>v_c</m> where <m>v = f(v_c)</m>. |
| Mechanical controller | Fuses control mechanism with measurement mechanism via mechanical coupling. Adaptation would require mechanical structure to change. Makes adaptation very difficult to implement. |
| Digital controllers | Separates the stages of measurement, analysis, and control. Makes adaptive control feasible. |
| Feedback | For a variable <m>v</m>, information of its value at time <m>t_1</m> is transmitted back to the controller through a feedback mechanism as <m>v{prime}</m>, where <m>v{prime}(t) > v(t)</m> that is, there is a latency in the transmission, which is a function of the speed of transmission (encoding (measurement) time + transmission time + decoding (read-back) time). |
| Latency | A measure for the size of the difference between <m>v</m> and <m>v{prime}</m>. |
| Jitter | The change in Latency over time. Second-order latency. |
Key Features of Feedforward (predicitive) Control
| Feedforward | Using prediction, the change of a control signal <m>v</m> can be done before perturbations of <m>v</m> happens, so that the output of the plant <m>o</m> stays constant. |
| What it requires | This requires information about the entity controlled in the form of a predictive model, and a second set of signals <m>p</m> that are antecedents of <m>o</m> and can thus be used to predict the behavior of <m>o</m>. |
| Signal behavior | When predicting a time-varying signal <m>v</m> the frequency of change, the possible patterns of change, and the magnitude of change of <m>v</m> are of key importance, as are these factors for the information used to predict its behavior <m>p</m>. |
| Learning predictive control | By deploying a learner capable of learning predictive control a more robust behavior can be achieved in the controller, even with low sampling rates. |
| Challenge | Unless we know beforehand which signals cause perturbations in <m>o</m> and can hard-wire these from the get-go in the controller, the controller must search for these signals. In task-domains where the number of available signals is vastly greater than the controller's search resources, it may take an unacceptable time for the controller to find good predictive variables. |
Intelligent Agents
 |
| An agent consists of a sensor, an effector, and a controller. |
 |
| A simple control pipeline consists of at least one sensor, at least one control process of some sort, and at least one end effector. a := afferent copy |
 |
| Since any sensory-transducer must be controllable, and any actuator-transducer must feed back information about its state, and since these co-vary in both cases, we can collapse these into a senctor. A senctor is a bi-directional transducer characterized by the content of its I/O data, physical characteristics of its operation (physical interface to the world, i.e. physical correlates of its I/O data), latency, transmission rate, bandwidth, and jitter, and the power characteristics for its transduction in either direction. |
 |
| Putting an agent in a world defines the set of possible inputs and outputs available to the agent. |
 |
| We can now see how the body of an agent belongs to the world it is placed in. The world defines the rules (physics) of the agent's task-environment, which inherits these, as well as the agent's body: Neither the task-environment nor the agent's body can break the laws of the world in which it runs. And since we are talking about real machines running real AI systems in the physical world, we must take time and energy into consideration. Of course, the mind of the agent will also be subject to the constraints of the world, since this defines the substrate on which the software of the mind runs. |
 | <m>v</m> := world variable <m>e</m> := energy <m>t</m> := time <m>dc</m> := decoding process <m>i</m> := information <m>ec</m> := encoding process <m>c</m> := controller <m>p</m> := processes |
| The time involved in a single loop through a cognitive agent can be decomposed into several parts: (a) the variables <m>v</m> that can be sensed, which the senctor turns into information of some sort, (b) the transmission of this information to relevant sources, (c ) the processing time of various cognitive processes, e.g. relating the information to previously stored relevant information, (d) the decoding of decisions into control signals for the senctor, and (e) the implementation of the senctor of the control signal. | |
| The fastest response time of any agent can never be shorter than the sum of the lowest possible of all of these. For human decision making involving the simplest choice possible (“A or B?”) this is around 70 ms. For dogs this is shorter. | |
AGI Controller
| Nested feedforward + feedback control | For a general intelligence, neither feedback alone nor feed-forward alone, is sufficient. They must be used in combination. |
| Coordination | In natural intelligence, and the agents we consider, use of feedback and prediction is expected to be used within the same system, in coordination. This coordination must be present through the right architecture. |
| Cognitive Architecture | The collected and integrated operation of a set of processes intended for intelligent control capable of learning. |
| Natural intelligence | Evidence exists for integrated feedback/feedforward control in natural systems, in particular with respect to motor control. |
EOF