public:t-720-atai:atai-16:lecture_notes_f-11_16.02.2016
Table of Contents
T-720-ATAI-2016
Lecture Notes, F-11 16.02.2016
Key Features of Feedforward Predictive Control
Feedforward | Based on correct and accurate predictions, the change of a control signal <m>v</m> can be done before perturbations of <m>v</m> happens, so that the output of the plant <m>o</m> stays constant. |
What it requires | This requires information about <m>v</m> in the form of a predictive model – or simply model, and a set of signals that can map this model's state to the current state of <m>v</m>. |
Learning predictive control | By deploying a learner capable of learning predictive control over a period of time, a more robust behavior can be achieved in the controller, even if each instance of the application of control is limited by sampling rates lower than the frequency of change in <m>v</m>. |
What Are Models?
Model | A model of something is an information structure that behaves in some ways like the thing being modeled. ‘Model’ here actually means exactly the same as the word when used in the vernacular — look up any dictionary defnition and that is what it means. A model of something is not the thing itself, it is in some way a ‘mirror image’ of it, typically with some unimportant details removed, and represented in a way that allows for various manipulations for the purpose of making predictions (answering questions), where the form of allowed manipulations are particular to the representation of the model and the questions to be answered. |
Example | A model of Earth sits on a shelf in my daughter’s room. With it I can answer questions about the gross layout of continents, and names assigned to various regions as they were around 1977 (because that’s when I got it for my confirmation |
Computational models | A typical type of question to be answered with computational (mathematical) models are what-if questions, and a typical method of manipulation is running simulations (producing deductions). Along with this we need the appropriate computational machine. |
Model (again) | A 'model' in this conception has a target phenomenon that it applies to, and it has a form of representation, comprehensiveness, and level of detail; these are the primary features that determine what a model is good for. A computational model of the world in raw machine-readable form is not very efficient for quickly identifying all the countries adjacent to Switzerland - for that a traditional globe is much better. |
Model acquisition | The ability to create models of (observed) phenomena. |
Causal Chains
Learner Observing Causal Chains
Autonomous Model Acquisition
What it is | The ability to create a model of some target phenomenon automatically. |
Challenge | Unless we know beforehand which signals cause perturbations in <m>o</m> and can hard-wire these from the get-go in the controller, the controller must search for these signals. In task-domains where the number of available signals is vastly greater than the controller's resources available to do such search, it may take an unacceptable time for the controller to find good predictive variables to create models with. |
Model Acquisition Function
Model Generation
Experimentation
Environmental Reinforcement
Two Nested Concurrent Loops
Model Operations
Creating | A model of <m>phi</m> does not spring forward automatically, someone or something must create it. | |
Retrieving | If you have a large collection of models in a large task-environment or world (i.e. the set of variables on the phenomena you are interacting with is rather large), to use any of the models you must know what model to use with respect to what phenomenon (which means you need to recognize a context and apply the appropriate pattern matching to retrieve the appropriate model(s)). You may even have different models for different kinds of questions. | |
Usage | Once you select the appropriate model you must set up the computational conditions such that the goal of the model manipulation may be met. This requires reading the current situation, setting the parameters of the model appropriately to match the condition, and then running (forward) simulations to deduce what may happen. | |
Example | If you are predicting where a pingpong ball will be 0.4 seconds from now so that you can hit it with your paddle you must a) run a simulation with the appropriate parameter settings, including the speed and direction of the ball, plus amount and direction of its spin if any, b) use the result to program the motor control sequence to get your hand and paddle in the vicinity of the ball at that future predicted state, 0.4 seconds in the future (which now may be 0.3 if it took 100 msecs to do all this), c) initiate the execution of that motor sequence, and d) hope that you met the goal of hitting the ball. The model appropriate for predicting ball direction need not be complex; it has only a few inputs (the 3D direction of the ball’s path so far, plus probably the angle at which your opponent hit the ball), and it need not be a giant “pingpong model” which may include the color of a typical pingpong table, size of typical paddles, etc, it could simply be a model using only the relevant parameters already mentioned. The context of the pingpong (everything else in your surrounding) gives rise to selecting the appropriate model(s) at the appropriate time(s), and make all of it time-dependent in a way that your arm’s motions are not lagging 10 minutes behind your intention to hit the ball, but rather, tend toward being synchronized with the event context in which they are intended to happen in. |
|
Evaluating | All models created for some phenomenon <m>phi</m> must be evaluated on their utility for predicting aspects of <m>phi</m>. This is done via experience. Note that experience can also help improve the model creation and evaluation process itself (more on this later). | |
Erasing | Useless and bad models must be removed. |
Requirements for Using Models
Effectiveness | Creation of models must be effective - otherwise a system will spend too much time creating useless or bad models. Making the model creation effective may require e.g. parallelizing the execution of operations on them. |
Efficiency | Operations on models listed above must be efficient lest they interfere with the normal operation of the system / agent. One way to achieve temporal efficiency is to parallelize their execution, and make them simple. |
Scalability | For any moderately interesting / complex environment, a vast number of models may be entertained and considered at any point in time, and thus a large set of potential models must be manipulatable by the system / agent. |
Problems with Feedback-Only Controllers
Thermostat | A cooling thermostat has a built-in supersimple model of its task-environment, one that is sufficient for it to do its job. It consists of a few variables, an on-off switch, two thresholds, and two simple rules that tie these together; the sensed temperature variable, the upper threshold for when to turn the heater on, and the lower threshold for when to turn the heater off. The thermostat never has to decide which model is appropriate, it is “baked into” it by the thermostat’s designer. It is not a predictive (forward) model, this is a strict feedback model. The thermostat cannot change its model, this can only be done by the user opening it and twiddling some thumbscrews. |
Limitation | Because the system designer knows beforehand which signals cause perturbations in <m>o</m> and can hard-wire these from the get-go in the thermostat, there is no motivation to create a model-creating controller (it is much harder). |
Other “state of the art” systems | The same is true for expert systems, subsumption robots, and general game playing machines: their model is to tightly baked into their architecture by the designer. Yes, there are some variables in thesethat can be changed automatically “after the machine leaves the lab” (without designer intervention), but they are parameters inside a (more or less) already-determined model. |
Benefits of Combined Feed-forward + Feedback Controllers
Greater Potential to Learn | A machine that is free to create, select, and evaluate models, on the other hand, has potential to learn anything (within the confines of the algorithms it has been given for these operations) because as long as the range of possible models is reasonably broad and general, the topics, tasks, domains, and worlds it could (in theory) operate in becomes vastly larger than systems where a particular model is given to the system a priori (I say ‘in theory’ because there are other factors, e.g. the ergodicity of the environment and resource constraints that must be favorable to e.g. the system’s speed of learning). |
Greater Potential for Cognitive Growth | A system that can build models of its own model creation, selection, and evaluation has the ability to improve its own nature. This is in some sense the ultimate AGI (depending on the original blueprint, original seed, and some other factors of course) and therefore we only need two levels of this, in theory, for a self-evolving potentially omniscient/omnipotent (as far as the universe allows) system. |
2016©K. R. Thórisson
EOF
/var/www/cadia.ru.is/wiki/data/pages/public/t-720-atai/atai-16/lecture_notes_f-11_16.02.2016.txt · Last modified: 2017/12/06 13:23 by thorisson