[[/public:t-713-mers:mers-23:main|DCS-T-713-MERS-2023 Main]] \\ [[http://cadia.ru.is/wiki/public:t-713-mers:mers-23:lecture_notes|Lecture Notes]] \\ \\ ======Learning & Knowledge====== \\ \\ \\ ====Key Learning Terms==== | What it is | Learning is a //**process**// that has the purpose of //generating actionable information//, a.k.a. **knowledge**. | | \\ \\ \\ Key Features | Inherits key features of any process: \\ - **Purpose**: To adapt, to respond in rational ways to problems / to achieve foreseen goals; this factor determines how the rest of the features in this list are measured. \\ - **Speed**: The speed of learning. \\ - **Data**: The data that the learning (and particular measured speed of learning) requires. \\ - **Quality**: How well something is learned. \\ - **Retention**: The robustness of what has been learned - how well it stays intact over time. \\ - **Transfer**: How general the learning is, how broadly what is learned can be employed for the purposes of adaptation or achievement of goals. \\ - **Meta-Learning**: A learner may improve its learning abilities - i.e. capable of meta-learning. \\ - **Progress Signal(s)**: A learner needs to know how its learning is going, and if there is improvement, how much. | | Evaluation | To know any of the above some parameters have to be //measured// somehow: All of the above factors can be measured in //many ways//. | | Major Caveat | Since learning interacts with (is affect by) the //task-environment and world// that the learning takes place in, as well as the nature of these in the learner's //subsequent deployment//, //none// of the above features can be assessed by //looking only at the learner//. \\ This is addressed by the //Pedagogical Pentagon// (see below). | \\ ==== Measurement, Data, Information, Knowledge ==== | Measurement | Sampling of a value of one or more variables over a particular temporal interval. \\ (Often simplified by considering it coming from a "point" in time.) | | \\ Data | Stored, committed-to //measurement//. \\ Anything that can be measured can be stored as data. Measurement takes time, and so does storing it as data. Data is therefore always old. \\ To be of any use it must contain how the measurement was made and when. \\ For instance, [//FI399 17:00 KEF//]. | | \\ Information | Data that is stored in a particular way for a particular purpose. //Contextualized data.// \\ In some sense, all data is information, because the data must be stored in //some// way, and the particular way will be better suited for some purpose than others. However, we typically only speak of "information" if there is something to the data format beyond simply the value measured and the time of measurement. \\ For instance, the //time of departure for your flight from Keflavik airport, FI399, is at 17:00 today//. | | \\ Knowledge | Actionable information. Information that can be used to get stuff done. \\ A set of interlinked information that can be used to plan, produce action, and interpret new information. \\ "Multi-purpose" information (that can be applied in many ways to many situations) -- //requires specialized mechanisms for manipulating the information in a context-sensitive way (i.e. reasoning methods).// | | Representation | To be accessed after a measurement has been made, it must be //represented// somehow. The way it is represented has an effect on //how it can be used.// This is why //representation// is a key topic in AI. | \\ ====The Pedagogical Pentagon==== | What is Needed | There exists no //universal theory of learning// -- nor of //teaching//, //training//, //task-environments//, and //evaluation//. \\ This means that experimentation, exploration, and blind search are the only ways to answer questions about a learner's performance, curriculum design, training requirements, etc., and that we can never get more than partial, limited answers to such questions. | | That Said... | The Pedagogical Pentagon captures the five pillars of education: Learning, Teaching, Training, Environments, and Testing. \\ It's not a theory, but rather, a conceptual framework for capturing all key aspects of education. | | {{http://cadia.ru.is/wiki/_media/pedagogical_pentagon_full1.png?850}} || | The Pedagogical Pentagon (left) captures the five main pillars of any learning/teaching situation. The relationships between its contents can be seen from various perspectives: (a) As information flow between processes. (b) As relations between systems. (c ) As dependencies between (largely missing!) theories. [[http://alumni.media.mit.edu/~kris/ftp/AGI17-pedagogical-pentagon.pdf|REF]] || | \\ Tasks | Learning systems adjust their knowledge as a result of interactions with a task- environment. Defined by (possibly a variety of) objective functions, as well as (possibly) instructions (i.e. knowledge provided at the start of the task, e.g. as a "seed", or continuously or intermittently throughout its duration). Since tasks can only be defined w.r.t. some environment, we often refer to the combination of a task and its environment as a single unit: the task-environment. | | \\ Teacher | The goal of the teacher is to help a learner learn. This is done by influencing the learner’s task-environment in such a way that progress towards the learning goals is facilitated. Teaching, as opposed to training, typically involves information about the //What, Why & How:// \\ - What to pay attention to. \\ - Relationships between observables (causal, part-whole, etc.). \\ - Sub-goals, negative goals and their relationships (strategy). \\ - Background-foreground separation. | | Environment & Task | The learner and the teacher each interact with their own view of the world (i.e. their own “environments”) which are typically different, but overlapping to some degree. | | Training | Viewed from a teacher’s and intentional learner’s point of view, “training” means the actions taken (repeatedly) over time with the goal of becoming better at some task, by avoiding learning erroneous skills/things and avoid forgetting or unlearning desirable skills/things. | | \\ Test | Testing - or //evaluation// - is meant to obtain information about the structural, epistemic and emergent properties of learners, as they progress on a learning task. Testing can be done for different purposes: e.g. to ensure that a learner has good-enough performance on a range of tasks, to identify strengths and weaknesses for an AI designer to improve or an adversary to exploit, or to ensure that a learner has understood a certain concept so that we can trust it will use it correctly in the future. | | Source | [[http://alumni.media.mit.edu/~kris/ftp/AGI17-pedagogical-pentagon.pdf|The Pedagogical Pentagon: A Conceptual Framework for Artificial Pedagogy]] by Bieger et al. | \\ ====Learning Controllers==== | \\ A Learner | Adaptive/intelligent system/controller, embodied and situated in a task-environment, that continually receives inputs/observations (measurements) from its environment and sends outputs/actions back (signals to its manipulators). \\ Some of the learner’s inputs may be treated specially — e.g. as feedback or a reward signal, possibly provided by a teacher or a specially-rigged training task-environment. Since action can only be evaluated as "intelligent" in light of what it is trying to achieve - we model intelligent agents as imperfect optimizers of some (possibly unknown) real-valued objective function. \\ Note that this working definition fits //experience-based// learning. | | Embodiment | The interface between a learning controller and the task-environment. | \\ ====Experience-Based Learning==== | What It Is | Learning is the acquisition of knowledge for particular purposes. When this acquisition happens via interaction with an environment it is experience-based. | | Why It Is Important | Any environment which cannot be fully known a-priori requires experimentation of some sort, in the form of interaction with the world. This is what we call //experience//. | | \\ The Real World | The physical world we live in, often referred to as the "real world", is highly complex, and rarely if ever do we have perfect models of how it behaves when we interact with it, whether it is to experiment with how it works or simply achieve some goal like buying bread. | | \\ Limited Time & Resources | An important limitation on any agent's ability to model the real world is its enormous state space, which vastly outdoes any known agent's memory capacity, even for relatively simple environments. Even if the models were sufficiently detailed, pre-computing everything beforehand is prohibited due to memory. On top of that, even if memory would suffice for pre-computing everything and anything necessary to go about our tasks, we would have to retrieve the pre-computed data in time when it's needed - the larger the state space the more demands on retrieval times this puts. | | Why Experience-Based Learning is Relevant | Under LTE (limited time and energy) in a plentiful task-environment it is impossible to know everything all at once, including causal relations. Therefore, most of the time an intelligent agent capable of some reasoning will be working with uncertain assumptions where nothing is certain, only some things are more probable than others. | | \\ Bottom Line | The physical world has infinite variety that cannot be catalogued beforehand. \\ As a result the fundamental rules of the world are //not known// and that there is a guarantee of //uncertainty//. \\ This means that for any learner in the physical world the //learning will be non-axiomatic//. | \\ ==== Probability ==== | \\ What It Is | Probability is a concept that is relevant to a situation where information is missing, which means it is a concept relevant to //knowledge//. \\ A common conceptualization of probability is that it is a measure of the likelihood that an event will occur [[https://en.wikipedia.org/wiki/Probability|REF]]. \\ If it is not know whether event **X** will be (or has been) observed in situation **Y** or not, the //probability// of **X** is the percentage of time **X** would be observed if the same situation **Y** occurred an infinite number of times. | | \\ Why It Is Important \\ in AI | Probability enters into our knowledge of anything for which the knowledge is //**incomplete**//. \\ As in, //everything that humans do every day in every real-world environment//. \\ With incomplete knowledge it is in principle //impossible to know what may happen//. However, if we have very good models for some //limited// (small, simple) phenomenon, we can expect our prediction of what may happen to be pretty good, or at least //**practically useful**//. This is especially true for knowledge acquired through the scientific method, in which empirical evidence and human reason is systematically brought to bear on the validity of the models. | | How To Compute Probabilities | Most common method is Bayesian networks, which encode the concept of probability in which probability is interpreted as reasonable expectation representing a state of knowledge or as quantification of a personal belief [[https://en.wikipedia.org/wiki/Bayesian_probability|REF]]. Which makes it useful for representing an (intelligent) agent's knowledge of some environment, task or phenomenon. | | \\ Beyes' Theorem | {{public:t-713-mers:bayes_theorem.svg?200}} \\ A,B:=events \\ P(A/B):=probability of A given B is true \\ P(B/A):=probability of B given A is true \\ P(A),P(B):=the independent probabilities of A and B | | Judea Pearl | Most fervent advocate (and self-proclaimed inventor) of Bayesian Networks in AI [[http://ftp.cs.ucla.edu/pub/stat_ser/R246.pdf|REF]]. | | \\ Conceptualization of Probability in This Course | It is useful, in the context of this course, to think about probability as 'that which is not fully known': \\ The World contains a mechanism, **M**, whose operation is not fully known, **K(M)