| |
| public:t-720-atai:atai-20:learning [2020/09/03 16:04] – [Key Learning Terms] thorisson | public:t-720-atai:atai-20:learning [2024/04/29 13:33] (current) – external edit 127.0.0.1 |
|---|
| ====Key Learning Terms==== | ====Key Learning Terms==== |
| | What it is | Learning is a //**process**// that has the intent of acquiring actionable information, a.k.a. //knowledge//. | | | What it is | Learning is a //**process**// that has the intent of acquiring actionable information, a.k.a. //knowledge//. | |
| | Key Features | Inherits key features of any process: \\ - **Purpose**: To adapt, to respond in rational ways to problems / to achieve foreseen goals; this factor determines how the rest of the features in this list are measured. \\ - **Speed**: The speed of learning. \\ - **Data**: The data that the learning (and particular measured speed of learning) requires. \\ - **Quality**: How well something is learned. \\ - **Retention**: The robustness of what has been learned - how well it stays intact over time. \\ - **Transfer**: How general the learning is, how broadly what is learned can be employed for the purposes of adaptation or achievement of goals. \\ - **Meta-Learning**: A learner may improve its learning abilities - i.e. capable of meta-learning. \\ - **Progress Signal(s)**: A learner needs to know how its learning is going, and if there is improvement, how much. | | | \\ \\ \\ Key Features | Inherits key features of any process: \\ - **Purpose**: To adapt, to respond in rational ways to problems / to achieve foreseen goals; this factor determines how the rest of the features in this list are measured. \\ - **Speed**: The speed of learning. \\ - **Data**: The data that the learning (and particular measured speed of learning) requires. \\ - **Quality**: How well something is learned. \\ - **Retention**: The robustness of what has been learned - how well it stays intact over time. \\ - **Transfer**: How general the learning is, how broadly what is learned can be employed for the purposes of adaptation or achievement of goals. \\ - **Meta-Learning**: A learner may improve its learning abilities - i.e. capable of meta-learning. \\ - **Progress Signal(s)**: A learner needs to know how its learning is going, and if there is improvement, how much. | |
| | Measurements | To know any of the above some parameters have to be //measured//: All of the above factors can be measured in //many ways//. | | | Measurements | To know any of the above some parameters have to be //measured//: All of the above factors can be measured in //many ways//. | |
| | Major Caveat | Since learning interacts with (is affect by) the //task-environment and world// that the learning takes place in, as well as the nature of these in the learner's //subsequent deployment//, //none// of the above features can be assessed by //looking only at the learner//. \\ This is addressed by the //[[/public:t-720-atai:atai-20:teaching|Pedagogical Pentagon]]//. | | | Major Caveat | Since learning interacts with (is affect by) the //task-environment and world// that the learning takes place in, as well as the nature of these in the learner's //subsequent deployment//, //none// of the above features can be assessed by //looking only at the learner//. \\ This is addressed by the //[[/public:t-720-atai:atai-20:teaching|Pedagogical Pentagon]]//. | |
| | Reinforcement Learning | Learning proceeds through discrete steps whereby a step is of the kind A-R pair, A being an action and R being a reward. || | | Reinforcement Learning | Learning proceeds through discrete steps whereby a step is of the kind A-R pair, A being an action and R being a reward. || |
| | \\ Observational Learning || Many animals have been observed to learn by observation (no pun intended). In some cases by observing members of the same species doing that particular thing (called "conspecifics"), in other cases by watching events unfold. | | | \\ Observational Learning || Many animals have been observed to learn by observation (no pun intended). In some cases by observing members of the same species doing that particular thing (called "conspecifics"), in other cases by watching events unfold. | |
| | | Morphological \\ Observational Learning | Aka Structural-. Learning is restricted to the morphology (structure) of the movements and/or actions being observed. May be sufficient when learning to dance, but certainly less useful when learning to conduct an orchestra. | | | | Morphological \\ Observational Learning | A.k.a. Structural-. Learning is restricted to the morphology (structure) of the movements and/or actions being observed. May be sufficient when learning to dance, but certainly less useful when learning to conduct an orchestra. | |
| | | Goal-Level Observational \\ Learning | Learning includes learning the //purpose// for which the observed actions are performed. | | | | Goal-Level Observational \\ Learning | \\ Learning includes learning the //purpose// for which the observed actions are performed. | |
| | \\ Life-Long Learning | Colloquially: Learning throughout one's lifetime. \\ In AI: A particular focus of learning research targeting how systems can change their learning over //long periods// of time. "Duration" doesn't refer to a particular number of hours or years but rather indicates the expectations on the system being engineered that it learn over long periods of time, "long" relative to prior such machine learners, and "long" relative to the system's operational lifetime. || | | \\ Life-Long Learning | Colloquially: Learning throughout one's lifetime. \\ In AI: A particular focus of learning research targeting how systems can change their learning over //long periods// of time. "Duration" doesn't refer to a particular number of hours or years but rather indicates the expectations on the system being engineered that it learn over long periods of time, "long" relative to prior such machine learners, and "long" relative to the system's operational lifetime. || |
| | Online Learning | Aka "continuous"- or "simultaneous"-. Learning while doing (same or other) things. || | | Online Learning | A.k.a. "continuous"- or "simultaneous"-. Learning while doing (same or other) things. || |
| | Multi-task Learning | Aka "multi-goal"-. The same system learning many tasks/things without forgetting what was learned before. || | | Multi-task Learning | A.k.a. "multi-goal"-. The same system learning many tasks/things without forgetting what was learned before. || |
| | Transfer Learning | The ability to benefit from something already learned when learning something new. || | | Transfer Learning | The ability to benefit from something already learned when learning something new. || |
| | Single-Shot Learning | Aka "few-shot"-. The ability to learn something new from one example. || | | Single-Shot Learning | A.k.a. "few-shot"-. The ability to learn something new from one example. || |
| | Cumulative Learning | New things learned are //integrated// with things learned prior. The two are fused so as to create a more coherent, more easily-verifiable knowledge set. || | | Cumulative Learning | New things learned are //integrated// with things learned prior. The two are fused so as to create a more coherent, more easily-verifiable knowledge set. || |
| | |
| | Learning by observation | A method for learning. An Agent <m>A</m> learns how to achieve Goal <m>G</m> by receiving realtime information about some other Agent <m>A'</m> achieving Goal <m>G</m> by doing action <m>a</m>. | | | Learning by observation | A method for learning. An Agent <m>A</m> learns how to achieve Goal <m>G</m> by receiving realtime information about some other Agent <m>A'</m> achieving Goal <m>G</m> by doing action <m>a</m>. | |
| | Learning from reasoning | A method for learning. Using deduction, induction and/or abduction to simulate, generalize, and infer, respectively, new information from acquired information. \\ Most effectively used in combination with Learning from Experience. | | | Learning from reasoning | A method for learning. Using deduction, induction and/or abduction to simulate, generalize, and infer, respectively, new information from acquired information. \\ Most effectively used in combination with Learning from Experience. | |
| | Learning from Teaching | Use of //Instructions//, provided by a //Teacher//, to improve knowledge. \\ Teaching is typically situated, i.e. provided on-demand (during learning/training). | | | Learning from Teaching | A method for learning. Use of //Instructions//, provided by a //Teacher//, to improve knowledge. \\ Teaching is typically situated, i.e. provided on-demand (during learning/training). | |
| | Multi-objective learning | Learning while aiming to achieve more than one Goal. \\ //(There is a strange concept usage in some circles, where 'multi-objective learning' means 1-5 objectives (goals), and 'many-objective learning' means >5 objectives.) This is a clear example of how terminology can get twisted in the "fight" for attention between academics. It comes about in part because of the combinatorial explosion with more than 5 objectives.)// | | | \\ Multi-objective learning | Learning while aiming to / learning how to achieve more than one Goal. \\ //(There is a strange concept usage in some circles, where 'multi-objective learning' means 1-5 objectives (goals), and 'many-objective learning' means >5 objectives.) This is a clear example of how terminology can get twisted in the "fight" for attention between academics. It comes about in part because of the combinatorial explosion with more than 5 objectives.)// | |
| | Transfer learning | Applying already-acquired knowledge to a new or newish //Problem//. \\ A method for learning faster based on similarity identification. \\ By not having to re-learn highly similar things to what has already been learned and adapting/mapping (modifying) existing knowledge to new problems. | | | \\ Transfer learning | Applying already-acquired knowledge to a new or newish //Problem//. \\ A method for learning faster based on similarity identification. \\ By not having to re-learn highly similar things to what has already been learned and adapting/mapping (modifying) existing knowledge to new problems. | |
| | Transversal (i.e. System-Wide) Ampliative Learning | \\ What we could call a combination of all of the above. | | | Transversal (i.e. System-Wide) Ampliative Learning | \\ What we could call a combination of all of the above. | |
| |
| | \\ What is Being Learned | Categories: \\ - Tool (body) \\ - Task-environment (the task at hand) \\ - Domain-bound strategies \\ - Domain-independent learning \\ - Domain-independent learning strategies ("cognitive development") \\ //Each one subsumes the ones above.// | | | \\ What is Being Learned | Categories: \\ - Tool (body) \\ - Task-environment (the task at hand) \\ - Domain-bound strategies \\ - Domain-independent learning \\ - Domain-independent learning strategies ("cognitive development") \\ //Each one subsumes the ones above.// | |
| | Tool (body) | A controller needs to be embodied to affect the world; learning what the body does, irrespective of the task-environment, domain, or other issues. | | | Tool (body) | A controller needs to be embodied to affect the world; learning what the body does, irrespective of the task-environment, domain, or other issues. | |
| | Task-Environment | The proverbial "task" that an agent has been assigned in a particular "environment". \\ Typically, 'task' is anything that would stay unchanged between environments, and 'environment' refers to anything else that still may affect task performance but should ideally stay out of the way (or be unchanged during task exectuion). | | | \\ Task-Environment | The proverbial "task" that an agent has been assigned in a particular "environment". \\ Typically, 'task' is anything that would stay unchanged between environments, and 'environment' refers to anything else that still may affect task performance but should ideally stay out of the way (or be unchanged during task exectuion). | |
| | Domain-Bound Strategies | Strategies related to specific issues in the task-domain but learning may temporarily slow down learning the task-environment. | | | Domain-Bound Strategies | Strategies related to specific issues in the task-domain but learning may temporarily slow down learning the task-environment. | |
| | Domain-Independent Learning | Refers to the concept of "learning to learn" - learning that is transferrable between domains. | | | Domain-Independent Learning | Refers to the concept of "learning to learn" - learning that is transferrable between domains. | |
| ====Learning Paradigms==== | ====Learning Paradigms==== |
| | Learning From Input/Output Pairs \\ "Supervised Learning" | \\ The ability to learn a mapping from inputs to outputs based on examples of input-output pairs. This requires having a way to perceive what the output to a particular input should have been. | | | Learning From Input/Output Pairs \\ "Supervised Learning" | \\ The ability to learn a mapping from inputs to outputs based on examples of input-output pairs. This requires having a way to perceive what the output to a particular input should have been. | |
| | No Feedback \\ "Unsupervised Learning" | The ability to learn patterns in the input even though no external feedback is given. Examples include clustering, anomaly detection and dimensionality reduction. | | | No Feedback \\ "Unsupervised Learning" | The ability to learn patterns in the input even though no external feedback is given. Examples include clustering, anomaly detection and dimensionality reduction. \\ Note that without some sort of goal, this kind of learning is without purpose and objectives (which means it should not, strictly speaking, be called "learning", because there is no way to answer the question //'have you learned it yet?'//). | |
| | Learning from Rewards \\ "Reinforcement Learning" | The ability to learn from a series of (positive and negative) rewards. This is usually used to learn how to behave in multi-step control problems. It requires machinery to treat certain perceptions (i.e.~the rewards) as "special" and something to be optimized for. | | | Learning from Rewards \\ "Reinforcement Learning" | The ability to learn from a series of (positive and negative) rewards. This is usually used to learn how to behave in multi-step control problems. It requires machinery to treat certain perceptions (i.e.~the rewards) as "special" and something to be optimized for. | |
| | \\ Learning from Teaching | Can be done in a wide variety of ways, each of which might impose their own requirements on the AI architecture. For instance, imitation learning -- the ability to learn behaviors by observing another agent carry them out -- requires a deep understanding of the perceived actions to be imitated, meaning the system must not only be able to observe those actions, but also recognize those actions, map them to its own perspective and body, and possibly infer their intent. | | | \\ Learning from Teaching | Can be done in a wide variety of ways, each of which might impose their own requirements on the AI architecture. For instance, imitation learning -- the ability to learn behaviors by observing another agent carry them out -- requires a deep understanding of the perceived actions to be imitated, meaning the system must not only be able to observe those actions, but also recognize those actions, map them to its own perspective and body, and possibly infer their intent. | |
