T-719-NXAI-2025 Main
Links to Lecture Notes
Key concepts: Reasoning, Causality, Non-Axiomatic Logic, Defeasibility
Related readings:
| What is Reasoning? | A systematic way of thinking about antecedents, implications, conditionals and inevitables. |
| Empirical Reasoning | A systematic way of thinking about how the world hangs together. |
| How is it done? | Via processes that observe rules. |
| Types of reasoning operations | Deduction: All men are mortal. Socrates is a man. Hence, Socrates is mortal Abduction: How did this come about? (Sherlock Holmes) Induction: “What is the general rule?” Analogy: 'This' is like 'that' (in 'this' way). |
| How is it used in science? | In empirical science to unearth the “rules of the universe”. In mathematics as axioms. In philosophy as a way to construct arguments. In computer science to write code. In AI: To model how intelligent agents learn. |
| Why does it work? | Because the world's behavior is/seems rule-based. |
| But the world is not deterministic! | The world is “pragmatically non-deterministic”, meaning that for all practical purposes we must approach it as if it is non-deterministic. Also, we do not know the world's “axioms” – and cannot ever be sure we will have them. For these reasons, reasoning about the world as if it was axiomatic will not suffice – we must deal with the unknowns ('noise', 'known unknowns' and 'unknown unknowns') in a systematic manner. This is done through defeasibility methods and non-axiomatic logic. |
| EMPIRICAL REASONING | Defeasible and non-axiomatic reasoning where the data, rules, and results are restricted to the physical world. |
| What it is | The idea that all knowledge comes from experience – the senses. In AI it also means that this experience comes from the physical world, through physical sensors. |
| Why it matters | Before the emphasis on empirical knowledge, science did not have a chance to rise in any obvious way above “other sources of knowledge,” including old scriptures, intuition, religious beliefs, or information produced by oracles. |
| Empiricism & Science | The fundamental source of information in (empirical, i.e. experimental) science is experience, which eventually became the formalized comparative experiment. |
| Comparative Experiment | A method whereby two experimental conditions are compared, were they are identical except for one or a few strategic differences that the experimenters make. The outcome of the comparison is used to infer causal relations. Often called “the scientific method”, this is the most dependable method for creating reliable, sharable knowledge that humanity has come up with. REF |
| Logical Positivism | Philosophical school of thought closely related to empirical science. REF |
| Rationalism | Historically a philosophically opposing view to empiricism, contending that knowledge is produced through innate knowledge, not through experience. REF |
| Empirical Rationalism | Both pure empiricism and rationalism are exaggerated views on where knowledge comes from. The sensible happy medium is that knowledge creation is started by innate bootstrapping processes on which knowledge is built through experience. This is the philosophical view that we take in this course. |
| Theory | A scientific (empirical) theory is a “story” about how certain phenomena relate to each other. The more details, the more accurately, and the larger scope the theory covers, the better it is. |
| Hypothesis | A statement about how the world works, derived from a theory. |
| Experimental design | A planned interference in the natural order of events. |
| Subject(s) | Subject of interest - that to be studied, whether people, technology, natural phenomena, or other |
| Sample | Typically you can't study all the individuals of a particular subject pool (set), so in your experiment you use a sample (subset) and hope that the results gathered using this subset generalize to the rest of the set (subject pool). |
| Dependent variable | The measured variable(s) of the phenomenon which you are studying |
| Independent variable | The variable(s) that you manipulate in order to systematically affect (or avoid affecting) the dependent variable(s) |
| Internal validity | How likely is it that the manipulation of the independent variables caused the effect in dependent variables? |
| External validity | How likely is it that the results generalize to other instances of the phenomenon under study? |
| What is it? | A fairly recent research method, historically speaking, for testing hypotheses / theories |
| Why is it Important? | The most reliable way humanity has found to create reliable sharable knowledge. |
| Why is it Relevant Here? | Like individual learning, it involves dealing with new phenomena and figuring them out. |
| Comparative experiment = everyday human learning? | More similiar than you'd think. Science is a systematized, organized approach to what individuals do when they learn. This involves: - Exploring - Hypothesizing - Trying out - Recording results - Unifying and consolidating related information - Creating coherent stories about what has been found (theorizing) |
| When is it used? | When it is possible to control and select everything of importance to the subject of study |
| How it's used | Select subjects freely, randomize samples, remove experimenter effect through double-blind procedure, use control groups, select independent and dependent variables as necessary to answer the questions raised. |
| What does 'random' mean? | A truly random selection means that there are no causal relations between what is being measured and how the entities are selected. |
| Why randomize? | Given a complex phenomenon, it is impossible to know all potential causal chains that may exist between the various elements under study. Randomization lessens the probability that there is systematic bias in any factors that are not under study but could affect the results and thus imply different conclusions. |
| What is randomized? | 1. The entities in the sample should be randomly selected; subjects should be randomly assigned to control group versus experimental group. 2. Any independent variable identified which could affect the results but is not considered of interest to the research at hand. |
| Bottom line | Empirical controlled experiment: The most powerful mechanism for generating reliable knowledge known to mankind. |
| Even more bottom line | The most powerful mechanism for generating reliable knowledge known to mankind has high relevance for A(G)I research. |
| What It Is | Learning is the acquisition of knowledge for particular purposes. When this acquisition happens via interaction with an environment it is experience-based. |
| Why It Is Important | Any environment which cannot be fully known a-priori requires experimentation of some sort, in the form of interaction with the world. This is what we call experience. |
| The Real World | The physical world we live in, often referred to as the “real world”, is highly complex, and rarely if ever do we have perfect models of how it behaves when we interact with it, whether it is to experiment with how it works or simply achieve some goal like buying bread. |
| Limited Time & Resources | An important limitation on any agent's ability to model the real world is its enormous state space, which vastly outdoes any known agent's memory capacity, even for relatively simple environments. Even if the models were sufficiently detailed, pre-computing everything beforehand is prohibited due to memory. On top of that, even if memory would suffice for pre-computing everything and anything necessary to go about our tasks, we would have to retrieve the pre-computed data in time when it's needed - the larger the state space the more demands on retrieval times this puts. |
| Why Experience-Based Learning is Relevant | Under LTE (limited time and energy) in a plentiful task-environment it is impossible to know everything all at once, including causal relations. Therefore, most of the time an intelligent agent capable of some reasoning will be working with uncertain assumptions where nothing is certain, only some things are more probable than others. |
| What It Is | Probability is a concept that is relevant to a situation where information is missing, which means it is a concept relevant to knowledge. A common conceptualization of probability is that it is a measure of the likelihood that an event will occur REF. If it is not know whether event X will be (or has been) observed in situation Y or not, the probability of X is the percentage of time X would be observed if the same situation Y occurred an infinite number of times. |
| Why It Is Important in AI | Probability enters into our knowledge of anything for which the knowledge is incomplete. As in, everything that humans do every day in every real-world environment. With incomplete knowledge it is in principle impossible to know what may happen. However, if we have very good models for some limited (small, simple) phenomenon, we can expect our prediction of what may happen to be pretty good, or at least practically useful. This is especially true for knowledge acquired through the scientific method, in which empirical evidence and human reason is systematically brought to bear on the validity of the models. |
| How To Compute Probabilities | Most common method is Bayesian networks, which encode the concept of probability in which probability is interpreted as reasonable expectation representing a state of knowledge or as quantification of a personal belief REF. Which makes it useful for representing an (intelligent) agent's knowledge of some environment, task or phenomenon. |
| How It Works | P(a#b) = {P(b#a) P(a)} / {P(b)} where # means 'given'. |
| Judea Pearl | Most Fervent Advocate (and self-proclaimed inventor) of Bayesian Networks in AI REF. |
| What It Is | A Platonic model for the relationship between two or more variables, whereby changes in one always precede changes in the other. |
| In More Detail | A causal relationship between variables <m>A,B</m> can be defined as a relationship such that changes in <m>A</m> always lead to a change in <m>B</m> and where the timing relationship is such that the former happens always before the latter, forall {A, B}: t(A) < t(B). Example: A light switch is designed specifically to cause the light to turn on and off. In a causal analysis based on abduction one may reason that, given that light switches don't tend to flip randomly, a light that was off but is now on may indicate that someone or something flipped the light switch. (The inverse - a light that was on but is now off - has a larger set of reasonable causes, in addition to someone turning it off, a power outage or bulb burnout.) |
| Why It Is Important in Science | Causation is the foundation of empirical science. Without knowledge about causal relations it is impossible to get anything systematically done. |
| Why It Is Important in AI | The main purpose of intelligence is to figure out how to get new stuff done, given limited time and energy (LTE), i.e. to get stuff done cheaply but well. To get stuff done means knowing how to produce effects. In this case reliable methods for getting stuff done are worth more to an intelligence than unreliable ones. A relationship that approximates a Platonic cause-effect is worth more than one that does not. |
| History | David Hume (1711-1776) is one of the most influential philosophers addressing the topic. From the Encyclopedia of Philosophy: “…advocate[s] … that there are no innate ideas and that all knowledge comes from experience, Hume is known for applying this standard rigorously to causation and necessity.” REF This makes Hume an empiricist. |
| More Recent History | Causation has been cast by the wayside in statistics for the past 120 years, saying instead that all we can claim about the relationship of any variables is that they correlate. Needless to say this has lead to significant confusion as to what science can and cannot say about causal relationships, such as whether mobile phones cause cancer. Equally badly, the statistical stance has infected some scientific fields to view causation as “unscientific”. |
| Spurious Correlation | Non-zero correlation due to complete coincidence. |
| Causation & Correlation | What is the relation between causation and correlation? There is no (non-spurious) correlation without causation. There is no causation without correlation. However, correlation between two variables does not necessitate one of them to be the cause of the other: They can have a shared (possibly hidden) common cause. |
| Correlation Supports Prediction | Correlation is sufficient for simple prediction (if A and B correlate highly, then it does not matter if we see an A OR a B, we can predict that the other is likely on the scene). |
| Knowledge of Causation Supports Action | We may know that A and B correlate, but if we don't know whether B is a result of A or vice versa, and we want B to disappear, we don't know whether it will suffice to modify A. Example: The position of the light switch and the state of the light bulb correlate. Only by knowing that the light switch controls the bulb can we go directly to the switch if we want the light to turn on. |
| Causal Models Are Necessary To Guide Action | While correlation gives us indication of causation, the direction of the “causal arrow” is critically necessary for guiding action. Luckily, knowing which way the arrows point in any large set of correlated variables is usually not too hard to find out, by empirical experimentation. |
| Judea Pearl | Most Fervent Advocate of causality in AI, and the inventor of the Do Calculus. C.f. BAYESIANISM AND CAUSALITY, OR, WHY I AM ONLY A HALF-BAYESIAN. |
| State Of The Art | Recent work by Judea Pearl demonstrates clearly the fallaciousness of the statistical stance, and fixes some important gaps in our knowledge on this subject which hopefully will rectify the situation in the coming years. YouTube lecture by J. Pearl on causation. |
| What It Is | The ability (of anyone or anything) to explain, after the fact or before, why something happened the way it did, how it could have happened differently but didn't (in general and/or this time). |
| In AI | The ability of a controller to explain, after the fact or before, why it did something or intends to do it. |
| Explanation Depends On Causation | It is impossible to explain anything in any useful way without referring to general causal relations. |
| Why It Is Important | If a controller does something we don't want it to repeat - e.g. crash an airplane full of people (in simulation mode, hopefully!) - it needs to be able to explain why it did what it did. If it can't, it means it - and we - can never be sure of why it did what it did, whether it had any other choice, or under what conditions it might do it again. |
| Explanation Depends on Causation | No explanation is without reference to causes; discernible causal structure is a prerequisite for explainability. |
| Bottom Line for Human-Level AI | To grow and learn and self-inspect an AI must be able to sort out causal chains. If it can't it will not only be incapable of explaining to others why it is like it is, it will be incapable of explaining to itself why things are the way they are, and thus, it will be incapable of sorting out whether something it did is better for its own growth than something else. Explanation is the big black hole of ANNs: In principle ANNs are black boxes, and thus they are in principle unexplainable - whether to themselves or others. One way to address this is by encapsulating knowledge as hierarchical models that are built up over time, and can be de-constructed at any time (like AERA does). |
2025©K. R. Thórisson