Empirical Reasoning (I)

Foundational Considerations

Why Empirical?	The concept 'empirical' refers to the physical world: We (humans) live in a physical world, which is to some extent governed by rules, some of which we know something about.
Why Reasoning?	For interpreting, managing, understanding, creating and changing rules, logic-governed operations are highly efficient and effective. We call such operations 'reasoning'. Since we want to make machines that can operate more autonomously (e.g. in the physical world), reasoning skills is one of those features that such systems should be provided with.
Why Empirical Reasoning?	The physical world is uncertain because we only know part of the rules that govern it. Even where we have good rules, like the fact that heavy things fall down, applying such rules is a challenge, especially when faced with the passage of time. The term 'empirical' refers to the fact that the reasoning needed for intelligent agents in the physical world are - at all times - subject to limitations in energy, time, space and knowledge (also called the “assumption of insufficient knowledge and resources (AIKR)” by AI researcher Pei Wang).

Agents in Worlds

Agent	A system that can sense and act in an environment to do tasks. Sensing and actuating is done via the agent's transducers, which are part of its embodiment. https://en.wikipedia.org/wiki/Intelligent_agent
World / Environment	We call a particular implementation of a set of processes, variables and relationships such that certain values of variables are possible and others cannot, a World. An Environment in a World is a subset of the World, where the list of what is possible is shorter.
Perception	A process that is part of the cognitive system of intelligent systems and whose purpose is to measure and produce outcomes of measurements, in a format that can be used by the control apparatus of which it is part.
Percept	The product of perception – produced outcomes of measurements.
Goal	A substate of a World. A (steady-)state that could be achieved by an agent, if assigned.
Sub-Goal	A substate of a World that can serve as an intermediate state towards achieving a (higher-level) goal.
Constraints	A set of limitations that serve to reduce the degrees of freedom in performing a task or achieve a goal.
Task	A set of Goals and Constraints that could be assigned to an agent to be performed.
Plan	The partial set of actions that an agent assumes may achieve the goal.
Planning	The act of producing a plan.
Knowledge	Actionable information. Information that can be used for various purposes.
Knowledge Acquisition	The production of information-based models from experience.
Empirical Learning Agent	An agent that can get better at doing tasks and achieving goals through its own experience. The term 'empirical' refers to the physical world.

Principles of Empirical Reasoning vs. Mathematical Reasoning

TOPIC	MATHEMATICAL REASONING	EMPIRICAL REASONING
Target Use	Specify/define complete ruleset/system for closed worlds. Intended for use with necessary and sufficient info. Meant for dealing with mathematical domains.	Figure out how to get new things done in open worlds. Intended for use with incomplete and insufficient info. Meant for dealing with physical domains.
World Assumption	Closed and certain. Axioms fully known and enumerated. Axiomatic and Platonic (hypothetical)	Open and uncertain. At least one unknown axiom exist at all times; every known axiom is defeasible¹ (not guaranteed).
Energy, Time, Space	Independent of energy, space and time (unless specifically put into focus).	Limited by energy, space & time (LEST); This is of central concern.
Source of Data	Mostly hand-picked by humans from a pre-defined World.	Mostly measured by reasoning system itself, from a mostly undefined World.
Human-Generated Info	Large ratio of human to machine-generated info (> 1). Human-generated info is detailed and targets specific topics and tasks.	Small ratio of human to machine-generated info (« 1). Human-generated info is provided in a small 'seed' and targets general bootstrapping.
System-Produced Output	Guaranteed to be correct if the premises and rule use is correct.	Not guaranteed to be correct - always defeasible.
Data Reliability	Always trusted.	Never fully trusted; always incomplete.
Data Availability	Most data is available. No hidden data.	Most data is unavailable and/or hidden.
Data Types	Known a-priori. Statements always syntactically correct; pre-defined syntax.	Mostly not known; tiny dataset provided a-priori.
Permitted Rule Values	Primarily Bool (True, False) Rarely augmented by “unknown”.	Highly variable combinations of Bool, N, Z, Q, R, C, as well as 'uncertain' and 'not known'.
Information Amount	Inevitably sparse (due to being fully known).	Always vastly larger than available processing - overwhelming.
Statements	Clear, clean and complete.	Most statements are incomplete; rarely clear and clean.
Incorrect Statements	Guaranteed to be identifiable.	Cannot be guaranteed to be identifiable.
Deduction	Safe² and complete³ (due to complete and clean data and semantics).	Defeasible (always, due to incomplete data and semantics).
Abduction	Safe and complete (always, due to complete knowledge).	Defeasible (always, due to incomplete knowledge).
Induction	Defeasible (always, due to incomplete data).	Defeasible (always, due to incomplete data and semantics).
Analogy	Complete (always, due to complete knowledge of data and semantics).	Defeasible (always, due to With incomplete data and semantics).

¹ By 'defeasible' is meant that it may be found to be incorrect, at any time, given additional data, reconsideration of background assumptions or discovery of logic errors.
² By 'safe' is meant that the output of a reasoning process is provably correct and can be trusted.
³ By 'complete' is meant that the output of a reasoning process leaves nothing unprocessed.

Challenges in Empirical Reasoning

Data Diversity	Data types in the physical world are many and diverse - and cannot be known beforehand.
Noisy Data	Data in the physical world is extremely noisy - we say that there is a 'low signal-to-noise ratio'.
Missing Data/Evidence	Key information that is necessary for achieving goals may be missing at any time, in any context - most of the time it is not known that the data is missing.
Missing Axioms	The rules of the world are not known. The rules that are known are often incorrect.
Bottom Line	Due to the above, empirical reasoning always deals with a high degree of uncertainty.
Result	This is why closed-world axiomatic reasoning cannot work for dealing with partially-known open worlds.
Knowing What is Relevant	Any general learner will, after learning for some time, hold knowledge about a diverse set of phenomena. Thus, any general learner is, at any point, faced with the question of what parts of its knowledge are relevant.
Hence	Empirical reasoning must be different from mathematical (axiomatic) reasoning.

Evidence

What it Is	“Evidence” is what we call information that supports a particular claim, conclusion, model, causal relation, or explanation. We must always ask “evidence of what?” because there is no such thing as “just evidence” – evidence is always relative to a particular claim, conclusion, etc.
What it Is (take 2)	Evidence comes in several forms; it is common to distinguish it based on its source: First-person experience is information that comes directly through the senses, third-person evidence is what we hear from others. A third source that is important in science (and hence for reliable knowledge) but is not very often discussed in everyday life is what we could call 'coherence' – or, how well some evidence fits known facts, other pieces of evidence, or background assumptions.
Forms of Evidence	There are two forms of evidence, *positive* and *negative; the former is evidence that supports* a claim, the second is evidence that refutes it. There is a third category of evidence that we can call “partial”. This is evidence that might, could, may, or possibly can be seen as supporting or refuting, depending on other information.
Why it Matters	Evidence-based knowledge creation is the only knowledge creation method that can be trusted.

Note on Claims and {T,F}	It is important to remember that claims are statements, and statements must be represented in some language, and languages have a syntax. A statement in a language using a particular syntax has limitations in what it can represent. Knowledge in the physical world is about physical phenomena, and these do not come in the form of claims; instead, they follow what we call “laws of nature”. To make claims about physical things the latter must be transcribed into the former. To collect evidence for and against claims means that the evidence must either be translated into the same language, or the claims must be re-interpreted back into physical form. Either way, there is often information loss in this process. This one reason why 'claims', 'evidence', 'truth' and 'falsehoods' are not always straight forward to deal with. This is part of what makes intelligence such a slippery phenomenon.

Non-Axiomatic Reasoning

NAL	Distinguishes itself from other reasoning languages in that it is intended for knowledge in worlds where the axioms are unknown, not guaranteed, and/or fallible. NAL is itself axiomatic, but it is designed for domains that are non-axiomatic.
NAL Features	Instead of being either {T,F}, statements have a degree of truth to them, represented by a value between 0 and 1. NAL uses term logic, which is different from propositional logic in the way it expresses statements.
Evidence	w⁺ is positive evidence; w^- is negative evidence.
Uncertainty	Frequency: f = w⁺ / w, where w = w⁺ + w^- (total evidence). Confidence: c = w/(w + k), where k ≥ 1. Ignorance: i = k/(w + k).
Deduction	The premises are given. Figuring out the implication of facts (or predicting what may come). Producing implications from premises. E.g. “The last domino will fall when all the other dominos between the first and the last have fallen”.
Abduction	A particular outcome X is given. Figuring out how things came to be the way they are (or how particular outcomes could be made to come about, or how particular outcomes could be prevented). E.g. Sherlock Holmes, who is a genius abducer.
Induction	A small set of examples is given. Figuring out the general case. Making general rules from a (small) set of examples. E.g. “The sun has risen in the East every morning up until now, hence, the sun will also rise in the East tomorrow”.
Analogy	A set of two (or more) things is given. Figuring out how things are similar or different. Making inferences about how something X may be (or is) through a comparison to something else Y, where X and Y share some observed properties. E.g. “What does a pen have in common with an arrow?” “What is the difference between a rock and a ball?”
	^{Author of the Non-Axiomatic Reasoning covered here: Pei Wang}

Center for Analysis and Design of Intelligent Agents

Table of Contents

Empirical Reasoning (I)

Foundational Considerations

Agents in Worlds

Principles of Empirical Reasoning vs. Mathematical Reasoning

Challenges in Empirical Reasoning

Evidence

Non-Axiomatic Reasoning