Center for Analysis and Design of Intelligent Agents

Readings README (Do not skip!)

What is intelligence?
How do experts talk about it?
What has been uncovered?
What uniquely separates the phenomenon of intelligence from other similar phenomena in the world?

• What is Thought? by Ezequiel Morsella
• Thought on Wikipedia.
• The g factor on Wikipedia.
• Multiple theory of intelligence on Wikipedia.
• Discretionarily Constrained Adaptation Under Insufficient Knowledge & Resources by K. R. Thórisson
• Animal Cognition on Wikipedia.
• A Computational Foundation for the Study of Cognition by D. Chalmers
• Computing Machinery and Intelligence by A. M. Turing.
• Evolution, Learning & Culture: Computational Metaphors for Adaptive Algorithms by R. K. Belew
  • (esp. Section 1, Section 2.2. and Figure 19)
• Podcast interview with Jeff Hawkins by L. Friedman
• A Collection of Definitions of Intelligence by Legg & Hutter.

Are animals other than humans also intelligent?
Note: You are encouraged to find other material on this topic (please let instructor know if you find interesting things).

• Parrots [ 3,3 ]
  • Alex the Parrot on YouTube (video repeats halfway, skip rest).
  • Alex on Wikipedia
  • Parrots vs. Children BBC.
• Gorillas [ 1,2 ]
  • Koko the Gorilla on YouTube.
  • Koko on Wikipedia
• Crows [ 2,3 ]
  • Crow solving an 8-step puzzle on YouTube
  • Crow demonstrates causal understanding on YouTube
  • TED talk on crow intelligence by John Marzluff
• Bumblebees [ 1,2]
  • Bumblebees learn by observation on YouTube.
  • Paper on Bumblebees learning by observation by Loukola et al.
• Elephants [ 0,0 ]
  • Why Aren't Elephants Smarter Than Humans Since Their Brains Are Bigger? by Fabian van den Berg

When engineers make an artifact, like a bridge or a space rocket, they start by listing the artifact's requirements. This way, for any proposed implementation, they can check their progress by comparing the performance of a prototype to these. The below papers consider what are necessary and sufficient requirements for a machine with real intelligence. (Therefore, these speak to defining what 'intelligent systems' in fact really means.)

• On Defining Artificial Intelligence by Pei Wang
• Architectural Requirements for Human-like Agents Both Natural and Artificial by A. Sloman
• Requirements for deliberative systems by A. Sloman – key sections: from 8 onwards.
• Cognitive Architecture Requirements for Achieving AGI by J.E. Laird et al.
• Cognitive architectures: Research issues and challenges. by Langley, P., Laird, J.E., Rogers, S. (2009). Cognitive Systems Research Vol 10(2), pp. 141-160.
• Holistic Intelligence: Transversal Skills & Current Methodologies by K.R. Thórisson & E. Nivel.
• Towards a Complete, Multi-level Cognitive Architecture by R. Wray et al.
• Introduction: Aspects of Artificial General Intelligence by P. Wang & B. Goerzel (first 3 sections)
• Mapping the Landscape of Human-Level Artificial General Intelligence by Adams et al.

Artificial intelligence (AI) started as a research field. It still is. Just like research results in physics are useful for engineering, results in AI are useful for industry. AI is still in formation, much like computer science. It is a knowledge-generating enterprise funded by the public through universities and competitive research grants. Applications of AI are funded by companies and through various other means (including competitive grants for applied research). The knowledge generated in AI research is in part determined by the nature of the enterprise - how it's organized, who are the influencers, what are low-hanging fruit, etc.

• What Is AI? by J. McCarthy.
• Four Basic Questions about Artificial Intelligence by P. Wang.
• Three Fundamental Misconceptions of Artificial Intelligence by P. Wang.

• What is GMI? by K. R. Thórisson.
• Artificial General Intelligence on Wikipedia.
• The very laws of physics imply that artificial intelligence must be possible. What's holding us up? by D. Deutch.
• https://www.youtube.com/watch?v=lcUk1cYWY9I Can artificial intelligence become sentient, or smarter than we are - and then what? | Techtopia @ Deutsche Welle

• Earth Offers Great Variety by K.R. Thórisson
• Why Artificial Intelligence Needs a Task Theory — And What It Might Look Like by K. R. Thórisson et al.
• Laplace's demon on Wikipedia
• Maxwell's demon on Wikipedia
• On the Decrease of Entropy in a Thermodynamic System by the Intervention of Intelligent Beings by L. Szilard

• Bayesianism and Causality, or, Why I Am Only A Half-Bayesian by J. Pearl
• Cumulative Learning With Causal-Relational Models by K.R.Thórisson et al.

• Evaluation of General-Purpose Artificial Intelligence: Why, What & How by J. Bieger et al.
• A New AI Evaluation Cosmos: Ready to Play the Game? by J. Hernandez-Orallo et al.
• Towards Flexible Task-Environments For Comprehensive Evaluation of Artificial Intelligent Systems & Automatic Leaners by K. R. Thórisson et al.
• About the Intricacy of Tasks by Belanchia et al.
• The Toy Box Problem by Johnston
• Loebner Prize article by Charlie Moloney.
• SAGE: Task-Environment Platform for Autonomy and Generality Evaluation by L. Eberding et al.
• FraMoTEC: Modular Task-Environment Construction Framework for Evaluating Adaptive Control Systems by Thorarensen et al.
• The Lovelace Test by M. Reidl.
• Psychometric Artificial General Intelligence: The Piaget-MacGuyver Room by Bringsjord, S. & Licato, J. In Theoretical Foundations of Artificial General Intelligence, edited by P. Wang and B. Goertzel (Atlantis Press).
• AGI Preschool: A Framework for Evaluating Early-Stage Human-like AGIs by Goertzel, B. & Bugaj, S. V. Proceedings of the Second Conference on Artificial General Intelligence, Atlantis Press.
• Hybrid Systems: Generalized Solutions & Robust Stability by R. Goebel et al.

• A Brief Intro to Reinforcement Learning by Kevin Murphy.
• Cumulative Learning by K.R. Thórisson et al.
• Comparison of Machine Learners on an ABA Experiment Format of the Cart-Pole Task by Eberding et al.
• Growing Recursive Self-Improvers by B. Steunebrink et al.
• The Logic of Learning by P. Wang.
• The Emergence of Artificial Intelligence: Learning to Learn by P. Bock.
• Deep learning on Wikipedia (Sections: Intro, Overview, and Neural Networks).
• Reinforcement Learning: An Introduction by Rich Sutton and Andrew Barto (1998), First chapter. This is “the” introductory text on RL.
  • Alternative to Murphy: Reinforcement Learning in the Encyclopedia of Cognitive Science by P. Dayan and C. Watkins.

• Bounded Seed-AGI by E. Nivel et al.
• From Seed AI to Technological Singularity via Recursively Self-Improving Software by R. V. Yampolskiy.
• Creating a Child Machine: Reflections on Turing’s Proposal by R. Reddy.
• Self-Programming: Operationalizing Autonomy by E. Nivel & K.R. Thórisson.
• Von Neumann Universal Constructor on Wikipedia.
• Towards a Programming Paradigm for Control Systems With High Levels of Existential Autonomy by E. Nivel & K.R.Thórisson.
• Behavioral Self-Programming by Reasoning by P. Wang.
• Cumulative Learning by K.R. Thórisson et al.
• Growing Recursive Self-Improvers by B. Steunebrink et al.
• Self-awareness in Real-Time Cognitive Control Architectures by Sanz, R., López, I. & Hernández, C.

• The Pedagogical Pentagon by J. Bieger et al.
• Raising AI: Tutoring Matters by J. Bieger, K.R.Thórisson and D. Garrett.
• Machine teaching: An inverse problem to machine learning and an approach toward optimal education by X. Zhu
• Bringing up Turing's 'Child-Machine' by S.G.Sterrett.
• Matching Learning Style to Instructional Method: Effects on Comprehension by Rogowsky et al.
• Models of cooperative teaching and learning by S. Zilles, S. Lange, R. Holte and M. Zinkevich.
• Teaching on a Budget: Agents Advising Agents in Reinforcement Learning by L. Torrey and M. E. Taylor.
• Curriculum learning by Y. Bengio, J. Louradour, R. Collobert and J. Weston.

• On Defining Artificial Intelligence, Journal of Artificial General Intelligence 10(2):1-37 by Pei Wang.
• An integrated theory of mind by Anderson et al.
• (ACT-R) An Integrated Theory of the Mind by Anderson, J. R.; Bothell, D.; Byrne, M.D.; Douglass, S.; Lebiere, C. & Qin, Y.
• Universal Intelligence: A Definition of Machine Intelligence by Shane Legg and Marcus Hutter.
• A Mind Model for Multimodal Communicative Creatures and Humanoids by Thórisson, K. R.

• Constructionist Design Methdology paper by K.R. Thórisson.
• Subsumption Architecture on Wikipedia.
  • (supplementary) How to Build Complete Creatures Rather than Isolated Cognitive Simulators by Rodney Brooks.
  • (supplementary) A Robust Layered Control System for a Mobile Robot by R. Brooks.
• Architectural Mismatch or Why it’s hard to build systems out of existing parts by Garlan, D., R. Allen and J. Ockerbloom.
  • Also available here.
• BDI Architecture on Wikipedia.
• BDI Agents: From Theory to Practice by A.S. Rao & M.P. Georgeff.
• Whatever happened to the subsumption architecture? by Simon Birrell.

• A New Constructivist AI: From Manual Construction to Self-Constructive Systems by K.R. Thórisson
• Introduction to Software Architecture by Garlan & Shaw.
• Reductio ad Absurdum: On Oversimplification in Computer Science & its Pernicious Effect on Artificial Intelligence Research by K.R. Thórisson.
• Constructivist AI Methodology by K.R. Thórisson
• Can there be a science of complex systems? by H. A. Simon
• A Primer For Conant & Ashby's “Good-Regulator Theorem” by D.L. Scholten
• Every Good Regulator of a System Must be a Model of that System by Conant & Ashby
• Approaches & Assumptions of Self-Programming in Achieving Artificial General Intelligence by K.R. Thórisson et al.
• Essentials of General Intelligence: The Path Towards AGI by P. Voss.
• Cybernetics and Second-Order Cybernetics by Heylighen & Joslyn
• Does the Future of AGI Lie in Cognitive Synergy? by B. Goertzel

• Introduction to Control Theory
• Robert Rosen’s Anticipatory Systems Theory: The Art and Science of Thinking Ahead by J. Rosen.
• Hybrid systems on Wikipedia.
• Time, Clocks, and the Ordering of Events in a Distributed System by L. Lamport.
• Hybrid Dynamical Systems by R. Goebel et al.
• Theory of Hybrid Automata by T. A. Henzinger.
• Cybernetics and Second-Order Cybernetics by Heylighen & Joslyn.
• Control Theory by Bellman.

• A Primer For Conant & Ashby's “Good-Regulator Theorem” by Daniel L. Scholten.
• Every Good Regulator of a System Must be a Model of that System by Conant & Ashby.
• Cumulative Learning With Causal Relational Models by Thórisson & Talbot.

• What is GMI? by K. R. Thórisson.
• Seed-Programmed Autonomous General Learning (sections 1 and 2) by K.R. Thórisson

• Fridges, Elephants, and the Meaning of Autonomy and Intelligence by R. Sanz et al.
• Cognitive Architectures & Autonomy: A Comparative Review by K.R. Thórisson & H.P. Helgason.
• Syntactic Autonomy - Or Why There is no Autonomy Without Symbols and how Self-Organizing Systems Might Evolve Them by L. M. Rocha.
• The Emergence of Artificial Intelligence: Learning to Learn by P. Bock.
• A Framework for Autonomy Levels for Unmanned Systems (ALFUS) by Huang et al.

• Principles of Integrated Cognitive Control by R. Sanz et al.
• On Attention Mechanisms for AGI Architectures: A Design Proposal by Helgason et al.; accompanying video can be found here.
• A model of attention that takes global scene factors into account by Torralba.
• Towards a Programming Paradigm for Control Systems with High Levels of Existential Autonomy by E. Nivel et al.
• Predictive Heuristics for Decision-Making in Real-World Environments by H. Helgason et al.
• Towards a General Attention Mechanism for Embedded Intelligent Systems by H. P. Helgason et al.
• Attention Capabilities for AI Systems by H. P. Helgason & K. R. Thórisson.

• The Physical Symbol System Hypothesis: Status & Prospects by N. J. Nilsson.
• Minds, Brains & Programs by John Searle
• Subsymbolic Computation and the Chinese Room by D. Chalmers
• The Physics of Symbols: Bridging the Epistemic Cut by H. H. Pattee
• The 'Cybernetic Cut': Progressing from Description to Prescription in Systems Theory by D. L. Abel

• Cybernetics and Second-Order Cybernetics by Heylighen & Joslyn
• Cell psychology: An evolutionary approach to the symbol-matter problem by H. Pattee

• About Understanding by K. R. Thorisson et al.
• Evaluating Understanding by K.R. Thórisson & J. Bieger
• The ‘Explanation Hypothesis’ in General Self-Supervised Learning by K.R. Thórisson
• Do Machines Understand? A Short Review of Understanding & Common Sense in Artificial Intelligence by K.R. Thórisson & D. Kremelberg
• Understanding & Common Sense: Two Sides of the Same Coin? by K. R. Thórisson & D. Kremelberg
• Understanding is a Process by Blaha et al.

• The Logic of Learning by P. Wang.
• A New Approach for Induction: From a Non-Axiomatic Logical Point of View by Pei Wang
• Abduction & Deduction With Causal-Relational Models by K.R. Thórisson & A. Talbot
• Critical Reasoning by Marianne Talbot
• Wason's Cards: What is Wrong? by P. Wang
• Return to Term Logic by Pei Wang
• Abduction in Non-Axiomatic Logic by Pei Wang

• When is a cognitive system embodied? by Riegler
• Does a Laptop Have a Body? by P. Wang
• Social situatedness by Lindblom & Ziemke
• Towards a Programming Paradigm for Control Systems With High Levels of Existential Autonomy by E. Nivel & K. R. Thórisson

• From NARS to a thinking machine by P. Wang
• What is NARS by K. R. Thórisson
• Introduction to NARS by P. Wang
• NARS intro paper by P. Wang
• NARS demos by P. Wang

• Achieving Artificial General Intelligence Through Peewee Granularity by Thórisson, K. R. & Nivel, E.
• Self-Programming: Operationalizing Autonomy by Nivel, E. & K. R. Thórisson
• Bounded Seed-AGI by E. Nivel et al.
• Autonomous Acquisition of Natural Communication by K. R. Thórisson et al.
• Anytime Bounded Rationality by E. Nivel et al.
• Bounded Recursive Self-Improvement by E. Nivel et al.

• Made-Up Minds: A Constructivist Approach to Artificial Intelligence by G. Drescher
  • Read chapters 1 & 2 for understanding Drescher's (and Piaget's) psychological motivation; Ch. 3 & 4 for an overview of the computational framework.

• The Sigma Cognitive Architecture and System by P. S. Rosenbloom

• OPENCOG PRIME: A COGNITIVE SYNERGY BASED ARCHITECTURE FOR ARTIFICIAL GENERAL INTELLIGENCE by Ben Goertzel
• Getting Started with Open Cog by Ben Goertzel

• Franklin, S. (2007). (LIDA) A Foundational Architecture for Artificial General Intelligence. Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms. IOS Press, Amsterdam, The Netherlands, The Netherlands, pp. 36-54. PDF
• Anderson, J.R. & Schunn, C.D. (2000). Implications of the ACT-R learning theory: No magic bullets. Advances in instructional psychology. 5:1-34. Lawrence Erlbaum | PDF
• Laird, J.E.; Newell, A. & Rosenbloom, P.S. (1987). SOAR: An architecture for general intelligence. Artificial Intelligence, Volume 33, Issue 1, Pages 1-64 PDF
• Snaider, J; McCall, R. & Franklin, S. (2011). The LIDA framework as a general tool for AGI. Artificial General Intelligence, Lecture Notes in Computer Science. 2011. Volume 6830/2011. pp. 133-142 PDF
• J. Schmidhuber (2016). Learning how to Learn Learning Algorithms: Recursive Self-Improvement. - slides

• Searle's Chinese Room Argument on the Internet Encyclopedia of Philosophy
• The Chinese Room on Wikipedia

• Selected Self-Organization and the Semiotics of Evolutionary Systems by L. M. Rocha
• Weak Emergence by M. A. Bedau
• Strong and Weak Emergence by D. Chalmers
• THE SCIENCE OF SELF-ORGANIZATION AND ADAPTIVITY by Heylighen

• What's it Like to be a Bat? by T. Nagel
• Consciousness and its Place in Nature by D. J. Chalmers
• The Future of Consciousness - TEDx lecture by R. Hameroff
• Consciousness is a Mathematical Pattern - TEDx lecture by M. Tegmark
• Consciousness and the Brain - TEDx lecture by J. Searle

• Leave no dark corner by Matthew Carney.
• THE FUTURE OF EMPLOYMENT: HOW SUSCEPTIBLE ARE JOBS TO COMPUTERISATION? by Frey and Osborne

• Reinforcement Learning: An Introduction by Rich Sutton and Andrew Barto (1998) is the introductory text on RL.
• Machine Super Intelligence is Shane Legg's 2008 PhD thesis. While it is not on reinforcement learning, it does connect the concepts of RL to AI, what he calls “universal AI”.
• Reinforcement Learning is a full Udacity course on RL from Georgia Tech.
• A Short Course on Reinforcement Learning by S. Singh at MLSS'11 introduces RL and discusses some important shortcomings and proposed first steps to solving them.
• Advanced Topics: RL by D. Silver is a more in-depth, modern RL course from one of the people who worked on Google DeepMind's Atari playing system that received a lot of (media) attention.
• Model-Based Reinforcement Learning is a tutorial given by Michael Littman at NeurIPS'09 about model-based RL, which is a lot less common than model-free RL, but not less interesting.
• Resource-Bounded Machines are Motivated to be Effective, Efficient & Curious by B. Steunebrink

• A gentle introduction to neural networks by J. Le - gives a good overview of the different approaches
• Form, function, and the giant gulf between drawing a picture and understanding the world by G. Marcus

• Kognit: Intelligent Cognitive Enhancement Technology by Cognitive Models and Mixed Reality for Dementia Patients by D. Sonntag and accompanying video
• Comparison table of cognitive architectures, courtesy of BICA Society / A. Samsonovich
• Why are so many smart people such idiots about philosophy? by O. Goldhill
• Cognitive Map Architecture: Facilitation of Human-Robot Interaction in Humanoid Robots by V. Ng-Thow-Hing et al. (2009).
• Toward a unified artificial intelligence by Wang, P.
• Judea Pearl lecture on Causation on YouTube
• Creativity in a Nutshell by M. Boden.
• The Role of Cognitive Systems Engineering in the Systems Engineering Design Process by Militello, L.G., Dominguez, C.O., Lintern, G. & Klein, G. (2010). In Systems Engineering, Vol 13(3), pp. 261-273.
• A survey on transfer learning by Pan, S. J. & Yang, Q. (2011). IEEE Transactions on Knowledge and Data Engineering, 22(10), pp. 1345–1359.
• Systems, models and self-awareness: Towards architectural models of consciousness by Sanz, R., Hernandez, C., Gomez, J., Bermejo-Alonso, J., Rodriguez, M., Hernando, A. & Sanchez, G. (2009). International Journal Of Machine Consciousness, 1(2), pp.255–279.
• Requirements for Machine Lifelong Learning by Silver, D.L. & Poirier, R. (2007). IWINAC, LNCS (4527), pp.313-319.
• Physicalism, Emergence and Downward Causation by Campbell and Bickhard
• Man as Machine - J. Bach's AGI intro lecture at AGI Summer School 2009
• Mastering the game of Go with deep neural networks and tree search by Silver, Huang et al. (DeepMind)
• Why Philosophers Should Care About Computational Complexity by S. Aaronsson
• A Vision for Computer Science - the System Perspective by J. Sifakis

• Journal of Artificial General Intelligence
• Introduction to Artificial Intelligence - A Modern Approach by S. Russell & P. Norvig
• Chapter 1: An Introduction to Philosophy of Science by Malcolm Forster

• Algorithms for Reinforcement Learning by Csaba Szepesvári (2010) is a much more recent, shorter book that discusses the strengths and weaknesses of various RL algorithms. See also: Rich Sutton's FAQ.
• Deep learning on Wikipedia (Sections: Intro, Overview, and Neural Networks).

How to Use This Page

Note: DO NOT SKIP READING THE BELOW TEXT

Papers under each section are ordered from most to least important, so start counting from the top.

[ x,y ]
x: necessary mandatory number of papers to be read – absolute minimum number.
y: the recommended number.
No number: Read all the papers listed.

It is your responsibility to ensure that you grasp the concepts covered; the readings are my top choices for getting this done. However, if you are aware of alternative sources of treatment of the concepts covered in these you may prefer to read about them from your preferred source. If in doubt, ask me.

You are expected to read a lot of papers in this course, at least 3-4 papers per week (5 recommended). Keep at it and you'll be fine!

Assigned readings should be read before class.
If you do so you will already have some familiarity with the subject matter, which not only means you will remember it better but also that you can ask questions for clarification during the lecture and partially steer its direction.

Reading the papers after class is less effective.

Warning: Do not attempt to read them during class - this is absolutely the worst way to cover this material (but of course you may have it open for reference).

Reading the assigned readings not at all should generally be avoided.

As you read papers from each of the following categories I want you ask yourself a few questions:

• For each paper in each category X, ask yourself:
  • What is X?
  • How does the human mind do X?
  • Do current computers do X?
  • …and …
  • Do we need (to replicate or capture) what the human mind does to achieve X to create a machine that rivals the ability of humans to do X?

If you can answer them satisfactorily when you're done reading you're good! Even if you can't you'll be fine if you: Write down the discrepancies and bring them to class in the form of questions. There is no such thing as a 'stupid question' when you're learning something new.

EOF