FAIM Workshop on

Architectures and Evaluation for Generality, Autonomy & Progress in AI

July 13-19, 2018, Stockholm, Sweden

1st International Workshop held in conjunction with IJCAI-ECAI 2018, AAMAS 2018 and ICML 2018

Call for Papers

Architectures and Evaluation for Generality, Autonomy & Progress in AI

Call for Papers

The Joint Workshop on Architectures and Evaluation for Generality, Autonomy and Progress in AI (AEGAP) focuses on our field's original grand dream: the creation of cognitive autonomous agents with general intelligence that matches (or exceeds) that of humans. We want AI that understands its users and their values so it we can form beneficial and satisfying relationships with them.

In 2018, it is about three decades since John McCarthy published a new version of his 1971 Turing Award Lecture on “Generality in Artificial Intelligence”. Since he coined the term "Artificial Intelligence", the field has come a long way. Progress has certainly been made as AI grew from a niche science to a multi-billion dollar endeavor that solves many tasks and a household term that is often viewed to be the future of everything. However, it is not clear how much progress has been made exactly, and especially with respect to AI's grand dream.

As the task turned out to be more difficult than anticipated in the 1950s, a divide-and-conquer approach was adopted that has resulted in a very successful but fractured field. AEGAP aims to bring together researchers from different sub-disciplines to discuss how the different approaches and techniques can contribute to the goal of building beneficial AI with high levels of generality and autonomy. To achieve this goal we will likely need to build large-scale, complex and dynamic architectures that can integrate bottom-up and top-down approaches. One hopeful avenue may be to combine logic- or rule-based top-down approaches with neuroscience-inspired bottom-up approaches, so that intelligence might emerge from their interplay.

This cannot be done without methods for evaluating the different approaches to AI as they exist now and are developed in the future. While we can readily see the performance of AI systems in specific domains, it is more difficult to assess progress in AI, ML and autonomous agents when we put the focus on generality and autonomy. Real progress in this direction only takes place when a system exhibits enough autonomous flexibility to find a diversity of solutions for a range of tasks, some of which may not be known until after the system is deployed. Many evaluation platforms exist (see here), but open research questions remain about how to define batteries or curricula of tasks that capture notions such as generality, transfer or learning to learn, with gradients of difficulty that actually represent the progress we want to make in several directions. The question of fully autonomous reproducibility must also be understood as the goals become more open and general.

We welcome regular papers, short papers, demo papers about benchmarks or tools, and position papers, and encourage discussions over a broad list of topics. As AEGAP is the result of a merger between the Third Workshop on Evaluating Generality and Progress in Artificial Intelligence (EGPAI), the Second Workshop on Architectures for Generality & Autonomy (AGA) and the First Workshop on General AI Architecture of Emergence and Autonomy (AAEA), we are interested in submissions on both evaluation and architectures:


Key Information:

July 13/14/15 (TBD)
Stockholm, Sweden

Paper & Demo submission:

Due date:
April 26th
Notification date:
May 20th
Camera-ready date:
Submission system:





  • Analysis, comparisons and proposals of AI/ML benchmarks and competitions. Lessons learnt.
  • Theoretical or experimental accounts of the space of tasks, abilities and their dependencies.
  • Tasks and methods for evaluating: transfer learning, cognitive growth, development, cumulative learning, structural self-modification and self-programming.
  • Conceptualisations and definitions of generality or abstraction in AI / ML systems.
  • Unified theories for evaluating intelligence and other cognitive abilities, independently of the kind of subject (humans, animals or machines): universal psychometrics.
  • Evaluation of conversational bots, dialogue systems and personal assistants.
  • Evaluation of common sense, reasoning, understanding, causal relations.
  • Evaluation of multi-agent systems in competitive and cooperative scenarios, evaluation of teams, approaches from game theory.
  • Better understanding of the characterisation of task requirements and difficulty (energy, time, trials needed...), beyond algorithmic complexity. Item generation. Item Response Theory (IRT).
  • Evaluation of AI systems using generalised cognitive tests for humans. Computer models taking IQ tests. Psychometric AI.
  • Assessment of replicability, reproducibility and openness in AI / ML systems.
  • Evaluation methods for multiresolutional perception in AI systems and agents. Analysis of progress scenarios, AI progress forecasting, associated risks.


  • Analysis of requirements for autonomy and generality
  • Design proposals for cognitive architectures targeting generality and/or autonomy
  • Complex layered networked systems and architectures
  • Synergies between AI approaches
  • Integration of top-down and bottom-up approaches (e.g. logic-based and neural-inspired)
  • Emergence of (symbolic) logic from neural networks
  • New programming languages relevant to generality and autonomy
  • New methodologies relevant to generality and autonomy
  • New architectural principles relevant to generality and autonomy
  • Complex (e.g. layered, hierarchical or recursive) network architectures for generality and autonomy
  • New theoretical insights relevant to generality and autonomy
  • Motivation (intrinsic, extrinsic) for enabling autonomous behavior selection and learning
  • Analysis of the potential and limitations of existing approaches
  • Methods to achieve general ((super)human-like) performance
  • Methods for epigenetic development
  • Baby machines and experience-based, continuous, online learning
  • Seed-based programming and self-programming
  • Education for systems with general intelligence and high levels of autonomy
  • Understanding and comprehension
  • Reasoning and common-sense
  • Acquisition of causal models
  • Cumulative knowledge acquisition
  • Curiosity, emotion and motivation for enabling autonomous behavior and knowledge acquisition
  • Meta-planning, reflection and self-improvement
  • Principles of swarm intelligence for generality and autonomy



Dr. Oren Etzioni

Oren Etzioni is Chief Executive Officer of the Allen Institute for Artificial Intelligence. He has been a Professor at the University of Washington's Computer Science department since 1991, receiving several awards including Seattle's Geek of the Year (2013), the Robert Engelmore Memorial Award (2007), the IJCAI Distinguished Paper Award (2005), AAAI Fellow (2003), and a National Young Investigator Award (1993). He has been the founder or co-founder of several companies, including Farecast (sold to Microsoft in 2008) and Decide (sold to eBay in 2013). He has written commentary on AI for The New York Times, Nature, Wired, and the MIT Technology Review. He helped to pioneer meta-search (1994), online comparison shopping (1996), machine reading (2006), and Open Information Extraction (2007). He has authored over 100 technical papers that have garnered over 1,800 highly influential citations on Semantic Scholar. He received his Ph.D. from Carnegie Mellon University in 1991 and his B.A. from Harvard in 1986.

Talk: TBA

Dr. Joelle Pineau

Joelle Pineau is the head of the Facebook AI Research lab in Montreal, Canada. She is also an Associate Professor and William Dawson Scholar at McGill University where she co-directs the Reasoning and Learning Lab. Her research focuses on developing new models and algorithms for planning and learning in complex partially-observable domains. She also works on applying these algorithms to complex problems in robotics, healthcare, games and conversational agents. She serves on the editorial board of the Journal of Artificial Intelligence Research and the Journal of Machine Learning Research and am currently President-Elect of the International Machine Learning Society. Finally, he is also a Senior Fellow of the Canadian Institute for Advanced Research and in 2016 was named a member of the College of New Scholars, Artists and Scientists by the Royal Society of Canada.

Talk: TBA



The program will consist of invited talks, contributed talks, group discussions and a poster & demonstration session. The program is currently highly tentative as we are awaiting the FAIM Workshop Committee's decision on being allowed a two-day workshop slot.


General AI Architecture of Emergence & Autonomy

Dr. Satoshi Kurihara (contact)

University of Electro-Communications, Japan

Dr. Kenji Doya

Okinawa Institute for Science and Technology, Japan

Dr. Itsuki Noda

National Institute of Advanced Industrial Science and Technology, Japan

Dr. Hiroaki Wagatsuma

Kyushu Institute of Technology, Japan

Dr. Tadahiro Taniguchi

Ritsumeikan University, Japan

Dr. Hiroshi Yamakawa

University of Tokyo & Dwango AI Lab, Japan

Architectures for Generality & Autonomy

Dr. Kristinn R. Thórisson (contact)

Reykjavik University & Icelandic Institute for Intelligent Machines, Iceland

Dr. Pei Wang

Temple University, U.S.

Dr. Claes Strannegård

Chalmers University of Technology & University of Gothenburg, Sweden

Dr. Antonio Chella

University of Palermo, Italy

Dr. Lola Cañamero

University of Hertfordshire, U.K.

Jordi Bieger

Delft University of Technology, The Netherlands &
Reykjavik University, Iceland

Evaluation of Generality & Progress in AI

Dr. José Hernández-Orallo (contact)

Polytechnic University of Valencia, Spain

Dr. Seán Ó hÉigeartaigh

Centre for the Study of Existential Risk, U.K.

Dr. Nader Chmait

Victoria University, Australia

Dr. Fernando Martínez-Plumed

Polytechnic University of Valencia, Spain

Dr. Shahar Avin

Centre for the Study of Existential Risk, U.K.

Program Committee

Eizo Akiyama Tsukuba University, Japan
Joscha Bach Harvard University, U.S.
Marco Baroni Facebook AI Research, U.S.
Tarek Richard Besold City University of London, U.K.
Jordi Bieger Delft University of Technology, The Netherlands & Reykjavik University, Iceland
Selmer Bringsjord Rensselaer Polytechnic Institute, U.S.
Miles Brundage University of Oxford, U.K.
Lola Cañamero University of Hertfordshire, U.K.
Antonio Chella University of Palermo, Italy
Virginia Dignum Delft University of Technology, The Netherlands
Haris Dindo Yewno & University of Palermo, Italy
David Dowe Monash University, Australia
Kenji Doya Okinawa Institute of Science and Technology, Japan
Emmanuel Dupoux EHESS, France
Jan Feyereisl AI Roadmap Institute & GoodAI, Czech Republic
Patrick Hammer Temple University, U.S.
Helgi P. Helgason Activity Stream, Iceland
Bernhard Hengst University of New South Wales, Australia
Sean Holden University of Cambride, U.K.
Hidenori Kawamura Hokkaido University, Japan
David Kremelberg Icelandic Institute for Intelligent Machines, Iceland
Satoshi Kurihara Keio University, Japan
Othalia Larue Wright State University, U.S.
Ramon Lopez de Mantaras AI Research Institute or the Spanish National Research Council, Spain
Richard Mallah Future of Life Institute, U.S.
Tomas Mikolov Facebook AI Research, U.S.
Itsuki Noda National Institute of Advanced Industrial Science and Technology, Japan
Frans A. Oliehoek University Of Liverpool, U.K.
Satoshi Ono Kagoshima University, Japan
Laurent Orseau DeepMind, U.K.
Ricardo B.C. Prudencio Federal University of Pernambuco, Brazil
Gavin Rens University of Cape Town, South Africa
Hiroyuki Sato University of Electro-Communications, Japan
Ute Schmid Universität Bamberg, Germany
Murray Shanahan Imperial College London & DeepMind, U.K.
Carles Sierra IIIA-CSIC, Spain & UT Sydney, Australia
Jim Spohrer IBM Research, U.S.
Bas Steunebrink NNAISENSE, Switzerland
Claes Strannegård Chalmers University of Technology & University of Gothenburg, Sweden
Reiji Suzuki Nagoya University, Japan
Tadahiro Taniguchi Ritsumeikan University, Japan
Kristinn R. Thórisson Reykjavik University & Icelandic Institute for Intelligent Machines, Iceland
Hiroaki Wagatsuma Kyushu Institute of Technology, Japan
Pei Wang Temple University, U.S.
Hiroshi Yamakawa Dwango Artificial Intelligence Laboratory, Japan
Masahito Yamamoto Hokkaido University, Japan


Papers should be between 2 and 12 pages (excluding references) and describe the authors' original work in full (no extended abstracts). Formatting Guidelines, LaTeX Styles and MS Word Template can be downloaded from here. Papers will be subjected to peer-review and can be accepted for oral presentation and/or poster presentation. For papers that have previously been submitted to IJCAI and rejected, we ask authors to append the reviews and their responses to aid our review process.

Proposals for Demonstrations should be accompanied with a 2-page description for inclusion in the workshop's pre-proceedings. Examples include, but are not limited to: (interactively) demonstrating new tests or benchmarks, or the performance of a robot, (cognitive) architecture or design methodology.

Oral presentations should be given by one of the authors during one of the Contributed Talks Sessions. Posters and demonstrations will be presented during the Demonstrations and Posters Session.

Submission should be made before the deadline on April 26th on EasyChair using this link.

Accepted papers will be gathered into a volume of pre-proceedings and published on this website before the workshop. We are looking into the possibility of producing a special issue for an archival journal.