Center for Analysis and Design of Intelligent Agents

This is an old revision of the document!

A SHORT CRASH COURSE IN ANNs

What are ANNs?	A special way to create classification functions over large amounts of data without explicitly specifying the mathematical operations. Instead, a largely automated process called 'training' (based on an algorithm called “back propagation” or “backprop”) for achieving their production. The data sets can be for example images and text.
History	The name comes from their inspiration from natural neural networks like those in the central nervous system in many animals. ANNs are certainly “nets” but they are not really “neural” in any meaningful sense, as they don't try to mimic what neurons in nature do, except in a cartoonish way.
Current State	Very large ANNs of a particular kind, called Large Language Models and Deep Neural Networks, are used in a variety of industries to achieve the key property mentioned above, namely, to produce a (feed-forward) function based on large amounts of data, for a variety of classification tasks.

Input Data	Continuous and discrete variables.
Output Data	Continuous and discrete variables.
Max. # I/O Vars.	Very high (possibly limited by CPU and BW only).
Min. Training Cycles	Typical numbers 4k-10k. Depends on data complexity and number of layers in the ANN.
Training	Off-task. Learning turned off when fully trained.
Training Style	Training phase BILL (before it leaves the lab); discrete training steps.
Training Signal	Supervised learning: Explicit “error signal propagation” after every turn, generated from pre-categorized examples and outcomes. Unsupervised: explicit “error signal propagation” after every turn, auto-generated.
Hyper-Parameters	- Learning Rate, Exploration/Exploitation, and many others
Strengths	Handles complex data sets.
Scalability	Unpredictable behavior under data drift AILL (after it leaves the lab). Must be trained BILL (unpredictable learning AILL).