Simple Agents and their Behavior

Sensor-motor couplings

• A simple, but often very effective way of controlling agents is to use direct "sensor-motor" couplings (i.e., direct connections between sensors and motors)
• simplest kind: one-sensor, one motor (e.g., Braitenberg vehicle 1)
• what can we do with it, i.e., what kinds of behaviors can we implement?
• what could we do with two motors?
• better: multiple sensors
• still: sensors can be independent (e.g., Braitenberg vehicle 2)
• what is the space of possible behaviors?
• Issue: to what extent does the environment matter for your answer?
• once sensors are dependent on each other, we get the problem of "behavior integration" (i.e., behavior arbitration and action-selection)
• depending on the arbitration mechanisms, there are different options:
• Braitenberg vehicle 3c uses a "cooperative method" (e.g., addition of sensory inputs)
• "dominance" of one sensory input over another (e.g., proximity completely dominates light, again in Braitenberg vehicle 3c)
• increase complexity of possible actions by allowing for complex functional dependencies between sensors and motors (e.g., non-linear functions, thresholds, etc.)
• Issue: mechanism vs. behavior
• distinction makes as difference in either direction:
• what kinds of behaviors a given mechanisms can bring about
• what kinds of mechanisms a given behavior requires (i.e., what can we conclude must be true of an architecture if we know that it can give rise to a certain behavior)
• in particular, Braitenberg's vehicles demonstrate that it is possible to get very complicated behaviors out of simple mechanisms!
• CAREFUL: do not be tempted to hypothesize complicated architectural structures automatically only because the observed behavior seems complex:
• sometimes the complex behavior is a result of the interplay of architectural mechanisms and environmental influences
• Problem: how much of the environment do we have to include in our analysis and consequently in our design considerations?

Architectures again

• Start with Braitenberg vehicles and lay out the architectures:
• E.g., vehicle 2--what does the architecture look like?
• Consider various options for control components:
• "direct wire" connections
• modulating elements on wires (e.g., threshold, inverters, delays)
• combination elements for wires (e.g., summation, logical combination, max-min, priority, amplifiers, etc.)
• feedback loops

Simple Architectures and with Simple Components

• Distinguish architectures with
• homogeneous components (e.g., neural networks, schema-based, subsumption, etc.)
• heterogeneous components (e.g., SOAR, hybrid architectures, etc.)
• Note: often the term "architecture" is used in the sense of "design methodology" or "architecture schema" (it is important to distinguish these three senses!)

Neurons and Connectionist Units

• Basic structure:
• cell body
• dendrites
• axon (axon hillock)
• synapses
• Basic mode of operation:
• all incoming inputs (potentials) are summed
• if threshold potential at axon hillock is exceeded, an action potential is created that travels down the axon
• after a short refractory period (during which it cannot fire), the neuron can fire again (determines maximum firing rate)
• Artificial neurons:
• abstract over various aspects of real neurons (e.g., all the low-level chemical processes)
• model the real neuron as a computational summation unit:
• take the weighted sum of inputs netinput
• apply activation function act
• apply output function out
• mathematically:
• the activation(t+1)=act(Sigma(input_i(t)* weight_i(t))) for all inputs i
• the output(t+1)=out(activation(t+1))
• Different options for activation and output function (identity, linear, constant, step/threshold, sigmoid, etc.)

Simple Connectionist Networks and Architectures

• Connecting connectionist units yields connectionist networks (that may or may not resemble to varying degrees natural neural nets)
• Distinguish different classes of networks depending on their connectivity (network architectures):
• layered
• strictly feedforward (only one layer to the next)
• feedforward (possibly skipping layers)
• recurrent
• non-layered
• Often:
• input layer/units
• output layer/units
• all layers in-between are hidden layers/units
• Implementation detail: think of a network of n units as being defined by a n x n  matrix
• non-existent connections from unit j to unit  i are represented by weight_ij= 0
• Some advantage of connectionist networks:
• linear algebra applies, hence we have a well-worked out theory for (some of) them
• algorithms for adjustment of connection weights exist to "train" networks (i.e., to find a weight matrix that makes the network exhibit the desired behavior)
• one possibility: learning associations
• i.e., increasing the connection weight between two units that tend to be active at the same time
• use "Hebbian learning" do adjust weights: w_ij(t+1)-w_ij(t) = eta * o_j(t) * a_j(t)
• Issue: how to connect the network to sensors and motors in the agent