From Reactive to Deliberative Architectures

Extending Reactivity

• First, why do we need to extend it in the first place?
• Claim: every robot behavior can be implemented solely in reactive behaviors--do you believe this?
• Because we want to improve the performance of the agent (i.e., add new capacities), make it "smarter"
• In what directions can we/do we want to extend reactive behaviors?
• Remember: "avoid-past scheme", to keep track of past locations without explicitly keeping track of environmental states
• Analysis: what exactly did we keep track off?
• Hence, one possible extension: explicitly allow for "internal states" to keep track of environmental states-->memory!
• Distinguish: short term vs. long term memory
• More generally: allow for "internal states" to "stand in" for something else-->representation!
• Careful: this is a very complex notion, lots of dispute in philosophy alone about it (and different disciplines use the term "representation" in very different ways)
• Also note: state /= representation

Representations

• Why representations?
• E.g., because they allow us to acquire knowledge and use reasoning methods (e.g., to predict future developments or infer facts that cannot be observed)
• What are other reasons?
• What is required for representation?
• Systematic correlation of a virtual machine state with another state (either in the environment or within the agent)
• Why not "state in the architecture"?
• Note: maintaining such a correlation is not trivial--what is involved?
• What to represent?
• "Location" (of the agent or of a landmark in the environment)
• E.g., use "triangulation" from known landmarks to locate yourself or other objects
• Prerequisite to making maps
• Evidence from biology that lots of animals are capable of localization (using triangulation
• Also: animals use "cognitive maps"
• Distinguish: agent-centric coordinate system - world coordinate system
• In general: want to represent knowledge (about the world)--why?
• Remember the different kinds of "knowledge" we distinguished earlier?
• implicit vs. explicit
• positive vs. negative
• declarative vs. procedural
• innate vs. acquired
• direct vs. indirect

Maps

• Why are maps advantageous?
• Short term maps
• Improve perceptions (e.g., save recent sensory readings and use them to constrain what could be "out there")
• Use for navigation or manipulations of objects
• Behavioral memory: keep track of sensor readings (in reasonably "stable" environments), typically using "grid representations" of the space around the agent
• Distinguish different kinds of grid representations:
  • regular
  • sector
  • quadtree
• Long term maps
• Represent "knowledge" of environment
• Different encoding methods:
• metric
• qualitative

Summary

• Main extensions:
• short and long-term memory components (e.g., maps containing landmarks)
• representational capacities (e.g., of states in the environment, or states of the agents itself such as "goals")
• why can it be advantageous to represent one's own goals?
• methods that operate on representations
• planning components (e.g., AuRA, Atlantis, Planner-Reactor, etc.)
• this is what most BB architectures have focussed on--why?
• reasoning components (e.g., BDI architectures, PRS, etc.)
• reflective components (e.g., probabilistic modal logics plus belief nets, Koller et al.)
• Hybrid architectures (see section 6.6 in the Arkin book for a nice overview)
• typically: reactive + deliberative
• better: reactive + non-reactive
• biological evidence: contention scheduling (Norman and Shallice, 1980, Cooper and Shallice, 2000)
• often: layered ("layers of competence")
• layers may model time frames
• immediate: reactive
• short-term: action sequences
• long-term: deliberation (e.g., with explicit goal representations)

Case Study: Contention Scheduling

• Three layers:
• supervisory system
• schema layer
• schema network
• schemas
• goals
• object network
• resource network
• low-level motor actions
• Influences:
• top-down excitation
• lateral inhibition
• self-influence
• external influence
• random
• Intended to model human action sequencing ("coffee making example")