COGS 211
Taught by: Ken Livingston
The Course
What is your course “Perception and Action” about?
One of the key themes of Cognitive Science 211, Perception and Action is that much of what makes us smart is embodied in the way we keep track of and respond to events in the environment. The capacities for abstract thought and complex reasoning are certainly key to what is distinctive about being human, but those capacities are late developments in evolutionary history. They would be useless without the ability to track the state of the world, and to use that information to guide movement. A second key theme of the course is that neither systems for perceiving nor those for acting can be understood apart from one another. The ways in which we see, hear, touch, and so forth are tightly coupled with the ways in which we move. In this class, we begin by exploring very simple organisms, and very simple machines (robots) that sense and act without elaborate mental processing. As the term progresses, the agents we study become increasingly complex, until we are focused on human beings and on cutting edge robots by the end of the term. The course is designed to give students an appreciation for the power of multiple methodological perspectives in grappling with complex problems about mind and behavior, so we read from literatures in philosophy, experimental psychology, neuroscience, biomechanics, and robotics throughout the term.
The Technology
How does this particular software tool enhance your teaching goals?
Beginning over a decade ago I began to introduce hands-on robotics in addition to readings and lectures about robotics to make some of the themes and issues more immediate for students. The robots and the ways in which we use them have become more and more sophisticated as the years have gone by. In 1998, Chris Welty, formerly of the Computer Science Department, and I obtained an NSF grant to build a computer interface to allow students to construct programs for robots using simple point-and-click screens, without having to know a programming language. This allowed us to do a number of interesting experiments in robot-ethology. Students study the behavior of a robot and attempt to infer the logic of its control structure from observation and simple experimentation. They can test these hypotheses using the programming interface. The code they generate in this way then becomes the focus for efforts to teach the programming language itself. Instead of starting with the classic task of getting a program to print “Hello, world,” we start from the top down by systematically examining code that they have seen in action in a real robot. Over half of the lab sections in the course are devoted to learning to program well enough to build a new behavior for the robot. In order to do that, students have to learn to think in very precise ways about how sensors and motors get coordinated in the real world, a process that teaches general principles that are applicable to biological P&A systems as well. The result of introducing these technologies has been a much deeper engagement in both the theoretical and the technical problems associated with understanding perception and action in particular, and intelligence more generally.
The Student Response
How have your students responded to your use of technology?
Early on in particular I think the response was very mixed. Technophiles thought it was great, and technophobes saw it as a reason to delay taking the course, or to avoid it all together. Adding the top-down approach to programming has definitely made the technology more accessible, and more immediately interesting. There is still a wide variety of technical backgrounds among students in the course, and the robots are more intimidating for some than for others, but I’ve found ways of adjusting expectations accordingly, and for many students the robot labs become a high point of the semester. One indication of the success of this approach is the number of students from the course who have gone on to participate in our annual robotics competition, including students who had done no programming at all prior to working with the robots.
The Challenges
What are the challenges you faced teaching this class for the very first time?
The greatest challenges are (1) staying familiar enough with what’s happening across such a broad range of contributing disciplines to do at least an adequate job of representing new discoveries and controversies, and (2) keeping all the robots working! Software upgrades on computers that have to talk to robots may leave the robots behind, electronics fail and don’t tell you why or even how, wheels fall off, new light bulbs suddenly confuse sensors… the list of odd things that can go wrong seems always to have new entries. At least some of the time these failures themselves teach interesting lessons about perception-action systems, but the fact that we use technologies that are not available off the shelf presents special challenges.
New Directions
Are there any new directions you would like to explore next Spring, other new things you would like to try out?
I’m looking for ways to add technologies for studying P&A processes in humans using those of us involved in the course as our own subjects. We already do labs on psychophysics (how are the physical properties of an event in the world related to the experience of those events) and skill learning, and this year I hope to add a lab using Fresnel-lens glasses to change visual input so that we can explore short-term adaptation to a change in sensor inputs, but there are a number of other interesting ideas to explore. For example, our work on skill learning would be greatly enhanced by technologies for getting pre- and post-learning brain scans, and there are very sophisticated but expensive technologies for studying movement that would be ideal for showing just what it is that changes in a movement pattern as it becomes more skilled. The new Interdisciplinary Robotics Research Laboratory also promises to add some new capabilities. This is an enormously fertile area for the use of technology in teaching, and will continue to be so for the foreseeable future.