Fishes of the Silicon Sea

But these creatures do not exist in three-dimensional space, the undersea world has no reality, and behaviors, though lifelike, are not intended to precisely mimic those of a particular species. Instead, each creature has a virtual existence, being represented by patterns of electrons racing through the silicon circuitry of a small but powerful computer.

The textured, colorful images of the active denizens of the fish world are reminiscent of the realistic, animated images of the Tyrannosaurus and Velociraptors in the film version of Michael Crichton's science fiction novel Jurassic Park. Yet as realistic looking as they may be, the dinosaurs of Jurassic Park are mere graphic puppets that require highly skilled human animators to plot their motions from one second to the next. In contrast, each fish in this community, known as ARTIFICIAL FISH, exists as an independent, self-governing computer program.

None of the action here is programmed in advance. When the program is initiated, the operator specifies only which fish are present and their initial locations. The creatures do the rest. Tu, who designed the behavior system, emphasizes that each fish is an autonomous agent, with actions driven by unique perceptions, instincts, and experiences. Each agent responds to a hierarchy of needs, and its brain considers the urgency of each situation. If there is an obstacle in the way, the fish will avoid it at all costs, as a collision could be fatal. Next come threats to survival, such as predators. If the need for food has been satisfied and no predators are in sight, it might check out another fish, perhaps a potential mate. Such autonomy is an important part of what distinguishes the fish from their Jurassic Park counterparts.

Artificial life

Development of the aquatic ecosystem embodied by ARTIFICIAL FISH marks a milestone in efforts to simulate living systems in computers. There is far more to this software than you would find in a video game.

To create fully autonomous fishes that exhibit a large repertoire of lifelike behaviors, the team has assembled_and, where necessary, invented_a large collection of innovative software techniques.

Some of this work parallels advanced research in robotics, with one important feature being the construction of individuals whose acute computer vision can sense relevant aspects of their environment [see ``Seeing-Eye Machines,'' T^he^ W^orld^ & I, June 1994, p. 189]. A second feature is the ability to rapidly process those images in an artificial brain, sift out relevant features, and set appropriate behavioral priorities. A third aspect is the development of artificial muscles and streamlined bodies that can navigate with the agility and grace of living fish. Fourth, each fish is a fully self-governing individual, capable of independent responses to the challenges of its microenvironment.

Advanced roboticists might protest that these artificial creatures have no physical reality. They have no impact on the real world and cannot be harnessed to perform useful actions. But Terzopoulos and colleagues would correctly assert that their virtual existence permits the fish to play an important and significantly different role than their hulking mechanical cousins: They can serve as a controlled reference to guide construction of a functional physical system. The ARTIFICIAL FISH world is a virtual laboratory and so has all the advantages derived from being able to simulate a complex system before attempting to build a physical model. Freed from the minutiae that often plague physical prototypes, the designers can set their sights as high as their imagination takes them.

Once a working model is constructed in silicon it becomes feasible to cast the model in metal and plastic, because the overall, holistic integration of the components has already been accomplished. In this, the focus on software in the ARTIFICIAL FISH world is part of a trend that is rapidly transforming most areas of modern technology: the use of computer simulations to design and troubleshoot hardware before it is even built.

A real-world challenge for robotics

Designing and building a robot that can move around, sense its environment, and decide how to respond has proven to be much more difficult than was envisioned some 30 years ago, when the field began to blossom. Hence commercially successful robots tend to perform a limited movement repertoire from a fixed position.

It is one thing to build a computer-driven robot that can be programmed to weld the chassis of a car. The task is repetitive and predictable, the parts are uniform, and both chassis and robot are anchored to the spot. It is quite another to build a robot that moves about and encounters novel situations in the real world.

A traditional approach to creating mobile robots has been to program them with a response to every object and circumstance they are likely to encounter. However, this approach places heavy burdens on the human programmer while creating agents that are vulnerable to the vagaries of the real world. In the United States, the NASA Ames Intelligent Mechanisms Group (IMG) has chosen to build robots whose actions are guided by remote human operators responding to a virtual display of the robot's environment. In July 1994, Dante II, a remote guided robot developed at Carnegie-Mellon University, completed a successful exploration of the Mount Spurr volcano in Alaska. A remote guided robot that combines the highly capable Russian rover Marsokhod with sophisticated software developed by the IMG is being developed for exploring the surface of Mars [see ``Robot's Gear Up to Invade Mars!'' T^he^ W^orld^ & I, February 1994, p. 196].

Rodney Brooks, a director of research in the mobile robot lab at MIT, has taken a nontraditional approach to the creation of fully autonomous robots. He has provided his mobile robots with a number of simple ``reflexes'' that are executed according to local perceptions and that interact to generate more complex behaviors. For example, one of the simple programs ensures that the robot can ``go down a corridor without hitting stuff.'' This is a radical departure from the centralized, highly complex, memory-intensive symbolic programming of the past.

Artificial intelligence

Grzeszczuk has taken another approach to simplification of complex behavior. He has imbued the denizens of ARTIFICIAL FISH with the capacity to learn from their experiences. For efficient swimming, for example, the fish learn_by trial and error_how to coordinate their muscles, much like a human baby learns to walk. This circumvents the need for a detailed instruction set to accomplish complex behaviors in diverse environments.

Terzopoulos describes a sort of ``shark training camp'' in which virtual leopard sharks learn to gradually refine their movements and ultimately swim with great efficiency. First, he says, ``You define a goal_efficient swimming motions_and specify the sets of muscles to be controlled.'' Then you need a way to select for improvements in swimming motion. The algorithms (computer instructions) for trial-and-error learning can be visualized as a sort of miniature, three-dimensional ``fitness landscape,'' with rolling hills and peaks and valleys. The valleys represent efficient swimming behavior, that is, the least effort for the most speed and distance, as represented by equations of motion. Each swimming attempt can be viewed as a marble that comes to rest at a local minimum_low effort with high mobility_on the landscape. In each learning attempt, you ``jiggle the marbles,'' says Terzopoulos: The programmer encourages the marbles to roll into lower valleys without undoing the learning that has already been accomplished.

The group showed me a video display in which the oldest, most experienced sharks were swimming with a beautiful, sinusoidal motion, while the ``newbies'' were still flailing around without getting anywhere. With the pride of paternity, Terzopoulos says, ``Whether or not this is the way that real fish learn to locomote is beside the point. The important point is that it is now possible to program a machine to learn how to move efficiently, without the need to explicitly tell it how to do so.''

The speed with which visual images can be processed is one of the chief limitations for autonomous robots. For this reason, simplification of those images has become the goal of Rabie, who has given the fish a simplified computer vision system to save on unnecessary computation. The fish have a high-resolution view of their world only at the center of their visual field, the fovea, much as we do. Peripheral vision is at lower resolution, which makes visual responses much faster. Images are processed by the brain and direct the eyes to focus where the action is_much like the real eyes of other vertebrates, such as ourselves.

Looking to the future

Will ARTIFICIAL FISH always be confined to the virtual display of a computer monitor? This would probably upset its benefactors; the work has been funded by grants from the Natural Sciences and Engineering Research Council of Canada and Precarn Associates. In Canada, as in other countries, funding agencies are pushing for concrete applications that will advance industry and stimulate the economy.

So I asked Terzopoulos, Tu, and Grzeszczuk why, as they now have a bug-free, working virtual system that is based on physical laws, it would not be possible to build a physical implementation of ARTIFICIAL FISH, with robotic individuals that could actually swim_and, in general, act like a real fish_in a real body of water? If swimming, learning, and autonomy could be realized in a physical model, this could pave the way for the development of agile, intelligent marine-transport vehicles.

In fact, the Toronto group firmly believes that autonomous underwater vehicles, or AUVs, could be built with existing technology. Terzopoulos also hastened to point out that this is precisely the goal of a research team at MIT led by Michael Triantafyllou, professor of ocean engineering. This group has succeeded in building a prototypic piscine robot, dubbed RoboTuna. The long-term goal of this project: a fully autonomous fish that ``we could throw into Boston Harbor, tell to go somewhere, and have it come back,'' according to David Barrett, a graduate student in ocean engineering who is developing RoboTuna for his doctoral thesis. The engineers hope that such a fish could be a reality in about four years.

Meanwhile, Terzopoulos' group is laying the groundwork for a project it hopes will bridge the gap between the piscine world and our own. ``People wonder why we don't focus more on human behaviors and intelligence,'' says Terzopoulos. Yet, human senses, movement options, and cognitive processes are an integrated package of capabilities that lies far beyond the range of any robotic models today.

Not being ready to tackle the full complexity of humans, Terzopoulos' group instead is moving toward what it hopes will be a viable and publicly appealing intermediate creation: a virtual mermaid. ``In developing a mermaid,'' says Terzopoulos, ``we can build on our solid experience with fish and move gradually into the much more complex realm of human behavior, facial expressions, and even motivations.

``We humans are so critical of any model that attempts to represent human characteristics that researchers have shied away from modeling humans too precisely The hybrid virtual mermaid may be just what is needed to begin to bridge the gap between advanced robotics and the human realm.''

Sidebar to Fishes of the Silicon Sea

How to Build a Fish

As an air-breathing observer, you may contact the ARTIFICIAL FISH world through the medium of a high-resolution video display. In so doing, you will find a variety of colorful fish doing what fish do naturally. These animated images, striking in their realism, are built up in the computer, starting with a three-dimensional geometric body. Natural textures are extracted from photographs and superimposed on the digital 3-D bodies. These images can then be captured from any point of view in the virtual 3-D space as the fish go about their business.

Though the computer graphics of ARTIFICIAL FISH are stunning, an even more profound beauty lies beneath the surface. Reading through the scientific description of this work that is currently in press, I was struck by the many principles of physics and biology involved in the creation of each fish. Xiaoyuan Tu, the creator of these physics-based, dynamic fish models, explained that each fish has a set of virtual muscles that allows it to navigate with the grace of a true aquatic creature, flexing its torso and controlling its fins; it responds in real time to its inner drives and to obstacles or targets in its immediate environment. The equations that describe the thrust of the fish's muscles and the opposite force exerted by the water are indistinguishable from equations that describe physical laws.

Even more impressive is the ``brain'' that Tu created for these artificial creatures. There are instincts for avoidance, escape, schooling, eating, mating, leaving, and wandering, as appropriate for both predators and prey. A hierarchical ``intention generator'' is designed so that the fish will first attend to collision avoidance and then to other priorities, such as predator detection for prey, or vice versa. These fish respond appropriately to a variety of survival needs and are able to prioritize, based on their inner drives, capacity to learn, and subjective perceptions.

These fish also have been endowed with the capacity for group behaviors, but, as in real life, these behaviors depend on individual capabilities. Species endowed with the schooling instinct form larger collections and move together when they meet. Once assembled, the school veers up or down, left or right in concert, deftly avoiding underwater obstacles as it races through the virtual 3-D world. Each fish responds instantaneously to its neighbors, and the result is a glittering dance of light. Yet there is no choreography here. There is no leader. Each fish responds only to the visual cues that it alone perceives.

At the local level, the rules that govern schooling behavior are remarkably simple, involving instincts such as ``speed up if there is only one neighbor within two body lengths to your front and turn if there are fewer than two to your side.'' Add two more rules (``swim at the standard speed and turn so you match the average orientation of your neighbors'') and the schooling algorithms, or computer instruction set, is complete.

Because each fish has also been endowed with the instinct to avoid underwater obstacles at all costs, the school smoothly bifurcates when, for example, it encounters a cylindrical object in its path. Such agile behavior would be of great value in the field of robotics: Imagine, for example, a ``herd'' of computer-controlled, driverless vehicles, or perhaps even an end to automotive accidents and gridlock on the highways. Again, the value of virtual simulations is that the principles involved in the design of such hardware can be explored within the safety of the virtual laboratory.

So fishlike are these creatures that ichthyologists are intrigued. I asked Terzopoulos if his creatures provide insights into the biological world. ``I hadn't thought so at first,'' he replied, ``but several ichthyologists who have seen the work seem to be excited by the possibility of testing theories using the models_such as strategies for optimal foraging, or the relation of form to function and locomotion. How might swimming be affected, for example, if we move a fin forward on the body?'' --G.L. ##

Additional Reading

David Freedman, Brainmakers, Simon & Schuster, New York, 1994.

Raymond Kurzweil, The Age of Intelligent Machines, MIT Press, Cambridge, Mass., 1992.

Mitchel Resnick, Turtles, Termites, and Traffic Jams, MIT Press, Cambridge, Mass., 1995.

Michael and George Triantafyllou, ``An Efficient Swimming Machine,'' Scientific American, March 1995, pp. 64-70.

About the author: Gene Levinson directs research in the Microgenetics laboratory of the Genetics and IVF Institute in Fairfax, Virginia, which offers state-of-the-art medical services in the areas of human genetics and infertility.

Two brief addenda:

• Please note that other work in robotics and alife-- especially ongoing work at M.I.T.-- is also discussed briefly in the article, as space permitted.

• I wish to thank Glenn C. Strait, Natural Science editor of World & I**, whose suggested revisions of early drafts made a huge difference in the quality of the final draft.

**World & I: a Washington Times periodical, available at bookstores and libraries; color illustrations of Artificial Fish in the June article.