Sleepwalkers is an interactive installation about beings that live inside the walls of a historic building. It was commissioned by Urban Putt, an indoor miniature golf course built by artists and designers in San Francisco.
Sleepwalkers combines a host of techniques to make it seem that a three inch tall luminous being is interacting with the physical world—including the hands of participants—while illuminating its environment.
The interaction works as follows:
Participants are encouraged to put their golf ball into an entrance in the wall. After a few seconds of rattling, the ball becomes visible through an opening in the broken lathe, stuck inside the wall. The goal is to figure out how to get the ball out from behind the wall so it can be used to play the adjacent mini golf course.
A source of light seems to be moving inside, towards an opening in the bottom of the wall. Inside that opening, a small luminous being is visible, standing on piece of wood, slumped over and depleted.
Participants interact with the being by putting their arms through ceramic portholes in the wall, using their hands to first provide "energy" to the being and then to provide a physical platform for it to jump onto.
In the top opening, the being seems to push the ball, dislodging it and ejecting it from a hole in the wall.
Sleepwalkers employs a new visual technique, developed for this piece, that combines the pepper's ghost illusion and projection mapping to enable a computer generated character to seem to illuminate and interact physically with real-world objects. In each setting, the two effects are created simultaneously from a single projector. (The piece uses four projectors in total--one for each of four diorama-like settings.)
In addition the piece incorporates the following components:
- Depth cameras and custom signal processing code to detect hand shape
- C# computer code running in the Unity game engine
- A custom ball feeder mechanism (incl. actuators, sensor, and microcontroller)
- A hidden linear actuator to help the sleepwalker push the ball
- Found objects
- Custom made objects, including CNC machined and 3d printed parts
The most difficult aspect of this project was achieving the durability and reliability necessary for a (sometimes rowdy) environment, open to the public 7 days a week. A substantial portion of the total effort was spent developing systems to detect and recover from hardware and software errors.
Kathrin Feser, an enormously talented visual artist, built the interior scenes, did much of the exterior finish, and was an invaluable consultant throughout the project.
