Most of the work ingeniously combines low and hi-tech to devise products and environments that allow 'children of all ages' to learn, create, and share in playful and collaborative ways. All very inspiring, very impressive and most importantly very, very FUN! One such project produced by David and collaborators Edwina Portocarrero, Michelle Chung and Sean Follmer, is The Never-Ending Drawing Machine whose application is built in TouchDesigner which provides the framework for the integration of the machine's various subsystems.
The Never-Ending Drawing Machine (NEDM)
NEDMs are collaborative creativity stations that enable users to co-create and edit each other's tangible, paper sketchbooks in real-time. For each page, the system loads the appropriate background content and lets you take a picture and send it back and forth to your friend or collaborator using an identical table somewhere else on the network (co-located or remote). Your collaborator also has an enhanced sketchbook and if it's on the same page each table will see everyone's latest additions.
Participants don't have to be on the same page. The sketchbooks allow non-linear, asynchronous access to the evolving, co-created content with a physical editing interface. It's a cool analog/digital hybrid model that requires no expertise and is fun just to use. Sound may also be recorded - on a blank page for example, and sent to inspire someone else's drawings. The project is very wide open and left up to individual interpretation.
Never-Ending Drawing Machine was chosen as one of the six projects among 200+ to be shown live during MIT Media Lab's Research Update televised show.
How it Works
TouchDesigner takes in live data from a camera, tracking software, TCP/IP messages, internet files and buttons connected via Arduino. It implements all the logic of the page-switching, communication to the other table, image retrieval, image snapshots, compositing, corner-pinning, perspective correction, time delays, animation sequence playback and audio playback. This is an all in one TouchDesigner “network” where you can see all your data and images flowing through the “operator” nodes in real-time. This makes for excellent troubleshooting and facilitates experimentation and what-if scenarios.
The Arduino physical computing platform is programmed separately using the Arduino/Processing IDE. The simple program uploaded into the ATMEL ATMega328 AVR chip on the Arduino was configured to listen for physical button presses and communicate their activity via serial protocol to TouchDesigner. In turn, TouchDesigner takes the raw button signals and conditions them by incorporating simple logic to make the button presses meaningful and routes the signal to the appropriate subsystem (i.e. Red button takes a picture of the creation station's desk).
Live video input from the webcam is run through a simple homography process, extracting corner-pin coordinates and re-projecting them to fit the sketch book on the table. A secondary camera uses the open-source ReacTIVision system to detect “fiducial markers” placed on each book page spread. The fiducial marker IDs are sent over TUIO as OSC packets which are read by TouchDesigner. The final resulting video projection sent from TouchDesigner is a composite image formed by the pre-made, background content (unique per book page spread) as well as the latest camera snapshots from both (or all participating) creation stations.
The creation stations are networked to each other and access each other's file systems to read images and sounds. Additionally, metadata is sent over the network to inform each participating creation station of each other's current page.
Finally, video can be optionally broadcast over the network to enable a viewing station to watch the content evolving organically. All of the applications are built in a scalable, modular structure to support the expansion of the project based on site-specific needs.
The visual programming environment: TouchDesigner is available free of cost for non-commercial use and can be downloaded at derivative.ca.
If anyone wants to contact David Robert about this project or other ideas, he'd be happy to share info. David's Derivative forum user name is 'seed' and his email address is lifeform@mit.edu
Thanks David!
In the Press
Since the writing of this article the NEDM has been published in BoingBoing and Fast Company. In the latter publication Kit Keaton describes:
"The NEDM comes partly from David Robert, formerly of Side Effects Software, and Derivative.ca's TouchDesigner system. This is a hand-built device powered by Webcams, network connections, Web sources, video projectors, and manual controls all synced up through some Arduino circuitry. TouchDesigner is something like Microsoft's Surface, something like the G-Speak Minority Report user interface, and something totally new, and it's at the core of the NEDM.
Remember that we're rapidly getting used to tablet PCs, that kid-friendly or educational apps are big news on the iPhone and iPad, and that ever-faster broadband is penetrating across the nation, this sort of technology--while it's still a prototype--is a definite sign for the future direction of education."
Woah.. couldn't have said it better!
