Back in 2006 I wrote about Tape, an commission by Someth;ng for our Cybersonica exhibition. There is something authentic about playing in an easy way with real audio tape, rather than software and digital interfaces.
Instead of moving the tape over a play head, the tape is fixed stationary to a wall. As a participant, you wear a glove that has the playhead on your finger trip, allowing you to drag your finger over the tape to play the audio.
I really like that the shadows are used to reveal hidden worlds, augmented over the existing shadows from light. As you move the light source around, you see the shadow of trees and houses within the blocks. If there is no light near a house, a person comes out to capture the light.
“Augmented Shadow utilizes this unique interface metaphor for interactive storytelling. Maximizing the magical amusement of AR, it is embedding an ecosystem where imaginary objects and organic beings co-exist while each of them influences on each other’s life-cycle, even though it is not in use by users. Light and shadow play critical roles in this world’s functions causing chain reactions between virtual people, trees, birds, and houses as shown below”.
“Hello Haptic is a flash card kit for the blind children to learn various haptical experiences about nature.
Visually impaired children are able to self-educate themselves about different parts of nature with this learning aid. They will be properly stimulated about diverse characters of nature as well as fulfilling their curiosity through their first-hand tactile knowledge.
Created by Rhea Jeong in collaboration with Sunmin Lee, Saehee Lee, Youngsoo Hong”
“The bugs analyse the shape (topography) of the sand around them, preferring to move gently downhill. This means they can be shepherded, enclosed within walls of sand, encouraged to meet each other (at which point strange metamorphoses happen; they merge into larger caterpillars then, if you’re lucky, into butterflies. If they get frightened, they pop and disappear”.
“A surface covered with black sand turns into a pool full of life when people grab and remove a handful of sand away. In this micro-world, virtual creatures are born, live and perish.They recognize their spatial boundaries and obstacles of living and respond to peoples’ touch in various ways”.
I’ve had this post in my draft items since October 2006, now slowly finding time to write on Pixelsumo again. In August 2005 I posted ZXZX, a device by Crispin Jones that you cheat the computer at button tapping games, asking “What does it mean to cheat this unseeing opponent?”
Periborg are a series of game devices by Eiji Morikawa that also follow along a similar theme, including cheating your human opponent and playing without distraction.
1) Obacha-Break: Hang this air horn around your neck, and you can unleash a klaxon to disorient your opponents during multiplayer matches. Hardly sporting, but preferable to a humiliating defeat in King of Fighters.
2) Cocolo-Con: “Remember when you first played a game? Remember how excited you were, how you stayed up all night?” Morikawa writes on the Periborg site. “The Cocolo-Con is here to remind you of that.” It does this by, uh, tracking heart rate and administering wrist massages.
3) Ore-Commander: What do trigger-happy old-school gamers want most? Faster firing capability. This thumb-mounted vibrator helps you pound buttons 20 times a second.
4) Shock-C: Why pause the action to eat? This utensil holder slips over two fingers and can be fitted with chopsticks or a fork and spoon. Your thumb is free to work a controller, which “never gets greasy,” Morikawa says.
5) Electric-Wang Show: This unfortunately named device can display scrolling messages like “Looking for opponents” or “Bring me a Coke.” It’s a great way to communicate with other humans without looking up from your precious, precious game.
Tim Rogers did an interesting & insightful interview with Eiji for Insertcredit (and Wired), so I highly recommend you read it here.
Last week in London, Philips held an event to preview new potential products ‘that demonstrate its sense and simplicity brand positioning’. Unfortunately I couldn’t make it to the event, but below are a few products that I find interesting. Many of these follow approaches that I have seen from interaction designers and students in the past.
Drag & Draw turns the whole house into a canvas for a child to draw on. Very little technical info, but by the looks of it, the bucket contains a laser projector and tracks the ‘magic wand’ (a pen with a light on), projecting the augmented drawing back on to the walls. Includes an eraser and basic animation frame features.
Storyteller aims to bring story time to life. Whilst the parent reads the story, the child shines a torch on the wall that contains moving images relating to that part of the story.
Chameleon is a floor lamp that uses a colour sensor to change its colour to whatever object you hold up to it. Like a larger version of RGBy.
Wake-up Light is an alarm clock with a light that glows brighter nearer the time of wake-up, making you wake up naturally like a sunrise. I could do with one of these in the winter. Related -> light sleeper; bedding that lights up in the same way from loop.ph
The amBX system aims to give PC gamers a more immersive experience. This usb devices allows games to control rgb lights, air fans and a vibrating wrist rest. Video (WMV)/ Interactive Demo
“A simple stroke of the uWand allows users to intuitively point at a device and to scroll, select, play and move elements. The uWand is a simple instrument to control all content at home; from music to films and from TV channels to family photographs. Philips hopes that the uWand will stimulate product designers to rethink user interfaces to make them more simple and designed around the consumer”. This seems very much like a Nintendo Wiimote.
Trans Light – handheld sun-like light that shines energizing blue light into the face as soon as opened, helping to compensate for the lack of natural light – anytime, anywhere
Ambient Experience Catheterization Lab (Ambi Cat Lab) – designed to improve the workflow of physicians and reduce anxiety of heart patients undergoing catheterization.
Simon Elvins, a graduate from MA Communication Art & Design at the RCA, has created a number of interesting works dealing with print as an interactive interface for sound.
Notation : “This is part of an ongoing exploration into sound, print and notation, and looks at ways of linking sound to the printed page. Using a simple sounder, the project aims to directly relate tonal values of pattern and drawing, into a tonal scales of
sound and music”.
FM Radio Map : “This map plots the location of FM commercial and pirate radio stations within London.The poster works in its own right as a piece of information design, but when connected to the modified radio it becomes part of the interface. Each map is made site specific by connecting only the stations that can be received in that location. This is done by drawing power lines in pencil on the back of the map, which conducts electricity from the radio to the front of the poster.Placing a metal contact onto each point enables us to listen to the sound broadcast live from that location”.
Sound Book : “Writings on the use of sound within Art and Design. Includes interactive typography and audio CD which can be played through the book when connected to a CD player”.
Between Blinks & Buttons are two projects about the camera as a networked object. Through making their photos public on the internet, individuals create traces of themselves. In addition to their value as a memory, each image contains a multitude of information about the context of its creation. Cameras become context-recorders which create references that go well beyond taking a photo.
Through this metainformation, every image is linked to the precise moment in time when it was taken, making it possible to see what happened simultaneously in the world at that instant. This work tries to focus the user’s imagination on that other, to create narratives that run between one’s own memory and a stranger’s moment which happened to coincide in time.
Blinks is a table-top interactive, where projected photos are scattered on the surface. Placing a glass prism over a photo causes it to refract the light to the sides of the table. The really clever part, is that this light contains projections of other photos taken at exactly the same moment in other locations (the software searches Flickr using the api). The user can browse photos through fragments of time, and also upload their own photos via Bluetooth.
Buttons is camera that takes other peoples photos. Buttons has no optical input, by pressing the button you remember your moment, but it also retrieves the stored moment of others from the internet on Flickr. You then wait for a Flickr user to upload a photo taken at the same time as your moment. Created using a mobile phone, Mobile Processing and custom PHP code. Watch video. Great stuff.
[update1 - Jimmi got in touch and has posted his video on YouTube in five parts -> one, two, three, four, five]
I’ve had the pleasure and honor of seeing Toshio Iwai do the keynote lecture at Futuresonic 2006. As an inspiration for many years, having only read about his previous work and seen static images, it was fantastic to actually see videos of his older work, truly amazing.
This post is not a transcript, but some observations and details of things that really stood out.
Here is a little video (shot on broken photo camera, I need a DV cam):
Maria has also posted a video of the Tenori-On demonstration here.
Toshio began talking about his childhood and obvious early inspiration. First he showed books that were bought for him as a young child, one of insects and one about the science of light, sound and heat. At the age of 9 or 10, his mother said he would get no more toys. Instead she gave him materials (paper craft etc) and he started using his imagination to make his own toys and games. Then a video of his first animation was shown, a series of flipbook doodles in the side panel of his school notebooks, ranging from cartoon characters to planets in orbit. Then he began to get technology into his life, like the hifi, super famicom, microscope etc.
He was inspired by pre-cinema moving image. In 82 and 83 he made Phenakistiscopes, animations on a circular spinning disc, using a photo copy machine (more here). At university he made more animated flipbooks, this time printed and then stuck together. One shown was a 3d vector shape animation (XY Plotter Flipbookk) and another printed from traditional film (Video Books 1984).
In 1988 Toshio created zoetropes, but instead of using flat images to create animation, using real clay models. Like the mechanical music box, the 3D zoetrope was spun using a crank handle. At the age of just 23, he made a 3D zoetrope titled Time Stratum I. This led on to another three Time Stratum works, winning him the grand prix at the 17th Contemporary Japanese Arts Exhibition, making him the youngest winner.
Using his experience of 3D zoetropes, Iwai was asked by Studio Ghibli (famous Anime film makers) to help with zoetropes for the Ghibli Museum. A little of ‘Bouncing Totoro‘ can be seen in the above video (around 01:00). Breathtaking! This website says Toshio conceived an LED strobe system, as well as using computer to work out positions of the models. The 2nd zoetrope was titled Rising Sea Stream.
He discussed how for his moving films he could never compose music through lack of understanding traditional score. He wanted to create every part of this work, but was not from a traditional music background. A mechanical toy music box allowed him to punch holes in the paper, feed it through to create a tune, making a very visual way of creating sound.
In 1992 he created an installation called Music Insects. This was then turned into a game for Nintendo called Sound Fantasy (unfortunately it wasn’t released by Nintendo, later turned into Sim Tunes from Maxis I believe). We were lucky enough to see a video of this, an early precursor to Electroplankton. These music insects react to color dots on the screen. When they pass over them such dots, they trigger musical scales, sounds, and different light patterns. The user selects colors from a palette with a track ball, and paints in their path, and
the insects perform the colors when they pass over them. The insects’ direction can be changed with certain colors, and colors can be painted to achieve less random musical performances.
The score of the mechanical music box is something that clearly inspired the score in Piano as Image Media (1995). Players would use a trackball, drawing dots of light on to a projected score. These dots travelled up and hit the keys of a real grand piano, playing that key and the shooting into the air as projected shapes and visuals.
He then went on to describe Composition on the Table (see below) and his installation for Bloomberg, written about previously (using 800 infrared sensors).
Toshio collaborated with composer Ryuichi Sakamoto to create ‘Music Plays Images X Images Play Music‘, a series of works for live performance using the piano in 1997 (Ars Electronica Archive has described all 10). Thanks to youtube, we have some videos!
This created a really interesting relationship between Sakamoto, the visuals he created which then went into a feedback loop, playing the second piano. There was also the interaction between Sakamoto and Toshio during the performance, each playing off each other and the medium in which they were using.
Nintendo finally saw sense :) this time releasing Electroplankton, his latest sound based game. Electroplankton is really like an archive of his previous artworks. The tiny creatures reminiscent of Music Insects. Two plankton Lumiloop & Luminaria being portable game versions of his installation Composition on the Table from 1999. It’s great to see that a media artist is now taken seriously by a game publisher, creating not musical sequencer or games with rules and objectives, but a toy for creative play and exploration. Electroplankton gave Toshio room to experiment with ideas that may have not been possible in interactive installation.
The keynote ended with a demonstration of Tenori-On, a new instrument in development at Yamaha. He talked about the concept, rather than any specific technical achievements. You can read some of those details here, and I won’t bore you with an introduction to Tenori-On as you should hopefully know it by now. Nothing new to report on this, although I did see the laptop screen showing Tenori-On software. It took up a quarter of the screen, was a photo style copy of the instrument, with the grid lighting up at the same rate of play (with no apparent latency).
He wants to create a new kind of electronic musical instrument. Traditional musical instruments have been around much longer than digital media, so this is a difficult task. Although there have been many new controllers for synthesizers, he believes the Theremin is a true new instrument invention, as the movement from the performer and the sound it creates is very unique.
Tenori-On is a fantastic instrument for electronic music. Toshio has really thought about this from a performers viewpoint during development, allowing us to save performances and playback, perform with other Tenori-On and communication via Midi. Enabling the display of lights on the rear lets the audience see what the performer is doing. Many of the play modes borrow from earlier work (like Chess, Electroplankton and Music Insects), but can be layered on top of each other. No details yet as to possible release date or price, but I am certainly looking forward to it.
I think that the way that the performer holds Tenori-On is like playing a portable game system in your hands. I wanted to ask this but we ran out of time for questions.
Toshio has posted details of his trip to Manchester on this blog.
A project in a similar concept to RGB Player is spinCycle from Spencer Kiser (an NYU ITP student).
As the turntable rotates, a camera reads the fluorescent tinted plexiglass (red, yellow and blue). The camera acts as the input detection, picking up the colours and positions of the pucks, but also as the visual feedback for the audience (displayed behind the performer). The speed of the turntable can be adjusted, as well as which colours are being detected. Whilst the input remains the same, experiments in using it as a drum machine and granular synthesis have been tried by Spencer.