All posts filed under “Games


Sorry I am a bit slow posting this one, but I am really behind on my to-blog list.


British TV station Channel 4 has never shied away from being controversial, and thankfully they are putting money and energy into video game development too.

Enter the recently announced Privates, a game by Zombie Cow Studios for Channel 4, aiming to educate teenagers on safe sex…. “a funky little game about tiny little condom-hatted marines going right up peoples’ rude areas and shooting all the nasty chompy things that tend to live there if you’ve been carelessly putting bits of yourself in silly places”.


“Where pregnant, waddling teenagers take up the full width of the pavement with their oversized triplet pushchairs, unaware that their rampant, perpetual humping has filled them to the brim with all manner of grotty infections.

Privates is a platform twin-stick shooter in which you lead a teeny-tiny gang of condom-hatted marines as they delve into peoples’ vaginas and bottoms and blast away at all manner of oozy, shouty monsters. It’s rude, funny, bitingly satirical and technically pretty accurate if you don’t count the tiny people or the germs with teeth.”


Will it stop teenagers having sex, or educate them on STDs? Who knows. But you could argue that whilst they are busy playing the game, they at least aren’t spending their time having sex ;) The style of game is definitely more suited towards boys. Sort of Earth Worm Jim meets Inner Space. Maybe they will remember the smelly monsters later.

I’m glad this is being made and look forward to having a play on it.

Watch this video, making sure your sound is on, but not too loud :)


The Artvertiser

The Artvertiser
I am a big fan of the works Berlin based Julian Oliver (see Fijuu2, levelHead & q3apd) so I am very pleased to see his latest work, The Artvertiser, come to life at the recent Transmediale.

“The Artvertiser is an urban, hand-held Improved Reality project exploring on-site substitution of advertising content for the purposes of exhibiting art.”

The project was initiated by Julian Oliver in February 2008 and is being developed in collaboration with Clara Boj, Diego Diaz (hardware) and Damian Stewart (software).

“The Artvertiser considers Puerta del Sol Madrid, Times Square New York, Shibuya Tokyo and other sites dense with advertisements as potential exhibition space. An instrument of conversion and reclamation, The Artvertiser takes imagery seen by millions and re-purposes it as a surface for the presentation of art”

“The Artvertiser software is trained to recognise individual advertisements, each of which become a virtual ‘canvas’ on which an artist can exhibit images or video when viewed through the hand-held device. After training, where ever the advertisement appears, the chosen art will appear instead when viewed live through the hand-held device. It doesn’t matter whether the advertisement is on a building, in a magazine or on the side of a vehicle.”

Watch video

Its a fantastic use of Augmented Reality. Of course this software could work on netbooks, smart phones / iPhones etc, but what interests me most is the custom hardware device (shown below)…

Billboard Intercept Prototype
“A set of urban and weather-proof digital binoculars have been built. This device guarantees high-quality immersive advertisement substitution and is be more performant for AR applications than any hand-held device currently available; equipped with a high-quality wide-angle lens, fast CPU and GPU, powerful wireless adapter, long battery life and plenty of solid state storage space.”

The Artvertiser

I would much rather have a portable device like that than view it on a phone. The styling is great too, retro games machine feel (like these).

The Artvertiser

Inside Project Natal

Project Natal
(image Popular Science)

Previously on Pixelsumo I posted a closer look at Project Natal, from hardware status & origins. As the post suggests, the key to the technical success is bringing complex algorithms that estimate body joint positions to a mass consumer level. Getting this right in every home & lighting condition is no easy task.

The processing of the depth image was going to be on a chip in the camera, but it’s now been reported that the processing will be done in software, to keep the costs of the hardware down.

Recently lots more behind the scenes videos and press revealed insights into how the body processing works.

Project Natal
This video shows some of the body estimation processing in the making.

Originally it was thought that Natal might use time of flight (like the 3DV ZCam) to measure the time it takes for infrared light to bounce from objects.

“Frances MacDougall, chief technology officer at GestureTek, said his company was also working on Project Natal. Asked why there were so many vendors on Natal, he said that Microsoft will be using a low-cost 3-D camera from PrimeSense. But it purchased 3DV because it had a strong patent portfolio. And GestureTek itself is providing a software layer that helps interpret the data coming in from the 3-D camera and makes it useful for the game machine”. (source Venture Beat)

Instead of time of flight, the PrimeSense camera projects a pattern (like a barcode) of infrared light, with the sensor reading back this pattern and computing the depth map (which they are calling Light Coding).

This video (linked above) shows how impressive the tracking is, calculating accurately body parts from a side angle, and when joints are occluded out of view.

Project Natal
Popular Science has a great article about how Microsoft are training the brain behind the pose estimation & how the algorithms work. I really hope to get my hands on this one day…

“Step 2: Then the brain guesses which parts of your body are which. It does this based on all of its experience with body poses—the experience described above. Depending on how similar your pose is to things it’s seen before, Natal can be more or less confident of its guesses. In the color-coded person above [bottom center], the darkness, lightness, and size of different squares represent how certain Natal is that it knows what body-part that area belongs to. (For example, the three large red squares indicate that it’s highly probable that those parts are “left shoulder,” “left elbow” and “left knee”; as the pixels become smaller and muddier in color, such as the grayish pixels around the hands, that’s an indication that Natal is hedging its bets and isn’t very sure of its identity.)”

Augmented Reality Roundup

I’ve never really written very much about augmented reality on Pixelsumo, just an alternative reality. The other week I was talking at an Augmented Planet event, where various demos were shown & discussed.

I am slightly cynical towards AR because many things I have seen don’t work, so I decided to post some projects that I actually do like.

For me, bad uses of augmented reality are where it is used just as a gimmick. Showing a 3D model on a marker? I mean common, the ARToolKit was developed nearly 10 years ago! The experience has to be meaningful, with the technology being part of the context in which it is used, and enhance an experience, not making it harder. If you are aware of the technology as the main focus, then it hasn’t worked.

Watch this parody video
Me too (doing some AR stuff)

Or how bad is this? :)
Transformers Augmented Reality

A lot of marker based stuff doesn’t work, due to lighting conditions, obscuring the marker or just being too slow. The problem is that we get used to these flaws as part of the experience of using AR, so we just accept them.

To be honest, I’m not keen on the concepts of augmented magazines, corporate brochures, band videos & business cards. What happens when the novelty of AR wears off?

So what do I like?

Le monde des montagnes
Written about previously already. Lovely.

levelHead by Julian Oliver is a great spatial memory game. Players use a plastic cube, each side covered in an AR marker, acting as a window into a virtual room, whilst tilting the cube to make the character walk. I like this because the display cube is used as a tangible link into the virtual world, and the markers themselves help act as visual helpers to remembering which rooms belong to the cube sides.

Augmented Reality Magic
Marco Tempest - Augmented Reality Magic
Augmented illusions from the ‘virtual magician’ Marco Tempest, watch video. Whilst in contemporary magic you have no idea how a trick is done, here its easy to say ‘oh its done in the computer’. However there are traditional card tricks linked into some AR graphics here. The system recognises the cards (rather than needing AR markers), with the software for this magic being created by Zach Lieberman & Theo Watson. Pictures of the software in action here.

Also check out the Magic Projection (created by Zach for Marco).

In EyePet, Sony have brought AR to the masses in a game, much more accessible than any other attempt at AR for homes before it. More of an advanced toy than fully fledged game, you control your virtual pet via the PSEye camera as the interface. As well as motion vectors from your hands movement, they have used ‘magic card’ as AR markers to let you interact with your pet (shown here). Most impressive is the technology to turn your drawings into 3d objects that the pet can ride & play with (shown above). Watch this video, and this one. I suspect when the new PS3 motion controller is released this spring will bring a host of improvements to EyePet.

Lego Digital Box
Lego Digital Box
When you take a child to buy a box of Lego, the box doesn’t just show the pieces that are inside in a random pile, it shows that these pieces can (or should) be made into. Watch video. So its a very logical idea to let children see what their creations could look like when finished & also animate them. Why is it good? In store you get to instantly see how the Lego pieces animate, and the technology doesn’t rely on large ugly AR markers. Created by Metaio for Lego (via Notcot)

Augmented Reality Pool
“The RCVLab at Queen’s University demonstrates Deep Green, a pool playing robot, and ARPool, an augmented reality system for teaching the science of pool”. Watch video. Sure it works as a training aid, but also opens up ideas for creating new rules & games on a pool table. Although, this Obscura Digital product might be too distracting?

Total Immersion
Total Immersion
French company Total Immersion are one of the leaders in this field for years. They have done too much to mention (3d face mapping, markerless space tracking, pattern recognition), but check out their videos here. Also check out the videos of team member Emmanuel here for tonnes of great behind the scenes & experiments.

AR on iphone
Yes there have been a few apps that overlay arrows pointing at tube stations, and demos of using the AR toolkit, but the marker-less space tracking is really going to be the key to success. Apple, please allow developers to use the camera in your sdk!

The Active Vision Group of the Oxford University Engineering Department created PTAM (Parallel Tracking and Mapping) by Georg Klein & David Murray, watch this awesome video (from 2007). Now supports multiple cameras & multiple maps (with source code), see PTAMM. As you can see from this video, they have PTAM running on iPhone.

I suspect that other companies, such as Total Immersion & Metaio have iPhone technology ready to go when Apple buck up their ideas. Why is this important? Well the popularity of the device could see interesting AR apps and games appearing that really push the boundaries of outdoor AR and pervasive gaming.

Augmented Reality Toys
Augmented Reality Toys
For his final project of Interaction Design Master degree, Frantz Lasorne created a toy prototype, asking the question how we might play if wearing a head mounted AR viewer device (view videos). Read about the project here.


The Nintendo Wii is a great step in the right direction for games for everyone, from little babies to my grand parents. Taking away the mass of complicated controls and adding gesture control into a shape most consumers are familiar with (remote controls) was clever. Yet I still feel there could be issues of very young children playing with the WiiRemote as its still an electronic looking device with buttons and safety straps.

So I was happy to see WiiWaa from Swedish developers Zoink Games today.

Put your WiiRemote inside the mouth (not bottom) of a soft toy…

And the toy then becomes a sort of virtual puppet, using the tilt & accelerometer sensors to control the on screen character in a variety of ways & situations. Watch video


Apparently to be released next year, although little details so far. Conceived & created by Klaus Lyngeled. Funded by the Nordic Game Program.

A closer look at Project Natal

I pretty much always use cameras and computer vision in my work, so I am always keeping a close eye on game technology developments in this area. Once these devices come out, we are then able to hack and use them for our own artworks.

I wanted to take a closer look Natal (Sony controlled and Ubisoft ‘Your Body’ coming another time), in regards to its origins, whats real, what is marketing and whats exciting.

In case someone from Microsoft is readings this post, I am not being negative, I am super impressed by the software behind it and the accuracy of the tracking algorithms, I am looking forward to getting my hands on it as a consumer & hopefully developer.

If you haven’t seen Project Natal, read here and go watch this video
project natal
First of all, its not everything shown in the video real. There are a few instances where the tracking wouldn’t work (hands over skateboard, dad getting in the way of daughter etc). The clue is the text shown clearly at the beginning “Product vision: actual features and functionality may vary”.

This video shows a demo on stage at E3 is the live demo…
project natal
Very good estimation of body points, works very fast too. Microsoft Research you have some very clever people.

This is a concept of what the hardware *could* look like come release…
project natal
(was white before e3)

But right now it doesn’t (at least publicly). The camera has an rgb sensor just like your webcam, but also a depth perception, in the form of an infrared sensor. Infrared light needs to be projected out from the hardware, it hits the player and the room and is recorded by an infrared imaging sensor. This is called time of flight. The software gets a grayscale image, with the brightest pixels being closest to the camera, before being analyzed by the software.

I was wondering where the infrared emitters are on that hardware image above? The green xbox coloured circle on the left?

This is a closer look at the live demo. It looks like probably 3 devices, rgb cam, infrared emitter and infrared sensor…
project natal

Engadget said
“The first thing to note is that Microsoft is very protective of the actual technology right now, so they weren’t letting us film or photograph any of the box itself, though what they had was an extremely rough version of what the device will look like (not at all like the press shot above). It consisted of a small, black box aimed out into the room — about the size of a Roku Player — with sensors along the front. It almost looked a bit like a mid-size (between pico and full size) projector.”

That means that the Milo interaction demo shown here from Lionhead is pre-recorded, naughty naughty :) (p.s funny parody video)
project natal
Of course its real though, hands on writeups from eurogamer, kotaku and wired.

“The Milo demo was partially being manipulated by a developer who was sitting nearby, and I couldn’t tell if he was merely calibrating the game or how much he was pulling its strings.” – kotaku

These videos show the camera hardware blurred out even :)

Ok so the hardware isn’t finished, fair enough. The magic really is in the software. How good will the tracking work under different lighting conditions in my house?

Its all about getting a good clean depth image, unlike normal webcam games that are based on solely movement detection or background subtraction (nightmare). So how big will the infrared emitter need to be?

This is the zcam by 3DV Systems…. (pic shows depth image and hardware)
3dv zcam
Watch video, see how it works.

Microsoft bought 3DV Systems. I instantly thought they were going to use this camera and their own sdk after seeing the demos, however Eurogamer reports:

“Aaron Greenberg was even more direct. Asked whether Natal was derived from 3DV technology, he told Eurogamer: ‘No, we built this in house.’”

“Kim admitted ‘it’s a combination of partners and our own software’, and some have theorised that acquisitions like 3DV’s were designed to insure the company against similar patents. ‘You have to be very aware,’ Kim said. ‘We want to ensure that we have great intellectual property protection. You have to have a strong legal approach, and this is not easy stuff. It has to be all buttoned up, legally. We have had a very concerted focus on this.’ ”

An interview on Venture Beat…

“VB: Some people are very curious about the patents in the gesture control technology. Is there freedom to innovate here, or do you have to be very aware of the patents out there?

SK: You have to be very aware. We want to ensure that we have great intellectual property protection. You have to have a strong legal approach, and this is not easy stuff. It has to be all buttoned up, legally. We have had a very concerted focus on this.”

Eurogamer reports today… “All of the key image-processing is done by Natal’s in-built silicon, leaving the Xbox 360 free to power the game itself”.

There are rumors that Microsoft are working with & licensing patents from Prime Sense, another Israel company like 3DV making such technology. This image shows their hardware, which the Natal model looks much similar to…
prime sense

Funnily enough, both joystiq and n4g reported in 2006 that Prime Sense might have been working with Sony Eyetoy for the ps3 for 3d depth sensing :)

I don’t know if that is true or not, but there is a very detailed video on youtube of the history of eyetoy with tech demos shown to university students, along with demos Sony were working on using the 3DV Zcam, shown below (1 hour in to video) with full skeletal estimation and ball batting games for PS2.

Its a competitive market out there.

Er, anyway. This video from Eurogamer came out today, a hands on with Natal. No camera, but notice the large light source in the corner (*probably* normal light with filter on to only let through infrared spectrum, maybe to help ensure good coverage for the press).
project natal

I feel the hardware or who got it to release first isn’t as important as the software tracking. This is fairly interesting…

“Tsunoda made the point that Natal will continue to work even if someone walks in front of a player because it knows how the human body works. So, if a player had his or her arms blocked, but Natal’s cameras could still see part of their arm, it can fill in the rest based on algorithms that tell it how that arm should look.”

This splat video is great. At 3:05 it shows how the depth image threshold adjusts to a 2nd person entering the space. See at 3:31 he adjusts his mic and the avatar does the same :)
project natal

For me, Johnny Chung Lee has hit the nail on the head here

“The human tracking algorithms that the teams have developed are well ahead of the state of the art in computer vision in this domain. The sophistication and performance of the algorithms rival or exceed anything that I’ve seen in academic research, never mind a consumer product.” (source)

Its exciting times ahead for the consumer level of computer vision, can’t wait.

p.s small disclaimer. Everything posted above is publicly available by google, and is just speculation.

Gamers Reactions

There was some talk of a Sony campaign copying the work of photographer Phillip Toledano on the Kotaku blog.

I had intended to post his and Robbies work ages ago but they have been sat in my blog draft items for eternity, so here you go…

We Will Destroy You

2002, see my previous post from project from Chris Evans.

Video Gamers
Phillip Toledano
An excellent photo series my Phillip Toledano (2002) looking at hidden parts of peoples character whilst playing games.

Robbie Cooper
A really fantastic film by Robbie Cooper, watch here. Robbie will be doing more of a scientific study into this soon.

Sony PlayFace
Sony Playface
The Sony ps3 PlayFace campaign (directed by Timothy Saccenti, agency: Wieden + Kennedy Tokyo), watch here (2009).

Artistically I feel the cuts are too quick in this ad, the speed of time changing etc. They could have gone with the Immersion approach and have raw footage and it would come across as more honest and less acted.

The Graveyard

The Graveyard

The Graveyard is a very short computer game designed by Auriea Harvey and Michaël Samyn (A Tale of Tales). You play an old lady who visits a graveyard. You walk around, sit on a bench and listen to a song. It’s more like an explorable painting than an actual game. An experiment with realtime poetry, with storytelling without words.”

“Buying the full version of The Graveyard adds only one feature, the possibility of death. The full version of the game is exactly the same as the trial, except, every time you play she may die.”

I really like the work of A Tale of Tales, and they are constantly debating what a ‘game’ could or should be. To be honest I only played this game through twice, due to the slow moving story that ends the same way. My main reason for posting the above project is the really fantastic Postmortem articles about Graveyard, taking you through the process of creating this artwork from beginning to end, split into…

1. Birth of an idea: History & Theory
2. How to make an idea come to life: Funding & Technology
3. Making the game: what went right and what went wrong
4. Spotlight on character animator Laura Raines Smith: interview
5. Spotlight on music composer Gerry De Mol: interview & lyrics
6. Spotlight on sound designer Kris Force: interview
7. Out in the world: Critical Response, Downloads & Sales

Its a valuable insight for artists and game creators out there.

The Graveyard

Tuttuki Bako

Tuttuki Bako

Bandai are releasing an interesting sort of portable augmented reality toy called Tuttuki Bako, most likely only in Japan. This small box has a low resolution lcd screen on the front, and a hole on the side in which you insert your finger. Your finger then appears on the screen augmented into the game.

Watch this official video:

(youtube embedded video)

Other videos show the menu and octopus game, annoy a girl game & squishy blob game.

I look forward to getting one to see how well it works, and possibly take it apart.

(found on CrunchGear)

Ping Pong Door

Ping Pong Door

Following on from the previous ping pong post…

No space in your house for a games or ping pong table? Then the Ping Pong Door could be for you, turning an existing doorway into an occasional use play space.

Designed by Tobias Fraenzel (originally for a DesignBoom contest) who tells me:

“The PingPong Door is ready to put in a standard sized door-frame. Just take out the old one and put this one in. The PingPong Table can be flipped open in a second allowing you to play between rooms. Even though the size is smaller, the game is ultra-fun. The door has had an incredible debut at the Tendence Lifestyle in Frankfurt and will be in production this year. It is not available yet”.

(via YankoDesign)