Torben Sko - Projects

Overview

For my PhD, I'm exploring how face tracking can used to augment traditional forms of computer game interaction, using commercially viable approaches. Through this research, I've been working closely with Seeing Machines, who produce a face tracking program called FaceAPI. The program is capable of tracking a user's facial features using a standard webcamera, allowing the results of my research to be applicable to the common gamer, rather then just those who can afford expensive equipment.

To date, my work has focused on two key areas, the first of which is an exploration into how the user's head position and orientation can be used to enhance first-person-shooters. In undertaking this work, we considered how it could be used for both controlling and perceiving the game. Simultaneously, we've explored some of the social implications, such as how people respond to virtual characters that mimic their facial expressions. This later work has significant implications for online socialising, such as multiplayer games.

Recognition.

Since commencing my research at the start of 2008, it's received the following recognition:

My interactive portrait was selected as a finalist in the National Youth Self Portrait Prize (NYSPP), run by the Australian National Portrait Gallery (NPG). The work used a webcamera to mimic the facial expressions of the viewer, as demonstrated in the video below.

On the back of being selected in the NYSPP, the Canberra Weekly decided to feature an article on my work. A low-grade copy of the interview can be found here.

Similar to portrait I had exhibited at the NPG, I had another artwork exhibited in the Beginning, Middle, End exhibition, which was held at the ANU School of Art Gallery. The work let the viewer influence the artwork by allowing them to choose which face they preferred.

Want To Do Something Similar?

If you like this work and want to try doing it yourself, then give my integration guide a look over. If you get something up and running, let me know and I'll put something about it up on this site. We really appreciate hearing about how others are using this technology!

Supervisors

Dr Henry Gardner

Reader and head of the School of Computer Science

Prof. Tom Geddeon

Professor of computer science and head of the Information and Human Centred Computing (iHcc) research group at the ANU

Paul Kirwan (IMDb)

Digital compositor and video convenor for the department of Photography and Media Arts

Videos

Controlling the Facial Expressions of a Virtual Character using a Webcam

Using Head Gestures in PC Games

Creating a 3D Desktop using a Web Camera

Publications & Presentations.

Head Tracking in First-Person Games: Interaction using a Web-Camera

To be published in INTERACT 2009 (12th IFIP TC13 Conference in Human-Computer Interaction)
Co-author Dr Henry Gardner

Recent advances in face-tracking technology have made it possible to recognise head movements using a commodity web-camera. This development has created exciting possibilities for enhancing player enjoyment during computer game play. In order to ascertain the real-world potential for head gestural input to First Person Shooter games, we have developed seven diverse interaction techniques and integrated these with a modern games engine. Evaluation of the techniques was carried out with four focus groups made up of expert games developers and experienced end-users. One of the techniques was further refined and subjected to a follow-up comparison test with promising results. A set of guidelines for the future development of head interaction techniques for computer games has been derived from the studies. All of the techniques have been built upon freely available software and open-sourced to encourage further research in this area.

Consumer Face Tracking

Presentation at COFES 2009 in the MaieuticParataxis session

A presentation exploring the potential of using face tracking in an engineering context. Uses include using head-coupled-perspective to create a greater sense of depth to aid in viewing 3D models.

Camera-Tracked Head Gestures in First Person Shooter Games

Presentation at HCSNet Summerfest Speed Paper 2008

In light of recent advances made in the field of computer vision, it is possible to reliably track a user's head position using an off-the-shelf web camera. Whilst such an advance is exciting due to how commonplace cameras now are in desktop setups, the potential of this new input technique is largely unexplored.

In effort to rectify this, our work looked to systematically explore the commercial interaction space for head gesturing. In particular, we focused on their use in augmenting traditional input for first-person-shooter computer games, in an effort to increase user enjoyment through promoting greater levels of presence and realism. We developed 6 techniques which consisted of; zooming, peering, spinning, looking down the barrel of the weapon, a fishbowl effect, and replicating the appearance of a hand-held camera. A demonstration of these techniques can be seen at: http://www.youtube.com/watch?v=qWkpdtFZoBE

These techniques were analysed using several focus groups, during which both game developers and end-users were represented. From this study, the handy-cam technique was found to have immediate commercial potential without the need to modify the game content. Likewise, the peering and zooming techniques were shown to have potential, but it was felt that the game content would need to be focused around their use, to ensure their success. A follow up user study confirmed part these results, by showing that most participants preferred the use of the handycam technique over a standard setup. Through showing this, we have demonstrated that head tracking can be used to enhance computer games.

Downloads

Free Download

1.0 for Windows XP and above