VicTour, Virtual Interactive Character Tour Guide

VicTour is YLabs ‘s final proof-of-concept resulting from its latest software research project CHAMELEON. During the development process the project generated other examples already published such as the “mother of all depth sensing demos”, long before the whole Kinect frenzy (http://www.youtube.com/watch?v=qXcIZ1…). This depth sensing demo was distinguished with the first Auggie award (the Augmented Reality Oscars) at the Augmented Reality Event 2010.
CHAMELEON emerged from the strong belief that the research and development of next-generation intelligent interaction devices is expected to rely on integrative efforts across several research fields. The main objective of this project was therefore to explore and implement architectures and practical design methodologies for embodied intelligent interaction, with a focus on affective and cognitive computation models, as well as autonomous adaptation and learning of system components and parameters over dynamic multi-modal and multi-user environments. More specifically, research focused on:
• Situated cognition and human action models to support development of natural interaction systems;
• Embodied agent architectures suitable to the scalable integration of learning and affective computing algorithms;
• Unsupervised, reinforcement and evolutionary learning techniques for autonomous agents in the context of multi-modal natural interaction installations;
• Affective computing techniques for the design of emotion-based agents capable of interacting with users in a natural and believable way;
• Data mining techniques capable of analyzing and extracting high-level interaction trends and user features to assist in the self-adaptation of interactive systems

VicTour was created with the intention of validating the CHAMELEON project ‘s entire set of achievements . It inherited some of YVision´s previously developed augmented reality functionalities like depth extraction with 2D cameras, collision detection, dynamic occlusions, etc, and materialized into this intelligent and empathetic virtual character that now lives in YDreams’ showroom, and is partly responsible for presenting it to our visitors.
This application encompasses many of the actual computational trends like affective computing, distributed computation, augmented reality, ambient intelligence or smart surroundings, and opens up a large spectrum of possible commercial applications that may be explored in the near future.

YDreams’ Augmented Reality experience with depth-sensing camera

YDreams augmented-reality platform supports depth-sensing cameras. These types of cameras make it possible to pinpoint the 3D position of the users (it supports multiple users) and objects in the environment, allowing for a true marker less augmented reality experience. The virtual objects added in the video stream react to the user as well as the real objects´ positions and movements.

Unlike similar applications that use 2D cameras, this one is not sensitive to light changes, camera movements and noisy backgrounds. This is a simple demo to exemplify the user experience.

.txt

.txt is an interactive performance work, mediated by several sensorial technologies, that explores contemporary transversal forms of artistic languages. The result is the creation of a unique vocabulary articulated physically, through interactive soundscapes, visual composition and real time choreography; an array of artistic expressions that support the dramaturgic intention.

The project’s central ideas are:

1. “the big-bang” of (artistic) language: the birth of a gesture, a sound, an image and their evolution and relation to the performer throughout the piece

In .txt images and body are one. Words are as important as gestures or as images derived from words. They are there to spur strong reactions, by its value and movement. And motion is sometimes vital to captive the imaginary.

The images, movements and bodies should rise above the screen and scenic space alike, forgetting grammatical, syntaxic, semantic and semiologic formulations. They constitute a dramaturgic, emotional, sensitive whole, and should become a transcendental experience that can take you to another place, towards another emotion, and induce in each one’s body our (own) experiences.

2. Considering interactivity as a medium in itself, the interface between the performer and technology is explored as content

The non-instrusive tracking of the performer and of the performance by computer vision, and the gestural interpretation algorithms, along with the ability to simulate reactions through behavioural descriptors, expand the concept of responsive stage onto a living-like environment, where the performer is simply one of the components.

The piece also intends to reinforce the perception of real time interaction with participatory content by the audience. .txt audience encouraged to revolutionise power structures through the subversive manipulation of the electronic media serrounding us. Through Twitter this empowered “spectActor” can take part in the dialogue happening on stage by adding content to the performer interaction environment.
The scenic space and decoration design unfolds from the search for new scenographic solutions that can be effective at the visual and narrative, but also at the interactive level. Extending the array of interaction possibilities, scenic decoration becomes a physical interface available to the performer control.
Physical simulations like particles systems, bird flocking or potential fields applied to typography and its evanescence to abstract communication symbols constitute the performer’s universe. Its evolution, behavior and emotions are determined by the performer’s interaction with it while keeping an autonomous emergent behavior.

3. Critical reflection of convergences between art and science in its social, historic and aesthetic dimensions is a part of the performance itself

Centring the visual imagery in text and typography .txt expresses also homerage to such a plethora of great digital art works, from installations to performances and electronic poetry, in all its forms, that contributed to digitally explore this media.

“The Electronic Revolution” (1970) by Willliam S. Burroughs serves as a dramaturgic base to develop the narrative structure along the lines traced above. In his essay, Burroughs identifies essential questions about the use and abuse of media technologies. Furthermore, he proposes artistic ideas, as well as humanistic concerns, that have been inspiring the narrative dimension of “.txt” project. Some of the compositional devices shared by many artists of his time are re-visited, refined and transferred to other areas. His famous cut-ups, for example, can be compared to sampling techniques today applied to all kind of fragments of digital and analog media materials.