Description

Creating for Mobile Culture, Atau Tanaka

Impossible Geographies: Archaeology of the Instant, Petra Gemeinboeck

Extensions, Dérivations, Christophe Kihm [english] [français]

The Visual and Sonic Experience of Net_Dérive, Ali Momeni

Publications

Description
Net_Dérive is an artwork in the domain of location sensitive mobile media art. It is an installation piece extending beyond the confines of a gallery, to include the urban environment. Conceived for participative use from advanced mobile telephones, they call upon techniques from interactive music applied to new contexts that include the interplay of sound and image, an exchange between participants in the gallery and participants in the streets, and the creation of an abstract narrative from audiovisual media captured on multiple mobile devices. Deployed on portable, networked, location-aware computing devices, we seek to create a kind of musical instrument, thinking of the city-as-instrument.

Participants in the piece will borrow a mobile phone at the gallery site, then be given a “mission” to carry out in the surrounding neighborhood. They set out on their task, with the mobile phone in hand. The mobile phone serves as a graphical display, and with a pair of headphones becomes a connected audio device. Their geographical location is known to the server, allowing the system to trace the users’ movements. The mobile users takes a series of photographs using the mobilephone’s built-in camera, these images are auto-uploaded over the mobile network to the server.

The server generates an audiovisual stream based on this information, and is fed back live to each mobile client. Voice instructions suggesting paths to follow or turns to make are generated heard by the user, abstracted in a musical fashion. As the user chooses to heed or ignore these instructions, a trace of his path is carved out in the city. The simultaneity, history, and memory of the various users’ paths and uploaded images become an abstract narrative that is summed together and projected in the main gallery space. There is a feedback created as the users’ actions generate the collective narrative that in turn direct them.

Net_Dérive creates a tension field between a gallery space and the urban fabric into a playground on which the excavation of urban spaces is negotiated. This will create an archaeology of the instant, into which we will be able to carve grooves to uncover layers of shared memories. This shares experience develops alternative modes of reading and forms of representation that produce spaces in-between, hybrid spaces, from which other relations, yet unknown, can emerge.

Creating for Mobile Culture

We are living in an increasingly mobile culture. But while society has the means to be mobile, what are the real issues in creating a culture of mobility? Commercial offerings simply attempt to transpose existing media, such as television, to portable devices. Meanwhile unexpected usage emerges in a grassroots fashion, for example the explosion of SMS. How can we leverage these forces and channel these energies to create deeper cultural experiences for a mobile society?

Locative media is the name for a field of artistic practice concerned with
geography as artistic canvas. Artists such as Janet Cardiff create imaginary audio stories that accompany the listener walking through real places. The group Blast Theory turns the city into a game board inciting groups of users to unravel a mystery. My interest is to take interactive music practice off the stage and outside the concert hall into the urban sphere.

By transposing musical action from stage to street, we displace the locus of creation and creativity, not just physically, but socially. Mobile communications devices are meant to connect groups of people. Musical concerts, similarly, are situations that bring people together for a common purpose. Can we elicit commonalities to make a community-based musical process, creating a shared experience among users?

In the research arena, we have fields of study such as Social Computing, and notions such as Communities of Practice, and Social Capital. Ubiquitous and networked technologies are called upon and configured to amplify cooperative behavior. In this work, I attempt to take the fact that music is a collaborative human activity, combine that with contemporary practice of digital music production to create social music systems.

Creating working prototypes for such ideas are an act of system building. We must create an architecture of itinerant devices connected over wireless infrastructures, utilizing network level services such as localization, to feed a dynamic audio-visual content generation process. With all this, the primacy still resides in thinking about what the resulting music sounds like and what the ultimate experience will be for the listener!

Composing for such an environment is to look at the system as a kind of instrument, a musical instrument that is not an acoustic network of vibrating elements and air pulses, but a human/technological network of entities and elements coming into musical interaction.

Seen from am industrial point of view, this is a vision of convergence: convergence of content and device, of services and product. While some multi-nationals, including the mother structure of CSL, have all the components to make a convergence product, the reality of mobilizing forces to do so are daunting.

We have the possibility from the point of view of fundamental research, to take an alternative angle to tackle these problems. By building systems as artworks, we gain insight often overlooked in the rush of product development. By injecting creativity into the research process allows us to create results that have the potential to have a true cultural impact in society.

Petra Gemeinboeck

Impossible Geographies: Archaeology of the Instant

…in his dreams, cities as light as kites appear, pierced like laces, cities transparent as mosquito netting,…, filigree cities to be seen through their opaque and fictitious thickness.
Italo Calvino, Invisible Cities

The electro-magnetic flows of information and mobile technologies pervading our everyday life are often said to be spinning such “filigree cities.” Cities whose “opaque and fictitious thickness” can only be accessed if we co-inhabit these invisible spaces. And yet, this city is only one of the multiple cities, invisible and visible, we inhabit in our everyday life. The imbrication of these multiple spaces shapes a thickness that might appear opaque – invisible – and yet that is certainly not fictitious. This experiment explores the negotiation of multiple urban geographies by looking at one such invisible city, twined by the spatial and temporal pattern of diverse everyday practices, through the “fictitious thickness” of another, the virtual spaces spanned by mobile technologies. It imagines the city as a playground, where Calvino’s Invisible Cities meet Debord’s Naked City. Mapping this playground, we develop alternative modes of reading and forms of representation that produce spaces in-between, hybrid spaces, from which other relations, yet unknown, can emerge.

Motivated to dwell in the spaces between binary opposites, without mapping one onto the other, we call upon the Postcolonial concepts of hybridity or  “third space,” from which other positions can emerge. Looking for such “thirds spaces” evolving between the imbricated textures of the urban fabric, we find counter-spaces that are foreign to the traditional cartographic space. Fluid and transversal, their temporal spaces are always intrinsically connected to other temporalities elsewhere. Feminist discourse on spatialization and subjective heterogeneities provides the framework that propels our cartographic engagement with dislocating, unmapping and juxtaposing spatial and temporal geographic territories. The urban cartographies will thus unfold through drifts, ruptures, and hybrid connections and emerge from the situated, partial and interpreted knowledge of the “terrain.”

The socializing and spatializing practices of locative media deploy portable, networked, location-aware computing devices that involve participants in mapping processes, social networking or artistic interventions. Here geographical spaces become the virtual canvas to be inscribed with personal narratives, desires, and memories. There is, however, a double-edge to this notion of public authoring, as the technologies affording the sharing of collective location-based interventions paradoxically operate upon the same plane as surveillance. Furthermore, its reliance on positional precision and the emphasis on a location-based context critically link the field of locative media to the arena of cartography and its hegemonic practices of mapping.
The challenge is thus to unfix the subject from the grid in order to create a map that is open. The opportunity of locative media’s paradoxical and yet participatory nature can be seen in Deleuze’s and Guattari’s map that has “multiple entrances.”  As the map that is taken in hand by individuals, groups or a social formation, it eludes tracing, “which always returns to the ‘same’.” It creates thus alternative connections that question traditional forms of mapping, producing forces of diversification, rather than homogenization.
Net_Dérive presents a hybrid between a gallery installation and a mobile artwork. The work establishes a connection across two distinct types of spaces, the urban space and a gallery exhibition space. The latter is typically characterized as a “white box,” an enclosed controlled environment. The urban environment, on the other hand, shows an uncontrollable multiplicity of both independent and interrelated nodes of intensities, events and actors. Our approach interacts with the imbrications, and multiple forms of belonging that emerge from this fluid fabric. The resulting subjective cartography shapes “a migrational or metaphorical city [that] slips into the clear text of the planned and readable city” - as well as into the controlled space of the gallery.

The hybrid playground of this work emerges between the grooves inscribed and the spaces excavated by the mobile participants (equipped with mobile phones) and the movements of the participants that are surveyed in the gallery space. The “grooving” scenario and its translation and representation by the underlying system allude to the idea of the Situationist dérive, or drifting through the city, where one’s path is perpetually deviated. Net_Dérive creates an interplay between the participants drifting and the representations of their trace that is deviated based on external events that relate them to the other participants. So, in a way, the path detaches from the “drifter” and performs its own capability to “drift.” The permeability of the city skin is influenced by the relations between the grooves (paths), the negotiation between the gallery participants and the participants drifting through the city. As a result, the mobile participants are able to “excavate” earlier traces, left by previous participants, creating an archaeology of the instant. The city’s “memory” is materialized as a contact zone in which present and past, present and absent, virtual and real meet and interlace. Always flowing and neither clearly one nor the other, this “memory space” manifests itself as a leakage between time and space. Grooving into the layers of recent history, dérive is then horizontal as well as vertical.

Extensions, Dérivations

Net_Dérive is an art project situated at the intersection of four structural elements: mobile communications, the generation of sound/image projections, a specific-action time, and a place of experimentation (the Bastille district in Paris where we follow the meandering of three people carrying mobile phone prototypes). This project mobilizes and stretches the technical possibilities of 3G telephony, and imposes significant changes on current modes of production and dissemination of sounds and images (and consequently on the space-time fields generated by them). The visuals and sounds generated are, in effect, visual-auditory traces of geographic displacement, of presence and distance. They are the musical and graphical translators of the experiences of the “dérive protagonists”.

This experiment thus rests on a series of technical extensions of the mobile phone, a commercial object, that modify its possibilities and create freer usage. The term “extension” should not be seen simply as technical amelioration, but rather as a gauge of creative innovation, that is, a difference where advancement is made possible due to its necessity from an artistic point of view.

By determining the points linking these extensions, its modes of sound/image projection, time and space – we get a glimpse at the research process. First, it recasts a communication medium across an ensemble of connections, transmissions and of filters, which imply the constitution of a new object, of a new network and of new mediations. Here enters the choreography of the software Max/MSP-Jitter, of audio/video up and down streaming, of control tools, projection screens, headphones, etc. Parallel to creating this setup, is a reorganization and displacement of the constituent elements of a typical mobile phone by the addition of a GPS module.

Through these extensions, Petra Gemeinboeck and Atau Tanaka combine the processes of hybridization and mutation. They are distinguished from one other by the fact that hybridization implies a transformation of the exterior of the mobile phone (following the model of a graft, in this case as Walkman-style headphones), while mutation changes the interior of the object (on the model of genetic evolution, where the mobile phone’s software is reprogrammed).

Hybridization and mutation are not restricted to technical aspects. Net_Dérive transposes social software techniques (e.g. matchmaking sites) in terms of sounds (melodic and amplitude variations) and physical parameters (proximity or distance, out-of-sight or present). It creates a hybrid of social, musical and spatial relations, which produces music for groups of people while evolving in response to personal behavioural variables. By turning the mobile phone into a real time transmitter of audio and visual data, the object is transformed into a measurement device as well as a controlling device. The usage of this hybrid object is open-ended: it is a mutant because it does not give in to any predefined “function”, but completely turns itself to its “usage” as a pure mediation.

In this way, Net_Dérive creates a contact surface in the interstices between the real and the virtual: the real space of the city (the Bastille district) in which the work is activated, and the virtual space that is shared by the protagonists through their mobile phone prototypes. It hybridizes space itself, as seen in the images re-transcribed on the gallery screen.

Floating in this in-between space, the dérive offers a unique experience: that of a materialization of its own mediations in sounds and images. It thus invents ways of listening to music which create relations, and a ways to navigate urban environments that create intervals. These elements, not a score, nor a map, work together to create visual and sonic representations of real and virtual communities.

As the system is sensitive to presence and movement, that is, since it is informed by physical contingencies that determine its function, it becomes transparent to its own usage and allows insertion of the human into the musical space created. Here one last hybridization takes place where man, music, and space recompose themselves in infinite variation.

Christophe Kihm is managing editor of ArtPress magazine and the co-editor of the review Fresh Théorie. He also teaches at Le Fresnoy (the French National Studio of Contemporary Arts) and gives courses in the MA program in Public Sphere at the ECAV (Sierre, Switzerland). Kihm is also independent curator, whose recent work includes the Jeremy Deller exhibition at Villa Arson (Nice, France) and the Enseigner/Produire project at the Centre Georges Pompidou (Paris) and Le Fresnoy (Tourcoing, France).

Extensions, Dérivations

Net_Dérive est un projet artistique qui se détermine à l’intersection de quatre éléments structurants : trois prototypes – des téléphones mobiles –, une production et une diffusion de sons et d’images (à travers un circuit établi entre une galerie, un serveur et les trois objets-prototypes), un lieu et un temps d’action (la quartier Bastille, à Paris, terrain d’expérimentation, de déplacements et de déambulations de trois personnes munies de ces trois objets).
Ce projet, qui mobilise les conditions techniques du téléphone 3G, opère d’importantes modifications sur ses modes courants de production et de diffusion d’images et de sons, et par conséquent sur l’espace et le temps qu’ils génèrent. Les images et les sons produits et diffusés par les téléphones proposent en effet des relevés auditifs ou visuels des déplacements, des présences et des distances – des traducteurs graphiques et musicaux –, qui renvoient directement à l’expérience vécue par les protagonistes de ladérive.
Les conditions de cette expérience reposent donc sur une série d’extensions techniques apportées à un objet industriel, le téléphone mobile, qui en modifient les possibilités et en libèrent les usages. Ce terme d’extension n’est pas à entendre ici comme le gage d’un progrès, au sens d’une amélioration technique, mais comme celui d’une innovation, c’est-à-dire d’une différence ou d’un écart permis par les nécessités d’une perspective artistique.
En déterminant le, ou les points, qui relient les opérations d’extensions produites sur cet objet, ses modes de production et de diffusion d’images et de sons, leur temps et leur espace, il est possible d’entrevoir la nature de la recherche engagée par ce projet.
Il s’agit, en premier lieu, de rejouer un médium de communication à travers un ensemble de connexions, de transmissions et de filtres, qui impliquent la constitution d’un nouvel objet, d’un nouveau réseau et de nouvelles médiations. Ici interviennent des agencements entre les logiciels Max/MSP et Jitter, le streaming audio/vidéo dans le sens téléphone/serveur puis serveur/téléphone, des outils de contrôle, des écrans de projection, des casques audio, etc. Parallèlement à la création de ce dispositif spécifique est engagée une réorganisation et un déplacement des éléments constitutifs du téléphone portable 3G par l’ajout d’équipements : stream audio/vidéo, pose de capteurs, etc.
Dans l’une et l’autre de ces extensions, le projet de Petra Gemeinboeck et d’Atau Tanaka combine des procédés d’hybridation et de mutation, qu’il est possible de distinguer, pratiquement, en posant que les premiers impliquent une transformation de l’objet par l’extérieur (sur modèle de la greffe, emblématiquement matérialisée par le branchement d’un casque type walkman sur le téléphone), alors que les seconds produisent une transformation de l’intérieur (modèle de la re-formation génétique qui vaut ici pour la re-programmation informatique du téléphone).
Mais hybridation et mutation ne jouent pas uniquement sur un plan technique.En effet,Net_Dérive, en transposant les applications du social software (modèle des sites de rencontre) en des termes sonores (des mélodies, des variations d’intensités) et physiques (la proximité ou l’éloignement, la distance et la présence), réalise un hybride social, musical et spatial, qui propose d’écouter et de produire une musique à plusieurs, évoluant en fonction de variables comportementales personnelles. En transformant ensuite le téléphone mobile en transmetteur de données audio et visuelles en temps réel, l’outil de communication mute en appareil de mesure et donc, également, de contrôle.
L’objet hybride est un dispositif à activer au sein duquel les usages sont ouverts. Il est mutant dans sa capacité à ne pas admettre de fonction, à s’en remettre entièrement à ses usages, comme une pure médiation.
L’objet mutant fabrique des hybrides dans la navigation qu’il instruit entre deux dimensions : l’espace réel de la ville (quartier Bastille) au sein duquel il est activé, et l’espace virtuel que partagent les protagonistes via leurs téléphones portables, les images et les sons qu’ils émettent et reçoivent. Il crée ainsi une surface de contact, dans un interstice entre le réel et le virtuel : il hybride l’espace lui-même, comme le figurent les images retranscrites sur l’écran de la galerie.
Flottant dans cet entre-deux, la dérive médiatique offre une expérience inédite : celle d’une matérialisation de ses propres médiations en musique et en images : elle invente donc une manière d’écouter de la musique qui crée des relations et une manière de se déplacer qui crée des intervalles. Ces deux éléments, ni partition, ni carte, participent d’une représentation visuelle et sonore d’une communauté réelle et virtuelle.
C’est parce que l’objet/médiation est sensible à la présence et au mouvement, c’est-à-dire informé des contingences physiques qui déterminent son fonctionnement, qu’il devient transparent à son propre usage et permet l’insertion de l’humain dans la musique comme dans l’espace produits. C’est ici le lieu d’une dernière hybridation où l’homme, la musique et l’espace se recomposent dans des variables infinies.

Christophe Kihm est critique et théoricien. Membre de la rédaction d’art press et co-directeur de le revue Fresh Théorie, il est enseignant au Fresnoy, Studio national des arts contemporains et au programme MAPS (Master of Art in Public Sphere) de l’ECAV (Sierre, Suisse). Il est également commissaire d’expositions (Jeremy Deller, Villa Arson, Nice, mars-juin 2006 ; Enseigner/Produire, Centre Georges Pompidou/Le Fresnoy, novembre 2005-janvier 2006.

The Visual and Sonic Experience of Net_Dérive

Interdisciplinary, participatory and locative works like Net_Derive pose an interesting set of compositional, aesthetic and technological challenges.  First, the materials with which one must compose are largely derived from unpredictable sources.  Second, the incoming data must control the visual and sonic instruments that realize the piece in creative and perceivable ways.  Third, these audio-visual real-time instruments must respect and support the conceptual motivations described by Petra Gemeinboeck and the mobile-culture aesthetics and technologies gathered and analysed by Atau Tanaka.  Finally, working with mobile technologies is simultaneously a glance into the future and a few steps into the past; many amenities of modern personal-computing technology on which we depend as new media artists are simply not available on mobile platforms.

My approach to meeting these challenges has been to attempt to bridge the gap between the outdoor space and gallery space as much as possible.  The sounds and images surrounding the mobile participants are the principal raw materials.  Analysis-resynthesis techniques are applied to these raw materials create variations and augmentations of the source sound that still retain their connection to the original.  For example, street sounds from the neighborhood are spectrally analyzed in real-time.  The analysis results are then applied to other audio instruments in order to harmonically tune them to the environmental sounds. Cartography and data visualization play important roles in the installation’s gallery visuals, whereas virtual audio instruments similar to those used by Tanaka and myself in performance sonify the mobile participants behavior during their walk.  Collective parameters like the area covered by all of the mobile participants or their average distance from the gallery affect the overall diffusion, presence, periodicity and reverberation of the gallery installation.  Simultaneously, each mobile player is also sonified indepentendly by a number of dedicated instruments;  indivudual parameters like each mobile participants level of activity, speed, or proximity to the other players are used to control these dedicated instruments.

Publications

Tanaka, A., Gemeinboeck, P. "A Framework for Spatial Interaction in Locative Media." in Proceedings of New Interfaces for Musical Expression (NIME06), Paris, 2006. p. 26-30.

Gemeinboeck, P., Tanaka, A., Dong, A. "Instant Archaeologies: Digital Lenses to Probe and to Perforate the Urban Fabric." In Proc. ACM Multimedia, Santa Barbara. 2006.

Gemeinboeck, P., Tanaka, A., "Concepts in Locative Media: Instrumental and Theoretical Considerations in the Design of a Framework." ISEA2006 symposium, San Jose, CA.