A Big Slow Global Approach to VR: Silicon Valley’s Blind Spot
By Michael Naimark ● May 18, 2017 10:05
Empathy or Porn?
Last fall I had the honour and challenge of giving the opening keynote presentation for Open Immersion, the first Virtual Reality “Doc Lab” produced by the Canadian Film Centre’s Media Lab, the National Film Board of Canada, and the Ford Foundation. The Open Immersion participants, selected via nomination and vetting, consisted of six First Nations artists in Canada and six artists of colour from the American South. Ages ranged from twenty-something to sixty-something, eight of the 12 were female, and most had little or no experience in VR. All were accomplished artists.
After my presentation, a wide range of VR projects were briefly presented, usually by their creators. Some phrases brought up by the participants included “poverty porn,” “unearned access,” “politics of representation,” and “to be imaged is to be exploited.” One VR project, which attempted to place the viewer in the shoes of “travelling while black,” elicited a terse comment from an African-American participant from North Philadelphia: “Please at least admit that this wasn’t made for black people.” In all fairness, several straight-up educational VR pieces that were presented – including one on auto safety simulation — were enjoyed by this sharply-critical group.
The next morning, during a long shuttle ride with several participants from both countries, most of the conversation was about whether anyone can really empathize with the “other” in a 10-minute VR experience, or whether it’s simply folly for the privileged to feel good.
It was clear that I wasn’t in Silicon Valley anymore or the mainstream VR community.
Michael Naimark at Open Immersion
My name is Michael Naimark and I’ve managed to straddle between the worlds of research and corporations on one side, and art and activism on the other for over three decades. I can “pass” in both worlds as family (albeit perhaps more as a cousin than sibling) and have been fortunate to have participated in the “birth moments” of several creative research labs on both sides, going back to the MIT Media Lab. My practice centres on emergent media and immersive experiences, and my specialty is real-world “place representation.” I geek out on cameras, and I've designed and used many immersive rigs. I’m also a fan of ethnographic field practice and have been a member of the Society for Visual Anthropology mostly since 1980. I live in San Francisco and work out of Francis Coppola’s Zoetrope building (definitively “old guard”), and often also work in Silicon Valley. In 2015 I was appointed Google’s first-ever VR Resident Artist.
Not long ago, a virtual reality accelerator of a large international corporation asked on its online application: "Years of VR experience?” There were three options on the pulldown menu: “<1 Year, 1-2 Years, and >2 Years.”
I found this a bit alarming and contacted the head of the initiative, who happened to be a friend, and let him know. The next day, the pulldown menu had changed to six options: “<1, 1-2, 2-5, 6-10, 11-20, and 20+ years”. Perhaps better, but nevertheless reflecting a rather innocent, some might say clueless, perspective about timeframes.
Like VR, Silicon Valley prides itself on speed. But with speed often comes myopia. Most on the inside would say: “Our job is to build, market, scale, and ship!” In such environments, serendipitously saying “this is interesting” is sometimes interpreted as losing focus and moving off-mission. Many Silicon Valley veterans say that such short-term thinking has intensified in recent years. Some blame greed and the gladiator mentality of needing to pitch a “unicorn or nothing.” Others blame lack of critical skills, the inability to “think things through.”
In a sardonic way, the election of Donald Trump may be a wake-up call to Silicon Valley’s myopia, for example, of not thinking through the implications of easily-manipulated social media. To paraphrase historian Yuval Harari (before the election): “The most crucial choices about the future are made not by bureaucrats or lobbyists but by engineers, entrepreneurs, and scientists who are hardly aware of the implications of their decisions, and who certainly don’t represent anyone.” He continues: “I doubt whether the gurus of Silicon Valley have really thought through the full social and political consequences of their ideas, but at least they are thinking in fresh ways.”
Many of us would like to think, despite Silicon Valley’s endemic myopia, that VR is a radical new form and thus a venue for fresh new ways of thinking. We saw this with cinema in 1900, video games in 1975, the web in 1995, and mobile in 2005. As commonly noted, these new forms began by emulated old ones. So it’s no surprise that much of VR today are “linear movies in 360” and “video games in 360.”
So, where do we find fresh alternatives, ways where VR is explored as its own radical new form? I’d vote high for both location-based VR and immersive art installations, where compromise and making things “at scale” are less critical. I’d also give a nod to the sort of shared virtual spaces exemplified by Second Life (ostensibly why Facebook acquired Oculus). And given my years making “interactive movie maps,” precursors to Google Street View, there’s huge potential relating to virtual travel and tourism, an industry that employs 10 per cent of the global work force. As a professor friend, long active in sustainable indigenous eco-tourism says, “it’s a win-win-win-win proposition.”
A related model, one personally close to heart, encompasses several fresh areas of exploration, and it is far, far from myopic.
John Lomax, a pioneering musicologist and folklorist
An Epic Model
The story begins in the 1860s, in central Texas, cowboy country, where one John Lomax was born. Hearing the cowboys sing as they led their cows along the river valley every season, Lomax grew up and became a folklorist, an ethnomusicologist. In the 1920s, he made some of the world’s first field sound recordings, with his teenage son Alan assisting, with a recorder given to them by Thomas Edison’s widow. They made it portable and it weighed 500 pounds. The Lomax’s early field recordings became the basis for the U.S. Library of Congress Folksong Archive.
Alan Lomax continued his father's work for decades in other countries. People sent their field recordings to him. Thousands. Lomax thought he heard patterns. In the early 1960s at Columbia University, he began research to codify all of world music. His team came up with 37 parameters and began coding thousands of songs from hundreds of cultures. They called this work Cantometrics and later did the same for recorded dance, which they called Choreometrics.
Lomax's recording gear, 1930s
Even back then, with punch cards and mainframe computers, they looked for correlations. They found, for example, that the song styles of Central Africa and Georgian Russia were statistically similar, as were the song styles of Inuit and Patagonian music. They could cross-correlate their music database with their dance database, and could correlate both with the equally ambitious “Ethnographic Atlas” of George P. Murdock. What they found was strong, romantic, provocative stuff. Lomax was convinced it proved a “unifying theory” of the expressive arts, a theory as resilient as genetic evolution. By the 1980s, he wanted to “give this back” using personal computers and called his vision the Global Jukebox.
Around 1990, Lomax sent a cold letter of his vision to Apple Computer, where it found its way to the Apple Multimedia Lab, a small skunkworks outfit in San Francisco set up with Lucasfilm, where it found its way to me, a young member. I visited Alan in New York and helped him get support from Apple, and later from Paul Allen’s Interval Research, and produced his first and only Global Jukebox video demo. I was convinced that Alan Lomax was largely responsible for such concepts as metadata and folksonomy, and that the Global Jukebox was an early example of what epic-scale global media made for everyone might be like.
Alan Lomax died in 2002 and his daughter Anna Lomax Wood has taken over the project. A 400-page book on Alan came out in 2011, The Man Who Recorded the World, and a 2012 Kickstarter campaign was modest ($50k) but successful. Despite its scope and a long list of celebrity fans (Margaret Mead, Jerome Wiesner, Dylan, Clapton, Jagger, Eno), the Global Jukebox has always been a hard sell in the fast-paced, myopic world of high tech. From the book: “the Global Jukebox has fallen into an abyss between academic and pop culture, between world-saving and money-making, and between content and technology.”
An early demo video for the Global Jukebox Project from 1998
A Modest Proposal
Can the Global Jukebox and work of the Lomax’s inform a fresh new way of exploring VR’s radical new form? For example, can we make headway with the notions of metadata, folksonomy, and ethnographic field practice, where community buy-in is essential? Can we also make headway with the questions and issues surfaced by the Open Immersion Lab artists? How can their voices be part of the conversation? I went into my Google VR artist residency asking some of these questions.
We produced several VR cinematography studies, whose intentions were to explore how people are represented in VR.
From VR Cinematography Studies for Google – our intention was to explore how people are represented in VR
We learned about the limits of closeups, artifacts, and recognizability; about camera height and eyeline; about dialogue and positioning; about how to digitally add subjects (and hide crew); and how these technical aspects gave us access to start unpacking and understanding “privilege” as literal representation. We also started to play around with digital compositing which can result in an artificially overpopulated scene, perhaps with dozens of people whose words, actions and positions in space are all “tagged” with metadata, allowing an immersive VR experience which can be both deeply intimate and richly interactive.
The 360 degree by 180 degree “equirectangular” video format turns the radial lines into parallel lines. Here’s the left eye view from a stereo pair.
(Yep, that’s a 360 degree image of the same scene above!)
Toward this end, we propose a community-based ethnographic VR experiment, thinking globally but with a fast, lean, local prototype. Its basic (yet flexible) elements are:
- a stationary VR camera planted in a nicely “panocentric” public space from dawn to dusk
- shots of locals answering “everyday life” questions and expressing themselves
- breaking down the shots into short clips and tagging each with metadata
- digitally compositing an overpopulated “hyperimage” to serve as a home menu
- and making an interactive VR experience allowing “directed interactivity” through the clips.
When I include critical inquiries and challenges that surfaced during Open Immersion, we can start to add more nuance to this list, including:
- how to frame position in the 360 space as indicative of the dynamics of power and how the metadata can reflect such positions
- how to rethink what subject means in VR
- how digital compositing and interactivity can help us reframe subjectivity in VR.
We envision the project to be a conflation of Thornton Wilder’s Our Town, in its appreciation and passion for everyday life, and (don’t laugh) TedX, serving as an early and “just good enough” template that other folks could see it and say “we can do that too (and maybe even better)!”
Children's Games by Pieter Bruegel the Elder, 1560, an early example of a hyperimage
Back in Silicon Valley, I’m learning that such a project, like the Global Jukebox, is a hard sell. It’s simply too experimental and not commercial and short-term enough. (Joke at Google: better to do puppies!)
Since returning from the Canadian Film Centre, several Americans have noted that shocking, tragic images like the napalmed Cambodian girl may be considered “porn” to some but yet may have also changed the tide of public opinion about the Vietnam War.
So what will it take to make better connections between the tech-based commercial enterprises and the content-based creative activist communities? How can more bridges be built? Curiously, both sides often acknowledge the potential for symbiosis.
A map from the recently launched The Global Jukebox
There’s a bit of news, and inspiration. The Global Jukebox, decades in the making, officially launched last month. Perhaps it will inspire others to think big, think slow, think global, and practice accordingly.