A Virtual Reality Revolution, Coming to a Headset Near You

Virtual reality — once the stuff of science fiction — is still in its infancy. But there’s already a gold rush around the technology, which plunges viewers into a simulated 3-D environment and lets them explore their surroundings as if they were really there.

Technology and entertainment giants are betting billions that virtual reality is much more than a passing fad, one that will revolutionize the way we experience movies, news, sporting events, video games and more.

Meanwhile, filmmakers and other creators are grappling with an entirely new storytelling language and dealing with some formidable challenges — claustrophobic headsets that can make people cybersick.

The competition to dominate this space begins in earnest these next few months, with the arrival of newfangled, affordably priced headsets from Samsung, Sony, HTC and Facebook (which paid $2 billion last year for a virtual reality startup called Oculus VR). And Disney, Comcast, Time Warner and Legendary Entertainment are just a few of the entertainment companies plunking down millions of dollars in a mad dash to create content for these machines. By 2025, the market for virtual reality content will be $5.4 billion, according to the Piper Jaffray investment bank. The hardware component will be worth $62 billion.

“We’re at the brick-size cellphone days of VR,” said Ted Schilowitz, the in-house futurist at 20th Century Fox. “The technology works. It’s remarkable. But it is nowhere near good enough, on any front, to take on mass, mass adoption.”

Yet, he added, “every few months, we’re reaching closer to the target.”

Without compelling content, even the most impressive piece of technology won’t appeal to more than a hardy band of early adopters. One of the more high-profile experiments at filling that void is taking place over at Mr. Schilowitz’s studio, where the director Robert Stromberg (“Maleficent”), Ridley Scott and the Fox Innovation Lab are putting the final touches on a virtual reality companion to “The Martian,” Mr. Scott’s hit film. In the 15-to-20-minute film, to be released early next year, viewers will become the stranded astronaut (played by Matt Damon in the feature film) as they navigate the planet and attempt tasks to stay alive. They will even get to experience zero gravity in space and drive the rover on Mars.

Here, some of the other pioneers in film, journalism, sports and gaming talk about the potential and struggles of building a new art form from the ground up.

Better Than Backstage

For the director Mark Romanek (“One Hour Photo,” “Never Let Me Go”), virtual reality has long been a fixation. About 25 years ago, he tried on a VR rig at a convention sponsored by the early cyberculture magazine Mondo 2000 and was disappointed by how huge and uncomfortable it was.

So in 1991, when he was directing the music video for “Love Conquers All” for the British pop band ABC, he jury-rigged his own contraption.

He covered a scuba mask in black spandex, paired that with a fake interface glove, and then concocted a 15-second piece of computer graphic imagery.

“I just sort of wished it into reality, even though the technology wasn’t there yet,” he said.

Last year reality caught up, when he was involved in an actual VR project with none other than Paul McCartney.

While he was talking to Mr. McCartney about collaborating on a short music film, the conversation turned to virtual reality. The former Beatle had never seen any footage, so Mr. Romanek asked the people at Jaunt VR, where Mr. Romanek is on the board of advisers, to supply a demonstration. Mr. McCartney was so enthralled, he urged them to film the concert he was performing the next day, at Candlestick Park in San Francisco.

Mr. Romanek, in Europe at the time, suggested camera angles to the Jaunt VR crew via FaceTime, and a performance of “Live and Let Die” was captured in 360-degree glory. Viewers start off right next to Sir Paul’s piano, where they can swivel to take in the rest of the stage or glance upward to find a sky filled with pyrotechnics. Moments later, they’re in the front row, but a glimpse behind reveals a crowd 70,000-strong.

Now, as he preps more ambitious mini-films, Mr. Romanek said he’s grappling with how to allow the viewer to feel that he or she is affecting the story in some way (often called “agency” in VR circles) while retaining some semblance of directorial control.

“When the viewer can look anywhere at any time, composition and montage goes out the window,” Mr. Romanek said. “Do you want Steven Spielberg or Alfred Hitchcock modulating your viewing experience, or your Uncle Morty?”

But a director’s willing and enthusiastic ceding of control may come to define the medium.

“I don’t think the question is: How do we make ‘The Godfather’ or ‘Jaws’ in VR?,” Mr. Romanek said. “I think it’s something else.”

“Who knows?,” he added. “In the end, we may discover that VR will turn out to be an essentially ineffective medium for narrative and be better suited to gaming, live events, news coverage or more purely ambient or fine art experiences.”

Inventing New Tricks of the Trade

As film students at Concordia University in Montreal last decade, Félix Lajeunesse and Paul Raphael studied the tools of the director’s trade. The zoom pan, the Dutch tilt, the extreme close-up.

Now, as virtual reality filmmakers whose Félix & Paul Studios has created works for Cirque du Soleil, Universal Pictures and LeBron James, they’ve had to forget, or retrofit, most of those techniques.

People inside a virtual reality universe are able to look in any direction, a freedom that can be disorienting at first. So filmmakers need to find new tricks to guide their gaze, essentially sprinkling breadcrumbs of sounds, images and transitions throughout the films.

“It’s like playing musical notes that didn’t exist up until a few years ago,” Mr. Raphael said.

Sound is the nudge in “Wild — The Experience,” a three-minute VR film that cemented the duo’s status as groundbreakers in the fledgling field. Based on the 2014 movie “Wild,” about a grieving woman hiking the Pacific Crest National Scenic Trail, the film places viewers alone in the wilderness until Reese Witherspoon’s character appears. Then, a voice materializes off to the side, out of sight. Prompted to turn, viewers will discover Laura Dern, who plays the mother of Ms. Witherspoon’s character. If you keep your eye on her for the rest of the experience, she’ll stick around. But turn back to Ms. Witherspoon, and Ms. Dern will be gone the next time you look over.

It’s a subtle bit of interactivity.

“We didn’t want to see her walk in or dissolve in,” Mr. Raphael said, referring to Ms. Dern’s character. “We wanted to hide that cut in a way you can’t do in a film.”

The filmmakers warn against succumbing to the temptations of the 360-degree frame and staging constant action all around. Grounding viewers in their surroundings, engineering a sense of presence, is more important. Also, a frantic, jittery camera, particularly when it moves out of sync with a viewers’ head motion, can induce nausea.

“An experience that’s not great in a movie theater is just a boring movie; an experience that’s not great in virtual reality can ruin the rest of your afternoon,” said the filmmaker Chris Milk, a leader in the field who’s worked on several virtual reality projects with The New York Times.

Mr. Lajeunesse and Mr. Raphael have incorporated the lessons of the past two years in their latest project, “LeBron James: Striving for Greatness,” which follows the basketball star’s off-season training regimen as he prepares for his 13th season. (The 13-minute virtual reality experience will be released next month.) Blending fly-on-every-wall scenes of Mr. James in the gym, on the court and at home with ones of him speaking directly to the viewer, the filmmakers were aiming to steer viewers between moments both objective and subjective.

“When you tell a story closer to the way the mind operates, suddenly it feels like things are more loose,” Mr. Lajeunesse said.

Journalists and the ‘Empathy Machine’

Nonny de la Peña was a correspondent for Newsweek. She’s written for The Los Angeles Times. She’s made documentary films.

But nothing in her journalism career had quite satiated the need that drew her to the profession in the first place: telling stories that would inspire people to truly care about inequality and human rights abuses. Then she discovered virtual reality.

Her first project, “Hunger in Los Angeles,” explored food insecurity through the scene at a church’s overwhelmed food bank. She had originally hoped to capture what it would be like for a mother in line to discover that the food had run out. But when an intern recorded harrowing audio of a man collapsing into a diabetic coma while waiting in line, she opted to tell that story instead, in a novel way.

Using about $700 of her own money, teaching herself computer coding and cadging favors from friends, she spent about two years recreating the scene in a seven-minute virtual reality experience. “Hunger” melded computer-generated animation of the people and environs with real audio, and allowed viewers to move around inside the story, even to kneel down to (futilely) help the collapsed man. In 2012, it made a splash at the Sundance Film Festival, where it was selected for the New Frontier showcase. Ms. de la Peña has focused on virtual reality ever since, with projects on the war in Syria and the plight of a migrant beaten and Tasered to death by the United States Border Patrol.

The theory behind this sort of immersive journalism, as it’s become known in academic and journalistic circles, is that the visceral nature of the experience makes a viewer a new kind of witness.

 

A two-dimensional view of “Hunger in Los Angeles.” Video by BradleyAllenNewman

“You really engage on scene in a way that gives you this incredible connection to where you are,” Ms. de la Peña said. “And that’s why, early on, I was calling it an empathy generator, an empathy machine.”

Ms. de la Peña may have been one of the first journalists to branch out into virtual reality, but she now has plenty of company. News organizations, including The Wall Street Journal, ABC, CNN, The Associated Press and Vice, have all done virtual reality projects. The New York Times signaled its commitment to the medium earlier this month when it released a virtual reality app with the premiere of “The Displaced,” an 11-minute documentary about the global refugee crisis produced with Mr. Milk’s company, Vrse.works.

Creating journalistic stories in virtual reality is expensive, often costing tens of thousands of dollars (or more), and some wonder if virtual reality will ever be more than a show horse to be trotted out occasionally for buzz.

But acolytes like Ms. de la Peña are convinced that virtual reality will become a regular part of news organizations’ storytelling arsenal, particularly as costs come down and the hardware improves.

“It is a natural extension for a generation growing up with gaming,” she said.

She believes the possibilities, though, are freighted with journalistic peril. “As much as or more than anything, this medium allows for propaganda and mistruth,” she said.

Filmmaker and subject often have more of a symbiotic relationship than in traditional video journalism, as the technical logistics require more coordination. And viewers, feeling as if they’re on the scene they’re watching, give virtual reality a credibility they may not give other media.

“What does transparency look like when you have goggles on?” she said. “I don’t know the answer, but it is something I think about a lot.”

Courtside Seats and Immersive Games

Late last month the reigning N.B.A. champion Golden State Warriors opened their season at a sold-out Oracle Arena in Oakland, Calif. Sitting courtside were the wealthy and connected who had shelled out thousands of dollars for the most coveted seats. And thanks to a funky looking double-lensed camera sharing that real estate, some hard-core, tech-savvy fans at home got to enjoy the action as if they were right there, too.

That was the first time a professional sporting event was broadcast live and nationwide in virtual reality. And it was no accident that the Warriors were playing.

The team’s co-owner, the longtime entertainment executive Peter Guber, is also a major investor in NextVR, the virtual reality company that filmed the home opener. He, like the owner of the Sacramento Kings, is betting that fans who crave live experiences and sponsors looking to tap that passion, will be willing to pay for the experience. What form those experiences will take — through broadcasts of complete games or packages of highlights, on a subscription basis or pay-per-view — will become clear in time.

“There are untold ways to mix the brew,” said Mr. Guber, a former chief at Sony Pictures and PolyGram before setting up his own company, Mandalay Entertainment. “I just drank the Kool-Aid.”

The lucrative future isn’t here yet, of course. People watching the Warriors game on their smartphones while wearing Gear VR headsets complained about jerky reception, no virtual scoreboard, and the sound and visuals sometimes failing to match up.

But perhaps the biggest hurdle to overcome is the uncomfortable headgear. The arriving batch of contraptions (see sidebar), while sure to be more comfortable, will not yet be conducive to wearing for entire sporting events — or for video games, another market that deep-pocketed companies are beginning to flood.

Mr. Guber and Tim Sweeney, the founder and chief executive of Epic Games, which in September released a virtual reality demo called Bullet Train, believe that it won’t take long for today’s oversized goggles to morph into glasses no bigger than a pair of Oakleys.

Bullet Train Unreal Engine 4 Demo – Oculus Connect 2 A two-dimensional demo of Bullet Train.

(For Mr. Sweeney, the biggest challenge for virtual reality game designers is creating incredibly photorealistic scenes. “When you’re looking at a computer screen, you have a high tolerance for cartoony artwork,” he said. “But when you’re in it, it just feels wrong.”)

Improving the actual viewing experience of virtual reality can only help exploit the possibilities in live events besides sports, according to Mr. Guber. Like awards shows.

In August, MTV livestreamed the Video Music Awards red carpet show in virtual reality and followed that up in October with the entire European Music Awards. The American Music Awards will probably experiment with virtual reality soon. Not coincidentally, Mr. Guber owns part of the company producing that show, Dick Clark Productions, which itself is also an investor in NextVR.

When it comes to virtual reality, Mr. Guber — and his wallet — are all in.

“It may not have the same trajectory that we plan for,” Mr. Guber said. “It may not become omnipresent like we believe it will. But it’s here, and it’s going to play a prominent role in more than film and entertainment.”

The Headsets to Experience It

In the coming months, virtual reality will get its biggest mainstream push yet, as companies like Samsung, Sony and HTC release new headsets likely to be accompanied by huge marketing campaigns. Here’s a look at some of the major players’ offerings.

Samsung Gear VR

A collaboration between Oculus, the virtual reality company bought by Facebook, and Samsung. Powered by smartphones (though just high-end Samsung Galaxy models), the Gear VR headset offers a portable virtual reality experience. Available on Nov. 20, costing $99.99.

Oculus Rift

Powered by a PC, making it more robust than the Gear VR. Its positional technology gives users a wider range of physical interaction with the virtual environment, allowing them to crouch down and dodge bullets, for example. Available first quarter of 2016, pricing to be announced.

HTC Vive

Developed in conjunction with Valve, the creators of video games like Portal. The headset plugs into a PC, and two base stations encourage users to move around a 15-foot by 15-foot area as their actions are replicated in the virtual environment. Available first quarter 2016, pricing to be announced.

PlayStation VR

Designed to work with the PlayStation 4. Unlike the other entries, it creates two sets of images: one for the headset and one for a TV, so virtual reality can be more of a communal experience. Available first half of 2016, pricing to be announced.

Google Cardboard

The simplest — and most affordable — way to experience virtual reality. A foldable cardboard mount with plastic lenses and a fastening device, into which a smartphone is slotted horizontally, it requires a compatible app. Available now for as low as a few dollars, and often given away through corporate promotions (including one by The New York Times).

A version of this article appears in print on November 22, 2015, on page AR1 of the New York edition with the headline: And You Are There. Order Reprints