A $2.6 Million Bet To Turn Your Home Into A Giant Screen

Creatives from Microsoft, Disney, and Adobe are teaming up to launch a consumer product that could bring projection mapping into your home.

The technology enables a projector to wrap an image across a 3D environment like a pixelated skin. Perhaps you remember a Microsoft project called Illumiroom, which stretched your TV to become the size of a wall. Or maybe RoomAlive rings a bell, which turned 3D spaces into giant, glowing touch screens. Or then again, it’s possible that you just saw this super crazy demo where a Kuka robot arm acts like it’s David Copperfield. These are some of the most impressive feats of projection mapping in the last five years—and they were all created by the team behind Lightform.

Lightform is a small box, priced “less than most laptops,” that can be plugged into any off-the-shelf projector, automatically scan your room in a minute, and coat it with digital images and animated textures. Whereas projection mapping of yore has required complicated custom software, 3D scanners like the Microsoft Kinect, and powerful PCs to crunch all the numbers, Lightform is built to be simple enough for almost anyone to use, whether to create dazzling effects on a concert stage or turn your entire living room into a planetarium.

“If you can use Photoshop, you can use Lightform,” promises cofounder Brett Jones, who was also the lead researcher on Microsoft Illumiroom. “We want to create instant magic.”

But the designers aren’t saying much about the software. Nor are they revealing more about their hardware, at least not until Lightform goes on sale this summer. All we really know is that Lightform uses a high-resolution camera system to analyze a room’s geometry, and handles all the calculations on board. It will also use some form of AI to automatically map a room in detail—and it will do so with enough intelligence that it can adjust its own map of a room if you move an object inside of it.

Lightform’s developers have shown demos that paint a room in rippling waves, and another that puts daily specials onto a restaurant menu. And it’s very easy to imagine Lightform simply bending a high-resolution movie around your living room, turning one entire wall, complete with side tables and book cases, into a seamless screen.

Unlike Room Alive, however, it will not create a fully interactive environment where you can touch the walls like some giant iPad screen. Instead, Jones suggests that the box is a more like a remote, or even a MIDI controller. A MIDI controller is something like a keyboard, or similar electronic instrument. And while that might sound strange, it makes sense: Projection mapping is common in stage performances. Lightform’s initial market may find a foothold in the music scene.

The company (also named Lightform) has big ambitions. The team has successfully raised $2.6 million so far from Lux Capital and Seven Seas, the former principal scientist at Oculus, and the National Science Foundation. And it has already put three years of work into the system. The team is pegging its invention, not as the world’s first turnkey projection mapping system, but as an alternative to bulky, augmented reality headsets, which are coming from companies like Microsoft and Magic Leap, and promise to place holograms right in front of your eyes.

“We are calling it a computer for ‘projected augmented reality’ because we think this is more than just regular projection mapping,” says Jones. “We make projectors understand their environment, and make effects and content adapt to the 3D environment. It is closer to Hololens than to traditional projection mapping tools.”

Indeed, companies like Argodesign have long argued that it’s environmental UI—not augmented reality—that will find a place in our homes. It’s what can happen when every surface can double as a screen, like cutting boards that list the next steps to a recipe, or a child’s room that can turn into a whimsical castle at a moment’s notice. The technology to make it all possible has actually existed for decades, but it’s been far too expensive and difficult for anyone but a few select researchers to wield.

Lightform could usher in a sort of augmented reality for the rest of us. Or at the very least, it could make your next Superbowl party very, very cool.