Skip to main content
Part of complete coverage on

A camera that can see around corners

By Ramesh Raskar, Special to CNN
August 19, 2012 -- Updated 1305 GMT (2105 HKT)
STORY HIGHLIGHTS
  • Ramesh Raskar's team at the MIT Media Lab is re-imagining what photography can do
  • By creating movies of light in motion at a trillion frames per second, camera can see around corner
  • Raskar: Technology is years away from applications in the real world
  • He says it could be helpful in preventing car crashes and aiding rescues

Editor's note: Ramesh Raskar is Associate Professor at MIT Media Lab and head of the Camera Culture research group. He is an Alfred P. Sloan Research Fellow (2009), a recipient of the DARPA Young Faculty award, the co-author of "Spatial Augmented Reality", and the holder of more than 50 US patents. He spoke at the TEDGlobal conference in June. TED is a nonprofit dedicated to "Ideas worth spreading" which it makes available through talks posted on its website.

(CNN) -- A camera that can see around a corner?

I know this sounds like something in a sci-fi movie or a superhero comic, but this is a real-world technology we've made possible with a camera that is aware of the travel time of light, an imaging technique that can create movies of light in motion with an effective rate approaching a trillion frames per second: the speed of light.

Before I joined the MIT faculty in 2008, I had done deep research in "computational photography," a field of new imaging techniques dramatically improving the capture and synthesis of photos. But, I knew there was more to photography than just depicting what the eye can see. I wanted to create a camera that could see beyond the line of sight. The speed of light isn't infinite: light travels about a foot per billionth of a second.

Photography could make X-rays obsolete

If I could build a camera fast enough to analyze light at high speeds in room-sized environments, I knew we could then create cameras to solve major problems in scientific and consumer imaging, and enable completely new functionality.

I spoke to top researchers in ultrafast lasers and photonics to understand what was currently possible. When I did, most of them asked some version of: "Why? Why spend years building a camera to look around corners when no commercial application is screaming for it and no funding agency has a call for it?" In addition, it's rare to shoot light pulses and analyze at such high speeds in large environments. Ultra-fast imaging experiments are usually limited to centimeter- or smaller-size samples.

I continued the work and in the spring of 2008, with James Davis from UC Santa Cruz, wrote a proposal that laid out the mathematical foundation and various experimental solutions for exploration.

TED.com: Boaz Almog 'levitates' a superconductor

I also began working with Media Lab graduate student Matthew Hirsch to build a prototype, hoping that we would have something to demonstrate within a few months. The grant proposal was rejected for administrative reasons (I made a paperwork error!), which meant we had to wait nearly a year to apply again. But those two years didn't yield any meaningful results, as our lab components weren't designed to be used the way we wanted.

After nearly three years of experimental work, the team -- especially postdoctoral associate Andreas Velten and MIT professor Moungi Bawendi, many students, and several collaborators -- cobbled together pieces of the puzzle and built a software program to create a first demonstration of looking around corners. Very soon afterward, we could also start creating surreal movies of light in motion.

We call this new imaging technology femto photography because we capture a segment of the image with a flashlight (in this case, a laser pulse) on for a few millionths of a billionth of a second (or a few femtoseconds) and an exposure time approaching a trillionth of a second.

Just how fast is femto photography? Think of it this way: if we took one-thousandth of a second of footage from the femto camera video output and slowed it down to the speed of 30 seconds per frame -- the approximate speed of a standard TV broadcast -- it would take us a lifetime to watch.

Photographers know that at very short exposures and even at the most sensitive setting for dark scenes, we will record barely any light. So what about in a trillionth of a second? We actually record and average millions of photos to get enough light, each photo made to look the same via carefully timed synchronization with the light pulse. So even if our exposure time is indeed nearly a trillionth of a second, to get sufficient light we must take an average. Thus, as of now, we can only record repeatable events, but this is not a fundamental limitation.

TED.com: A robot that flies like a bird

Unlike conventional cameras, our femto camera captures an image as one thin slice at a time of one-dimensional space using a "streak tube," a laboratory instrument that is commonly used by chemists to study light passing through chemical samples. We then take hundreds of these narrow videos (each shot at a slightly different angle) and create a carefully synchronized, slow-motion composite of the light pulses. It takes about an hour to collect and aggregate the data (view a demonstration of a light pulse as it travels through an ordinary Coca-Cola bottle).

To see around the corner, we use femto photography to analyze scattered light. We bounce light off of visible parts into hidden parts and then measure the time and direction of returned light.

Usually the scattering of energy is considered a nuisance -- whether driving in fog or poor reception from a cell phone tower -- and most techniques either try to avoid it (by turning on fog lights) or reduce the impact of scattering (by selecting energy for the phone only from direct paths).

In contrast, we exploit the scattering. For the camera, a laser pulse is fired at a wall, and the impact of hitting the wall causes the particles of light to scatter. Some of the scattering particles return to the camera at different times. This is repeated about 60 times per image as the camera measures how long it takes for the light to travel back and where the particles land. An algorithm then crunches the data to reconstruct the hidden image. This technique even allows us to see a three-dimensional object such as a mock-up of running person.

TED.com: Test for Parkinson's with a phone call

As exciting as this work is, don't look for this technology to be in practice tomorrow -- we're still years away from bringing this to market. But, we can already imagine multiple ways that it could have a significant, positive impact on our everyday lives.

By allowing us to "see" around a corner, for example, this technology added to our cars could let us know if there's another vehicle approaching around a blind curve. It also could give us a new way to look deep inside our bodies without X-rays, or to look through a window into a burning building from a distance to see if anyone is left inside --without risking a firefighter's life.

When I gave a TEDGlobal talk on femto photography in June, I began with a reference to Doc Edgerton, a very popular MIT professor of electrical engineering who, in 1964, wowed the world with an image of a bullet in midair, having just passed through an apple. He accomplished this by using a strobe light to freeze the action of the bullet at a millionth of a second.

What we're talking about here -- the speed of light --is a million times faster, and is opening the door to a complete rethinking of what we mean by, and can do with, photography. It is a first step toward a new world of imaging that far exceeds human ability, resynthesizing data and depicting it in ways that are within human comprehensibility.

Follow @CNNOpinion on Twitter

Join us at Facebook/CNNOpinion

The opinions expressed in this commentary are solely those of the author, Ramesh Raskar, who would like to thank his collaborators: Moungi G. Bawendi, Professor, MIT Department of Chemistry; Andreas Velten and Christopher Barsi, postdoctoral associates, MIT Media Lab; Everett Lawson, Otkrist Gupta, Nikhil Naik, Amy Fritz, Di Wu, MIT Media Lab; Matt O'toole, Kyros Kutulakos, University of Toronto; Diego Gutierrez, Belen Masia, and Elisa Amoros, Universidad de Zaragoza; Kavita Bala, Shuang Zhao, Cornell University; Ashok Veeraraghavan, Rice University.; Thomas Willwacher, Harvard University.

ADVERTISEMENT
Part of complete coverage on
October 21, 2014 -- Updated 2006 GMT (0406 HKT)
Timothy Stanley says Lewinsky is shamelessly playing the victim in her affair with Bill Clinton, humiliating Hillary Clinton again and aiding her critics
October 21, 2014 -- Updated 0102 GMT (0902 HKT)
Imagine being rescued from modern slavery, only to be charged with a crime, writes John Sutter
October 21, 2014 -- Updated 1600 GMT (0000 HKT)
Tidal flooding used to be a relatively rare occurrence along the East Coast. Not anymore, write Melanie Fitzpatrick and Erika Spanger-Siegfried.
October 21, 2014 -- Updated 1135 GMT (1935 HKT)
Carol Costello says activists, writers, politicians have begun discussing their abortions. But will that new approach make a difference on an old battleground?
October 21, 2014 -- Updated 1312 GMT (2112 HKT)
Sigrid Fry-Revere says the National Organ Transplant Act has caused more Americans to die waiting for an organ than died in both World Wars, Korea, Vietnam, Afghanistan and Iraq
October 21, 2014 -- Updated 1851 GMT (0251 HKT)
Crystal Wright says racist remarks like those made by black Republican actress Stacey Dash do nothing to get blacks to join the GOP
October 21, 2014 -- Updated 2207 GMT (0607 HKT)
Mel Robbins says by telling her story, Monica Lewinsky offers a lesson in confronting humiliating mistakes while keeping her head held high
October 20, 2014 -- Updated 1329 GMT (2129 HKT)
Cornell Belcher says the story of the "tea party wave" in 2010 was bogus; it was an election determined by ebbing Democratic turnout
October 20, 2014 -- Updated 2012 GMT (0412 HKT)
Les Abend says pilots want protocols, preparation and checklists for all contingencies; at the moment, controlling a deadly disease is out of their comfort zone
October 20, 2014 -- Updated 0336 GMT (1136 HKT)
David Weinberger says an online controversy that snowballed from a misogynist attack by gamers into a culture war is a preview of the way news is handled in a world of hashtag-fueled scandal
October 20, 2014 -- Updated 1223 GMT (2023 HKT)
Julian Zelizer says Paul Krugman makes some good points in his defense of President Obama but is premature in calling him one of the most successful presidents.
October 20, 2014 -- Updated 0221 GMT (1021 HKT)
Conservatives can't bash and slash government and then suddenly act surprised if government isn't there when we need it, writes Sally Kohn
October 22, 2014 -- Updated 1205 GMT (2005 HKT)
ISIS is looking to take over a good chunk of the Middle East -- if not the entire Muslim world, write Peter Bergen and Emily Schneider.
October 20, 2014 -- Updated 1300 GMT (2100 HKT)
The world's response to Ebola is its own sort of tragedy, writes John Sutter
October 17, 2014 -- Updated 2033 GMT (0433 HKT)
Hidden away in Russian orphanages are thousands of children with disabilities who aren't orphans, whose harmful treatment has long been hidden from public view, writes Andrea Mazzarino
October 18, 2014 -- Updated 1722 GMT (0122 HKT)
When you hear "trick or treat" this year, think "nudge," writes John Bare
October 18, 2014 -- Updated 0442 GMT (1242 HKT)
The more than 200 kidnapped Nigerian schoolgirls have become pawns in a larger drama, writes Richard Joseph.
October 17, 2014 -- Updated 1345 GMT (2145 HKT)
Peggy Drexler said Amal Alamuddin was accused of buying into the patriarchy when she changed her name to Clooney. But that was her choice.
October 16, 2014 -- Updated 2043 GMT (0443 HKT)
Ford Vox says the CDC's Thomas Frieden is a good man with a stellar resume who has shown he lacks the unique talents and vision needed to confront the Ebola crisis
October 18, 2014 -- Updated 0858 GMT (1658 HKT)
How can such a numerically small force as ISIS take control of vast swathes of Syria and Iraq?
October 17, 2014 -- Updated 1342 GMT (2142 HKT)
How big a threat do foreign fighters in Syria and Iraq pose to the West? It's a question that has been much on the mind of policymakers and commentators.
October 17, 2014 -- Updated 1221 GMT (2021 HKT)
More than a quarter-million American women served honorably in the Iraq and Afghanistan wars. Now they are home, we have an obligation to help them transition back to civilian life.
October 16, 2014 -- Updated 2027 GMT (0427 HKT)
Paul Begala says Rick Scott's deeply weird refusal to begin a debate because rival Charlie Crist had a fan under his podium spells disaster for the Florida governor--delighting Crist
October 16, 2014 -- Updated 0407 GMT (1207 HKT)
The longer we wait to engage on Ebola, the more limited our options will become, says Marco Rubio.
October 15, 2014 -- Updated 1153 GMT (1953 HKT)
Democratic candidates who run from President Obama in red states where he is unpopular are making a big mistake, says Donna Brazile
October 16, 2014 -- Updated 0429 GMT (1229 HKT)
At some 7 billion people, the world can sometimes seem like a crowded place. But if the latest estimates are to be believed, then in less than a century it is going to feel even more so -- about 50% more crowded, says Evan Fraser
October 20, 2014 -- Updated 1653 GMT (0053 HKT)
Paul Callan says the Ebola situation is pointing up the need for better leadership
October 15, 2014 -- Updated 2245 GMT (0645 HKT)
Nurses are the unsung heroes of the Ebola outbreak. Yet, there are troubling signs we're failing them, says John Sutter
October 15, 2014 -- Updated 1700 GMT (0100 HKT)
Dean Obeidallah says it's a mistake to give up a business name you've invested energy in, just because of a new terrorist group
October 15, 2014 -- Updated 2301 GMT (0701 HKT)
Fear of Ebola is contagious, writes Mel Robbins; but it's time to put the disease in perspective
October 14, 2014 -- Updated 1744 GMT (0144 HKT)
Oliver Kershaw says that if Big Tobacco is given monopoly of e-cigarette products, public health will suffer.
October 18, 2014 -- Updated 1335 GMT (2135 HKT)
Stop thinking your job will make you happy.
October 15, 2014 -- Updated 0208 GMT (1008 HKT)
Ruben Navarrette says it's time to deal with another scandal involving the Secret Service — one that leads directly into the White House.
October 14, 2014 -- Updated 1125 GMT (1925 HKT)
Americans who choose to fight for militant groups or support them are young and likely to be active in jihadist social media, says Peter Bergen
October 13, 2014 -- Updated 1303 GMT (2103 HKT)
Stephanie Coontz says 11 years ago only one state allowed same sex marriage. Soon, some 60% of Americans will live where gays can marry. How did attitudes change so quickly?
October 14, 2014 -- Updated 2004 GMT (0404 HKT)
Legalizing assisted suicide seems acceptable when focusing on individuals. But such laws would put many at risk of immense harm, writes Marilyn Golden.
October 13, 2014 -- Updated 1307 GMT (2107 HKT)
Julian Zelizer says the issues are huge, but both parties are wrestling with problems that alienate voters
October 13, 2014 -- Updated 2250 GMT (0650 HKT)
Mel Robbins says the town's school chief was right to cancel the season, but that's just the beginning of what needs to be done
October 11, 2014 -- Updated 1543 GMT (2343 HKT)
He didn't discover that the world was round, David Perry writes. So what did he do?
ADVERTISEMENT