Time-Lapse Mining: Watch the World Change Before Your Eyes!

The estimated reading time for this post is 4 minutes

While you’ve been busy living your daily life, scientists have been busy mining public photos to give you a new view of the world.

You can see the effect of the seasons, construction, geological changes, water currents, erosion, water evaporation, excavations, and more over years with a new technology for creating time-lapse photography.

Time-Lapse Videos

Millions of pictures are taken every year, but now scientists have combined publicly available photos to give us a new perspective of how our world is changing over time. Ricardo Martin-Brualla, David Gallup, and Steven M. Seitz from the University of Washington working with Google have developed a program that has combed through over 86 million photos to create videos showing the progression of time in various environments.

The scientists call this process Time-Lapse Mining and have gathered publicly available photos from websites such as Picassa, Flickr and others to create their videos. They studied the photos for perspective, time and location stamps. They then tried to accumulate the photos that were taken from the same viewpoint and order them in time to create the time-lapse videos.

You can read a detailed explanation of the process in their paper: Time-lapse Mining from Internet Photos:

We introduced an approach to mine time-lapses from Internet photos. Our system discovered 10,728 time-lapses that show how the world’s most popular landmarks are changing over time. Our method stabilizes the time-lapse video sequence so that the underlying changes in the scene become visible. The depicted changes include buildings under construction, glaciers retreating, plants growing, seasonal changes, and many geological processes. The scale and ubiquity of our mined time-lapses creates a new paradigm for visualizing global changes. As more photos become available online, mined time-lapses will visualize even longer time periods, showing more drastic changes.”

Over 120,000 landmarks were analyzed to find ones that had many publicly available photos suitable for combining into time-lapse videos. Cities, glaciers, waterfalls, construction projects and other places have been the subject of these videos.

The scientists gathered the 1000 best photos of each landmark, trying to achieve consistent views, photo resolutions and exposure. Their computer then took six hours to combine the photos into a time-lapse video that made it look like a single camera recorded the video over time.

Here are some examples of the project’s time-lapse videos:

This slideshow requires JavaScript.

“This capability is transformative; whereas before it took months or years to create one such time-lapse, [with our algorithms] we can now almost instantly create thousands of time-lapses covering the most popular places on earth. The challenge now is to find the interesting ones, from all of the public photos in the world.”

This video by Martin-Brualla shows the process of creating the time-lapse videos from photos:

Each time-lapse video includes a timeline at the bottom showing the progression of time.

From the paper:

“Our time-lapses cover a broad range of interesting transformations:

• Construction: from individual buildings to whole skylines. The time-lapse of the Goldman Sachs Tower,

• Changing cities: smaller changes in the appearance of cities, like billboards or changes in urban elements, like sidewalks, etc.

• Vegetation: plants and trees growing, like the trees in the Butchart Gardens,

• Waterfalls: we found that waterfalls are constantly changing, as branches dry up and new ones appear,

• Renovations: monuments being renovated, like the Basilica of St. Maria of Salute in Venice,

• Seasons: seasonal changes, like the blooming cycles of the flowers in Lombard Street,

• Geological changes: retreating glaciers, erosion or the growth of a hot spring in Yellowstone due to the deposit of minerals.

• Stationary: some scenes are interesting because of how little they change. For example, the Swiss Guard is so still, that it becomes part of the time-lapse of an entrance to the Vatican.”

Also see ⇒ Prosthetic Knowledge, a Tumbler site with time-lapse images from the project.

Refining

The scientists went a step beyond regular time-lapse photography by stabilizing the images so they appear seamless.

Note how the video shows you the constant background sky on the right, as opposed to the flickering of a regular time-lapse video on the left.

The Math

If your kids ever ask you what they will ever do with the math that they’re learning, point them to the paper that shows the formulas behind this Time-Lapse Mining Project. Show them algorithms used in the creation of this project, such as this one on page 4 of the paper:

where δ(·) is a loss function, λ is a temporal smoothing coefficient and xi = ∅ when p corresponds to pixel coordinates outside the image boundary of Ii, i.e., has no correspondence in Ii.

That should give them plenty of incentive to keep studying!

Failures

Not every time-lapse video is a success. The YouTube video above shows you some failures where the images blur or the change in lighting creates an artificial effect.

This time-lapse shows the Wall Street Bull statue moving over time.

The bull doesn’t really move, at least not that I’ve observed…

Your Thoughts

Do you like to see changes in the world over time? Which landmark would you like to see a time-lapse video of? Can you figure out math problems to create your own time-lapse videos? Share your thoughts with us in the Comments section below!

________________

* Waterfall image courtesy of Michael Matti via Flickr and Creative Commons

** Time-lapse images courtesy of University of Washington via Creative Commons