Sunday, December 30, 2012

Rethinking Time-lapse

Since I got my hands on the HERO3 Black Edition, I've been doing significantly more time-lapse.  This is not because the stills are so much better than HERO2, and they are, it is due to the new video modes at 2.7K and 4K, combined with Protune. There is no need to time-lapse with JPEG sequences with this camera, unless you know you need a very long interval.  Most time-lapse shoots that document human experiences, are better with shorter intervals between 0.5 and 10 seconds. Knowing what interval is best takes practice, but forget that! With the new camera shooting 4K at 12 or 15fps, basically a continuous 9MPixel motor drive sequence (stored in an MP), time-lapse guess work can be left to post. Resampling 4K video to your needed interval is a straightforward process in most video tools, and a standard feature in the free GoPro CineForm Studio software.

At 4K you get most of the spatial resolution of the still mode, at 6+ times the temporal resolution (12/15fps vs a maximum of 2fps for JPEG) at approximately half the data rate.  So there is a data rate saving using the video modes for simulating the shorter intervals, but for long intervals there are still reasons to shoot ultra-HD video over stills -- simulated longer shutter intervals.  The HERO cameras mainly use the shutter speed to control exposure, which is fine for high action moments, but for scenes best for time-lapse a fast shutter may not be desirable.  With a DSLR, you can stop down the camera's aperture, but that only gets you so far. For time-lapse exposures of 5 seconds of more, that would require a lot of neutral density filters for daylight shooting.  For a HERO camera, aperture control is not available, and adding neutral density is highly impractical, so we need do the camera operation in post.

So let's say you want to simulate a 5 second exposure with a 10 second interval in full daylight (simulating a 180 degree shutter at play speed.)  HERO's default exposure might be around 1/1000th of a second in strong daylight, nowhere near the 5 second exposure target.  Yet the camera could be recording 4K at 12fps over those 5 seconds, collecting 60 individual frames.  If you average those 60 frames, you get very close to the look of a single long exposure from a DSLR with a hell of a lot of ND filtration, without the setup headache. Typically blending over 30 frames for daylight simulates the motion blur of a single exposure.  With darker shots that might have the camera's shutter exposure near 360 degrees (1/30th for 30 fps video), far fewer frames can be blended for a natural look.  Of course, the more frames used in averaging, the smoother the results.  I have been asked how a GoPro achieved this high action shot with so much motion blur:

Now you know.  This was shot 1920x1440 at 24p, with 30 frames averaged for each single frame in the time-lapse output.

Continuing with the target of 5 second exposure and with 10 second interval, I was intending to model 180 degree shutter, however the CineForm Studio software with the Motion Blur enabled will simulate 360 degree (this was by design.)  So setting "SPEED UP" to 60, "FRAME BLEND (MOTION BLUR)" on and the output frame rate to 23.976p, the result will be a clip with 5 second exposure and a 5 second interval.
To get this to simulate a 10 second interval simply place it in your editing tool's timeline and double the playback speed (with frame blending off.)  Now every other 5 second exposure will be displayed for 180 degree shutter emulation.

Protune helps greatly, particularly in low light. Now that we are averaging frames together, we get an excellent side effect:  a large reduction in noise. Each doubling of the number of frames averaged will half the noise in the image. Combined with Protune, which preserves much more shadow detail, you can basically see new details that would normally be lost to noise with regular video or stills time-lapse.  Protune lifts the shadow detail so that it is no longer crushed to black. In standard mode, averaging crushed black only results in more crush black, yet in Protune averaging a noisy shadow detail results in more shadow detail.

I've used this technique in most of my recent videos, such as this one (the night time-lapses are very clean, because of HERO3 Black and this averaging technique using CineForm Studio):

24 Hours of Lemons at Chuckwalla Dec 2012. from David Newman on Vimeo.

Update Jan 5/2013: Example comparing classic and video blended time-lapse

P.S. For those who have been following my Instagram feed (http://instagram.com/0dan0) or via Twitter (@David_Newman), you are likely aware that I've been combining the above time-lapse technique with a motion controller I have been experimenting with.  This combined a GoPro with a 3D printed motion controller that runs on toy G-scale train track. I've just posted its design on thingiverse.com, for this project. Let me know if you successfully build one, and link me to your videos.

22 comments:

ES.Enrique said...

You state taht "(the night time-lapses are very clean, because of HERO3 Black and this averaging technique using CineForm Studio)"; but i have not been able to locate a night time lapse.
Could you please show us your Hero3 night time lapses?
Many thanks.

David said...

The middle for the Lemons clip shown above is shot in the middle of the night. All the car engine work.

The end of this clip is night-time time-lapse concert footage. All very clean.
https://vimeo.com/52572663

This was all shot at dusk, between 45-60 minutes afer sunset. https://vimeo.com/56884431

adzix said...

Excellent read, thanks.
I've got a question though.
Just recorded some footage in 4K for a timelapse. I import it in CineForm and it plays back just fine. Set it to your suggested conversion settings, but from there on things get wonky.
I can't play back the converted file in the edit view, I see it, it shows up fine, I can color correct it according to the frame I see, but if I hit the play button, nothing happens.
Also when I want to export it, the exporting bar never gets over the 0% mark. Any clue what might be the issue? Thank you.

David said...

That is not an error I'm familiar with. If you scale to 1440p or smaller, does it play? Maybe you system is hitting some memory limit. Also try change the default playback resolution from full to half or quarter.

adzix said...

Hello, just in case you're interested, I managed to solve it: my system has some pretty ok specs, the problem was elsewhere.
I searched various forums and found someone with the same issue. (the problem also occured when only working with 1080p or smaller).

I re-installed the latest version, now everything works like a charm.
Love how your settings make my timelapses look so much smoother. thanks.

One more question: what do you do when you have multiple files? Can you combine them within GoPro Cineform or do you use some other software?

David said...

I used something that is likely in your future, not yet released software.

Simon said...

Interesting article. When the GoPro uses a fast electronic shutter in bright light, does this mean that if I record footage in 1080/50 (50hz country) I could use Cineform Studio to perform motion blending to output a 25p sequence much closer to what I would get with a shutter set to 1/50th instead of 1/100th?

Obviously this will depend on the situation as if the GoPro goes to 1/500th of a second then even with motion blending it would only be the equivalent of 1/250th. But it would be interesting to know if this holds true?

David said...

Simon, 2 frame to 1 with motion blur will only work (somewhat) if the shutter is fully open (360 degree shutter in film terms.)

Anonymous said...

I love your timelapses. What would you suggest when making one with hero 3 silver?

David said...

HERO3 Sliver Edition is a great time-lapse camera, using stills or video technique above. Only limitation is in very low light conditions where Black Edition can't be beat. This time-lapse video was from HERO2 (uses the same sensor as Sliver Edition) https://vimeo.com/51877097 I shot 1080p24 with Protune on for most of this.

Unknown said...

This might be a dumb question, but what's the advantage of taking the video at 4K rather than just taking a 1080p video at a higher framerate and turning it into a timelapse? You're going to output a 1080p video at the end anyway so spatial resolution doesn't matter, and you have more frames to blend for motion blur.

David said...

The 4K is for post reframing. With 4K source you can do a 2X post zoom and still have 1080p resolution. I tend to find that overkill and use 2.7K for video and timelapse.

tom said...

Making a timelapse using a motion control unit and using this technique to add motion blur in post (like most other post blur methods), would it not add blur to the movement of the camera as well ?

Ive used an egg timer to do a few rotational gopro timelapses and whenever I try to use motion blur in post it makes the entire frame blur as the camera rotates as well as the subjects in the film, albeit not as strong but its definitely noticeable.

David said...

Tom,

Absolutely all motion is blurred, the trick is to have your rig moving slow enough compared to the subject. See
http://instagram.com/p/Y8ys_cLrGi/#
and
http://instagram.com/p/Y8w5kIrrC_/#
For linear motion, you have less of a problem, but the panning of egg timers can be too fast. I was advised by the GoPro Media team to use water timers (like those linked), they have a two hour cycle rather than the one of most egg/kitchen timers.

tom said...

aha, thanks for the info, David. I will give that a shot.

I have got one these on order with the dolly system which iam hoping will give me slow and fluid movement both linear and rotational for gopro tlapses.

http://www.kickstarter.com/projects/665986460/spinpod-seamless-panoramic-and-motion-time-lapse-p

Im guessing you're beta testing the cineform 2.0 software? anything new and cool for us timelapsers ? :-)

DanH said...
This comment has been removed by the author.
carter said...

Thanks for the helpful posts. I'm going to do some tests on my Hero 3 black - planning to shoot a bunch of driving TL for a credit sequence.

In the comparison video on this blog, did you use the same settings as described in the article above? 4K @ 12fps, then "speeding up" to 60 on conversion with frame blend?

I like the results in the video and wanted to use those settings as a starting point for my tests. Many thanks in advance!

David Newman said...

In the test, 30p was blended around 30x and played at 30fps - 1 shoot per second emulation. For 4kp12, played at 30, you will have a 2.5X speed up with any blend. To get 1s emulation, you would use a blend of 12X. This would look okay, but it might be better to should 2.7Kp30 with 30X blend.

Carter said...

Thanks, David. I will use those specs in my TL driving tests.. 2.7kp @ 30 and speed up/blend x 30.

I forgot to mention that my final output needs to be 1080p 23.98fps. Does this then dictate a better/different spec?

Thanks!

David Newman said...

2.7k 30p source with a 30X speed/blend, presented at 24p will look excellent. The extra advantage of use video for time-lapse, is the time speed-up doesn't look prefect you and easily adjust it.

Anonymous said...

I shot a beautiful winter sunset at 4k. I need to speed it up considerably to fit in video timeline. Sped up to 120x and blurred works best and looks very dramatic, but what framerate should I use? The master video will be 30fps.

David Newman said...

Set the new video to match (29.97)