The upcoming Pixel Cloud plugin for After Effects can be used to relight a source image affecting the diffuse and specular properties. Coming soon I’ll be posting a revealing demo of how easy it is to take a static image from a 3D package and use Pixel Cloud and After Effects’ own 3D lights to create a dynamic and believable composite with moving lights, shadows, and reflections. Here’s an end product of what it will look like!
3D model courtesy of Stanford 3D Model Repository
Pixel Cloud has a robust set of relighting tools. These include relighting tools for diffuse, specularity, reflections and the ability to relight the “Alpha” of a 3D displaced image using After Effects own lights. In Pixel Cloud we can specify lights that only affect the alpha channel of an image. Since the image is a 3D displaced Pixel Cloud, this means we can easily separate parts of an image according to their depth information. This is a powerful tool and a normal map isn’t a prerequisite, meaning you can relight non-CG generated images like photos and video. The video isn’t up yet but as a taste here is a preview of what the upcoming demo/tutorial will show how we can utilize this tool to relight the alpha of Jack, our cute little dog from the previous demo!
I just received my Lytro and have been having a blast taking photos with it. It’s absolutely brilliant. If you’re unfamiliar with this camera, it allows you to focus the picture after it’s already been taken. The move from in-camera to in-post is certainly being lead by technology and economics. My Pixel Cloud plugin is also trying to bring more of those capabilities into After Effects. Just look at the Microsoft Kinect and how innovative pioneers are using its ranging features to recreate environments from recorded point clouds. And Samsung has developed a sensor that not only records RGB but Depth pixels as well! These are amazing innovations for effects artists.
Imagine a future where we can record whole rooms as animated environments. Think 3D scanners that scan whole rooms at one time and at 24fps. We could completely eliminate conventional camera motion control and create everything in-post. We could change the lighting setups, create digital camera rigs all after the video has already been shot. Not as CG but as recorded pixels in 3D space; An accurate representation of reality that we can manipulate to our choosing. This opens up possibilities for interactive story-telling, not to mention subjective 3D stereography. I could imagine a dozen more uses.
This isn’t just virtual reality but recreated reality, “Recreality.”
[CORRECTION] Although we use the term depth map, the correct term for usage should be height map. The main difference being that a height map denotes distance from a flat surface and a depth map denotes distance from the camera.
I’ve shown how we can use a CG render and a Position Pass in After Effects to animate and relight a 3D displaced Pixel Cloud. But not all 3D programs can produce a Position Pass and photographic sources obviously do not come with specialized passes. Pixel Cloud can still get around these limitations.
Although not yet released, this demo/tutorial gives a quick look at how simple it is to use the Pixel Cloud plugin to create a realistic camera animation with a simple photograph. Pixel Cloud can use not only Position Passes but Depth Map passes as well within an 8-bpc project.
So keep an eye out for more updates on how close we are getting to release Pixel Cloud!
After a brief hiatus from postings, we’ll soon be coming back with a demo/tutorial on how to use the upcoming Pixel Cloud plugin to animate a photograph using depth maps and 3D displacement! It really is an amazing difference from just using flat 3D layers to simulate parallax. We’ll also be coming out with a couple of new tutorials with integrating this with a camera projection workflow as well as tips on how to get the most out your composites!
Here’s a look at the finished product looks like!
I’m excited to say that there will indeed be relighting functionality built into the Point Cloud plugin. Still in Beta, I’m happy to show off a little bit of what this relighting capability can do for your workflow! I’m still looking for a couple of beta testers who are serious and will be committed to helping me bring this plugin to release. In particular, I’m looking for Windows CS4 and CS5 After Effects users who use 3DSM and/or Maya and will be able to help me provide animated examples of relighting 3D renders in an After Effects workflow. You should be familiar enough to provide a workflow for generating position as well as normal passes. Please reply to betaprogram@blurrypixel.com.