A recent post on FXGuide regarding Pointcloud9, a European company that provides high quality 3D scanning services to the film industry, has me fascinated with how this technology is being used today. LIDAR is basically the process of using a laser to get the 3D information of an object or environment similar to the way desktop 3D scanners operate and it ties in perfectly with my previous Recreality post, about the future of cinema. These laser-based range finding cameras are ultra accurate. No Kinect hack here. Perhaps this is the way it will be done in the future? Incidentally, this technology was used 4 years ago for Radiohead’s House of Cards music video.
Pixel Cloud has a robust set of relighting tools. These include relighting tools for diffuse, specularity, reflections and the ability to relight the “Alpha” of a 3D displaced image using After Effects own lights. In Pixel Cloud we can specify lights that only affect the alpha channel of an image. Since the image is a 3D displaced Pixel Cloud, this means we can easily separate parts of an image according to their depth information. This is a powerful tool and a normal map isn’t a prerequisite, meaning you can relight non-CG generated images like photos and video. The video isn’t up yet but as a taste here is a preview of what the upcoming demo/tutorial will show how we can utilize this tool to relight the alpha of Jack, our cute little dog from the previous demo!
I just received my Lytro and have been having a blast taking photos with it. It’s absolutely brilliant. If you’re unfamiliar with this camera, it allows you to focus the picture after it’s already been taken. The move from in-camera to in-post is certainly being lead by technology and economics. My Pixel Cloud plugin is also trying to bring more of those capabilities into After Effects. Just look at the Microsoft Kinect and how innovative pioneers are using its ranging features to recreate environments from recorded point clouds. And Samsung has developed a sensor that not only records RGB but Depth pixels as well! These are amazing innovations for effects artists.
Imagine a future where we can record whole rooms as animated environments. Think 3D scanners that scan whole rooms at one time and at 24fps. We could completely eliminate conventional camera motion control and create everything in-post. We could change the lighting setups, create digital camera rigs all after the video has already been shot. Not as CG but as recorded pixels in 3D space; An accurate representation of reality that we can manipulate to our choosing. This opens up possibilities for interactive story-telling, not to mention subjective 3D stereography. I could imagine a dozen more uses.
This isn’t just virtual reality but recreated reality, “Recreality.”
[CORRECTION] Although we use the term depth map, the correct term for usage should be height map. The main difference being that a height map denotes distance from a flat surface and a depth map denotes distance from the camera.
I’ve shown how we can use a CG render and a Position Pass in After Effects to animate and relight a 3D displaced Pixel Cloud. But not all 3D programs can produce a Position Pass and photographic sources obviously do not come with specialized passes. Pixel Cloud can still get around these limitations.
Although not yet released, this demo/tutorial gives a quick look at how simple it is to use the Pixel Cloud plugin to create a realistic camera animation with a simple photograph. Pixel Cloud can use not only Position Passes but Depth Map passes as well within an 8-bpc project.
So keep an eye out for more updates on how close we are getting to release Pixel Cloud!
After a brief hiatus from postings, we’ll soon be coming back with a demo/tutorial on how to use the upcoming Pixel Cloud plugin to animate a photograph using depth maps and 3D displacement! It really is an amazing difference from just using flat 3D layers to simulate parallax. We’ll also be coming out with a couple of new tutorials with integrating this with a camera projection workflow as well as tips on how to get the most out your composites!
Yeah! After some soul searching, I’ve moved all the blog videos onto YouTube! Our YouTube channel is at TheBlurrypixel. Of course all the videos are still embedded in the posts so no need to go directly to the channel but I will be posting some bonus extra videos on there that may not be on the blog so show us your support and Subscribe!