Preparation
I started to create my background environment inside Unreal Engine 5. I used reference images from the 2021 movie Dune as my inspiration for the environment. When looking at the reference images I notice that all the mountains and details within the environment were very spread out with large sand expanses in between so I tried to recreate this in my environment. Using some rock photogrammetry scans from Quixel Megascans I started to fill in some backdrops for my environment, these will be moved later but this basic scene will help me in designing the details on the car that will be driving through the scene.
I collected various images and explored different interpretations of these types of desolate landscaped from films and games and other media. After reviewing all of these I decided that I wanted my project to combine the environments of Dune and the UE4 Rebirth cinematic creating a more rocky desert that is a mix of sand flats and rocky plains similar to canyons and Icelandic terrain. For my vehicle I wanted to create a mix of the Tesla Cybertruck and the cars in Blade Runner 2049.
Car Modeling
The process of texturing the vehicle started with marking the edges and seams within Blender, then UV unwrapping the object and trying to fit as much texel density as I could into the UV map. I did this by reducing the size that unseen parts of the vehicle such as the underside take up in the UVs and managing the space as efficiently as possible. I decided to use Blender instead of Maya for this step as I am more familiar with it and have access to plugins that make this step more efficient. After this is done the object can then be exported into Substance Painter. However after looking at the model and trying some textures I felt like it was lacking in detail, so I went back into Blender to make a high poly version with more details.
Using the sculpting features within blender I was able to add little dents and scratches into my model, this makes it feel like it belongs in the environment its going to be in instead of looking like it just arrived out of the factory. I sculpted these details in Blender instead of Zbrush as it was only small details being added to the model and for this Blender works fine.
Once the detailed high poly mesh was finished I brought it into substance painter to bake these details onto my low poly mesh. After these details were added and all the maps were baked I could create a metallic smart material for my car. I wanted to create a dark metal plate look that’s has the effect of being sand blasted from all its time spent in the desert.
The materials were then baked down and exported as base colour, normal and orm (occlusion, roughness, metallic) textures for use on my model within unreal engine.
Unreal Engine World Building
After the car was finalized I planed out what the layout for my environment was going to be, I made a quick sketch in paint based on my inspiration then recreated it within Unreal Engine using large backdrop assets.
Before I started to build out my landscape I created a small detailed scene so I could set up my textures and post processing. I used Megscans assets and rock prefabs from Epic Games Valley of the Ancient demo project to create a platform where I could place my model and create a look for the environment that was similar to my references.
I went into the materials that came with the assets and tweaked the hue, saturation and contrast until the assets blended together seamlessly.
To start building the ground of my environment I created a pallet of ground meshes and started to place these, scaling and rotating the meshes until I created a seamless ground area for my environment.
For the distant sand dunes I wanted to create the look of sand blowing across the landscape. I did this by getting a noise texture and using the panner node to pan the noise across the world space then merged this with the Megascans default ground material.
After creating this texture I felt like the sand dunes that I created were a bit too flat, so I created a new landscape layer and used the Landmass plugin’s material brush to create a heightmap in a material that could then be applied to the landscape. This material used a voronoi noise to create more pronounced sand dunes.
I continued to add more detail and realism to the environment through the use of static meshes, foliage and decals. Through the use of blueprints and level instances I was able to group smaller meshes into larger collections that allowed me to build up this environment relatively quicky by transforming and merging these grouped mesh actors.
I also experimented with a feature I have not used before, the Niagara particle system within Unreal Engine. I did some research into the system and managed to create a smoke/dust cloud simulation using the gas master fluid sim base. I manged to create a look I was pleased with but I could not find a good way to scale this system up into a cloud that could blow over the whole scene, when scaled up the particle system would lack detail and the lifetime of the particles would not last long enough to cover such a large space without large performance hits. I decided to add these types of effects later in Nuke instead. I still learned a lot from this experiment and will be coming back to this feature.
Rendering
I originally planned on using the default UE5 renderer but after exploring some of the rendering features available I quickly decided to use the built in path tracer instead of the default differed renderer. Using the path tracer increased the visual fidelity of my scene dramatically but also significantly increased the time cost of the project however I felt like this trade off was worth it.
I created a set of cameras at 3 different focal lengths all with a default aperture of 2.8, I chose this aperture as it created a very pleasing DOF on objects close to the camera and objects very far away whilst still keeping the exposure of the scene at desirable levels. When using the cameras in Unreal Engine before, I would just select numbers that made the best looking scene without thinking to deep into it, but after learning more about how cameras and exposure works I was able to adjust these settings according to the exposure triangle, where shutter speed is set, aperture is set on the camera and ISO set in the post process volume. After some tests I decided that these would be the camera settings I would use to achieve my desired look.
To set up the render for use in nuke I needed to change most of the default settings. I changed the export from .jpeg sequence to .exr so I can get a higher quality image and have layer data to use in Nuke, turned on Object Ids so I can select and roto objects without having to manually draw them later, changed the colour space to linear and enabled the path tracer.
After the renderer was set up I rendered an image including my model and a colour calibrator so I can test everything is working correctly and refine my render settings before I commit to a long render.
I imported this test into Nuke to make sure the render looked as intended and had all the necessary information.
I created multiple renders each with slightly difference settings so I could find the best settings to use based on visual fidelity, file size and compatibility. At first I started with high Spatial samples and low temporal but figured out that its best to balanced the two. The difference between 1024 and 2048 samples was slightly noticeable but it was not worth almost double the render time. I tried using 3 different EXR compressions, PIZ, DWAA and DWAB. PIZ uses lossless compression and DWAA/DWAB use lossy with respective file sizes of 20mb and 7/6mb. I could not see any visual differences between these compression so I wanted to use DWAB for the reduced file size, but the cryptomatte only worked with PIZ so that had to be used.
Compositing
After rendering out my environment in Unreal Engine I brought it into nuke to track. I graded the scene to allow for easier tracking then created a camera tracker node and adjusted the settings to match the camera setting from the render in Unreal. Then I added a mask as I don’t want to track the sky and adjusted the threshold and amount of features.
Then I solved the track and tweaked the settings until I got a solve with and error value that I was happy with (<1 for 1080p).
The next step in tracking was to set my world origin and ground plane, I did this by selecting a group of points on the main floor of my scene and set them to my ground plane. After that I set a known distance between two points to get the scale of my scene, for this I selected two points on a rock that I knew was 2m wide.
Now that I had a camera track and point cloud I created a scene in Nuke to test that everything was working correctly and help me place my objects later on. I created a plane on the ground and some axis at key points in my scene and at the locations I wanted the car that drives through the scene to start and end. This data was then exported into and FBX file using the write geo node.
The FBX object data was imported into an Unreal Engine level containing only the lighting from my environment, I then scaled up these objects by 100x to convert the scale from Nuke to Unreal. To import the camera data I created a new sequence and imported the FBX file into the camera, after setting the camera to world origin this lined up with my tracking objects perfectly.
Because the main source of lighting in my environment was a HDRI I was able to light the object as if it was in the environment. When rendering I tried to create an image with just the model and have a transparent alpha background, I could not seem to get this to work with the path tracer renderer so instead I used the cryptomatte that luckily also included the motion blur. The model was then masked and overlayed onto my environment to check everything lined up correctly.
Once all the tracking and rendering was complete I could start compositing my footage together in Nuke. I started by extracting the layer data that came with the render, this included depth info and cryptomatte object data that would be used to mask and separate parts of my scene for grading and adjustments. I then graded my scene to achieve the intended look, I tried to create a feel similar to the night scenes within my Dune references and the overcast foggy feel in the Quixel Rebirth reference. I also added a glow effect to the rear light of the car to make it feel more realistic and like it was emitting light instead of just being a solid red bar.
Whilst I was composting my shots I notice that in one of my shots you could see a part of the landscape I had forgot place an object in, this resulted in a very dark shadow that didn’t look right in the scene. To fix this I used the roto paint node and clone brush to paint over this dark patch and fix the shot.
I decided to create my smoke trail in Nuke instead of in Unreal Engine. To do this I made a a spline roto that followed the car then added some noise on top of it, this noise was then blurred and colour graded to fit in with the scene. After lots of tweaking with the noise values and adding another roto to blur out and dissipate the smoke I got it to a place I was happy with.
Video Editing
The final stage of the project was to combine all my all my shots together and make a VFX Break-down using Adobe Premiere Pro. I put all my shots together with some simple transitions, and added music and sound effects. For the sound design in this project I worked with a friend in FL Studio to create some futuristic vehicle sounds that add a lot to the finished video. Finally I added a backing track that helps the viewer feel like they are in this large desolate landscape.
Problems
During the project I encountered multiple errors with Unreal Engine. The first main problem I had was with rendering, Whenever I tried to render a shot with the MRQ (Movie Render Queue) using Path Tracing my VRAM and then RAM would slowly keep filling up until my computer froze and I would have to manually restart it. I tried many changes to the render settings but in the end I had to copy my work to a new project to fix this. The second issue I had was problems with DirectX and Megascans assets I was using. I Opened my project with Visual Studio to try and debug this issue and found that whenever the engine was calling Delete Recourses on certain Megascans textures it would crash the engine. I tried to fix this by deleting the affected asset but this didn’t help as it would just crash with a different asset instead. I was not able to fix this issue and other rendering issues even with help from people with a depth of knowledge in this area so for the final stretch of my project I had to deal with crashing and reopening my project very regularly.
Reflection
Looking back at this project I am very happy with the final outcome. I learned a lot using different software’s and techniques that I have not used before or not too familiar with and cant wait I’m improve more on these skills. If I was to create this again I would like to add more effects, such as better smoke trails on the car, dust in the environment and other effects to achieve a fore filmic look. I will try and improve on this in my next project by learning more about the Niagara system within Unreal Engine and learning how to make effects within nuke.