This week I developed the final abstract particles, composited and edited the final video.
When researching different particle styles and techniques I finally found a tutorial that would serve as a good base for what I wanted.
This system works by creating a pyro simulation, then transfering the velocity field from the pyro onto the particles, allowing the
Abstract Particle SetupPyro Sim
To have the abstract particles be the same colour as the environment I imported the pointcloud into the network, then copied the colour values onto the nearest points of the sphere.
Transfered ColoursAbstract Particle Sim
During the final week I collected all my rendered clips and began comping and editing in Nuke and Premier.
Nuke Node GraphInput, Grading And Glow
To make a glitch VHS effect I created a noise that was streched on one axis, this created lines varying in brightness, I then blurred it and graded the green channel to add variation in the distance certain colours get stretched. This was then copied to the forward u and v channels and used to distort the image. This not only helps to cover up some visual artifacts in the final render but also adds more to the effect of the data points being distored.
Glitch Effect, Camera Effects And Output
As I had to deviate from my storyboard when editing I tried to use what clips I had as efficiently as possible, trying to fit them to the initial music cues and cutting thr 2nd half of the music. When exporting the video I testesd H.264 and ProRes 422HQ. As the video consists of many moving points, when it is exported in H.264 there is a visibile loss in quality where the bitrate of the video is not enough to accurately show all the moving parts. 422HQ solves this issue but at the cost of almost 10x the file size and is not as widely supported as H.264.
As I planned to render the particles and Gaussian Splat separately I wanted to add the effect of the particles lighting up the environment. I did this by getting the transition particle band that was being used to spawn the particles and converted it to a VDB volume, this volume was then cleaned up and converted into a mesh, this allows for it to be procedural and change shape with the particles. The mesh is then used as an Octane Mesh light, the lights colour is also set to the average colour of the particles.
Mesh Light VDB
To add to this effect on the particle side, I used the age and life attributes. These are created when a POP sim is used and contain values related to how long the particle has been alive for. This data was then passed into the material where I could use it to multiply or set colour values.
Particle Colour Based Off Age
My initial particle simulation used a POP wind and custom velocity. This velocity came from a cube made to be the same shape as the hallway, the normals of this cube were then converted to velocity attributes and used to push the particles away from the center of the hallway and stop them covering the center of the screen. I spent a long time with this setup and went through many iterations and research attempts to try and find my desired look. After focusing of other aspects of the project I went back to this, I found a technique of creating a simple pyro smoke simulation from an object the same shape as your desired effect, caching it, then using an addvecfield node within the POP network. This node imports the smoke cloud data into the particle simulation and adds the velocities to their nearest particles, making them behave in the same way as the smoke. Finally I added attribute blurs after the particle simulation, this try’s to blur and group the particles positions making the trails of particles appear more condensed and feel like they have forces clumping them together instead of being widely dispersed.
With this new setup, I had the look I was going for, but realised that the camera angle I had initially planned didn’t showcase this to its fullest. I decided to flip the camera and have it pulling back down the hallway, this allows the viewer to see more of the environment dissolving while also removing the issue of the particles obscuring the cameras vision.
Pyro Based Particle SimSkylight Setup
I was having some issues with floating points and Gaussian Splats, so I went back into Postshot and cleaned up the Splat and reimported it into Houdini. As all of the processing on the points in Houdini is procedural, I didn’t need to change anything and it accepted the updated point cloud.
In my initial concepts I planned to have the Houdini Gaussian Splat transition into particles, then have those transition into a deconstructed version of the landscape within Unreal Engine.
Unreal Engine Environment Setup
I created a simple previs of the environment so I can use it as reference for what I need my last shot to transition into. But as the main portion of this project is more challenging than I anticipated for this will only be further developed if there is enough time to allow for it.
Test Details Added
Moving back to Houdini I started to tackle the challenge of rendering the Gaussian Splat. Gaussian Splats can be rendered in a variety of ways, some of these can even be fast enough to view entire scenes on a web page. All the lighting, colour and other required data has been packed into the points, so all of the heavy rendering is done when training the cloud. However I wanted to relight my scene and add effects.
The first method I tried was the default plug and play rendering with Houdini 21, This didn’t look very visually pleasing though, and I decided to Downgrade to Houdini 20.5 and use the GSOPs plugin and use custom renders.
To render with raytracing in Houdini’s Karma I got a sphere primitive and instanced it on every point.
Sphere Instance
With this sphere I could then apply a material. This material uses the same mathematical functions that make up the core rendering of Gaussian Splats in other applications.
USD Raytraced Gaussian Splat Material
This looked better than the default Houdini 21 Render but still left areas for improvement and didn’t give me the effect I wanted.
USD Karma Raytraced Render Test
I also did a quick test within blended using custom plugins, but yielded the same results.
Blender Gaussian Splat Addon Test
For my next render test I found a custom plugin for Redshift in Houdini. I saw some promising results of this but it didn’t work well for my application. After some more research on this renderer I found that people have said I can achieve good results, but only in very specific conditions and lighting.
Houdini Redshift Gaussian Splat Plugin Test
Finally I did some research in the GSOPs discord server. Here there are lots of people developing and experimenting with this technology every day. I found one person that has been doing a lot of renders with Octane. Their work looked amazing and was exactly the style I was going for, so after talking with them for a while and asking questions and getting help setting the renderer up I could finally lock in my renderer choice and start progressing the LookDev aspect of the project.
Houdini Octane Test
The latest version of Octane has a dedicated Gaussian Splat object setting, just by enabling this I already had a render I liked the aesthetics of more than any other I explored. The next step was to get the particle rendering side developed.
Octane Particle Render
As my base for the particles I just wanted them to inherit the base colour from their respective points. I did this by stripping all of the Gaussian Splat related data and just passed the particles Cd colour attribute into the material. Octane does this by packing all the data into a UV map then reading that inside the material.
Octane Particle Sim Test
With the colours working I could then focus on developing the look of the particle size, lighting and PBR elements.
Octane Particle Size and Lighting Test
Next I needed to test if the renderer could handle the animation of the Gaussian Splat and particles transitioning. When testing this I encountered an issue that wouldn’t render more than 1 frame. After spending a while trying to fix this, I went back to the person that showed me Octane, and they said that this was because Gaussian Splat and particle rendering within Octane does work work well when motion blur is enabled, so I turned this off and will add it back in post by adding an AOV motion vector layer.
Octane Gaussian Splat Animation Test
With this done I could then render out a motion test.
Octane Transfered Particle Colour Material
The next stage was relighting the scene. One of the many benefits of Gaussian Splatting is its ability to be relit with relative ease, so all I needed to do was as a light source and edit the scene as I would with any other 3D asset.
Octane Relighting Test
With the lighting, materials and animations working, I could then tweak my camera settings.
This week I began to apply the transition to my point cloud. I created the initial point group and passed it into the DOP network.
Point Activation Setup
I then used the attribute transfer technique, however this moved very fast and also had a problem of missing points. As it picks points within a certain distance, if a point is slightly too far away it will miss it and leave it out of the simulation, so I needed to find fix.
Transition DOP Network
I combined my previous development of a moving mask object, and the transition calculation. Now as the masked moved, any points that are active on the current frame and not active on the previous would be added to the transition.