Background
This project was developed in three and a half weeks for a Game Programming final at California State University Chico. The assignment had me recreate a mechanic from a game of my choosing.
I chose to recreate the movement system from Nintendo’s Splatoon franchise because it would allow me to practice HLSL shader programming, and try my hand at creating a character controller from scratch.
The biggest challenges were getting a painting system up and running, then getting the character controller to dynamically react to the paint.
There were some hiccups and shortcomings, but overall I’m satisfied that I was able to implement all the core features in the given time limit.
Paint System
At the heart of the paint system is SplatMask.shader [fig 1].
It draws to a render texture (henceforth referred to as the “splatmap”) using several properties
that control the stroke’s position, radius, color, and hardness.
The only hard requirements were that objects not owning the splatmap (IE projectiles) could issue a
request to paint with a specific location and color.
Since painting did not seem to be a significant bottleneck, I focused my optimization efforts
elsewhere.
v2f vert(appdata v) { v2f o; o.worldPos = mul(unity_ObjectToWorld, v.vertex); o.uv = v.uv; float4 uv = float4(0, 0, 0, 1); uv.xy = (v.uv.xy * 2 - 1) * float2(1, _ProjectionParams.x); o.vertex = uv; return o; } float4 frag(v2f i) : SV_Target { float4 col = tex2D(_MainTex, i.uv); float m = mask(i.worldPos, _SplatPos, _Radius, _Hardness); float edge = m * _Strength; return lerp(col, _InkColor, edge); }
public void DrawSplat(Vector3 worldPos, float radius, float hardness, float strength, Color inkColor) { splatMaterial.SetFloat(Shader.PropertyToID("_Radius"), radius); splatMaterial.SetFloat(Shader.PropertyToID("_Hardness"), hardness); splatMaterial.SetFloat(Shader.PropertyToID("_Strength"), strength); splatMaterial.SetVector(Shader.PropertyToID("_SplatPos"), worldPos); splatMaterial.SetVector(Shader.PropertyToID("_InkColor"), inkColor); cmd.SetRenderTarget(tempM); cmd.DrawRenderer(GetComponent<Renderer>(), splatMaterial, 0); cmd.SetRenderTarget(splatmap); cmd.Blit(tempM, splatmap, alphaCombiner); Graphics.ExecuteCommandBuffer(cmd); cmd.Clear(); }
For a terrain mesh to be paintable it must implement SplatableObject.cs [fig 2] as a component.
A Splatable Object component stores references to the splatmap, Inkable Surface material, and Splat
Mask shader.
Its public DrawSplat() function first sets parameters on the Splat Mask shader, then uses a command
buffer to draw the shader output to the splatmap.
More specifically, the output is first stored in a temporary variable, which is blended with the
actual splatmap using a simple blend shader.
Finally the splatmap is fed to the Inkable Surface material, which interprets it as a mask before
actually rendering the ink.
Reading the Splatmap
Read Pixels
Reading pixel data from the splatmap poses a challenge because the data needs to be moved from a render texture, which lives on the GPU, to a C# script on the CPU.
Unity provides an easy way to do this with the Texture2D.ReadPixels function, but this method is extremely slow.
dst = new Texture2D(1, 1, TextureFormat.RGBAFloat, false); RenderTexture.active = splatmap; dst.ReadPixels(copy, 0, 0);
We can make a couple optimizations.
First, raycasts are used to determine the texture coordinates of the pixel before making the call.
Meaning only one pixel needs to be sent back.
Secondly, we can use a coroutine to reduce the number of calls to Read Pixels.
Usually input delay is unacceptable for player controllers. But in this case we are reading the
splatmap beneath the player, and since the player capsule has some diameter there can be a small
delay before sinking into the ink and it won’t feel off.
These optimizations help, but ReadPixels is fundamentally slow because it stalls the CPU while
waiting on the render thread to finish before sending data back.
Async Gpu Readback
With further research I found Unity’s AsyncGpuReadback class.
This class does the same thing as ReadPixels, albeit on more generic data, and it allows the CPU to
continue running while it waits on said data.
This means the data will usually be a few frames old by the time the CPU gets it, but as established
earlier a slight delay is okay.
var rt = RenderTexture.GetTemporary(1, 1, 0, RenderTextureFormat.ARGBFloat); Graphics.CopyTexture(target, 0, 0, (int)(uv.x * target.width), (int)(uv.y * target.height), 1, 1, rt, 0, 0, 0, 0); AsyncGPUReadback.Request(rt, 0, TextureFormat.ARGB32, OnCompleteReadback);
Profiling each method in an isolated build shows that AsyncGpuReadback generally has a smaller
rendering load on the CPU compared to ReadPixels [fig. 6-7].
Furthermore, just like with ReadPixels an extra delay can be added between AsynGpuReadback calls.
The end result is that AsyncGpuReadback is far more scalable than ReadPixels. In the second
comparison each procedure gets called one hundred times per cycle using a for loop [fig. 8-9]
(albeit I can not think of a practical reason why you would want to run the procedure that
frequently).
Final Implementation
While AsyncGpuReadback gets significantly better results, it does introduce some more complexity to
the code.
I created a SplatmapReader.cs class so any object could use this functionality, however
AsyncGpuReadback works by making the request then passing the data to a callback function
implemented in the same class. This makes it hard to get the data back to the original caller.
To solve this issue I included a parameter for a delegate in Splatmap Reader’s public ReadPixel()
function. A copy of the delegate gets saved by Splatmap Reader, and when AsynGpuReadback finishes it
uses that delegate to pass the pixel Color back to the original caller.
In my case the original caller is the PlayerController, which implements splatmap reading procedures
in two functions.
UpdateMovementState() requests data from a SplatmapReader object, then FinishUpdateMovementState()
processes the returned data.
FinishUpdateMovementState() perpetuates the loop by calling UpdateMovementState() with a delay. The
delay time is exposed so it can be easily adjusted to maintain a good balance between accuracy and
performance.
Player Controller
Basic Approach
Rather than using the physics engine or a prebuilt character controller, I wrote simple character
and camera controls from scratch. I wanted to challenge myself with an unfamiliar approach to player
controls.
The character capsule does use Unity’s collision detection via a rigidbody component, but it does
not simulate any physics. Simple gravity, input, and collision corrections are all calculated in my
PlayerController.cs class.
Movement States
To handle different interactions with the ink, PlayerController.cs implements four movement states.
The movement states are defined in the public MovementState enum, which includes Walking, Swimming,
WallSwimming, and EnemyInk.
Player Controller continually updates a current movement state variable using the
UpdateMovementState() and FinishUpdateMovementState() functions, which were partially described in
the previous section.
When determining the movement state, UpdateMovementState() uses raycasts to probe the splatmap in
front of and beneath the player.
/// Forward probe Ray ray = new Ray(origin, mesh.transform.forward); bool isValidHit = Physics.Raycast(ray, out hit, capsule.radius + .1f); surfaceProbeHit = hit; if(isValidHit && currentMovementState != MovementState.EnemyInk) { splatObj = hit.collider.GetComponent<SplatableObject>(); if(splatObj) { splatmapReader.ReadPixel(splatObj.Splatmap, hit.textureCoord, FinishUpdateMovementSate); return; } }
If either the probe hits a Splatable Object, a request is sent to a Splatmap Reader object for the
color at the hit’s texture coordinates.
FinishUpdateMovementState() acts as the callback function that Splatmap Reader passes the color to.
Team Ink Colors
One challenge was getting FinishUpdateMovementState() to determine if the color corresponds with
friendly ink or enemy ink.
Comparing two RGB colors could be imprecise, while having multiple splatmaps would increase
complexity.
However, I was able to use the color’s RGBA values to my advantage. I chose to have the splatmap
alpha value represent if ink is present or not.
The red channel then represents “Team 1” ink, while the green channel represents “Team 2” ink.
The advantage to this approach is the player controller only needs to check which channel dominates
to determine the ink color. Of course, Splatoon has many colors other than red and green, so the
output shader uses channels as masks to lerp between different colors. This means designers can
freely change team colors via parameters on the material without the need to update data in the
player controller.
Default walking, swimming on the floor, and walking in enemy ink are all fairly trivial movement states (only the max speed and animation need updating). But if the movement state equals Swimming and the player starts pushing up against an inked wall, they begin swimming up the wall. To achieve this I conceptually separated horizontal and vertical movement calculations. Unfortunately this split is unclear in the code because I had to cut some corners in PlayerController.cs. More on this in the following section.
/// Wall Swimming if(surfaceProbeHit.collider && Vector3.Dot(surfaceProbeHit.normal, Vector3.up) - slopeCheckTolerance < minSlopeGradation && color.a > inkAlphaMinThreshold && color.r > color.g && isSquid) { currentMovementState = MovementState.WallSwimming; maxHorizontalSpeed = baseMaxHorizontalSpeed; grounded = false; Invoke("UpdateMovementState", updateMovementStateDelay); return; }
Limitations and What I Would Do Differently
Looking back on the project, my biggest mistake was spending too much time polishing the shaders
early on. Within a few days I had the basic elements of the paint system and splatmap reader worked
out. But then I got sidetracked working on purely visual things like normal maps and sheen.
I also had to spend a few days working on other classes, so by the time I started the character
controller I had under two weeks left. This caused a lot of stress in the second half of the
project. Had I focused on a minimal functional product for the shaders early on, I could’ve started
the controller with a lot more time and polished things later. This would have at least reduced
stress levels, and at best helped me to get more done.
As for the impact working on the shaders first did have, The character controller ended up a lot
messier than I would have liked. It still has some bugs and inconveniences like not being able
dynamically change the team channel, despite the shader being designed to support that.
However, as previously stated I am happy with the progress made given the time frame. I got all of
the major elements working and got to learn about shader programming and custom character
controllers along the way.