In the previous post, I spoke about the ability to perform hit detection using analytical functions.
This works extremely well when we can restrict our use case to it, but now we have some other cases where this is not as ideal:
This works extremely well when we can restrict our use case to it, but now we have some other cases where this is not as ideal:
- Perform detection on arbitrary shape/3d model.
- User input is not a pointer anymore, but can also be arbitrary (threshold camera texture, Kinect Body Index)
- Both previous cases combined together
While we can often perform detection for 3d model by using triangle raycast (I'll keep that one for next post), it can be pretty expensive (specially if we perform a 10 touch hit detection, we need to raycast 10 times).
So instead, one easy technique is to use ID map.
Concept is extremely simple, instead of performing hit with a function, we will render our scene into a UInt texture, where each pixel will be object ID.
Of course it means you have to render your scene another time, but in that case you can also easily use the following:
- Render to a downsized texture (512*512 is often sufficient)
- Render either bounding volumes, or simplified versions of our 3d models.
Great thing with this technique, our depth buffer already makes sure that we have closest object ID stored (so we get that for "free").
So now we have our ID map, picking objectID from pointer is trivial:
Code Snippet
- Texture2D<uint> ObjectIDTexture;
- RWStructuredBuffer<uint> RWObjectBuffer : BACKBUFFER;
- float2 MousePosition;
- int width = 512;
- int height = 424;
- [numthreads(1,1,1)]
- void CS(uint3 tid : SV_DispatchThreadID)
- {
- uint w,h;
- ObjectIDTexture.GetDimensions(w,h);
- float2 p = MousePosition;
- p = p * 0.5f + 0.5f;
- p.y = 1.0f-p.y;
- p.x *= (float)w;
- p.y *= (float)h;
- uint obj = ObjectIDTexture.Load(int3(p,0));
- RWObjectBuffer[0] = obj;
- }
Not much more is involved, we grab the pixel id, store in a buffer that we can retrieve in staging.
In case we need multiple pointer, we only need to grab N pixels instead, so process stays pretty simple (and we don't need to render scene for each pointer).
Now as mentioned before, we might need to perform detection against arbitrary texture.
As a starter, for simplicity, I will restrict the use case to single user texture.
So first we render user into a R8_Uint texture , where 0 means no active user and anything else = active.
We render our object map next in the same resolution.
We create a buffer (same size as object count, uint), that will store how many user pixel hit an object pixel.
Dispatch to perform this count.
Use another Append buffer, that select elements over a minimum account of pixel (this is generally important to avoid noise with camera/kinect textures).
Accumulating pixel hit count is done this way:
Make sure you use InterlockedAdd, as you need atomic operation in that case.
Next we can filter elements:
This is that easy, of course instead of only rendering ObjectID in the map, we can easily add some extra metadata (triangle ID, closest vertexID) for easier lookup.
Now in order to perform multi user detection (for example, using Kinect2 body Index texture), process is not much different.
Instead of having a buffer of ObjectCount, we create it of ObjectCount*UserCount
Accumulator becomes:
And filtering becomes:
We now have a tuple userid/object id instead, as shown in the following screenshot:
Please also note this technique can also easily be optimized with stencil, setting a bit per user. You get then limited to 8 users tho (7 users in case you also want to reserve one bit for object itself).
You will need one pass per user also (so 6 pass with proper depth stencil state/reference value).
If you lucky enough and can run on Windows10/DirectX11.3, and have a card that allows you, you can also simply do :
Here is a simple stencil test rig, to show all of the intermediates:
That's it for part 2 (that was simple no?)
For the next (and last) part, I'll explain a few more advanced cases (triangle raycast, scene precull....)
In case we need multiple pointer, we only need to grab N pixels instead, so process stays pretty simple (and we don't need to render scene for each pointer).
Now as mentioned before, we might need to perform detection against arbitrary texture.
As a starter, for simplicity, I will restrict the use case to single user texture.
So first we render user into a R8_Uint texture , where 0 means no active user and anything else = active.
We render our object map next in the same resolution.
We create a buffer (same size as object count, uint), that will store how many user pixel hit an object pixel.
Dispatch to perform this count.
Use another Append buffer, that select elements over a minimum account of pixel (this is generally important to avoid noise with camera/kinect textures).
Accumulating pixel hit count is done this way:
Code Snippet
- Texture2D<uint> ObjectIDTexture;
- Texture2D<float> InputTexture;
- RWStructuredBuffer<uint> RWObjectBuffer : BACKBUFFER;
- float Minvalue;
- int maxObjectID;
- [numthreads(8,8,1)]
- void CS(uint3 tid : SV_DispatchThreadID)
- {
- uint obj = ObjectIDTexture[tid.xy];
- float value = InputTexture[tid.xy];
- if (value > Minvalue && obj < maxObjectID)
- {
- uint oldValue;
- InterlockedAdd(RWObjectBuffer[obj],1,oldValue);
- }
- }
Make sure you use InterlockedAdd, as you need atomic operation in that case.
Next we can filter elements:
Code Snippet
- StructuredBuffer<uint> HitCountBuffer;
- AppendStructuredBuffer<uint> AppendObjectIDBuffer : BACKBUFFER;
- int minHitCount;
- [numthreads(64,1,1)]
- void CS(uint3 tid : SV_DispatchThreadID)
- {
- uint c,stride;
- HitCountBuffer.GetDimensions(c,stride);
- if (tid.x >= c)
- return;
- int hitcount = HitCountBuffer[tid.x];
- if (hitcount >= minHitCount)
- {
- AppendObjectIDBuffer.Append(tid.x);
- }
- }
This is that easy, of course instead of only rendering ObjectID in the map, we can easily add some extra metadata (triangle ID, closest vertexID) for easier lookup.
Now in order to perform multi user detection (for example, using Kinect2 body Index texture), process is not much different.
Instead of having a buffer of ObjectCount, we create it of ObjectCount*UserCount
Accumulator becomes:
Code Snippet
- Texture2D<uint> ObjectIDTexture;
- Texture2D<uint> UserIDTexture;
- RWStructuredBuffer<uint> RWObjectBuffer : BACKBUFFER;
- float Minvalue;
- int maxObjectID;
- int objectCount;
- [numthreads(8,8,1)]
- void CS(uint3 tid : SV_DispatchThreadID)
- {
- uint obj = ObjectIDTexture[tid.xy];
- uint pid = UserIDTexture[tid.xy];
- if (pid != 255 < maxObjectID)
- {
- uint oldValue;
- InterlockedAdd(RWObjectBuffer[pid*objectCount+obj],1,oldValue);
- }
- }
And filtering becomes:
Code Snippet
- StructuredBuffer<uint> HitCountBuffer;
- AppendStructuredBuffer<uint2> AppendObjectIDBuffer : BACKBUFFER;
- int minHitCount;
- int objectCount;
- [numthreads(64,1,1)]
- void CS(uint3 tid : SV_DispatchThreadID)
- {
- uint c,stride;
- HitCountBuffer.GetDimensions(c,stride);
- if (tid.x >= c)
- return;
- int hitcount = HitCountBuffer[tid.x];
- if (hitcount >= minHitCount)
- {
- uint2 result;
- result.x = tid.x % objectCount; //objectid;
- result.y = tid.x / objectCount;
- AppendObjectIDBuffer.Append(result);
- }
- }
We now have a tuple userid/object id instead, as shown in the following screenshot:
Please also note this technique can also easily be optimized with stencil, setting a bit per user. You get then limited to 8 users tho (7 users in case you also want to reserve one bit for object itself).
You will need one pass per user also (so 6 pass with proper depth stencil state/reference value).
If you lucky enough and can run on Windows10/DirectX11.3, and have a card that allows you, you can also simply do :
Code Snippet
- Texture2D<uint> BodyIndexTexture : register(t0);
- uint PS(float4 p : SV_Position) : SV_StencilRef
- {
- uint id = BodyIndexTexture.Load(int3(p.xy, 0));
- if (id = 255) //No user magic value provided by Kinect2
- discard;
- return id;
- }
Here is a simple stencil test rig, to show all of the intermediates:
That's it for part 2 (that was simple no?)
For the next (and last) part, I'll explain a few more advanced cases (triangle raycast, scene precull....)
