Timeline

One obvious very useful feature for most graphics and audio toolset is some form of timeline.

In 4v you have the native one, which with a little effort could become ok (add undo, move keyframes with keyboard, export/import, and better color editor), but in it's current state is at best irritating. It's not as bad as the hlsl code completion in 4v but that's another story ;)

Then you have more standalone ones, vezer looks pretty cool but mac only, so not for me.

You also have duration, which is likely the best but has 2 major flaws, missing scroll and groups. Also opengl rendering would not fit as easily to integrate in my tool.

Then you have this brand new Posh svg, new 4v timeliner, presented as usual as the next big thing which will make your grandmother dance rock and roll again, and so and so.

So first pre alpha release tests were let's call it.... disappointing. Only one track type, no interpolation, and an amount of ui bugs that it could just eat a piece of wood like piranhas would eat my bum.
Next first "public" release is well... appalling, I personally hardly see the point of releasing something in that state, except to show off your incompetence at building user interface.

So first thing you do when you build a timeliner, you prepare a decent track setup, there is NO point showing 5 key frames and one track. So I start to prepare 8 tracks, 40 key frames each (which is really a small setup), and then everything goes laggy, I get 40 % cpu usage (where the hell does that go), selecting and moving several frames just blow up everything, you can't even recover and have to restart the software, usability at it's best.

So well I start to report those issues, and happily suggest that some DirectX11/OpenGL rendering for that type of things would likely fit and scale much nicer, but only reply I get is "yes I know it's slow but I don't care Dx is not as interesting as svg". So I try to explain that well if your ui doesn't scale, you might look for something else, but I get the usual "I don't give a fuck type of attitude which pisses me off so much". We suggested that actually some of the changes could easily make it through the old one (some undo + key) , I get the same I don't give a fuck.

So well I don't really understand the point (and as a user slightly pissed off that you have to pay 500 euros a license to get that type of answer). And I feel really horrified that this took several month to produce, I'm getting really worried about user interface in the next vvvv50, since in current version since there's no will to improve the old version, then it gets decided to produce a brand new ui framework which is as shit, so I got more or less no faith that we will ever have a decently smooth user interface in previous or next gen vvvv.

But let's stop whining and go to the more fun part ;)

So I'm still without a timeliner, since now I got 2  10% baked unusable pieces of junk, and some others which are nice but I can't integrate them.

So since I also mentioned dx11 would be a good candidate, I decided to spend an afternoon doing track renderers. Also that would really prove a point that modern rendering wins against this browser jazz. I also decided to only focus on the rendering for now, since well it's not too hard to hittest and drag a point after all ;)

Main focus is also to of course have some fast rendering, I want a smooth user interface ;)

I just push my tracks into buffers and then render them. I decided to start with color, which is really simple.

Code Snippet

7. struct ColorKeyFrame
8. {
9.     float4 color;
10.     float time;
11.     int trackid;
12. };
13.  
14. struct KeyFrameLink
15. {
16.     int left;
17.     int right;
18. };
19.  
20. StructuredBuffer<ColorKeyFrame> ColorKeyFrameBuffer;
21. StructuredBuffer<KeyFrameLink> ColorLinkBuffer;
22. StructuredBuffer<float2> TrackOffsetBuffer; //x = top, y = height

Render a bulk of instanced quads, grab color on the left keyframe, color on the right keyframe, position with a map function.

Code Snippet

44. psInput VS(vsInput input)
45. {
46.     psInput output;
47.  
48.     KeyFrameLink cl = ColorLinkBuffer[input.ii];    
49.     ColorKeyFrame left = ColorKeyFrameBuffer[cl.left];
50.     ColorKeyFrame right = ColorKeyFrameBuffer[cl.right];
51.     int tid = input.ii;
52.     tid = tid / cpPerTrack;
53.     
54.     float2 pos = input.uv;
55.     float2 offset = TrackOffsetBuffer[tid];
56.     
57.     pos.x = map(pos.x,left.time,right.time);
58.     pos.x *= 2.0f;
59.     pos.x -= 1.0f;
60.     
61.     pos.y += offset.x;
62.     pos.y *= offset.y;
63.         
64.     output.pos = mul(float4(pos,0,1),tRange);
65.     output.colstart = left.color;
66.     output.colend = right.color;
67.     output.uv = input.uv.x;
68.     
69.     return output;
70. }

Send to the (hardcore) pixel shader:

Code Snippet

78. float4 PS(psInput input) : SV_Target
79.   {
80.     return lerp(input.colstart,input.colend, input.uv);
81. }

That was so hard ;)

Now let's go for value, positioning keyframe point, well one small Segment instance, just so simple that there's nothing to say about it, now let's build the connections, and let's add  the fact that I want more curves (aka tweens):

Code Snippet

7. struct ValueKeyFrame
8. {
9.     float value;
10.     float time;
11.     int trackid;
12.     int interpolation;
13. };
14.  
15. struct KeyFrameLink
16. {
17.     int left;
18.     int right;
19. };

Of course usual buffers are there too, we send a PatchListWithOneControlPoint batch for each connection (we don't need 2, that's beautiful).

Code Snippet

43. struct linkData
44. {
45.     int left : CPOINT0;
46.     int right : CPOINT1;
47. };

Now we only need to pass trough keyframe IDs till we reach the domain shader:

Code Snippet

144. [domain("isoline")]
145. psInput DS(hsConstantOutput input, OutputPatch<linkData, 1> op, float2 uv : SV_DomainLocation)
146. {
147.     psInput output;
148.     float t = uv.x;
149.     
150.     ValueKeyFrame kl = ValueKeyFrameBuffer[op[0].left];
151.     ValueKeyFrame kr = ValueKeyFrameBuffer[op[0].right];
152.     
153.     float2 start = ComputePosition(op[0].left);
154.     float2 end = ComputePosition(op[0].right);
155.  
156.     float x = lerp(start.x,end.x,t);
157.     float y = lerp(start.y,end.y,lerpFunc(t,kl.interpolation));
158.     
159.     float2 pos = float2(x,y);
160.     output.pos = mul(float4(pos,0.0f, 1.0f), tRange);
161.  
162.     return output;
163. }

lerpFunc selects interpolation function, which are all the basic tween modes.

Now well, bang track is simple instanced quad or line, nothing to speak about.

Then let's add wave track, I thought it would be a bit complicated, but that was so easy it's even embarrassing. So first get your favorite audio API (I used Bass). Load a music file, read all the samples as float and feed to a big fat structured buffer (float or float2). I chose float since at least i push any type of multi channel later.

Render a quad, then here is the insane pixel shader:

Code Snippet

38. float4 PS(psInput input) : SV_Target
39.   {
40.     uint cnt, stride;
41.     WaveDataBuffer.GetDimensions(cnt,stride);    
42.     float off = cnt;
43.     float xpos = input.uv.x * off;
44.     
45.     float sampleleft = WaveDataBuffer[xpos*2];
46.     float sampleright = WaveDataBuffer[xpos*2+1];
47.     float y = input.uv.y;    
48.     y *= 2.0f;
49.     y -= 1.0f;
50.     
51.     return (abs(sampleleft) > abs(y)) + 0.1f;
52. }

That was hardcore, I could do a bit of oversampling for sure, but it already looks pretty ok.

So here we go, then you add a small ruler.

And now pushing some 4000 keyframes:

Unzoomed view:

And let's push the ui a little bit, since I said I wanted it to scale, 14K keyframes:

3 hours well spent, I now got a pretty decent scalable timeline renderer, which is a pretty huge step forward (80 fps rendering every frame, no caching is rather good so far).

To conclude, hlsl > all :)