Execution Path and IO

I explained in previous post how you can build simple functions and convert them into IL code.

Now what is interesting is you can easily optimize those by using simple assumptions, which are things that you would easily forget while writing code.

You can consider 2 things:

• Immediate optimizations : For elements that you know at compile time
• Code Path : You need to define them at runtime depending on inputs size.

Let's see a few Immediate optimizations:

Code Snippet

13. [SlicewiseMethod(Name = "SimpleLerp", Category = "Value")]
14. public static void SimpleLerp(
15.     [Input("Input 1")] double d1,
16.     [Input("Input 2")] double d2,
17.     [Input("Amount")] double amt,
18.     [Output("Output")] out double result)
19. {
20.     result = VMath.Lerp(d1, d2, amt);
21. }

Here we just process a simple lerp.

Since in slicewise operations, our output size is always SpreadMax, Generator will automaticaly select a pointer output,and miss the mod operator.

This is a no brainer, easy as that, outputs never use mod and do direct storage.

Second test:

Code Snippet

126. [SlicewiseMethod(Name = "Sin", Category = "Value")]
127. public static void Sin(
128.     [Input("Input")] double d,
129.     [Output("Output")] out double dout)
130. {
131.     dout = Math.Sin(d);
132. }

Here we have one input and one output. The compiler can detect that, and automatically set the input read to safely ignore the mod operator too.

Now let's look at our lerp above, and let's say that we want only one lerp value for all elements:

Code Snippet

23. [SlicewiseMethod(Name = "SimpleLerp2", Category = "Value")]
24. public static void SimpleLerp(
25.     [Input("Input 1")] double d1,
26.     [Input("Input 2")] double d2,
27.     [Input("Amount", IsSingle=true)] double amt,
28.     [Output("Output")] out double result)
29. {
30.     result = VMath.Lerp(d1, d2, amt);
31. }

Here we did set the IsSingle Attribute to our variable. Compiler will detect that, read the variable into a local once before the for loop.

Now let's say we want to multiply a spread by a constant value:

Code Snippet

33. [SlicewiseMethod(Name = "MultiplyFixed", Category = "Value")]
34. public static void SimpleLerp(
35.     [Input("Input")] double d,
36.     [Input("Amount", IsSingle = true)] double amt,
37.     [Output("Output")] out double result)
38. {
39.     result = d * amt;
40. }

Here as before, our Amount variable will be stored into a local.
But now we also know that our Input Variable is alone (same as the Sine case).
So compiler will remove the mod operator on input value as well!

Now let's see a little bit more complex case:

Code Snippet

26. [Selector(Name = "Function", Category = "Value Inverse")]
27. public static Dictionary<string, Func<double, double>> TrigonometryInv
28. {
29.     get
30.     {
31.         var result = new Dictionary<string, Func<double, double>>();
32.         result.Add("Asin", (x) => Math.Asin(x));
33.         result.Add("Acos", (x) => Math.Acos(x));
34.         result.Add("Atan", (x) => Math.Atan(x));
35.         return result;
36.     }
37. }

Here we basically build a node that Has one input, one output , and and enum to select function.

Equivalent code:

Code Snippet

15. [PluginInfo(Name = "Function", Category = "Value", Version = "Simple")]
16. public unsafe class DummyFunc : IPluginEvaluate
17. {
18.     public enum eFunc { Sin,Cos, Tan}
19.     public delegate double FuncDelegate (double input);
20.  
21.     [Input("Input")]
22.     private ValueInput input;
23.     [Input("Function", IsSingle = true)]
24.     private ISpread<eFunc> function;
25.     [Output("Output")]
26.     private ValueOutput output;
27.     private Dictionary<eFunc, FuncDelegate> functable = new Dictionary<eFunc, FuncDelegate>();
28.  
29.     public DummyFunc()
30.     {
31.         functable.Add(eFunc.Sin, (x) => Math.Sin(x));
32.         functable.Add(eFunc.Cos, (x) => Math.Cos(x));
33.         functable.Add(eFunc.Tan, (x) => Math.Tan(x));
34.     }
35.  
36.     public void Evaluate(int SpreadMax)
37.     {
38.         FuncDelegate f = functable[function[0]];
39.         output.Length = SpreadMax;
40.         double* iptr = input.Data;
41.         double* optr = output.Data;
42.         for (int i = 0; i < SpreadMax; i++)
43.         {
44.             optr[i] = f(iptr[i]);
45.         }
46.     }
47. }

Please note that we could create a loop delegate, like this:

Code Snippet

50. [PluginInfo(Name = "Function", Category = "Value", Version = "Loopable")]
51. public unsafe class DummyFunc2 : IPluginEvaluate
52. {
53.     public enum eFunc { Sin, Cos, Tan }
54.     public delegate void FuncLoopDelegate(int cnt, double* input, double* output);
55.     [Input("Input")]
56.     private ValueInput input;
57.  
58.     [Input("Function", IsSingle = true)]
59.     private ISpread<eFunc> function;
60.  
61.     [Output("Output")]
62.     private ValueOutput output;
63.  
64.     private Dictionary<eFunc, FuncLoopDelegate> functable = new Dictionary<eFunc, FuncLoopDelegate>();
65.     public DummyFunc2()
66.     {
67.         functable.Add(eFunc.Sin, (cnt, input, output) => { for (int i = 0; i < cnt; i++) { output[i] = Math.Sin(input[i]); } });
68.         functable.Add(eFunc.Cos, (cnt, input, output) => { for (int i = 0; i < cnt; i++) { output[i] = Math.Cos(input[i]); } });
69.         functable.Add(eFunc.Tan, (cnt, input, output) => { for (int i = 0; i < cnt; i++) { output[i] = Math.Tan(input[i]); } });
70.     }
71.  
72.     public void Evaluate(int SpreadMax)
73.     {
74.         FuncLoopDelegate f = functable[function[0]];
75.         output.Length = SpreadMax;
76.         double* iptr = input.Data;
77.         double* optr = output.Data;
78.         f(SpreadMax,iptr,optr);
79.     }
80. }

This doesn't give a very good gain tho, so first option is easier to build.


Now let's look at code path.

We can eventually decide of our loop at runtime, let's take our Vector2 join again:

We have 5 cases we want to consider:

• Length (x) == Length(y) : We can ignore mod on each vector.
• Length (x) == 1 : We store x before the loop and sample y at full speed
• Length (y) == 1 : Same as above, just the other way round.
• Length (x) > Length(y) : Sample x at full speed, mod on y
• Length (y) < Length(y) : Opposite

Expressed as code:

Code Snippet

127. public delegate void VectorLoop(int cnt, double* d1, int i1, double* d2,int i2, Vector2D* output);
128.  
129. private Dictionary<eFuncSelector, VectorLoop> functable = new Dictionary<eFuncSelector, VectorLoop>();
130.  
131. public DummyVectorDynamic()
132. {
133.     functable.Add(eFuncSelector.XMax, (cnt, x, i1,y, i2, o) => { for (int i = 0; i < cnt; i++) { o[i].x = x[i]; o[i].y = y[i % i2]; } });
134.     functable.Add(eFuncSelector.YMax, (cnt, x, i1, y, i2, o) => { for (int i = 0; i < cnt; i++) { o[i].x = x[i]; o[i%i1].y = y[i]; } });
135.     functable.Add(eFuncSelector.FullSpeed, (cnt, x, i1, y, i2, o) => { for (int i = 0; i < cnt; i++) { o[i].x = x[i]; o[i].y = y[i]; } });
136.     functable.Add(eFuncSelector.SingleX, (cnt, x, i1, y, i2, o) => { double xval = x[0]; for (int i = 0; i < cnt; i++) { o[i].x = xval; o[i].y = y[i]; } });
137.     functable.Add(eFuncSelector.SingleY, (cnt, x, i1, y, i2, o) => { double yval = y[0]; for (int i = 0; i < cnt; i++) { o[i].x = x[i]; o[i].y = yval; } });
138. }

Now in our evaluate method:

Code Snippet

140. public void Evaluate(int SpreadMax)
141. {
142.     dout.Length = SpreadMax * 2;
143.     Vector2D* d = (Vector2D*)dout.Data;
144.  
145.     double* pt1 = d1.Data;
146.     double* pt2 = d2.Data;
147.     int l1 = d1.Length;
148.     int l2 = d2.Length;
149.  
150.     if (l1 == l2) {
151.         functable[eFuncSelector.FullSpeed](SpreadMax, pt1, l1, pt2, l2,d);
152.     } else if (l1 == 1) {
153.         functable[eFuncSelector.SingleX](SpreadMax, pt1, l1, pt2, l2, d);
154.     } else if (l2 == 1) {
155.         functable[eFuncSelector.SingleY](SpreadMax, pt1, l1, pt2, l2, d);
156.     } else if (l1 > l2) {
157.         functable[eFuncSelector.XMax](SpreadMax, pt1, l1, pt2, l2, d);
158.     } else {
159.         functable[eFuncSelector.YMax](SpreadMax, pt1, l1, pt2, l2, d);
160.     }
161. }

Now here are a few benchmarks (considering different cases):

X = 50000, Y = 20
Native: 0.35 ms
Zip : 0.28ms
Mutable : 0.2 ms

This is one of the 2 worst cases, (when X != Y), but we saved a mod on the biggest spread, which technically reflects.

X = 25000, Y = 25000
In that case the node will use : FullSpeed
Native: 0.17ms
Zip : 0.12ms
Mutable : 0.06 ms

In that case we simply ignore all the mod operators, and Mutable clearly wins.

X = 50000, Y = 1
Native : 0.36 ms
Zip : 0.25ms
Mutable : 0.16ms

We can also see that pushing a local and setting other at full speed gives a decent gain. So setting path at runtime is significant.

So technically, that gives us this information:

let function (p1,p2,p3,p4) = dosomething

Now what we have, to generalize our vector, is the following (for each parameter)
if length == 1 -> set local before loop and use that local
if length == SpreadMax -> read without mod
if length < SpreadMax -> read modded

Of course we can clearly see that we have a combinatorial explosion, a function with 8 parameters has a LOT of different cases.

Few options here:
generate the worst case scenario (eg: mod everything).

When you enter Evaluate:
build a mini hash (length == 1) -> 0, (length == SpreadMax) -> 1, (length < SpreadMax) -> 2

Ask the runtime if we already have the function, if not compile it and give it back.

Run the function.

Yes, clearly as you've seen, we have nodes that self compile ;)

So this is of course to use with caution, if your node input length vary a lot, you might have a lot of recompile.
If you have reasonably steady spread counts, you'll have a little overhead on startup. Please note that something like:

let function (a,b) => a+b
a = 2000, b = 1
next frame , a = 1000, b= 1

will not cause a recompile, since a is is still at SpreadMax.

On Some specific notes, node compiler is about 0.5 millisecond, which is pretty fast, but if you have 1000 nodes rebuilding on the fly, that's not too good ;)

What is good is that our node is now a basic container, so we save some il code. Second very cool thing, our logic becomes a delegate, which is much easier to wrap into a Task, but more on that later.

Into Code to IL to Node

I changed the title a bit, but yes I'll kinda speak again of code generation somehow.

I remember some time ago devvvvs shown some prototype to have simpler ways to makes nodes. Basically you create a function and it generates boilerplate code around.
Sadly it never really made it through (maybe they keep it for their new version).

Basically you build a function and you generate the node to fit it.

I started to work on this, I called the project MicroNodes ;)

Here is a simple example

Code Snippet

12. [SlicewiseMethod(Name = "+", Category = "Value")]
13. public static void Add([Input("Input 1")] double d1, [Input("Input 2")] double d2, [Output("Output")] out double dout)
14. {
15.     dout = d1 + d2;
16. }

Ok That's the infamous Add (without group), but you see the idea.

You basically have your operator, and then I build a plugin in intermediate language that prepares data for each slice, then calls the function.

So a dummy implementation looks like this:

Code Snippet

56. [PluginInfo(Name = "Add", Category = "Value", Version = "Slicewise Simple")]
57. public class DummyAdd3 : IPluginEvaluate
58. {
59.     [Input("Input 1")]
60.     private ISpread<double> d1;
61.  
62.     [Input("Input 2")]
63.     private ISpread<double> d2;
64.  
65.     [Output("Output")]
66.     private ISpread<double> dout;
67.  
68.     public void Evaluate(int SpreadMax)
69.     {
70.         dout.SliceCount = SpreadMax;
71.         double dou;
72.         for (int i = 0; i < SpreadMax; i++)
73.         {
74.             SliceWise.Add(d1[i], d2[i],out dou);
75.             dout[i] = dou;
76.         }
77.     }
78. }

You can notice I use output parameter in that case, that will fit some other parts. Here is the same implementation using operator:

Code Snippet

10. [PluginInfo(Name = "Add", Category="Value",Version="Operator Simple")]
11. public class DummyAdd : IPluginEvaluate
12. {
13.     [Input("Input 1")]
14.     private ISpread<double> d1;
15.  
16.     [Input("Input 2")]
17.     private ISpread<double> d2;
18.  
19.     [Output("Output")]
20.     private ISpread<double> dout;
21.  
22.     public void Evaluate(int SpreadMax)
23.     {
24.         dout.SliceCount = SpreadMax;
25.         for (int i = 0; i < SpreadMax; i++)
26.         {
27.             dout[i] = d1[i] + d2[i];
28.         }
29.     }
30. }

You can notice that it's very easy to generate intermediate language for Slicewise methods.

First interesting bit is that both of those are running at same speed. Reason being simple, the JIT, when it converts the Intermediate language into machine code will inline the function call, so assembly wise it's pretty similar.

That's good news, since it means I can easily use functions and call has no overhead.

Now let's see how we can optimize this.
First, any slicewise operation, output result count = spreadmax.
Doing dout[i] does the following in Intermediate Language:
virtcall ISpread<T>._setcyclic()

Setting a slice like this does (for every single write)

• Mark the Spread as dirty (eg: it needs to flush data back into vvvv) 
• Go access the internal DataStore (an array basically)
• Do a Mod to store, output[i%icount] = data

That's actually a lot.

So now we can access the internal store ourselves, this is exposed on Spread Type.

Here is a better version:

Code Snippet

80. [PluginInfo(Name = "Add", Category = "Value", Version = "Slicewise Buffer")]
81. public class DummyAdd4 : IPluginEvaluate
82. {
83.     [Input("Input 1")]
84.     private ISpread<double> d1;
85.  
86.     [Input("Input 2")]
87.     private ISpread<double> d2;
88.  
89.     [Output("Output")]
90.     private ISpread<double> dout;
91.  
92.     public void Evaluate(int SpreadMax)
93.     {
94.         dout.SliceCount = SpreadMax;
95.         double[] b = dout.Stream.Buffer;
96.         for (int i = 0; i < SpreadMax; i++)
97.         {
98.             SliceWise.Add(d1[i], d2[i], out b[i]);
99.         }
100.         dout.Flush(true);
101.     }
102. }

Now, since we access the buffer (and we know we never have to use Mod for output), we get access to the array, and can pass the index as output directly,this does this in IL:

  IL_0039:  ldloc.0 (This is our array local variable)
  IL_003a:  ldloc.1  (This is index)
  IL_003b:  ldelema    [mscorlib]System.Double

So instead it loads the address of the array and pass it to the function.

Pleace note that I call Flush(true), since now I write directly into the internal store, the dirty flag if never set.

So it's easy to add this in the IL generator, and it Gives you a decent substantial boost.

For 5000 elements:
ISpread virtual : 0.1 ms
ISpread buffer : 0.06 ms

For 50000 elements
ISpread virtual : 1 ms
ISpread buffer : 0.6 ms

It's pretty low values but it can easily add up.

So now we can use pointer output, since we can write directly back into memory.
This doesn't give a decent improvement, since a memcpy on 5000 elements is blazing fast, But at least we save some memory (40 kilobytes for a 5000 spread).

It seems kinda minimal, but in a reasonably large patch, it also adds up quickly.

So now the obvious next part is to modify our input. We also do a virtcall for access which also does a mod.

Operator version:

Code Snippet

152. [PluginInfo(Name = "Add", Category = "Value", Version = "Operator Ptr Full")]
153. public unsafe class DummyAdd8 : IPluginEvaluate
154. {
155.     [Input("Input 1")]
156.     private ValueInput d1;
157.  
158.     [Input("Input 2")]
159.     private ValueInput d2;
160.  
161.     [Output("Output")]
162.     private ValueOutput dout;
163.  
164.     public void Evaluate(int SpreadMax)
165.     {
166.         dout.Length = SpreadMax;
167.         double* d = dout.Data;
168.  
169.         double* pt1 = d1.Data;
170.         double* pt2 = d2.Data;
171.  
172.         int l1 = d1.Length;
173.         int l2 = d2.Length;
174.  
175.         for (int i = 0; i < SpreadMax; i++)
176.         {
177.             d[i] = pt1[i%l1] + pt2[i%l2];
178.         }
179.     }
180. }

And Static function version:

Code Snippet

182. [PluginInfo(Name = "Add", Category = "Value", Version = "Slicewise Ptr Full")]
183. public unsafe class DummyAdd10 : IPluginEvaluate
184. {
185.     [Input("Input 1")]
186.     private ValueInput d1;
187.  
188.     [Input("Input 2")]
189.     private ValueInput d2;
190.  
191.     [Output("Output")]
192.     private ValueOutput dout;
193.  
194.     public void Evaluate(int SpreadMax)
195.     {
196.         dout.Length = SpreadMax;
197.         double* d = dout.Data;
198.         double* pt1 = d1.Data;
199.         double* pt2 = d2.Data;
200.         int l1 = d1.Length;
201.         int l2 = d2.Length;
202.         for (int i = 0; i < SpreadMax; i++)
203.         {
204.             SliceWise.Add(pt1[i % l1], pt2[i % l2], out d[i]);
205.         }
206.     }
207. }

As before Operator vs Static function gets pretty much 0 difference, which is still good news, since it means that we can still easily generate that IL code.

It yields a difference compared to ISpread still, here is the table:

For 5000 elements:
ISpread virtual : 0.1 ms
ISpread buffer : 0.06 ms
Ful Pointer : 0.03 ms

For 50000 elements
ISpread virtual : 1 ms
ISpread buffer : 0.6 ms
Ful Pointer : 0.3 ms

Please note that we also save memory for each pin, so let's say we have 5000 elements in first spread and 500 in second, we gain : (5000*8*2) + (500*8) = 84 kilobytes

That's quite pretty decent specially also considering that we have a node that is much faster.

So technically speaking, one very fun thing is that by allowing the user to build this first function as above:

Code Snippet

10. public static void Add(double d1, double d2, out double dout)
11. {
12.     dout = d1 + d2;
13. }

By choosing a route in our IL generator we can make a node 3 times faster compared to the dummy implementation that comes from template.

Also please note that it's much easier to reuse that function or post optimize.
If you have 50 nodes like this, finding a new speedup is a decent amount of refactoring, here, none is needed.

Now I'll just give a second example, since now devvvs have presented their new 5x to 6x faster Vector Join, split, I might put them to the test :)

First thing is indeed, compared to last version they are much faster, no brainer, which is great!

Please note that there's also Zip (Value), which is normally pretty optimized.

So Let's see, Vector (2d Join)

5000 Elements
Native : 0.035 ms
Zip : 0.035 ms

50000 Elements
Native : 0.37 ms
Zip : 0.32 ms

So now I decided to test 3 techniques, so are more code generation friendly, some are more raw plugin, here we go.

First a Vector Join function is simply like this:

Code Snippet

14. [SlicewiseMethod(Name = "Vector", Category = "2d Join")]
15. public static void V2dJoin(
16.     [Input("X")] double x,
17.     [Input("Y")] double y,
18.     [Output("Output")] out Vector2D result)
19. {
20.     result = new Vector2D(x, y);
21. }

Really simple no?

Now here are the testers.

Method one:

Code Snippet

11.   [PluginInfo(Name = "Vector", Category = "2d Join", Version = "Ptr")]
12.   public unsafe class DummyVector: IPluginEvaluate
13.   {
14.       [Input("Input 1")]
15.       private ValueInput d1;
16.  
17.       [Input("Input 2")]
18.       private ValueInput d2;
19.  
20.       [Output("Output")]
21.       private ValueOutput dout;
22.  
23.       public void Evaluate(int SpreadMax)
24.       {
25.           dout.Length = SpreadMax*2;
26.           double* d = dout.Data;
27.  
28.           double* pt1 = d1.Data;
29.           double* pt2 = d2.Data;
30.  
31.           int l1 = d1.Length;
32.           int l2 = d2.Length;
33.  
34.           for (int i = 0; i < SpreadMax; i++)
35.           {
36.               *d = pt1[i % l1];
37.               d++;
38.                 *d=pt2[i % l2];
39.               d++;
40.           }
41.       }
42.   }

Pretty simple, we push per component.

Method 2:

Code Snippet

44. [PluginInfo(Name = "Vector", Category = "2d Join", Version = "Ptr Struct")]
45. public unsafe class DummyVector2 : IPluginEvaluate
46. {
47.     [Input("Input 1")]
48.     private ValueInput d1;
49.  
50.     [Input("Input 2")]
51.     private ValueInput d2;
52.  
53.     [Output("Output")]
54.     private ValueOutput dout;
55.  
56.     public void Evaluate(int SpreadMax)
57.     {
58.         dout.Length = SpreadMax * 2;
59.         Vector2D* d = (Vector2D*)dout.Data;
60.  
61.         double* pt1 = d1.Data;
62.         double* pt2 = d2.Data;
63.  
64.         int l1 = d1.Length;
65.         int l2 = d2.Length;
66.  
67.         for (int i = 0; i < SpreadMax; i++)
68.         {
69.             d->x = pt1[i % l1];
70.             d->y = pt2[i % l2];
71.             d++;
72.         }
73.     }
74. }

This is almost the same, but we build as struct.

Now actually we can also simply use our slicewise:

Code Snippet

76. [PluginInfo(Name = "Vector", Category = "2d Join", Version = "Ptr Out")]
77. public unsafe class DummyVector3 : IPluginEvaluate
78. {
79.     [Input("Input 1")]
80.     private ValueInput d1;
81.  
82.     [Input("Input 2")]
83.     private ValueInput d2;
84.  
85.     [Output("Output")]
86.     private ValueOutput dout;
87.  
88.     public void Evaluate(int SpreadMax)
89.     {
90.         dout.Length = SpreadMax * 2;
91.         Vector2D* d = (Vector2D*)dout.Data;
92.         double* pt1 = d1.Data;
93.         double* pt2 = d2.Data;
94.         int l1 = d1.Length;
95.         int l2 = d2.Length;
96.         for (int i = 0; i < SpreadMax; i++)
97.         {
98.             Joiner.Join(pt1[i % l1], pt2[i % l2], out d[i]);
99.         }
100.     }
101. }

So now the results:

5000 Elements
Native : 0.035 ms
Zip : 0.035 ms
Per Component: 0.035 ms
As Struct: 0.031 ms
Function Output:  0.038 ms

50000 Elements
Native : 0.37 ms
Zip : 0.32 ms
Per Component: 0.35 ms
As Struct: 0.33 ms
Function Output:  0.34 ms

Please note that values fluctuate a little bit.

But basically that means few things:

• Devvvvs did a good job at optimizing vectors
• On very low spread (50), zip seems slower, native is of course fastest (since you have COM object call). It is not as obvious tho.
• A generated version from function is as Fast as optimized native ;)

So to resume, IL code generation is pretty cool.

Now you have a lot of other use cases than SliceWise.

What I decided was to build different templates, so for example:

Code Snippet

12. [ReductorMethod(Name = "+", Category = "Value Spectral")]
13. public static void SpectralAdd([Input("Input")] IEnumerable<double> data,[Output("Result")] out double result)
14. {
15.     result = 0.0;
16.     foreach (double d in data) { result += d; }
17. }

You can also do Struct Join/Split like this:

Code Snippet

65. [Compactor(Name="Struct",Category="Test")]
66. public struct MyStruct2
67. {
68.     [CompactorProperty(Name = "Hello")]
69.     public double Hello;
70.  
71.     [CompactorProperty(Name = "Element2")]
72.     public double Element2;
73.  
74.     [CompactorProperty(Name="Vector",Flatten=true)]
75.     public Vector3 Vector;
76. }

Or have a visitor class in case you want to use an existing struct.

There is some other types/parts, but that will be for another post.