Into Code to IL to Node

I changed the title a bit, but yes I'll kinda speak again of code generation somehow.

I remember some time ago devvvvs shown some prototype to have simpler ways to makes nodes. Basically you create a function and it generates boilerplate code around.
Sadly it never really made it through (maybe they keep it for their new version).

Basically you build a function and you generate the node to fit it.

I started to work on this, I called the project MicroNodes ;)

Here is a simple example

Code Snippet

12. [SlicewiseMethod(Name = "+", Category = "Value")]
13. public static void Add([Input("Input 1")] double d1, [Input("Input 2")] double d2, [Output("Output")] out double dout)
14. {
15.     dout = d1 + d2;
16. }

Ok That's the infamous Add (without group), but you see the idea.

You basically have your operator, and then I build a plugin in intermediate language that prepares data for each slice, then calls the function.

So a dummy implementation looks like this:

Code Snippet

56. [PluginInfo(Name = "Add", Category = "Value", Version = "Slicewise Simple")]
57. public class DummyAdd3 : IPluginEvaluate
58. {
59.     [Input("Input 1")]
60.     private ISpread<double> d1;
61.  
62.     [Input("Input 2")]
63.     private ISpread<double> d2;
64.  
65.     [Output("Output")]
66.     private ISpread<double> dout;
67.  
68.     public void Evaluate(int SpreadMax)
69.     {
70.         dout.SliceCount = SpreadMax;
71.         double dou;
72.         for (int i = 0; i < SpreadMax; i++)
73.         {
74.             SliceWise.Add(d1[i], d2[i],out dou);
75.             dout[i] = dou;
76.         }
77.     }
78. }

You can notice I use output parameter in that case, that will fit some other parts. Here is the same implementation using operator:

Code Snippet

10. [PluginInfo(Name = "Add", Category="Value",Version="Operator Simple")]
11. public class DummyAdd : IPluginEvaluate
12. {
13.     [Input("Input 1")]
14.     private ISpread<double> d1;
15.  
16.     [Input("Input 2")]
17.     private ISpread<double> d2;
18.  
19.     [Output("Output")]
20.     private ISpread<double> dout;
21.  
22.     public void Evaluate(int SpreadMax)
23.     {
24.         dout.SliceCount = SpreadMax;
25.         for (int i = 0; i < SpreadMax; i++)
26.         {
27.             dout[i] = d1[i] + d2[i];
28.         }
29.     }
30. }

You can notice that it's very easy to generate intermediate language for Slicewise methods.

First interesting bit is that both of those are running at same speed. Reason being simple, the JIT, when it converts the Intermediate language into machine code will inline the function call, so assembly wise it's pretty similar.

That's good news, since it means I can easily use functions and call has no overhead.

Now let's see how we can optimize this.
First, any slicewise operation, output result count = spreadmax.
Doing dout[i] does the following in Intermediate Language:
virtcall ISpread<T>._setcyclic()

Setting a slice like this does (for every single write)

• Mark the Spread as dirty (eg: it needs to flush data back into vvvv) 
• Go access the internal DataStore (an array basically)
• Do a Mod to store, output[i%icount] = data

That's actually a lot.

So now we can access the internal store ourselves, this is exposed on Spread Type.

Here is a better version:

Code Snippet

80. [PluginInfo(Name = "Add", Category = "Value", Version = "Slicewise Buffer")]
81. public class DummyAdd4 : IPluginEvaluate
82. {
83.     [Input("Input 1")]
84.     private ISpread<double> d1;
85.  
86.     [Input("Input 2")]
87.     private ISpread<double> d2;
88.  
89.     [Output("Output")]
90.     private ISpread<double> dout;
91.  
92.     public void Evaluate(int SpreadMax)
93.     {
94.         dout.SliceCount = SpreadMax;
95.         double[] b = dout.Stream.Buffer;
96.         for (int i = 0; i < SpreadMax; i++)
97.         {
98.             SliceWise.Add(d1[i], d2[i], out b[i]);
99.         }
100.         dout.Flush(true);
101.     }
102. }

Now, since we access the buffer (and we know we never have to use Mod for output), we get access to the array, and can pass the index as output directly,this does this in IL:

  IL_0039:  ldloc.0 (This is our array local variable)
  IL_003a:  ldloc.1  (This is index)
  IL_003b:  ldelema    [mscorlib]System.Double

So instead it loads the address of the array and pass it to the function.

Pleace note that I call Flush(true), since now I write directly into the internal store, the dirty flag if never set.

So it's easy to add this in the IL generator, and it Gives you a decent substantial boost.

For 5000 elements:
ISpread virtual : 0.1 ms
ISpread buffer : 0.06 ms

For 50000 elements
ISpread virtual : 1 ms
ISpread buffer : 0.6 ms

It's pretty low values but it can easily add up.

So now we can use pointer output, since we can write directly back into memory.
This doesn't give a decent improvement, since a memcpy on 5000 elements is blazing fast, But at least we save some memory (40 kilobytes for a 5000 spread).

It seems kinda minimal, but in a reasonably large patch, it also adds up quickly.

So now the obvious next part is to modify our input. We also do a virtcall for access which also does a mod.

Operator version:

Code Snippet

152. [PluginInfo(Name = "Add", Category = "Value", Version = "Operator Ptr Full")]
153. public unsafe class DummyAdd8 : IPluginEvaluate
154. {
155.     [Input("Input 1")]
156.     private ValueInput d1;
157.  
158.     [Input("Input 2")]
159.     private ValueInput d2;
160.  
161.     [Output("Output")]
162.     private ValueOutput dout;
163.  
164.     public void Evaluate(int SpreadMax)
165.     {
166.         dout.Length = SpreadMax;
167.         double* d = dout.Data;
168.  
169.         double* pt1 = d1.Data;
170.         double* pt2 = d2.Data;
171.  
172.         int l1 = d1.Length;
173.         int l2 = d2.Length;
174.  
175.         for (int i = 0; i < SpreadMax; i++)
176.         {
177.             d[i] = pt1[i%l1] + pt2[i%l2];
178.         }
179.     }
180. }

And Static function version:

Code Snippet

182. [PluginInfo(Name = "Add", Category = "Value", Version = "Slicewise Ptr Full")]
183. public unsafe class DummyAdd10 : IPluginEvaluate
184. {
185.     [Input("Input 1")]
186.     private ValueInput d1;
187.  
188.     [Input("Input 2")]
189.     private ValueInput d2;
190.  
191.     [Output("Output")]
192.     private ValueOutput dout;
193.  
194.     public void Evaluate(int SpreadMax)
195.     {
196.         dout.Length = SpreadMax;
197.         double* d = dout.Data;
198.         double* pt1 = d1.Data;
199.         double* pt2 = d2.Data;
200.         int l1 = d1.Length;
201.         int l2 = d2.Length;
202.         for (int i = 0; i < SpreadMax; i++)
203.         {
204.             SliceWise.Add(pt1[i % l1], pt2[i % l2], out d[i]);
205.         }
206.     }
207. }

As before Operator vs Static function gets pretty much 0 difference, which is still good news, since it means that we can still easily generate that IL code.

It yields a difference compared to ISpread still, here is the table:

For 5000 elements:
ISpread virtual : 0.1 ms
ISpread buffer : 0.06 ms
Ful Pointer : 0.03 ms

For 50000 elements
ISpread virtual : 1 ms
ISpread buffer : 0.6 ms
Ful Pointer : 0.3 ms

Please note that we also save memory for each pin, so let's say we have 5000 elements in first spread and 500 in second, we gain : (5000*8*2) + (500*8) = 84 kilobytes

That's quite pretty decent specially also considering that we have a node that is much faster.

So technically speaking, one very fun thing is that by allowing the user to build this first function as above:

Code Snippet

10. public static void Add(double d1, double d2, out double dout)
11. {
12.     dout = d1 + d2;
13. }

By choosing a route in our IL generator we can make a node 3 times faster compared to the dummy implementation that comes from template.

Also please note that it's much easier to reuse that function or post optimize.
If you have 50 nodes like this, finding a new speedup is a decent amount of refactoring, here, none is needed.

Now I'll just give a second example, since now devvvs have presented their new 5x to 6x faster Vector Join, split, I might put them to the test :)

First thing is indeed, compared to last version they are much faster, no brainer, which is great!

Please note that there's also Zip (Value), which is normally pretty optimized.

So Let's see, Vector (2d Join)

5000 Elements
Native : 0.035 ms
Zip : 0.035 ms

50000 Elements
Native : 0.37 ms
Zip : 0.32 ms

So now I decided to test 3 techniques, so are more code generation friendly, some are more raw plugin, here we go.

First a Vector Join function is simply like this:

Code Snippet

14. [SlicewiseMethod(Name = "Vector", Category = "2d Join")]
15. public static void V2dJoin(
16.     [Input("X")] double x,
17.     [Input("Y")] double y,
18.     [Output("Output")] out Vector2D result)
19. {
20.     result = new Vector2D(x, y);
21. }

Really simple no?

Now here are the testers.

Method one:

Code Snippet

11.   [PluginInfo(Name = "Vector", Category = "2d Join", Version = "Ptr")]
12.   public unsafe class DummyVector: IPluginEvaluate
13.   {
14.       [Input("Input 1")]
15.       private ValueInput d1;
16.  
17.       [Input("Input 2")]
18.       private ValueInput d2;
19.  
20.       [Output("Output")]
21.       private ValueOutput dout;
22.  
23.       public void Evaluate(int SpreadMax)
24.       {
25.           dout.Length = SpreadMax*2;
26.           double* d = dout.Data;
27.  
28.           double* pt1 = d1.Data;
29.           double* pt2 = d2.Data;
30.  
31.           int l1 = d1.Length;
32.           int l2 = d2.Length;
33.  
34.           for (int i = 0; i < SpreadMax; i++)
35.           {
36.               *d = pt1[i % l1];
37.               d++;
38.                 *d=pt2[i % l2];
39.               d++;
40.           }
41.       }
42.   }

Pretty simple, we push per component.

Method 2:

Code Snippet

44. [PluginInfo(Name = "Vector", Category = "2d Join", Version = "Ptr Struct")]
45. public unsafe class DummyVector2 : IPluginEvaluate
46. {
47.     [Input("Input 1")]
48.     private ValueInput d1;
49.  
50.     [Input("Input 2")]
51.     private ValueInput d2;
52.  
53.     [Output("Output")]
54.     private ValueOutput dout;
55.  
56.     public void Evaluate(int SpreadMax)
57.     {
58.         dout.Length = SpreadMax * 2;
59.         Vector2D* d = (Vector2D*)dout.Data;
60.  
61.         double* pt1 = d1.Data;
62.         double* pt2 = d2.Data;
63.  
64.         int l1 = d1.Length;
65.         int l2 = d2.Length;
66.  
67.         for (int i = 0; i < SpreadMax; i++)
68.         {
69.             d->x = pt1[i % l1];
70.             d->y = pt2[i % l2];
71.             d++;
72.         }
73.     }
74. }

This is almost the same, but we build as struct.

Now actually we can also simply use our slicewise:

Code Snippet

76. [PluginInfo(Name = "Vector", Category = "2d Join", Version = "Ptr Out")]
77. public unsafe class DummyVector3 : IPluginEvaluate
78. {
79.     [Input("Input 1")]
80.     private ValueInput d1;
81.  
82.     [Input("Input 2")]
83.     private ValueInput d2;
84.  
85.     [Output("Output")]
86.     private ValueOutput dout;
87.  
88.     public void Evaluate(int SpreadMax)
89.     {
90.         dout.Length = SpreadMax * 2;
91.         Vector2D* d = (Vector2D*)dout.Data;
92.         double* pt1 = d1.Data;
93.         double* pt2 = d2.Data;
94.         int l1 = d1.Length;
95.         int l2 = d2.Length;
96.         for (int i = 0; i < SpreadMax; i++)
97.         {
98.             Joiner.Join(pt1[i % l1], pt2[i % l2], out d[i]);
99.         }
100.     }
101. }

So now the results:

5000 Elements
Native : 0.035 ms
Zip : 0.035 ms
Per Component: 0.035 ms
As Struct: 0.031 ms
Function Output:  0.038 ms

50000 Elements
Native : 0.37 ms
Zip : 0.32 ms
Per Component: 0.35 ms
As Struct: 0.33 ms
Function Output:  0.34 ms

Please note that values fluctuate a little bit.

But basically that means few things:

• Devvvvs did a good job at optimizing vectors
• On very low spread (50), zip seems slower, native is of course fastest (since you have COM object call). It is not as obvious tho.
• A generated version from function is as Fast as optimized native ;)

So to resume, IL code generation is pretty cool.

Now you have a lot of other use cases than SliceWise.

What I decided was to build different templates, so for example:

Code Snippet

12. [ReductorMethod(Name = "+", Category = "Value Spectral")]
13. public static void SpectralAdd([Input("Input")] IEnumerable<double> data,[Output("Result")] out double result)
14. {
15.     result = 0.0;
16.     foreach (double d in data) { result += d; }
17. }

You can also do Struct Join/Split like this:

Code Snippet

65. [Compactor(Name="Struct",Category="Test")]
66. public struct MyStruct2
67. {
68.     [CompactorProperty(Name = "Hello")]
69.     public double Hello;
70.  
71.     [CompactorProperty(Name = "Element2")]
72.     public double Element2;
73.  
74.     [CompactorProperty(Name="Vector",Flatten=true)]
75.     public Vector3 Vector;
76. }

Or have a visitor class in case you want to use an existing struct.

There is some other types/parts, but that will be for another post.