On plugin architecture

I was looking a bit a decoupling my code in a better way, for the moment Dx11 nodes follow the current way:

• Implement IPluginEvaluate
• Implement either ResourceProvider or ResourceContextProvider

Then when you Implement ResourceProvider you use : SetDevice(RenderDevice device)

In some way this forces you to register a SetDevice method (so you don't forget to grab device), but this is not ideal in many points.

One thing I don't like with this, you have too much cooking and so much work is wasted on initialization order. Let's take a very basic example:

Code Snippet

11. [PluginInfo(Name = "CopyCounter", Category = "DX11.Buffer", Version = "", Author = "vux")]
12. public class CopyCounterNode : IPluginEvaluate,IDxDeviceListener, IDxResourceContextProvider
13. {
14.     [Input("Buffer In", DefaultValue = 1,IsSingle=true)]
15.     protected Pin<DX11Resource<DX11StructuredBuffer>> FInBuffer;
16.  
17.     [Output("Buffer Out",IsSingle=true)]
18.     protected ISpread<DX11Resource<DX11RawBuffer>> FOutBuffer;
19.  
20.     private RenderDevice device;
21.  
22.     public void SetDevice(RenderDevice device)
23.     {
24.         this.device = device;
25.         this.FOutBuffer[0] = new DX11Resource<DX11RawBuffer>();
26.     }
27. }

Please note that I omitted the update/evaluate for clarity.

So what happens is First object constructor is called (here a default constrcutor since none is specified),
then Buffer In and Buffer Out are injected into the node, to finish, once node is ready, SetDevice is called to the node.

Now if I add a constructor, we here have the same:

Code Snippet

11. [PluginInfo(Name = "CopyCounter", Category = "DX11.Buffer", Version = "", Author = "vux")]
12. public class CopyCounterNode : IPluginEvaluate,IDxDeviceListener, IDxResourceContextProvider
13. {
14.     [Input("Buffer In", DefaultValue = 1,IsSingle=true)]
15.     protected Pin<DX11Resource<DX11StructuredBuffer>> FInBuffer;
16.  
17.     [Output("Buffer Out",IsSingle=true)]
18.     protected ISpread<DX11Resource<DX11RawBuffer>> FOutBuffer;
19.  
20.     private RenderDevice device;
21.     private IPluginHost host;
22.  
23.     [ImportingConstructor()]
24.     public CopyCounterNode(IPluginHost host)
25.     {
26.         this.host = host;
27.     }
28.  
29.     public void SetDevice(RenderDevice device)
30.     {
31.         this.device = device;
32.         this.FOutBuffer[0] = new DX11Resource<DX11RawBuffer>();
33.     }
34. }

Please note that first here there's 2 things which are pretty bad:

• I need to use ImportingConstructor otherwise MEF ignores injection (and plugin creation will fail). This is just really bad. That means I need to add an attribute on every single class of my model. Which, by propagating, would mean I'd need to use MEF even in my standalone library! Even worse, you also need to use attribute in every subclass, since they are not propagated.
• Using attributes for input/output sounds a good idea, saves some time, but that's reasonably true for simple nodes, where you use inheritance, that creates issue. If I mark my pins as private, I get compiler warning (rightly so), you can use pragma to disable, but then you need to do that on all nodes, really intrusive as well.

Now one cool thing, I can export my DX11 Services, and have them injected, so instead I can do :

Code Snippet

11. [PluginInfo(Name = "CopyCounter", Category = "DX11.Buffer", Version = "", Author = "vux")]
12. public class CopyCounterNode : IPluginEvaluate, IDxResourceContextProvider
13. {
14.     [Input("Buffer In", DefaultValue = 1,IsSingle=true)]
15.     protected Pin<DX11Resource<DX11StructuredBuffer>> FInBuffer;
16.  
17.     [Output("Buffer Out",IsSingle=true)]
18.     protected ISpread<DX11Resource<DX11RawBuffer>> FOutBuffer;
19.  
20.     private RenderDevice device;
21.     private IPluginHost host;
22.  
23.     [ImportingConstructor()]
24.     public CopyCounterNode(RenderDevice device, IPluginHost host)
25.     {
26.         this.host = host;
27.         this.device = device;
28.         this.FOutBuffer[0] = new DX11Resource<DX11RawBuffer>();
29.     }
30. }

But this will of course miserably fail.
The simple reason is that FOutBuffer is injected after the constructor. You can't get a proper working object.

2 Solutions for this:

• Implement IPartImportsSatisfiedNotification , and move the buffer initialization into this. Hurray,one more interface, one more remote method to implement, more intrusion.
• Create pins by hand in constructor, via IIOFactory. Example below:

Code Snippet

11.   [PluginInfo(Name = "CopyCounter", Category = "DX11.Buffer", Version = "", Author = "vux")]
12.   public class CopyCounterNode : IPluginEvaluate, IDxResourceContextProvider, IPartImportsSatisfiedNotification
13.   {
14.       private Pin<DX11Resource<DX11StructuredBuffer>> FInBuffer;
15.       private ISpread<DX11Resource<DX11RawBuffer>> FOutBuffer;
16.  
17.       private RenderDevice device;
19.       [ImportingConstructor()]
20.       public CopyCounterNode(RenderDevice device, IIOFactory iofactory)
21.       {
22.           this.device = device;
23.           this.FInBuffer = iofactory.CreateResourceInputPin<DX11StructuredBuffer>("Buffer In",true);
24.           this.FOutBuffer = iofactory.CreateResourceOutputPin<DX11RawBuffer>("Buffer Out",true);
25.           this.FOutBuffer[0] = new DX11Resource<DX11RawBuffer>();
26.       }
27.   }

This feels a little bit more verbose, but now you have something correct (eg : when your object is instanciated it is ready to use, no more dodgy ordering. Initialization is in one place, and you can easily add some extensions methods/static methods to IOFactory to reduce overhead.

Another great advantage, you can create some nicer extensions like :

Code Snippet

45. public static Pin<DX11Resource<DX11Layer>> CreateDX11LayerOut(
46.     this IIOFactory iofactory,
47.     RenderDevice device,
48.     RenderDelegate<IPluginIO, RenderSettings> rendermethod,
49.     string name = "Layer Out",
50.     bool subscribe = true)
51. {
52.     var attr = new OutputAttribute(name) { IsSingle = true };
53.     var pin = iofactory.CreateResourceOutputPin<DX11Layer>(name, subscribe);
54.     pin[0] = new DX11Resource<DX11Layer>();
55.     pin[0][device] = new DX11Layer();
56.     pin[0][device].Render = rendermethod;
57.     return pin;
58. }

Now if you call CreateDX11LayerOut, you even took care of the naming by providing a default, which is much easier to mess up with attributes (random typo, no idea of naming...)

Last issue then, this ImportingConstructor is annoying, specially it needs to be everywhere.

So I tried to check If I could get rid of MEF (partly), and build a small plugin interface with a far better container, eg : Autofac

One thing I like is all initialization code is centralized (and not scattered randomly like in MEF so you don't know what exported/imported anymore).

Building container is rather simple:

Code Snippet

96. ContainerBuilder cb = new ContainerBuilder();
97. cb.RegisterInstance<IHDEHost>(hdehost).ExternallyOwned();
98. cb.RegisterInstance<INodeInfoFactory>(ni).ExternallyOwned();
99. cb.RegisterInstance<IORegistry>(ioreg).ExternallyOwned().As<IIORegistry>();
100. cb.RegisterInstance<ILogger>(logger).ExternallyOwned();
101. cb.RegisterInstance(this.RenderDevice).As<DxDevice,RenderDevice>().ExternallyOwned();
102.  
103. this.container = cb.Build();

Please note that I didn't added all services here, but you also have a lot of nitfy features, and using AutoFac I can now technically even use composition on my core runtime, which was not possible with MEF (except adding attributes everywhere).

More fun, I can import all vvvv services in one go using their Integration library.

Now to register a plugin:

Code Snippet

40. container = parent.BeginLifetimeScope
41. (
42.     (cb) =>
43.     {
44.         cb.Register(c => pluginHost).As<IPluginHost, IPluginHost2, INode>().As<IInternalPluginHost>().ExternallyOwned();
45.         cb.RegisterType<AutoFacIOFactory>().As<AutoFacIOFactory, IIOFactory>().InstancePerLifetimeScope();
46.         cb.RegisterType(pluginType).As<IPluginEvaluate>().InstancePerLifetimeScope();
47.     }
48. );
49.  
50. this.iofactory = container.Resolve<AutoFacIOFactory>();
51. this.PluginBase = container.Resolve<IPluginEvaluate>();
52. autoevaluate = nodeInfo.AutoEvaluate;
53. iofactory.OnCreated(EventArgs.Empty);

Also really simple, and can be even more simplified, but I wanted to do a quick try.

Best of all now, I have proper inheritance support, keeping what I want private/protected and so on:

Code Snippet

11. [PluginInfo(Name = "CopyCounter", Category = "DX11.Buffer", Version = "", Author = "vux")]
12. public class CopyCounterNode : IPluginEvaluate, IDxResourceContextProvider
13. {
14.     private Pin<DX11Resource<DX11StructuredBuffer>> FInBuffer;
15.     private ISpread<DX11Resource<DX11RawBuffer>> FOutBuffer;
16.  
17.     protected RenderDevice Device { get; private set; }
18.  
19.     public CopyCounterNode(RenderDevice device, IIOFactory iofactory)
20.     {
21.         this.Device = device;
22.         this.FInBuffer = iofactory.CreateResourceInputPin<DX11StructuredBuffer>("Buffer In",true);
23.         this.FOutBuffer = iofactory.CreateResourceOutputPin<DX11RawBuffer>("Buffer Out",true);
24.         this.FOutBuffer[0] = new DX11Resource<DX11RawBuffer>();
25.     }
26. }
27.  
28. [PluginInfo(Name = "CopyCounter2", Category = "DX11.Buffer", Version = "", Author = "vux")]
29. public class CopyCounterNode2 : CopyCounterNode
30. {
31.     private ISpread<DX11Resource<DX11RawBuffer>> FOutBuffer2;
32.  
33.     public CopyCounterNode2(RenderDevice device, IIOFactory iofactory) : base(device,iofactory)
34.     {
35.         this.FOutBuffer2 = iofactory.CreateResourceOutputPin<DX11RawBuffer>("Buffer Out",true);
36.         this.FOutBuffer2[0] = new DX11Resource<DX11RawBuffer>();
37.     }
38. }

No more references to MEF, inheritance is streamlined. Also now you can see that device can't be written by then child class, and no more compiler warning.

Technically class is now perfectly isolated, and node don't know about autofac or using a container. The IOC manages it for you, you can replace it by another one if you want, or make you own barebone version.

Now you also have proper initialization at the place it should be : the constructor.

And as usual one very funny thing, this is MUCH less code than the .NET factory for the same features ;)

Happy new year, News, and thoughts

First Happy new year!

2013 was pretty great (first DX11 release, Node13, lots of interesting projects, birth of a new tool ;)

Hope 2014 will follow, and to be honest, it's starting quite in many interesting ways.

First thing I'm reasonably happy on adoption for DirectX11, still some parts are missing, but most times people going into it don't go back (which is a good sign).

There's still some nodes missing, but well I'm kinda more or less on my own writing it (thanks to the few people who contributed nodes), on top of various projects, so it's a lot of time (gladly I use it for projects, so at least I can add some bug fixes/Few more nodes, but nodes are generally so specific they not really much point pushing to core).

If a few wannabe c# programmers decide to pop in and help add some of the missing nodes (Text Geometry/XFile writer/Player...), please feel free to ping me, I'll happily help with basics of coding a Dx11 node (it's much easier than writing a DX9 one believe me ;)

Also of course thanks to people who started to post dx11 contributions, I believe that helps adoption, and at least I can spend more efforts on the runtime instead.

On the same way I'd love few people to write help patches and more examples, it also takes time, I rather enjoy to explain routines, but same thing = time.

So now 2014 started on a quite fast paced mode (I spent bit of time between xmas and new year too), let's see what's going on:

• Port to core runtime to SharpDX is pretty much complete, nicer, faster, simpler. That means support to latest versions, and support up to Win7 is there.
• Texture was bit of an issue, but thanks to directxtex.codeplex.com , few exports and P/Invoke, now have a portable Texture reader/writer, with tga support on the way, and BC7 encoder, if you survive the "very" slow export, in order to enjoy the very fast import ;)
• Most of old nodes are also done (i'd say 80% of the nodes are ported)
• Many type of nodes are now templated (using T4 templates), so most geometry/layer and others, are just reflected and node is generated from that, makes refactoring a hell of a lot easier).
• Shader nodes have been reasonably revamped.
• Lot of new interesting/experimental features on the way too ;)
• Many high level nodes, so non shader experts can start to use their gpu in a reasonable way :)

Now of course there's still work to do, Pin system (once greg will have sorted the convolution issue), will also get some revamp, Resource management is getting along more improvements, and still some more questions about some other bits (see below).

Please note that most work as been done to make API simpler. API now uses less code, is faster, and I find using it is also less code. Writing a node with new API is really simple, there's much less cooking around.

First session of rolling out (pre alpha early bird to not use for production you've been warned) small release is incoming, so will be able to see how things are getting along.

There is still some bits I'd like to work on, in no particular order:

• Code editor: Time to really start to think as fx files as projects, and just not a window with unusable code completion. 
• Improve elements decoupling: Some more core, vvvv being so tied to MEF it's gonna be fun, but there's some big rationale behind it, which deserves a post on it's own.
• Get rid of property injection : So i can really get proper inheritance control , and nodes could even be reused.
• More high level nodes: No comment, everyone likes high level nodes
• New transforms : The funny thing is actuallyafter some thoughts, moving transforms to a new type opens a hell of a lot of new doors, more on that later, that deserves another post.
• Continue to simplify: It doesn't mean lot of small files with 2 lines of code each (no offense to java people, which actually can't since their header is already filled with imports ;) Simpler is better, I don't need the most amazing configurable runtime, I need a lightweight, easy to use and fast runtime!
• Some more top secret things are also on the way ;)

As usual, thanks for reading and happy new year again !