Saturday, 31 May 2014

Reflecting Types

Lately I've been posting about using IL generation to connect elements.

I know I was about to post series about Scene Graph, but since I like to do things in an non ordered fashion (and at the end of the day, it's my blog ;) , I'll go a bit deeper into objects.

In most graph oriented architecture, we have to create nodes, and connectors, so we have to build inputs/outputs...

We have several ways to do it:

  • Big factory: You have calls like CreateIntParameter, CreateValueInput, or generic version like CreateInput<T> ...
  • Injection : You use decorators in order to inject some implementation. (So doing ISpread<int> internally generates a IntInputPin or something like that).
Going deeper, why do we even need pins? Let's take the following example:

Code Snippet
  1. public class Prop
  2. {
  3.     public double Hello { get; set; }
  5.     public int Integer { get; set; }
  7.     public string ReadOnly { get { return "ReadOnly"; } }
  8. }
  10. public class Prop2
  11. {
  12.     public double Double { get; set; }
  14.     public string DoubleOutput { get; set; }
  15. }

This is pretty easy to reflect properties, since we know about get/set, we can create a "virtual pin" automatically, so why even wrap the property?

In most systems we would work it like that:

  • Wrap hello as input and Output Pin. 
  • Wrap ReadOnly as Output pin.
  • Create a link that wraps InputPin<double> and OutputPin<double>

So let's do this in a simpler way, let's say we want to connect Hello (output) to Prop2 "DoubleOutput"

First we need a Func to access Property:

Code Snippet
  1. public static Func<T> BuildGetter<T>(object instance, PropertyInfo property)
  2. {
  3.     return (Func<T>)Delegate.CreateDelegate(typeof(Func<T>), instance, property.GetGetMethod());
  4. }

And a way to set property:

Code Snippet
  1. static Action<T> BuildSetter<T>(object instance, PropertyInfo property)
  2. {
  3.     return (Action<T>)Delegate.CreateDelegate(typeof(Action<T>), instance, property.GetSetMethod());
  4. }

Now we have to connect both, eg: read from source and write to destination:

Code Snippet
  1. static Action Connect<T>(object source, PropertyInfo getter, object dest, PropertyInfo setter)
  2. {
  3.     var getFunc = BuildGetter<T>(source, getter);
  4.     var setAction = BuildSetter<T>(dest, setter);
  5.     return () => setAction(getFunc());
  6. }

That was that easy, now we can simply have:

Code Snippet
  1. Prop p = new Prop()
  2. {
  3.     Hello = 20.0,
  4.     Integer = 20
  5. };
  7. Prop2 p2 = new Prop2();
  9. var connect = Connect<double>(p, typeof(Prop).GetProperty("Hello"), p2, typeof(Prop2).GetProperty("Double"));
  10. connect();

All done, out connector is ready ;)

If we need conversion, this is as easy, we can use a small interface:

Code Snippet
  1. public interface ITypeConverter<TSource,TDest>
  2. {
  3.     TDest Convert(TSource source);
  4. }
  6. public class DoubleToStringConverter : ITypeConverter<double, string>
  7. {
  8.     public string Convert(double source)
  9.     {
  10.         return source.ToString() + " from converter";
  11.     }
  12. }

Then our connector can either use this small interface, or we can just set a lambda is case we can't be bother ed adding one file + namespaces + new class, get rid of object oriented verbosity basically ;)

Code Snippet
  1. static Action Connect<TSource, TDest>(object source, PropertyInfo getter, object dest, PropertyInfo setter, Func<TSource,TDest> converter)
  2. {
  3.     var getFunc = BuildGetter<TSource>(source, getter);
  4.     var setAction = BuildSetter<TDest>(dest, setter);
  5.     return () => setAction(converter(getFunc()));
  6. }
  8. static Action Connect<TSource, TDest>(object source, PropertyInfo getter, object dest, PropertyInfo setter, ITypeConverter<TSource, TDest> converter)
  9. {
  10.     var getFunc = BuildGetter<TSource>(source, getter);
  11.     var setAction = BuildSetter<TDest>(dest, setter);
  12.     return () => setAction(converter.Convert(getFunc()));
  13. }

In action:

Code Snippet
  1. var connectConverter = Connect<double, string>(p, typeof(Prop).GetProperty("Hello"), p2, typeof(Prop2).GetProperty("DoubleOutput"), (d) => d.ToString() + " Hello");
  3. connectConverter();
  5. var connectInterface = Connect<double, string>(p, typeof(Prop).GetProperty("Hello"), p2, typeof(Prop2).GetProperty("DoubleOutput"), new DoubleToStringConverter());
  7. connectInterface();

Now we can actually simply reflect type, create IO on rules, but operate directly on properties.
Direct access costs us a few virtual calls, but since we haven't got a layer on top, this is even more minimized.

Once we have this, let's looks at it further, let's show how to display our element, with custom formatted string per type, in this example I just log to console, bu this is easily extendable to user interface, writing, serialization...

Code Snippet
  1. public interface IPropertyDisplayFactory
  2. {
  3.     Type PropertyType { get; }
  4.     IPropertyDisplay DisplayObject(object instance, PropertyInfo property);
  5. }
  7. public interface IPropertyDisplay
  8. {
  9.     void Display();
  10. }

First we do some simple defaults:

Code Snippet
  1. public class BasicDisplay : IPropertyDisplay
  2. {
  3.     private readonly Action display;
  5.     public BasicDisplay(Action display)
  6.     {
  7.         this.display = display;
  8.     }
  10.     public void Display()
  11.     {
  12.         display();
  13.     }
  14. }
  16. public abstract class TypedDisplayFactory<T> : IPropertyDisplayFactory
  17. {
  18.     public Type PropertyType
  19.     {
  20.         get { return typeof(T); }
  21.     }
  23.     public abstract IPropertyDisplay DisplayObject(object instance, PropertyInfo property);
  24. }

Then two stupid int and double implementations:

Code Snippet
  1. public class DoubleDisplayFactory : TypedDisplayFactory<double>
  2. {
  3.     public override IPropertyDisplay DisplayObject(object instance, PropertyInfo property)
  4.     {
  5.         Func<double> getter = Program.BuildGetter<double>(instance, property);
  7.         Action action = () =>
  8.             {
  9.                 Console.WriteLine(property.Name + " Double: " + getter().ToString());
  10.             };
  12.         return new BasicDisplay(action);
  13.     }
  14. }
  16. public class IntDisplayFactory : TypedDisplayFactory<int>
  17. {
  18.     public override IPropertyDisplay DisplayObject(object instance, PropertyInfo property)
  19.     {
  20.         Func<int> getter = Program.BuildGetter<int>(instance, property);
  22.         Action action = () =>
  23.         {
  24.             Console.WriteLine(property.Name + "Int: " + getter().ToString());
  25.         };
  27.         return new BasicDisplay(action);
  28.     }
  29. }

Now we need to know if a property is capable of display of not:

Code Snippet
  1. public class DisplayPropertyRegistry
  2. {
  3.     private readonly IEnumerable<IPropertyDisplayFactory> factories;
  5.     public DisplayPropertyRegistry(IEnumerable<IPropertyDisplayFactory> factories)
  6.     {
  7.         this.factories = factories;
  8.     }
  10.     public DisplayPropertyRegistry(params IPropertyDisplayFactory[] factories)
  11.     {
  12.         this.factories = factories;
  13.     }
  15.     private IEnumerable<IPropertyDisplayFactory> SearchCandidate(Type type)
  16.     {
  17.         return factories.Where(f => f.PropertyType == type);
  18.     }
  20.     public bool CanDisplay(Type type)
  21.     {
  22.         return SearchCandidate(type).FirstOrDefault() != null;
  23.     }
  25.     public IPropertyDisplayFactory GetFactory(Type type)
  26.     {
  27.         return SearchCandidate(type).First();
  28.     }
  29. }

Add filtering support:

Code Snippet
  1.   public class DisplayPropertyFilter
  2.   {
  3.       private readonly DisplayPropertyRegistry registry;
  5.       public DisplayPropertyFilter(DisplayPropertyRegistry registry)
  6.       {
  7.           this.registry = registry;
  8.       }
  10.       public IEnumerable<Tuple<PropertyInfo,IPropertyDisplayFactory>> ReflectType(Type type)
  11.       {
  12.           var props = type.GetProperties().Where(p => p.GetGetMethod() != null && registry.CanDisplay(p.PropertyType));
  14.             return props.Select(p => newTuple<PropertyInfo,IPropertyDisplayFactory>(p, registry.GetFactory(p.PropertyType)));
  15.       }
  16.   }

From here we know from a given type, every property that we can display, and how to create them. We only used object type so far, so now we need to add instance in the mix:

Code Snippet
  1. public class DisplayObjectFactory
  2. {
  3.     private readonly DisplayPropertyFilter filter;
  5.     public DisplayObjectFactory(DisplayPropertyFilter filter)
  6.     {
  7.         this.filter = filter;
  8.     }
  10.     public ObjectDisplay GetDisplay(object instance)
  11.     {
  12.         var factories = filter.ReflectType(instance.GetType());
  14.         var elements = factories.Select(f => f.Item2.DisplayObject(instance, f.Item1));
  16.         return new ObjectDisplay(elements);
  17.     }
  18. }

And the final object that displays our information:

Code Snippet
  1. public class ObjectDisplay
  2. {
  3.     private readonly IEnumerable<IPropertyDisplay> displayElements;
  5.     public ObjectDisplay(IEnumerable<IPropertyDisplay> displayElements)
  6.     {
  7.         this.displayElements = displayElements;
  8.     }
  10.     public void Display()
  11.     {
  12.         foreach (IPropertyDisplay display in displayElements)
  13.         {
  14.             display.Display();
  15.         }
  16.     }
  17. }

Our final code, that gets an instance and displays any supported data:

Code Snippet
  1. DisplayPropertyRegistry registry = new DisplayPropertyRegistry(
  2.     new DoubleDisplayFactory(),
  3.     new IntDisplayFactory());
  5. DisplayPropertyFilter filter = new DisplayPropertyFilter(registry);
  7. DisplayObjectFactory builder = new DisplayObjectFactory(filter);
  9. Prop p = new Prop()
  10. {
  11.     Hello = 20.0,
  12.     Integer = 20
  13. };
  15. var display = builder.GetDisplay(p);
  16. display.Display();

You can easily think that's a lot of classes and a lot of code to just display properties of a class (we could just iterate through property list and do some "tostring"


  • This is more extensible, and if you need to access your properties a lot, you get a pretty big gain over reflection.
  • You can register any type, replace as you wish, so you avoid the dreaded massive switch.
  • By having more classes, and less work per class, you have much more invariants, pretty much every class is ready to go once constructed, no temporal coupling (only edge case is CanDisplay/GetFactory which could create a related exception).
  • You are not limited to just a "tostring", create a controller, network serializer, add a proxy for automatic property change dispatch, possibilities are endless ;)
Here we are for now, promised I'll get back into Scene Graph next post (or maybe not ;)

Wednesday, 28 May 2014

(Reasonably) Modern SceneGraph (Introduction)

I decided to go a bit deeper into how FlareTic internals are handled, and decided that next posts will be some series about SceneGraph.

Since my tool is not designed to be a game engine, but for procedural graphics, I have some different requirements/limitations that need a different thinking.

So let's see what we will cover, and what we need for a reasonably modern rendering system. Most code will be in c#, but you'll likely see a bit of F# appearing too ;)

1/Asset management

This is a no brainer, asset managers are necessary everywhere, whereas it is from textures/models.

Main required features are:

  • Background loading: You don't want to block your rendering while you load this fat 3d distance field.
  • Converters : You will likely want to convert your data into some internal representation. For example, prepare a 3d model in a way that you can directly upload to gpu, or take a bulk of textures and convert to texture array.
  • Generators : Most times we also need more procedural assets, so a simple generator function is also eligible as an asset (random buffers is something I widely use).
  • Child asset : For example, from a 3d model asset, you want a distance field representation.
  • Metadata : 2D Texture, 3D Texture don't make any sense on it's own, so we also need to indicate what data is contained in the texture (distance field/potential field/velocity/normal map/height map....)
  • Drag drop handler so when you drop on the patch it creates appropriate node.
  • Hierarchy : so if you have a texture folder (namespace), you can prefetch all resources in that folder for fast switching.

2/Render Elements

At some point in our scene, we need some resources which are not part of our asset system (some shaders, some buffers....)

Here are all render elements in FlareTic
  • Fully immutable: Those resources are loaded once per node, and will never change. (Shaders are a very good example for it).
  • Low update frequency : Those are the ones that don't change very often (actually mostly at design time). So they are built as immutable in GPU, and dumped/recreated when required by the user. Very common example will be standard primitives. Since resource is immutable in GPU, it can be uploaded async without a problem.
  • Dynamic Resources (CPU) : Those are resources which change pretty often, like any camera textures. In FlareTic they are built as dynamic, and uploaded to gpu using MapSubResource. They require a rendercontext, so note that they do not fit async loading model very well. As a side note, please Microsoft and beloved DirectX team, can we just async upload content in an existing resource one day, without the need for a render context? Seems (hope) DirectX12 is getting this way.
  • Dynamic Resources (GPU) : This is increasingly important (or I could call it fundamental) in FlareTic. A lot of resources are updated once per frame, either via Stream Output,Compute Shaders (for example, geometry via modifiers, particle systems,splines...). They also require Render Context, but no upload, so they are quite easier to schedule.
  • Renderables : Once we have our particle system ready, we might want to render it on the screen as well ;) Renderables can be rendered several times per frame (shadow map/depth pre pass...)
  • Post processors : Who can survive without it ;) Or course you want to also provide a forward path and a deferred path. 


From the time we start our application to the time our scene is rendered on the screen, we of course have several stages.


This is when we load a scene graph patch, and this is more tricky than many people think.
We of course want to be able to load/dump a scene graph patch in async way.
So let's see the load process

  • Create all nodes: We first create all nodes, there is no mix of node/link load, and json files format reflects this pretty well (no messed xml). One place for all nodes, one place for all links (life can be so simple sometimes ;) 
  • Assign parameters : We parameter values for nodes, please note in (most) cases, this is immediate, bt in some cases they can be retained (see below)
  • Connect nodes : Since our nodes already told on loading what thy expect, we can safely connect them together. Since some connections can also create a compile task, this is also important to keep in background.
  • Apply retained parameters : As mentioned, a compile task can be created at link time (feels a bit reverse here ;) So some shader parameters are created at this stage. Once this happens, retained parameters are finally flushed to the node. On a side note, since dx11.2, this is now deprecated feature (since I can directly reflect a function instead).
So now what we really want is modify that to make it more flexible:
  • Load behaviour: We want to load patch without any resource this can easily be done in a thread and is pretty fast. Loading only the object graph doesn't consume much memory either, so it's rather simple to have a lot of scene graphs ready to show off their coolness.
  • Load resources : We indicate to this scene that we want it ready to show off, so it should get dressed. Same, we want this async (and ideally with a little progress bar as an eye candy ;)


I'll keep rendering as one stage, but obviously this is divided is several sub stages:
  • Update low frequency resources : Those are trivial to do async, some concurrency control is still useful
  • Update per frame resources : those do not depend on graph in general so they can also be pushed, but in this case in serial (render context).
  • Update GPU resource : Particles, Geometry modifiers.... All the bits that needs to be made look good after
  • Render : Make look good
  • Apply post processing : Make look better 

C/Fine tuned priority

Nodes in FlareTic scene graphs have no centralized priority system (like a pull based dataflow for example).

As opposite, a node has full decision power on how to process it's own children. 

This gives some pretty nice advantages:
  • Since a node can decide, we have full flexibility and can make complex decisions trivially.
  • Easy to cut a part of the graph, just don't call child ;)
  • Easy for user to just connect things and they work, instead of having a mess because you connected something in the wrong order.
Of course there is also some drawbacks:
  • A lot of work is dealt by the node. This can of course be offset using abstract classes, but still quite some work.
  • It's harder to have a more global control (logging/error handling is very easy when you have centralized traversal)
  • No global control also makes it harder to process parts in parallel, which is something that increasingly needs to be taken into account.
So we need the best of both worlds in some ways, dataflow doesn't give the flexibility we need, but we still want this global control.

Luckily, as we will see this can the solved rather elegantly.

Of course there's a lot of other features (that are already there, like Shader caching, Dynamic compilation, Handlers, Resource pools...) that are part of a scene graph, but they are not so much "core features", so they might be discussed in separate posts at some point.

That's it for now, quite a large program to come, stay tuned.