Catflier

Saturday, 31 May 2014

Reflecting Types

Lately I've been posting about using IL generation to connect elements.

I know I was about to post series about Scene Graph, but since I like to do things in an non ordered fashion (and at the end of the day, it's my blog ;) , I'll go a bit deeper into objects.

In most graph oriented architecture, we have to create nodes, and connectors, so we have to build inputs/outputs...

We have several ways to do it:

Big factory: You have calls like CreateIntParameter, CreateValueInput, or generic version like CreateInput<T> ...
Injection : You use decorators in order to inject some implementation. (So doing ISpread<int> internally generates a IntInputPin or something like that).

Going deeper, why do we even need pins? Let's take the following example:

Code Snippet

public class Prop
{
    public double Hello { get; set; }
 
    public int Integer { get; set; }
 
    public string ReadOnly { get { return "ReadOnly"; } }
}
 
public class Prop2
{
    public double Double { get; set; }
 
    public string DoubleOutput { get; set; }
}

This is pretty easy to reflect properties, since we know about get/set, we can create a "virtual pin" automatically, so why even wrap the property?

In most systems we would work it like that:

Wrap hello as input and Output Pin.
Wrap ReadOnly as Output pin.
Create a link that wraps InputPin<double> and OutputPin<double>

So let's do this in a simpler way, let's say we want to connect Hello (output) to Prop2 "DoubleOutput"

First we need a Func to access Property:

Code Snippet

public static Func<T> BuildGetter<T>(object instance, PropertyInfo property)
{
    return (Func<T>)Delegate.CreateDelegate(typeof(Func<T>), instance, property.GetGetMethod());
}

And a way to set property:

Code Snippet

static Action<T> BuildSetter<T>(object instance, PropertyInfo property)
{
    return (Action<T>)Delegate.CreateDelegate(typeof(Action<T>), instance, property.GetSetMethod());
}

Now we have to connect both, eg: read from source and write to destination:

Code Snippet

static Action Connect<T>(object source, PropertyInfo getter, object dest, PropertyInfo setter)
{
    var getFunc = BuildGetter<T>(source, getter);
    var setAction = BuildSetter<T>(dest, setter);
    return () => setAction(getFunc());
}

That was that easy, now we can simply have:

Code Snippet

Prop p = new Prop()
{
    Hello = 20.0,
    Integer = 20
};
 
Prop2 p2 = new Prop2();
 
var connect = Connect<double>(p, typeof(Prop).GetProperty("Hello"), p2, typeof(Prop2).GetProperty("Double"));
connect();

All done, out connector is ready ;)

If we need conversion, this is as easy, we can use a small interface:

Code Snippet

public interface ITypeConverter<TSource,TDest>
{
    TDest Convert(TSource source);
}
 
public class DoubleToStringConverter : ITypeConverter<double, string>
{
    public string Convert(double source)
    {
        return source.ToString() + " from converter";
    }
}

Then our connector can either use this small interface, or we can just set a lambda is case we can't be bother ed adding one file + namespaces + new class, get rid of object oriented verbosity basically ;)

Code Snippet

static Action Connect<TSource, TDest>(object source, PropertyInfo getter, object dest, PropertyInfo setter, Func<TSource,TDest> converter)
{
    var getFunc = BuildGetter<TSource>(source, getter);
    var setAction = BuildSetter<TDest>(dest, setter);
    return () => setAction(converter(getFunc()));
}
 
static Action Connect<TSource, TDest>(object source, PropertyInfo getter, object dest, PropertyInfo setter, ITypeConverter<TSource, TDest> converter)
{
    var getFunc = BuildGetter<TSource>(source, getter);
    var setAction = BuildSetter<TDest>(dest, setter);
    return () => setAction(converter.Convert(getFunc()));
}

In action:

Code Snippet

var connectConverter = Connect<double, string>(p, typeof(Prop).GetProperty("Hello"), p2, typeof(Prop2).GetProperty("DoubleOutput"), (d) => d.ToString() + " Hello");
 
connectConverter();
 
var connectInterface = Connect<double, string>(p, typeof(Prop).GetProperty("Hello"), p2, typeof(Prop2).GetProperty("DoubleOutput"), new DoubleToStringConverter());
 
connectInterface();

Now we can actually simply reflect type, create IO on rules, but operate directly on properties.
Direct access costs us a few virtual calls, but since we haven't got a layer on top, this is even more minimized.

Once we have this, let's looks at it further, let's show how to display our element, with custom formatted string per type, in this example I just log to console, bu this is easily extendable to user interface, writing, serialization...

Code Snippet

public interface IPropertyDisplayFactory
{
    Type PropertyType { get; }
    IPropertyDisplay DisplayObject(object instance, PropertyInfo property);
}
 
public interface IPropertyDisplay
{
    void Display();
}

First we do some simple defaults:

Code Snippet

public class BasicDisplay : IPropertyDisplay
{
    private readonly Action display;
 
    public BasicDisplay(Action display)
    {
        this.display = display;
    }
 
    public void Display()
    {
        display();
    }
}
 
public abstract class TypedDisplayFactory<T> : IPropertyDisplayFactory
{
    public Type PropertyType
    {
        get { return typeof(T); }
    }
 
    public abstract IPropertyDisplay DisplayObject(object instance, PropertyInfo property);
}

Then two stupid int and double implementations:

Code Snippet

public class DoubleDisplayFactory : TypedDisplayFactory<double>
{
    public override IPropertyDisplay DisplayObject(object instance, PropertyInfo property)
    {
        Func<double> getter = Program.BuildGetter<double>(instance, property);
 
        Action action = () =>
            {
                Console.WriteLine(property.Name + " Double: " + getter().ToString());
            };
 
        return new BasicDisplay(action);
    }
}
 
public class IntDisplayFactory : TypedDisplayFactory<int>
{
    public override IPropertyDisplay DisplayObject(object instance, PropertyInfo property)
    {
        Func<int> getter = Program.BuildGetter<int>(instance, property);
 
        Action action = () =>
        {
            Console.WriteLine(property.Name + "Int: " + getter().ToString());
        };
 
        return new BasicDisplay(action);
    }
}

Now we need to know if a property is capable of display of not:

Code Snippet

public class DisplayPropertyRegistry
{
    private readonly IEnumerable<IPropertyDisplayFactory> factories;
 
    public DisplayPropertyRegistry(IEnumerable<IPropertyDisplayFactory> factories)
    {
        this.factories = factories;
    }
 
    public DisplayPropertyRegistry(params IPropertyDisplayFactory[] factories)
    {
        this.factories = factories;
    }
 
    private IEnumerable<IPropertyDisplayFactory> SearchCandidate(Type type)
    {
        return factories.Where(f => f.PropertyType == type);
    }
 
    public bool CanDisplay(Type type)
    {
        return SearchCandidate(type).FirstOrDefault() != null;
    }
 
    public IPropertyDisplayFactory GetFactory(Type type)
    {
        return SearchCandidate(type).First();
    }
}

Add filtering support:

Code Snippet

  public class DisplayPropertyFilter
  {
      private readonly DisplayPropertyRegistry registry;
 
      public DisplayPropertyFilter(DisplayPropertyRegistry registry)
      {
          this.registry = registry;
      }
 
      public IEnumerable<Tuple<PropertyInfo,IPropertyDisplayFactory>> ReflectType(Type type)
      {
          var props = type.GetProperties().Where(p => p.GetGetMethod() != null && registry.CanDisplay(p.PropertyType));
 
            return props.Select(p => newTuple<PropertyInfo,IPropertyDisplayFactory>(p, registry.GetFactory(p.PropertyType)));
      }
  }

From here we know from a given type, every property that we can display, and how to create them. We only used object type so far, so now we need to add instance in the mix:

Code Snippet

public class DisplayObjectFactory
{
    private readonly DisplayPropertyFilter filter;
 
    public DisplayObjectFactory(DisplayPropertyFilter filter)
    {
        this.filter = filter;
    }
 
    public ObjectDisplay GetDisplay(object instance)
    {
        var factories = filter.ReflectType(instance.GetType());
 
        var elements = factories.Select(f => f.Item2.DisplayObject(instance, f.Item1));
 
        return new ObjectDisplay(elements);
    }
}

And the final object that displays our information:

Code Snippet

public class ObjectDisplay
{
    private readonly IEnumerable<IPropertyDisplay> displayElements;
 
    public ObjectDisplay(IEnumerable<IPropertyDisplay> displayElements)
    {
        this.displayElements = displayElements;
    }
 
    public void Display()
    {
        foreach (IPropertyDisplay display in displayElements)
        {
            display.Display();
        }
    }
}

Our final code, that gets an instance and displays any supported data:

Code Snippet

DisplayPropertyRegistry registry = new DisplayPropertyRegistry(
    new DoubleDisplayFactory(),
    new IntDisplayFactory());
 
DisplayPropertyFilter filter = new DisplayPropertyFilter(registry);
 
DisplayObjectFactory builder = new DisplayObjectFactory(filter);
 
Prop p = new Prop()
{
    Hello = 20.0,
    Integer = 20
};
 
var display = builder.GetDisplay(p);
display.Display();

You can easily think that's a lot of classes and a lot of code to just display properties of a class (we could just iterate through property list and do some "tostring"

But...

This is more extensible, and if you need to access your properties a lot, you get a pretty big gain over reflection.
You can register any type, replace as you wish, so you avoid the dreaded massive switch.
By having more classes, and less work per class, you have much more invariants, pretty much every class is ready to go once constructed, no temporal coupling (only edge case is CanDisplay/GetFactory which could create a related exception).
You are not limited to just a "tostring", create a controller, network serializer, add a proxy for automatic property change dispatch, possibilities are endless ;)

Here we are for now, promised I'll get back into Scene Graph next post (or maybe not ;)

Wednesday, 28 May 2014

(Reasonably) Modern SceneGraph (Introduction)

I decided to go a bit deeper into how FlareTic internals are handled, and decided that next posts will be some series about SceneGraph.

Since my tool is not designed to be a game engine, but for procedural graphics, I have some different requirements/limitations that need a different thinking.

So let's see what we will cover, and what we need for a reasonably modern rendering system. Most code will be in c#, but you'll likely see a bit of F# appearing too ;)

1/Asset management

This is a no brainer, asset managers are necessary everywhere, whereas it is from textures/models.

Main required features are:

Background loading: You don't want to block your rendering while you load this fat 3d distance field.
Converters : You will likely want to convert your data into some internal representation. For example, prepare a 3d model in a way that you can directly upload to gpu, or take a bulk of textures and convert to texture array.
Generators : Most times we also need more procedural assets, so a simple generator function is also eligible as an asset (random buffers is something I widely use).
Child asset : For example, from a 3d model asset, you want a distance field representation.
Metadata : 2D Texture, 3D Texture don't make any sense on it's own, so we also need to indicate what data is contained in the texture (distance field/potential field/velocity/normal map/height map....)
Drag drop handler so when you drop on the patch it creates appropriate node.
Hierarchy : so if you have a texture folder (namespace), you can prefetch all resources in that folder for fast switching.

2/Render Elements

At some point in our scene, we need some resources which are not part of our asset system (some shaders, some buffers....)

Here are all render elements in FlareTic

Fully immutable: Those resources are loaded once per node, and will never change. (Shaders are a very good example for it).
Low update frequency : Those are the ones that don't change very often (actually mostly at design time). So they are built as immutable in GPU, and dumped/recreated when required by the user. Very common example will be standard primitives. Since resource is immutable in GPU, it can be uploaded async without a problem.
Dynamic Resources (CPU) : Those are resources which change pretty often, like any camera textures. In FlareTic they are built as dynamic, and uploaded to gpu using MapSubResource. They require a rendercontext, so note that they do not fit async loading model very well. As a side note, please Microsoft and beloved DirectX team, can we just async upload content in an existing resource one day, without the need for a render context? Seems (hope) DirectX12 is getting this way.
Dynamic Resources (GPU) : This is increasingly important (or I could call it fundamental) in FlareTic. A lot of resources are updated once per frame, either via Stream Output,Compute Shaders (for example, geometry via modifiers, particle systems,splines...). They also require Render Context, but no upload, so they are quite easier to schedule.
Renderables : Once we have our particle system ready, we might want to render it on the screen as well ;) Renderables can be rendered several times per frame (shadow map/depth pre pass...)
Post processors : Who can survive without it ;) Or course you want to also provide a forward path and a deferred path.

3/Stages

From the time we start our application to the time our scene is rendered on the screen, we of course have several stages.

A/Load/Unload

This is when we load a scene graph patch, and this is more tricky than many people think.

We of course want to be able to load/dump a scene graph patch in async way.

So let's see the load process

Create all nodes: We first create all nodes, there is no mix of node/link load, and json files format reflects this pretty well (no messed xml). One place for all nodes, one place for all links (life can be so simple sometimes ;)
Assign parameters : We parameter values for nodes, please note in (most) cases, this is immediate, bt in some cases they can be retained (see below)
Connect nodes : Since our nodes already told on loading what thy expect, we can safely connect them together. Since some connections can also create a compile task, this is also important to keep in background.
Apply retained parameters : As mentioned, a compile task can be created at link time (feels a bit reverse here ;) So some shader parameters are created at this stage. Once this happens, retained parameters are finally flushed to the node. On a side note, since dx11.2, this is now deprecated feature (since I can directly reflect a function instead).

So now what we really want is modify that to make it more flexible:

Load behaviour: We want to load patch without any resource this can easily be done in a thread and is pretty fast. Loading only the object graph doesn't consume much memory either, so it's rather simple to have a lot of scene graphs ready to show off their coolness.
Load resources : We indicate to this scene that we want it ready to show off, so it should get dressed. Same, we want this async (and ideally with a little progress bar as an eye candy ;)

B/Render

I'll keep rendering as one stage, but obviously this is divided is several sub stages:

Update low frequency resources : Those are trivial to do async, some concurrency control is still useful
Update per frame resources : those do not depend on graph in general so they can also be pushed, but in this case in serial (render context).
Update GPU resource : Particles, Geometry modifiers.... All the bits that needs to be made look good after
Render : Make look good
Apply post processing : Make look better

C/Fine tuned priority

Nodes in FlareTic scene graphs have no centralized priority system (like a pull based dataflow for example).

As opposite, a node has full decision power on how to process it's own children.

This gives some pretty nice advantages:

Since a node can decide, we have full flexibility and can make complex decisions trivially.
Easy to cut a part of the graph, just don't call child ;)
Easy for user to just connect things and they work, instead of having a mess because you connected something in the wrong order.

Of course there is also some drawbacks:

A lot of work is dealt by the node. This can of course be offset using abstract classes, but still quite some work.
It's harder to have a more global control (logging/error handling is very easy when you have centralized traversal)
No global control also makes it harder to process parts in parallel, which is something that increasingly needs to be taken into account.

So we need the best of both worlds in some ways, dataflow doesn't give the flexibility we need, but we still want this global control.

Luckily, as we will see this can the solved rather elegantly.

Of course there's a lot of other features (that are already there, like Shader caching, Dynamic compilation, Handlers, Resource pools...) that are part of a scene graph, but they are not so much "core features", so they might be discussed in separate posts at some point.

That's it for now, quite a large program to come, stay tuned.

Thursday, 17 April 2014

Geometry and Functions

After finishing my projects, I finally have time to work on some way long overdue features.

I'm doing a lot of work on my user interface revamp, and also doing a lot of code cleanup since now Windows Phone gives us access (also long overdue) to all nice features like P/Invoke, Direct2d....

This means after a bit of tweaking on SharpDX code base, windows phone is now AnyCpu, which is really great, having x86, x64, ARM and lot of targets to maintain is not pleasing at all.

Also I decided to move on and remove the old DirectX11/DirectX11.1 targets on my toolbox. No more Windows 7 support, I prefer to focus on having a better build for Desktop / RT / Phone instead.

Now after all this I thought it was time to work on another (well overdue as well) feature.

I've always been interested into processing geometry in compute shaders instead of using StreamOut.

This gives some pretty useful advantages:

Write in place : RWStructuredBuffer rocks, no need for ping/pong
Writing Indexed geometry using Geometry Shader is doable, but rather painful.
Much easier to do more advanced post processing
Access to indirect Draw, which means it's much easier to instance generated geometry, StreamOut does not give us access to internal counter, except using a query, eg: not ideal.

Here are my structures:

Code Snippet

private DX11StructuredBuffer positionbuffer;
private DX11StructuredBuffer normalsbuffer;
private DX11StructuredBuffer uvbuffer;
 
private DX11StructuredBuffer appendindexbuffer;
 
private DX11InstancedIndexedDrawer drawer;
private DispatchIndirectBuffer vertexIndirectDispatch;
private DispatchIndirectBuffer indexIndirectDispacth;

Pretty simple, with two things to note:

Position buffer is created with the counter flag
Index Buffer is created with the append flag

Now there are a few bits to consider:

Structured Buffer is not bindable as Vertex Buffer: This is more or less a non issue in my case, since most of my shaders already fetch data from StructuredBuffers using SV_VertexID
StructuredBuffer is not bindable as Index Buffer. This is more annoying, since you need to bind it to the pipeline, no way to fetch. This is quite easy to sort still, as you can just create a standard index buffer (which allow raw view and give it UAV access), then just use a compute shader to copy indices.

So with this setup we are more or less set, you can process per vertex using VertexIndirectDispatcher, process per face using IndexIndirectDispatcher, and since you have access to counter/append, you can also easily emit/remove Geometry.

Now one main issue with this setup, you need to feed your buffers with some initial geometry.

You have a very simple way to do this in DirectX11.1 , but the feature is only supported on ATI.

So instead, let's replace my old monolithic geometry builders by something more flexible

For now my geometry builders look like this:

Code Snippet

public DX11IndexedGeometry QuadNormals(Quad settings)
{
    DX11IndexedGeometry geom = new DX11IndexedGeometry(device);
    geom.Tag = settings;
    geom.PrimitiveType = settings.PrimitiveType;
 
    List<Pos4Norm3Tex2Vertex> vertices = new List<Pos4Norm3Tex2Vertex>();
    List<int> indices = new List<int>();
 
    //Build your array
 
    geom.VertexBuffer = DX11VertexBuffer.CreateImmutable(device, vertices.ToArray());
    geom.IndexBuffer = DX11IndexBuffer.CreateImmutable(device, indices.ToArray());
    geom.InputLayout = Pos4Norm3Tex2Vertex.Layout;
    geom.VertexBuffer.InputLayout = geom.InputLayout;
    geom.Topology = PrimitiveTopology.TriangleList;
    geom.HasBoundingBox = true;
    return geom;
}

This has a bit too many things to do:

Read settings
Build Vertices/Indices list
Create Geometry resource

The big problem is that all is in one function, so let's improve and decouple this a little bit

First geometry builders do some pretty simple things, append vertices and append faces, so let's change the construction code like this.

Code Snippet

public class SegmentBuilder : IGeometryBuilder<Segment>
{
    public PrimitiveInfo GetPrimitiveInfo(Segment settings)
    {
        float phase = settings.Phase;
        float cycles = settings.Cycles;
        float inner = settings.InnerRadius;
        int res = settings.Resolution;
        bool flat = settings.Flat;
 
        int vcount = res * 2;
        int icount = (res - 1) * 6;
 
        return new PrimitiveInfo(vcount, icount);
    }
 
    public void Construct(Segment settings, Action<Vector3, Vector3, Vector2> appendVertex, Action<Int3> appendIndex)
    {
 
    }
}

Now we have one builder class responsible to build geometry, but it has no idea of which data structure we use anymore, which is great, since now it's very easy to:

Use different data back end (DataStream, List...)
Pack data the way we want
Completely bypass something (if resolution parameter does not change for example, we can set appendIndex function to null, so we completely ignore building faces).

Here is a simple example to append data into lists:

Code Snippet

public class ListGeometryAppender
{
    private List<Pos4Norm3Tex2Vertex> vertices = new List<Pos4Norm3Tex2Vertex>();
    private List<int> indices = new List<int>();
 
    public List<Pos4Norm3Tex2Vertex> Vertices { get { return this.vertices; } }
    public List<int> Indices { get { return this.indices; } }
 
    public void AppendVertex(Vector3 position, Vector3 normal, Vector2 uv)
    {
        Pos4Norm3Tex2Vertex v = new Pos4Norm3Tex2Vertex()
        {
            Position = new Vector4(position.X, position.Y, position.Z, 1.0f),
            Normals = normal,
            TexCoords = uv
        };
        this.vertices.Add(v);
    }
 
    public void AppendIndex(Int3 index)
    {
        this.indices.Add(index.X);
        this.indices.Add(index.Y);
        this.indices.Add(index.Z);
    }
 
    public void TransformVertices(Func<Pos4Norm3Tex2Vertex,Pos4Norm3Tex2Vertex> transformFunction)
    {
        for (int i = 0; i < this.vertices.Count; i++)
        {
            Pos4Norm3Tex2Vertex v = vertices[i];
            vertices[i] = transformFunction(v);
        }
    }
}

Now to construct our geometry, we can simply do:

Code Snippet

public partial class PrimitivesManager
{
    public DX11IndexedGeometry Cylinder(Cylinder settings)
    {
        CylinderBuilder builder = new CylinderBuilder();
        ListGeometryAppender appender = new ListGeometryAppender();
        PrimitiveInfo info = builder.GetPrimitiveInfo(settings);
        builder.Construct(settings, appender.AppendVertex, appender.AppendIndex);
        return FromAppender(settings, appender, info);
    }
 
    private DX11IndexedGeometry FromAppender(AbstractPrimitiveDescriptor descriptor, ListGeometryAppender appender, PrimitiveInfo info)
    {
        DX11IndexedGeometry geom = new DX11IndexedGeometry(device);
        geom.Tag = descriptor;
        geom.PrimitiveType = descriptor.PrimitiveType;
        geom.VertexBuffer = DX11VertexBuffer.CreateImmutable(device, appender.Vertices.ToArray());
        geom.IndexBuffer = DX11IndexBuffer.CreateImmutable(device, appender.Indices.ToArray());
        geom.InputLayout = Pos4Norm3Tex2Vertex.Layout;
        geom.Topology = PrimitiveTopology.TriangleList;
        geom.HasBoundingBox = info.IsBoundingBoxKnown;
        geom.BoundingBox = info.BoundingBox;
        return geom;
    }
}

What is great by passing functions instead of interfaces, it's much easier to composite actions, for example, we can easily compute bounding box while we create our List.

Code Snippet

SegmentBuilder builder = new SegmentBuilder();
ListGeometryAppender appender = new ListGeometryAppender();
PrimitiveInfo info = builder.GetPrimitiveInfo(settings);
 
Vector3 max = new Vector3(float.MinValue, float.MinValue, float.MinValue);
Vector3 min = new Vector3(float.MaxValue, float.MaxValue, float.MaxValue);
 
builder.Construct(settings, (v, n, u) =>
    { appender.AppendVertex(v, n, u); min = Vector3.Min(min, v); max = Vector3.Max(max, v); }, appender.AppendIndex);

We can also attach several Appenders, like:

Code Snippet

SegmentBuilder builder = new SegmentBuilder();
 
ListGeometryAppender appender = new ListGeometryAppender();
MultiListGeometryAppender multilist = new MultiListGeometryAppender();
builder.Construct(settings, (v, n, u) =>
    { 
        appender.AppendVertex(v, n, u);
        multilist.AppendVertex(v, n, u);
    }
    ,appender.AppendIndex);

And you noticed we can optimize by only filling one index buffer ;)

To build StructuredBuffers instead of geometry, here we are:

Code Snippet

SegmentBuilder builder = new SegmentBuilder();
MultiListGeometryAppender appender = new MultiListGeometryAppender();
 
builder.Construct(settings, appender.AppendVertex, appender.AppendIndex);
 
DX11StructuredBuffer intialposition = DX11StructuredBuffer.CreateImmutable<Vector3>(device, appender.Positions.ToArray());
DX11StructuredBuffer initialindices = DX11StructuredBuffer.CreateImmutable<Int3>(device, appender.Indices.ToArray());

Also, we can write to a dynamic buffer directly:

Code Snippet

SegmentBuilder builder = new SegmentBuilder();
PrimitiveInfo info = builder.GetPrimitiveInfo(settings);
DX11StructuredBuffer dynamicposition = DX11StructuredBuffer.CreateDynamic<Vector3>(device, info.VerticesCount);
DataStream ds = dynamicposition.MapForWrite(context);
builder.Construct(settings, (p,n,u) => ds.Write(p) , null);
dynamicposition.Unmap(context);

Here we are, next post I'll show a few examples of what we can now do with compute geometry (and some new idea for particle system ;)