Project: Vector Graphics on Video Games – Introduction

Hi! I plan to make a series of posts on a project I always dreamed to achieve after the Flash platform decline and the rise of other platforms: an easy and efficient way to use vector art on video games. This would be achieved possibly leveraging the GPU and some optimization strategies, and possibly by pre-processing the files to a more workable format, so the runtime only need to feed the data to the GPU and orchestrate the drawing order.

Vector art used to be used a good deal in some games for the Flash platform. While a lot professional games used bitmaps (even for better performance), the excellent vector drawing facilities present in Flash Professional provided some developers the ability to design directly inside the tool. Furthermore, vector graphics can be scaled indefinitely, which is one of the factors of SVG being widespread on the web, and could be used to make some simpler games “future-proof”, so they can be shown beautifully on infinitely large displays.

The drawback of current vector graphics (more of the SVG specification) is that is very complex to process, and currently there are non-existent vector processors on the GPU or they rely on scanline dividing of the paths. My proposal is to derive a technique (along with pre-processing) that allows the path geometries to be rendered as GPU primitives.

Roadmap

This project is very complex and involves a lot of geometry (the vector part of the name is because of vector algebra after all). My plan is to divide the process in two great parts: pre-processing and rendering.

1. Pre-processing

This is where great part of the math will be placed, since here we are dealing with the raw mathematical constructs of the lines, arcs and Bézier curves. The pre-processing phase has two phases: path processing and view list processing.

The path processing phase has three substeps:

  1. Surface subdivision
  2. Primitive generation
  3. Paint generation

Each one of the parts will have an entire length of topics, because they are a bit complex. The surface subdivision employs a set of curve intersection algorithms and a special data structure designed to handle this, called doubly-connected edge list. Why is the surface subdivision necessary? To handle non-simple paths, that intersect themselves, and to handle properly loops and cusps in case of cubic Bézier primitives. The primitive generation step uses the Loop–Charles–Blinn technique to compute the fill primitives for quadratic and cubic Bézier curves and some technique that I have yet to research for arcs. Finally, the paint generation will generate the texture coordinates for the fill primitives in order to allow them to be filled with arbitrary fills, since the SVG specification allows solid colors, gradients of various complexities, textures and even patterns that are themselves paths!

The view list generation process is just a post-processing step to organize the paths into the tree view that a SVG file exposes, so we can properly separate the paths by name and allow them to be selected by the renderer (think of a “tileset” SVG in which we pick the tiles as separate groups in the file).

2. Rendering

The actual rendering can be arbitrarily complex, since a SVG group can contain paths, other primitives and other groups; the paths can contain other paths as patterns, and they have nontrivial properties to care, such as the different rules that govern the behavior of fills in scaled paths and other intricacies of the SVG specification.

The primary steps of a rendering phase would be:

  1. Pattern and paint auxiliary textures
  2. Drawing of lines
  3. Drawing of fills

The first step is particularly complicated since it can recurse the entire rendering phase (because of pattern generation). Since the pre-processing phase already did most of the heavy math to us, we can abuse the generated data here to make the rendering step most efficient possible (some steps can not be properly optimized, like the radial gradients – they still will require square roots in the fragment shaders).

Technology

For this project, I will use C# alongside .NET. I am using some internal MonoGame types, because I will build a content pipeline extension for it for the pre-processing phase. Even though, the most important structures I will use are vectors (mainly 2-dimensional) and structures that describe the geometric primitives I will be working with. The code should be portable to C++, addressing some memory management issues on the way, but the core of the mathematical routines should not have a lot of effort to port.

Furthermore, since C# is being used, it might not be very difficult to port this to Unity platforms; however, I do not know Unity enough to figure out where should the pre-processing phase be placed.

Conclusion

This is a long and complex project, and it will possibly take a while to be completed. So far, I have not created a GitHub repo for the project, but I have some code done and I want to wait for the articles to be finished and then polish the code to publish it.

I do not plan to support the entire SVG specification, not yet. They have a lot of complicated features such as text-on-path, image importing, custom CSS and animation, and the project is oriented more towards an interface to use simple vector graphics on video games, so they can achieve resolution independence.

I hope to put a good deal of effort on this project. I might turn it into one (or diverse) NuGet package later, so more MonoGame developers can use this library in their games.