In early 2020 I created ofxGPULightmapper, an OpenFrameworks Addon to calculate and bake high fidelity light into textures.

Halfway through the project, I realized some visual inconsistencies in which the geometry edges had black borders. Highly noticeable on shared edges of flat faces.

The light mapper draws in GPU the result of the calculation of the lights and shadows into the textures using geometry data of the mesh.

OpenGL draws triangle faces and it uses a native rasterization process that admits or discards fragments that are inside or outside of the drawing triangle space. Rasterization discards texels whose centroid is outside of the primitive and sends to the fragment shader those whose centroid fits inside of the primitive, even if the entire texel doesn’t fully fit.

When that newly rasterized image is used as texture to forward render the geometry, the OpenGL texture filtering algorithms do not use the same centroid logic but interpolate the values of the pixels.

This causes that, when a pixel touches, even partially, the area described by the UV coordinates, the value of that pixel will be used to render the fragments corresponding to such segment of the geometry.

To solve this visual glitch on the ofxGPULightmapper addon, the ideal solution is to utilize conservative rasterization.

Conservative Rasterization is a technique in which all pixels that are at least partially covered by a rendered primitives are rasterized, which means that the fragment shader is invoked.

Conservative Rasterization is useful in a number of situations, including for certainty in collision detection, occlusion culling, and tiled rendering.

Conservative rasterization can be used as part of the dedicated extension API of the graphics card. But such capabilities are only supported on certain models and must be implemented differently depending o the hardware manufacturer.

GL_CONSERVATIVE_RASTERIZATION_NV

GL_CONSERVATIVE_RASTERIZATION_INTEL

VK_EXT_conservative_rasterization

SPV_EXT_fragment_fully_covered

Since this extension is not available on all graphics cards, I decided to create an alternative that does not depend on the brand or on the specific model.

Implementation

The first idea was to create a shader that would do a binary dilation on the shadow texture. With the first tries, I could stretch the textures by one pixel and this largely solved the problem of black lines at common edges. But it gave inaccurate results in situations where the geometry was too small or complex.

So starting from dilation’s original idea, I implemented a solution that extends the geometry of each triangle by one-pixel perpendicular to the edge. The solution was to create a geometry shader that inputs a vertex triangle and emits 9 vertices in triangle_strip layout drawing 6 extra triangles

	layout (triangles) in;
	layout (triangle_strip, max_vertices = 9) out;

You can look at the code in the project’s github

When creating the triangle strip, order is very important. The algorithm uses the first 3 vertices to represent the original geometry and concatenates the rest of the vertices to wrap the primitive around. It uses 2 triangles per edge to create rectangles that extend the drawing area. As the comment I left in the code to clarify the algorithm indicates, the strip shares the values of the vertices [0-4], [1-6], and [3-8].

Using trigonometry, the perpendicular of each side is calculated and extended a pixel using a uniform value with the overall dimensions of the texture.

The UV coordinates of the new geometry are exactly the same as those that shared data with the vertices of the original triangle. This extends the color of the edge throughout the new geometry, making the dilation effect.

Once we extend the render area by one pixel on each side of the triangle, the rendered image will no longer have unpainted pixels touching the area described by those UV coordinates. And thus, black borders will no longer appear when forward rendering the geometry.

This technique requires that the UV coordinates of each triangle have at least 2 pixels of separation between each other, otherwise, dilation will overwrite the shared space and start painting the edges of other triangles. This triangular packing solution is common in baking and light packing techniques.

Once the UV triangle packing was implemented together with the conservative rasterization. The same testing scene that was displaying the visual glitches, achieves a seamless transition between faces. Successfully displaying the smooth complex shadows calculated by the addon.

The solution to the Conservative Rasterization was inspired by this article from Nvidia by Jon Hasselgren, Tomas Akenine-Möller, and Lennart Ohlsson