How to test a game engine

If you were to ask “What are the most difficult aspects of testing a game engine,” an answer you’d hear a lot is “rendering.” What things look like when they’re drawn to screen–how do you test that? Well, to know that things drawn to screen look right you need to compare the rendered image to an image you can expect to be static, unchanging. The first approach is to create an image test fixture of the expected render image and compare that against the render image itself. However, this approach is extremely limited. As unfortunate as is is, we must leave the domain of simple static tests, and enter into the wacky world of dynamically testing.

Pygame provides a number of things that make testing parts of a game engine easy. It provides access to Surface pixel data, meaning if you have a surface that the engine has created, and a surface which you’re testing against, you can compare the values of the two surfaces to check that the engine rendering is working. We use this in Sappho’s SurfaceLayers module’s tests to great effect:

def test_render(self):

    subsurface_size = (150, 150)

    # Create our test surfaces
    background = pygame.surface.Surface(self.TARGET_SURFACE_SIZE)
    rect1 = pygame.surface.Surface(subsurface_size)
    rect1pos = (100, 100)
    rect2 = pygame.surface.Surface(subsurface_size)
    rect2pos = (200, 200)
    rect3 = pygame.surface.Surface(subsurface_size)
    rect3pos = (300, 300)

    # Fill the surfaces
    background.fill((255, 255, 255))
    rect1.fill((255, 0, 0))
    rect2.fill((0, 255, 0))
    rect3.fill((0, 0, 255))

    # Create a surface to compare with and blit our test surfaces
    test_surface = pygame.surface.Surface(self.TARGET_SURFACE_SIZE)
    test_surface.blit(background, (0, 0))
    test_surface.blit(rect1, rect1pos)
    test_surface.blit(rect2, rect2pos)
    test_surface.blit(rect3, rect3pos)

    # Create the SurfaceLayers object and fill it with our layers
    surface_layers = sappho.SurfaceLayers(self.target_surface, 4)
    surface_layers[0].blit(background, (0, 0))
    surface_layers[1].blit(rect1, rect1pos)
    surface_layers[2].blit(rect2, rect2pos)
    surface_layers[3].blit(rect3, rect3pos)

    # Render to the target surface
    surface_layers.render()

    # Compare the two surfaces
    target_view = self.target_surface.get_view().raw
    test_view = test_surface.get_view().raw

    # The returned value is a bytes (str in python2) object so we can 
    # just do a straight compare
    assert(target_view == test_view)

Let’s break this down. First, we create some test surfaces – one background layer and three rectangles which we fill with colors to differentiate them.

# Create our test surfaces
background = pygame.surface.Surface(self.TARGET_SURFACE_SIZE)
rect1 = pygame.surface.Surface(subsurface_size)
rect1pos = (100, 100)
rect2 = pygame.surface.Surface(subsurface_size)
rect2pos = (200, 200)
rect3 = pygame.surface.Surface(subsurface_size)
rect3pos = (300, 300)

# Fill the surfaces
background.fill((255, 255, 255))
rect1.fill((255, 0, 0))
rect2.fill((0, 255, 0))
rect3.fill((0, 0, 255))

We then create a surface, which we will later use compare against the surface rendered by the engine, and render our background and colored rectangles to it.

test_surface = pygame.surface.Surface(self.TARGET_SURFACE_SIZE)
test_surface.blit(background, (0, 0))
test_surface.blit(rect1, rect1pos)
test_surface.blit(rect2, rect2pos)
test_surface.blit(rect3, rect3pos)

Then, we instantiate our SurfaceLayers object and populate it’s layers with our test surfaces…

surface_layers = sappho.SurfaceLayers(self.target_surface, 4)
surface_layers[0].blit(background, (0, 0))
surface_layers[1].blit(rect1, rect1pos)
surface_layers[2].blit(rect2, rect2pos)
surface_layers[3].blit(rect3, rect3pos)

… and render it to our target surface.

surface_layers.render()

Finally, we compare the two surfaces, using Pygame’s get_view() function, which returns a buffer of the raw pixel content of each surface.

target_view = self.target_surface.get_view().raw
test_view = test_surface.get_view().raw

# The returned value is a bytes (str in python2) object so we can just do a straight compare
assert(target_view == test_view)

And we’re done! Let’s make sure it passes with py.test -v…

Success!

A cool thing to do would be to assert that the logic we use to dynamically create an image to test if the layering system works, is test it against known dimensions with a static fixture, before using the dynamically generated image to test against the actual layer’s rendered/generated image.