What is FlatBuffers?

FlatBuffers is a serialization library developed by Google and similar to Protobuf. In this article, I will talk a bit more about them and also a bit more about binary serialization approaches.

Table of Contents

• Introduction
  • What about performance?
  • Textual data is human-readable though…
• How does the Schema work?
• Why not use Protobuf?
• Conclusion

Introduction

Not so long ago I talked about binary vs. JSON serialization approaches. In summary when you need to save data for later reading or you need to send data for another application you will need to serialize your favorite programming language objects or functional structs of data. The problem is that you “cannot” store the in-memory objects in a file in the same way they are stored in the memory.

Well here is where FlatBuffers comes into play. FlatBuffers is a serialization and deserialization library that actually stores and loads objects in the same way they are stored in memory. The library has its own schema language that allows for code generation for quite a few other languages.

Essentially you would write an “.fbs” file and using the provided flatc compiler you can then generate header files for C or C++ but also for TypeScript, PHP, Go, and a few other languages.

What about performance?

Well first of all – doing binary serialization in general is times more performant than serializing to JSON or XML which require parsing and then constructing objects out of the strings. Also, flatbuffers data is loaded directly into the memory and doesn’t require any parsing. The objects are ordered with the correct offsets without the need for construction. Since the generated classes are read-only FlatBuffers also allows you to map to your own C++ objects or to generate objects that can be mutated which will bring native performance again.

Textual data is human-readable though…

Well FlatBuffers covers that front as well. Along with the binary serialization you can also use it to serialize to JSON. The JSON serializer is also one of the fastest out there and it might be worth it to use this library even just for the JSON serializer and the ability to generate one very performant code and objects out of its schema files.

How does the Schema work?

The schema file is a common language format. It describes typical objects and things that you can find in any language. There is still the need to depend on the flatbuffers library but that is the only dependency and it is really lightweight. The compiler will then give you control over what you want to generate out of the schema and you can also opt-in for some advanced features from your schema classes like reflection.

So let’s stop talking and get into it:

// This is a comment
// Custom 3D ojbect file

namespace obj;

// Structs are optimized but can only contain
// scalar values or other structs
struct Vec3 {
    x: float;
    y: float;
    z: float;
}

table Vertex {
    position: Vec3;
    color: Vec3;
}

enum Layout {
    LINE,
    TRIANGLES,
}

table Object {
    vertices: [Vertex];
    layout: Layout;
}

root_type Object;

In this file you will notice some of the most important things in a schema file. The file can contain one of the two most important object types: struct and table. Structs are constructible and usable as-is. But they must have a fixed size so no arrays, strings, or other tables. Tables are your typical object but they are also read only when they are generated. You can also have enumerations, namespaces, unions (aka variants), etc.

Why not use Protobuf?

Protobuf is also a library developed by Google but unlike FlatBuffers it is not appropriate in every case. Protobuf is heavily used with gRPC which is another technology within Google for network communication. Having said that the Protobuf is a bigger library that adds more features for validation, updates, and deprecation (which is a bit more limited in FlatBuffers). But also with being a bigger library, it adds a bigger footprint and also takes more time to serialize and deserialize objects.

Since this is a blog about game development I would always prefer FlatBuffers due to it being straightforward and also not directly tied to network. You can use it for assets, save files, setting files, network communication, interprocess communication, or communication between your main and scripting languages. The binary serialization is also so easy that you can probably make your own custom schema compiler addition to compile to more languages than the initially listed.

Conclusion

Even though serialization is not the hottest topic in game development it just has so many uses that one cannot just go by it without thinking about the consequences of using JSON files. You should always serialize the files to the most appropriate format before using them in your game. Take the GLTF format for example – it is one of the most popular file formats for 3D objects as being “easy to transfer over the web”… Well all of this is a lie. GLTF is actually just a fancy JSON file. It is still a textual file format. Parsing files faster might mean faster or no loading screens at all.

Data Serialization

You can get my course on Data Serialization which covers the basics of binary serialization through the flatbuffers library. You can get the best price from here:

Enroll Today