2023-09-05

The Developer's Guide To TOML

Everything you need to know to get started with TOML - examples, config files, serialization and more.

Example JSON

TOML (Tom's Obvious Minimal Language) is a language designed to create simple, human readable configuration files. It's 2013 release makes it a relatively new language yet TOML has gained significant traction across software development. This is part due to it's syntax being reminiscent of '.ini' configuration files but with a single formal specification.

How is TOML structured?

TOML is built around the concept of Key/Value pairs. Tables can then be used group and structure the pairs into a hierarchy, much like JSON and YAML. The idea is that a TOML file can easily become a table/map/dictionary in whichever programming language you're using. TOML also has arrays which can be used to store lists of values or further nest arrays and tables. As with YAML, tables can be represented as their own structure or inline in a style similar to JSON.

[example-table]
example_key = "example_value"

inline-table = { example_key = "example_value" }

To make your file as readable as possible you'll want to ensure you only use inline tables when necessary for small, easily represented data. Arrays are a little more complex. A basic array can contain data of various types,

example-list = [1, 2, "3"]

but, if you wish to combine the lists and tables, the syntax is a little more complicated and we'll dive into that later.

TOML Types

Key/Value pairs in TOML must map to one of the following types.

TOML also has three different types of key too.

Comments are allowed and should be written after a '#'.

TOML Examples

The example below shows off everything you can do within a '.toml' file.


# Key/Value pair in root table of file
boolean = true

[strings] # Example of defining a table
basic = "This is a basic string"
multiline = """This is a \
               multiline string"""
literal = 'C:\This is a literal string'
multiline-literal = '''This is a multiline \ 
                       literal!'''

[integers]
standard = 100
underscored = 100_000_000
hexadecimal = 0x7b
octal = 0o14
binary = 0b1100100

[floats]
fractional = 100.00
exponent = 1e3
combination = 1.001e3

[offset-datetimes] # RFC 3339 
example-1 = 2023-09-03T02:11:01Z
example-2 = 2023-09-03T02:11:02-01:00
example-3 = 2023-09-03T02:11:03.1234-01:00
example-4 = 2023-09-03 02:11:05Z

[other-datetimes]
# RFC 3339 without the timezone.
local-datetime = 2023-09-03T02:10:00.987 
# RFC 3339 without the time.
local-date = 2023-09-03 
# RFC 3339 without the date.
local-time = 23:22:21.0123

[arrays]
basic-array = [1, 2, 3, 4, 5]
nested-array = [[1, 2, 3], [4, 5, 6]]

[nested.table.definition]
some-val = 1

[nested.table.definition.another.layer]
some-val = 1
inline-table = {inline = "table"}

# This is the syntax for an array of tables

[[array-of-tables]]
x = 1

[[array-of-tables]]
y = 1

[[array-of-tables]]
z = 1

While the example above is a helpful little cheatsheet it lacks the context to give you and understanding of TOML. So, let's say I want to open a garden centre and need to keep track of all the different plants, where they are, and when they're about to flower. I need this file to be readable as I'll be handing it out to all the employees as well as ingesting it into the cash registers. I could write out this data in TOML as follows.


# Top level description
name = "Andrew's Garden Centre"
address = "75 Iris Avenue"
openAtWeekends = false
openHoursPerDay = 6

# Some information about the plants!
[plants.fuchsia]
garden = "rear"
varieties = [
    "Marinka",
    "Swingtime",
    "Dollar Princess"
]
flowers = 2023-06-01

[plants.allium]
garden = "front"
varieties = [
    "tuberosum",
    "neapolitanum",
    "flavum",
]
flowers = 2023-05-20

Using a converter we can take a look at what this would look like as a JSON.

{
  "name": "Andrew's Garden Centre",
  "address": "75 Iris Avenue",
  "openAtWeekends": false,
  "openHoursPerDay": 6,
  "plants": {
    "fuchsia": {
      "garden": "rear",
      "varieties": [
        "Marinka",
        "Swingtime",
        "Dollar Princess"
      ],
      "flowers": "2023-06-01"
    },
    "allium": {
      "garden": "front",
      "varieties": [
        "tuberosum",
        "neapolitanum",
        "flavum"
      ],
      "flowers": "2023-05-20"
    }
  }
}

As you can see, the comments have been removed and the dates have been turned into strings, as they're not supported in JSON, and the overall readability of the file has decreased. This is an excellent demonstration of why TOML is a favorite of mine for writing configuration files.

Let's take a quick look at a TOML file in the real world now. The programming language Rust uses TOML in its manifest files. A manifest file is where all the metadata about your code is stored - for example, dependencies, authors, licenses, and a variety of compiler options. A basic 'Cargo.toml' file is seen below.

[package]
name = "rust_Example"
version = "0.1.0"
edition = "2021"
authors = ["Andrew <[email protected]>", "Mel <[email protected]>"]

[dependencies]
tokio = { version = "1.15.0", features = ["full"] }
tera = "1.15.0"
rocket = "0.5.0-rc.1"
rocket_codegen = "0.4.4"
rocket_dyn_templates = {version = "0.1.0-rc.1", features=["tera"]}

[profile.release] # Some compiler options for release.
lto = true
opt-level = 0

Defining dependencies and compiler options within a TOML file makes it extraordinary clear to the next developer what dependencies were used and which options the code was compiled with.

Stringify, Parse, Serialize, and Deserialize

To start using TOML in your projects, you'll need to know about the process of parsing/deserializing a TOML file as you'll need it in a usable format in your chosen programming language. Generally, dynamically and strongly typed languages complete this process in two different ways. In languages languages such as Python or JavaScript where the interpreter infers what type a variable is, a TOML file is parsed directly into a dictionary/object. This can be nested, contain combinations of any types, and be used however you wish. Let's check out an example of this using Python to calculate a simple polynomial equation.

should_calculate = true

[polynomial]
a = 0.003
b = 0.6
c = 0.4

import toml #pip install toml

GLOBAL_CONFIG = {}


def load_config():
    # Open the file and parse it with the toml package.
    with open("configuration.toml", "r") as toml_file:
        global GLOBAL_CONFIG
        GLOBAL_CONFIG = toml.load(toml_file)


def calculate():

    x = 2.0
    # The toml package parses the structure into a dictionary.
    my_poly_coeffs = GLOBAL_CONFIG["polynomial"] 
    y = my_poly_coeffs["a"] * x**2 + my_poly_coeffs["b"] * x + my_poly_coeffs["c"]

    return y


if __name__ == "__main__":
    load_config()

    if GLOBAL_CONFIG["should_calculate"]:
        answer = calculate()
        print(answer)

As you can see, it's super simple to load in the file and use it as a standard dictionary type in Python. Let's take a look at doing this in a strongly typed, compiled language. Using Rust we'll recreate the same example. First, we'll use the Cargo.toml manifest file to declare our dependencies.

[package]
name = "example"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
serde = { verison = "1.0.188", features = ["derive"] }
toml = "0.7.6"



use serde::Deserialize;
use std::fs;
use std::process::exit;

// Create structs which match the same structure as your TOML file.
// Ensure you apply the Deserialize macro from the serde crate.

#[derive(Deserialize, Debug)]
struct Configuration {
    should_calculate: bool,
    polynomial: PolynomialConfig,
}

#[derive(Deserialize, Debug)]
struct PolynomialConfig {
    a: f64,
    b: f64,
    c: f64,
}

fn main() {
    let toml_string = match fs::read_to_string("configuration.toml") {
        Ok(toml_string) => toml_string,
        Err(e) => {
            println!("Could not find or open the configuration file.");
            exit(2);
        }
    };

    let parsed_toml: Configuration = match toml::from_str(&toml_string) {
        Ok(x) => x,
        Err(e) => {
            println!("Could not parse the TOML string: {:?}", e);
            exit(1);
        }
    };

    println!("Configuration struct: {:?}", parsed_toml);
}

Now let's say you're not sure what the exact format of the TOML data will be. It may change each and every time you use it and you want a way to read it in generically like in Python. You can do this using an enumeration. The toml package includes one called Value and it can contain all possible types within the TOML file.

pub enum Value {
    String(String),
    Integer(i64),
    Float(f64),
    Boolean(bool),
    Datetime(Datetime),
    Array(Array),
    Table(Table),
}

So, we can remove our structs and read the data into this enumeration.

use std::fs;
use std::process::exit;
use toml::Value;

fn main() {
    let toml_string = match fs::read_to_string("configuration.toml") {
        Ok(toml_string) => toml_string,
        Err(e) => {
            println!("Could not find or open the configuration file.");
            exit(2);
        }
    };

    let parsed_toml: Value = match toml::from_str(&toml_string) {
        Ok(x) => x,
        Err(e) => {
            println!("Could not parse the TOML string: {:?}", e);
            exit(1);
        }
    };

    match parsed_toml {
        Value::String(x) => println!("This is a string: {}", x),
        Value::Integer(x) => println!("This is an integer: {}", x),
        Value::Float(x) => println!("This is a float: {}", x),
        Value::Boolean(x) => println!("This is a boolean: {}", x),
        Value::Datetime(x) => println!("This is a datetime: {}", x),
        Value::Array(x) => println!(
            "This is an array and it contains more Value enumerations:\n{:?}",
            x
        ),
        Value::Table(x) => println!(
            "This is a table and it contains more Value enumerations:\n{}",
            x
        ),
    }
}

If you use a generic enumeration like this the downside is you'll have to check the type of each Value every time you want to use it. More often than not it's easier to use optional struct fields to try and coerce your data into a somewhat known structure and just check whether or not certain fields were present.

Useful Libraries

As seen in the Python and Rust examples, most programming languages will have packages which help you parse and stringify TOML data. I've listed some popular ones below.

Useful Tools

In addition to earlier blogs on JSON and YAML I've also created some online tools which help you validate and convert TOML files!

Thanks for taking the time to read this article. You can also find this post on Medium. Next up I'll be comparing the differences between JSON, YAML, and TOML.



Software
TOML