This is the second part of my little series about tinc and NixOS, where I first shortly introduced tinc and then explained how to set it up on Linux in general as well as on NixOS. This part is more or less a rewrite of a question I posted to the NixOS discourse a while ago.

For a while I’ve been using tinc on a bunch of infrastructure and so far it’s basically been rock solid: once set up nodes anywhere simply join their network and become reachable for anyone who needs to access them. The one downside I noticed was the fact that at some point maintaining and updating the node configurations became increasingly cumbersome and at some point I ended up with my laptop connected to two different VPN networks (or tunnels - hence the title) consisting of all in all almost 30 nodes.

As a result of tinc providing its so far excellent service, the powers that be handed me the exiting quest of scaling up the existing networks as well as adding a whole bunch of new networks to the growing mass of configuration I already had lying around. This is the tale of how I battled a bunch of long and confusing configuration files and wrote cargo culted my first NixOS module in the process, enjoy..

Introduction and Recap

While I’ve found that using NixOS usually results in a whole lot less (and all in all an usually different) kind of headaches than running infrastructure on any other Linux distribution, I quickly came to a point, where I wasn’t really happy with the way the tinc configurations looked. As a quick recap, here is an example configuration for a small tinc network:

{ config, lib, pkgs, ... }:

{
  networking.firewall.allowedTCPPorts = [ 655 ];
  networking.firewall.allowedUDPPorts = [ 655 ];

  networking.interfaces."tinc.example".ipv4.addresses = [
    { address = "10.0.0.0"; prefixLength = 24; } ];

  services.tinc.networks = {
    example = {
      name = "node0";
      hosts = {
        node0 = ''
          Address = 192.168.122.1
          Subnet = 10.0.0.0
          Subnet = 192.168.0.0/24
          Ed25519PublicKey = ...
          -----BEGIN RSA PUBLIC KEY-----
          ...
          -----END RSA PUBLIC KEY-----
        '';
        node1 = ''
          Subnet = 10.0.0.1
          Port = 655
          Ed25519PublicKey = ...
          -----BEGIN RSA PUBLIC KEY-----
          ...
          -----END RSA PUBLIC KEY-----
        '';
        node2 = ''
          Address = 192.168.122.3
          Subnet = 10.0.0.2
          Ed25519PublicKey = ...
          -----BEGIN RSA PUBLIC KEY-----
          ...
          -----END RSA PUBLIC KEY-----
        '';

        ...  # more nodes

      };
    };
  };
}

Initially a port (655) is opened on the firewall, then the network interface tinc expects for the network called example (tinc.example) is created and assigned an IP address. Then the name of the node is defined and finally the hosts is filled with some / all other nodes configuration information of the example network (as well as the one of the local node).

Using this way of setting up tinc networks for some time, I noticed two things:

  1. My config files grew into an unmaintainable mess as the number of hosts inside the VPNs increased
  2. Since tinc is a mesh VPN it is a nice Idea to deploy the keys of all participating nodes to all participating nodes, but doing this inside the configuration means that increasing the number of nodes inside a network results in the need to change the configuration of all other nodes in the network (However at this point I should clarify: adding the node to any reachable other nodes configuration and rebuilding it is sufficient for a new node to join the network)

While the first thing is just a bit confusing, it also feels like it is some sort of a future accident or outage waiting to happen and especially the latter one felt kind of important to set up, but was at the same time something my lazy self never really got around to do.

builtins.readfile to the rescue

Initially I looked into how to make configuration files shorter and at some point I stumbled over builtins.readFile, which you can use to read in configuration bits and pieces from files. Instead of having all the node configurations lying around somewhere inside my config, I could create a per node configuration file containing something like this:

Address = 192.168.122.1
Subnet = 10.0.0.1
Subnet = 192.168.0.0/24
Ed25519PublicKey = ...
-----BEGIN RSA PUBLIC KEY-----
...
-----END RSA PUBLIC KEY-----

Then using builtin.readfiles inside the configuration everything immediately becomes much more readable:

{ config, lib, pkgs, ... }:

{
  networking.firewall.allowedTCPPorts = [ 655 ];
  networking.firewall.allowedUDPPorts = [ 655 ];

  networking.interfaces."tinc.example".ipv4.addresses = [
    { address = "10.0.0.1"; prefixLength = 24; }
  ];

  services.tinc.networks = {
    example = {
      name = "node0";
      hosts = {
        node0 = (builtin.readFile /path/to/node0-config);
        node1 = (builtin.readFile /path/to/node1-config);
        node2 = (builtin.readFile /path/to/node2-config);
      };
    };
  };
}

Using let

The next thing to do was to look at everything and try to make my code a bit more generic, so I initially started out using the let statement in order to make replacing a bunch of values inside a tinc configuration easier by at least spatially grouping them together:

{ config, lib, pkgs, ... }:
let

  node_name    = "node0";
  vpn_name     = "example";
  port         = 655;
  ipv4_address = "10.0.0.1";
  ipv4_prefix  = 24;

in
{

  networking.firewall.allowedTCPPorts = [ port ];
  networking.firewall.allowedUDPPorts = [ port ];

  networking.interfaces.("tinc." + vpn_name).ipv4.addresses = [
    { address      = ipv4_address; prefixLength = ipv4_prefix; }
  ];

  services.tinc.networks = {
    example = {
      name = "node0";
      hosts = {
        node0 = (builtin.readFile /path/to/node0-config);
        node1 = (builtin.readFile /path/to/node1-config);
        node2 = (builtin.readFile /path/to/node2-config);
      };
    };
  };

}

Great! At this point a new network can be set up by easily copying the above code from another network configuration, then changing 5 lines at the top and finally explicitly defining each node inside the network inside hosts.

Reading all files from a directory

At this point I decided to switch from defining every single host inside the configuration to simply adding the hosts configurations from a directory.

I basically started a nix repl and begun writing some code until everything looked like it could work inside my configuration, then I pasted whatever I had into a config file and hoped nixos-rebuild switch wouldn’t throw any errors. This is probably not the best way to develop nix files, but having the repl to quickly try out a bunch of nix lines was a definite step up from just working inside the configuration files.

The only thing that was a bit frustrating was the fact that nix is really lazy and quickly stops evaluating things, so you frequently end up with something like this:

$ nix repl
Welcome to Nix version 2.3.10. Type :? for help.

nix-repl> a = [ 1 2 3 4 ]

nix-repl> b = [ 5 6 7 8 ]

nix-repl> c = [ a b ]

nix-repl> c
[ [ ... ] [ ... ] ]

On the other hand it’s really great to have things like :t in order to find out what type a value has:

nix-repl> :t c
a list

nix-repl> :t 1
an integer

nix-repl> :t "definitely not a string :)"
a string

nix-repl> :t {}
a set

Initially I wanted to read in every file from some directory and then fill up services.tinc.network.<name>.hosts with key-value pairs created from the files name and content. Later on however I noticed that hosts is an attrset and due to that discarded the idea. Instead I simply created a filled attrset that I then supplied to hosts. In order to do that, I switched from keeping the nodes config in regular files to using JSON:

$ tree /etc/nixos/vpn/tinc/example
/etc/nixos/vpn/tinc/example
├── node0.json
├── node1.json
└── node2.json
$ bat /etc/nixos/vpn/tinc/example/node0.json
───────┬───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: /etc/nixos/vpn/tinc/example/node0.json
───────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ {
   2   │   "name" : "node0",
   3   │   "value" : "Subnet = 10.0.0.0\nPort = 655\nEd25519PublicKey = ...\n-----BEGIN RSA PUBLIC KEY-----\n...\n-----END RSA PUBLIC KEY-----"
   4   │ }
───────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

After a bit of trial and error, as well the revelation of nix not having loops, I arrived at the following code:

    { config, lib, pkgs, ... }:

    let

      node_name    = "node0";
      vpn_name     = "example";
      port     = 655;
      ipv4_address = "10.0.0.0";
      ipv4_prefix  = 24;

    in
    {

      networking.firewall.allowedTCPPorts = [ port ];
      networking.firewall.allowedUDPPorts = [ port ];

      networking.interfaces.("tinc." + vpn_name).ipv4.addresses = [
        { address      = ipv4_address; prefixLength = ipv4_prefix; }
      ];

      services.tinc.networks = {
        mcrn0 = {
          name = node_name;
          hosts = let
            files        = builtins.readDir ("/etc/nixos/vpn/tinc/" + vpn_name);
            filenames    = builtins.attrNames files;
            filepaths    = map (x: "/etc/nixos/vpn/tinc/" + vpn_name + "/" + x) filenames;
            filecontents = map builtins.readFile filepaths;
            jsondata     = map (x: builtins.fromJSON x) filecontents;
            attrsetdata  = builtins.listToAttrs jsondata;
          in attrsetdata;
        };
      };

    }

Now a tinc network can be created by changing 5 lines and supplying a folder that contains the node configurations. This folder can also be shared between all nodes and adding a new node only requires a rebuild on at least one (better of course on every) other node of the network.

Managing multiple networks however still requires copying the file and creates duplicates of configuration code, which is another future accident waiting to happen.

Creating a Module

At this point I decided to look into Modules and decided to try and write my own. Initially I searched for some kind of tutorial on how to create a module in NixOS (especially writing one that is using the <name> feature was quite a bit confusing to be honest), but I didn’t really find anything, so instead I decided to take a look at some existing modules and basically ended up copying and changing the original tinc module. Initially I simply ripped out pretty much everything and set up some options I could use to verify if everything worked, namely I created a new network device with an IP address as well as an opened a port on the firewall. I started basically setting up the following options, but in a generic way:

networking.firewall.allowedTCPPorts = [ port ];
networking.firewall.allowedUDPPorts = [ port ];

networking.interfaces.("tinc." + vpn_name).ipv4.addresses = [
  { address      = ipv4_address; prefixLength = ipv4_prefix; }
];

Here is the code for this initial “dumb” version of the module:

{ config, lib, pkgs, ... }:

with lib;

let

  cfg = config.services.tincDifferent;

in

{

  options = {

    services.tincDifferent = {

      networks = mkOption {
        default = { };
        type = with types; attrsOf (submodule {
          options = {

            nodeName = mkOption {
              default = null;
              type = types.nullOr types.str;
              description = ''
                Name of the Node in the tinc network.
              '';
            };

            port = mkOption {
              default = 655;
              type = types.int;
              description = ''
                TCP port of the tinc network.
              '';
            };

            ipv4Address = mkOption {
              default = null;
              type = types.nullOr types.str;
              description = ''
                IPv4 Address of the machine on the tinc network.
              '';
              example = "10.0.0.1";
            };

            ipv4Prefix = mkOption {
              default = null;
              type = types.nullOr types.int;
              description = ''
                IPv4 Prefix of the machine on the tinc network.
              '';
              example = 24;
            };

          };
        });

        description = ''
          Defines the tinc networks which will be started.
          Each network invokes a different daemon.
        '';
      };
    };

  };

  config = {

    networking.firewall = fold (a: b: a // b) { }
      (flip mapAttrsToList cfg.networks (network: data:
        {
          allowedTCPPorts = [ data.port ];
          allowedUDPPorts = [ data.port ];
        }
      ));

    networking.interfaces = fold (a: b: a // b) { }
      (flip mapAttrsToList cfg.networks (network: data:
      {
      "tinc.${network}".ipv4.addresses = [
        {
          address      = data.ipv4Address;
          prefixLength = data.ipv4Prefix;
        }
      ];
    }
    ));

  };
}

At this point I was able to check if everything worked by simply importing the module, filling out the options, then looking if there was a new network interface and checking for open ports using something like this:

{ config, lib, pkgs, ... }:

{
  imports = [ ./modules/tincDifferent.nix ];

  services.tincDifferent.networks.example.nodeName = "node0";
  services.tincDifferent.networks.example.ipv4Address = "10.0.0.1";
  services.tincDifferent.networks.example.ipv4Prefix = 24;
  # Port 655 is a default value, but it could of course be defined like this:
  # services.tincDifferent.networks.example.port = 655;
}

Note that, the code above is using a nested (unnamed) option in order to create the generic *.<name>.* functionality. If I understand correctly this can generally be achieved by doing something like this in the options part of the module:

...
options = {

  services.exampleService = {

    exampleOption = mkOption {
      default = { };
      type = with types; attrsOf (submodule {
        options = {

          enable = mkOption {
            default = false;
            type = types.bool;
            description = ''
              just an example
            '';
          };

        };
      });

      description = ''
        <name> will be available under exampleOption..
      '';
    };
  };

};
...

Then the enable option can be set for several exampleService services using e.g.:

services.exampleService.exampleOption.first.enable  = true;
services.exampleService.exampleOption.second.enable = true;
services.exampleService.exampleOption.third.enable  = false;

Later on in the config part of the module, the <name> under exampleOption has to be retrieved somehow. This is also the point, where I asked for help on the NixOS discourse and while I still haven’t totally wrapped my head around, it is possible to do so by accessing cfg.exampleOption (in our example) and then using something like this to apply the value somehow:

services = fold (a: b: a // b) { }
  (flip mapAttrsToList cfg.exampleOption (serviceName: nestedOptions:
    serviceName = nestedOptions.enable;
  ));

# or

services = builtins.mapAttrs (serviceName: nestedOptions:
  { serviceName = nestedOptions.enable; }) cfg.networks;

All in all I finally ended up with this code for the Module:

{ config, lib, pkgs, ... }:

with lib;

let

  cfg = config.services.tincDifferent;

in

{

  options = {

    services.tincDifferent = {

      networks = mkOption {
        default = { };
        type = with types; attrsOf (submodule {
          options = {

            nodeName = mkOption {
              default = null;
              type = types.nullOr types.str;
              description = ''
                Name of the Node in the tinc network.
              '';
            };

            port = mkOption {
              default = 655;
              type = types.int;
              description = ''
                TCP / UDP port used byt the tinc network (The Port has to be supplied in the node configuration as well, since the original tinc module takes the Port from there).
              '';
            };

            ipv4Address = mkOption {
              default = null;
              type = types.nullOr types.str;
              description = ''
                IPv4 Address of the machine on the tinc network.
              '';
              example = "10.0.0.1";
            };

            ipv4Prefix = mkOption {
              default = null;
              type = types.nullOr types.int;
              description = ''
                IPv4 Prefix of the machine on the tinc network.
              '';
              example = 24;
            };

          };
        });

        description = ''
          Defines the tinc networks which will be started.
          Each network invokes a different daemon.
        '';
      };
    };

  };

  config = {

    networking.firewall = fold (a: b: a // b) { }
      (flip mapAttrsToList cfg.networks (network: data:
        {
          allowedTCPPorts = [ data.port ];
          allowedUDPPorts = [ data.port ];
        }
      ));

    services.tinc.networks = builtins.mapAttrs (network: data: {
      name = data.nodeName;
      hosts = let
        files        = builtins.readDir ("/etc/nixos/vpn/tinc/" + network);
        filenames    = builtins.attrNames files;
        filepaths    = map (x: "/etc/nixos/vpn/tinc/" + network + "/" + x) filenames;
        filecontents = map builtins.readFile filepaths;
        jsondata     = map (x: builtins.fromJSON x) filecontents;
        attrsetdata = builtins.listToAttrs jsondata;
      in attrsetdata;
    }) cfg.networks;

    networking.interfaces = fold (a: b: a // b) { }
      (flip mapAttrsToList cfg.networks (network: data:
      {
      "tinc.${network}".ipv4.addresses = [
        {
          address      = data.ipv4Address;
          prefixLength = data.ipv4Prefix;
        }
      ];
    }
    ));

  };
}

Using this module, I can now create networks using only a few lines. The following would for example create two tinc networks, called 0 and 1, with their configuration files located in /etc/nixos/vpn/tinc/0 respectively /etc/nixos/vpn/tinc/1:

{ config, lib, pkgs, ... }:

{
  imports = [ ./modules/tincDifferent.nix ];

  services.tincDifferent.networks.0.nodeName = "node0";
  services.tincDifferent.networks.0.ipv4Address = "10.0.0.1";
  services.tincDifferent.networks.0.ipv4Prefix = 24;
  services.tincDifferent.networks.0.port = 655;

  services.tincDifferent.networks.1.nodeName = "node0";
  services.tincDifferent.networks.1.ipv4Address = "10.0.1.2";
  services.tincDifferent.networks.1.ipv4Prefix = 24;
  services.tincDifferent.networks.1.port = 656;
}

Improving the Module - Switching from JSON to nix

At this point let’s take a minute and recap a little: Initially we started out with a normal tinc configuration on NixOS. Next we started using the builtins.readfile function in order to read in the config of the VPN nodes from files (instead of putting everything tinc-specific into the nix configuration). This resulted in a way shorter and more readable configuration.

Then we switched from explicitly defining nodes to reading in all configuration files inside a directory. While this added a bunch of lines to the configuration, it also solves the problem of having to manually change the configs of every node whenever a node is added or removed from a network. The distribution itself is at this point still up to the user, but there are numerous ways to take care of that problem, e.g. git, git-annex or syncthing.

Finally we created a Module, which enabled us to configure not only a single but multiple networks with only a few little lines of code.

If you remember my last post, then you may also remember that in the example from last time, one of the nodes shared a local subnet with the rest of the network.

Sharing the subnet of a node using the module from above would still require every node to set up the subnet explicitly, resulting in losing the benefit of not having to rewrite the configuration of every node on the network, so in order to keep that advantage, the information has to be shared together with the rest of a nodes configuration.

The JSON file our module is using contains a name-value pair, that is used by the builtins.listToAttrs function in order to construct the hosts attrset and as far as I understand it, due to this, the JSON file does not support any additional content. So I switched from using JSON to plain nix to configure a node:

$ bat /etc/nixos/vpn/tinc/example/node0.json
───────┬───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: /etc/nixos/vpn/tinc/example/node0.json
───────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ {
   2   │   "name" : "node0",
   3   │   "value" : "Subnet = 10.0.0.0\nPort = 655\nEd25519PublicKey = ...\n-----BEGIN RSA PUBLIC KEY-----\n...\n-----END RSA PUBLIC KEY-----"
   4   │ }
───────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
$ bat /etc/nixos/vpn/tinc/example-nix/node0.nix
───────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: /etc/nixos/vpn/tinc/example-nix/node0.nix
───────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ {
   2   │   tinc = {
   3   │     name  = "node0";
   4   │     config = ''
   5   │       Subnet = 10.0.0.0
   6   │       Port = 655
   7   │       Ed25519PublicKey = ...
   8   │       -----BEGIN RSA PUBLIC KEY-----
       │       ...
  20   │       -----END RSA PUBLIC KEY-----
  21   │     '';
  22   │   };
  23   │ }
───────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Now when using nix files, the import (not imports) function can be used in order to read in a nix file and sections of that file, such as e.g. tinc in the example above can be directly accessed:

$ nix repl
Welcome to Nix version 2.3.10. Type :? for help.

nix-repl> import ./node0.nix
{ tinc = { ... }; }

nix-repl> (import ./node0.nix).tinc
{ config = "Subnet = 10.0.0.0\nPort = 655\nEd25519PublicKey = ...\n-----BEGIN RSA PUBLIC KEY-----\n...\n-----END RSA PUBLIC KEY-----\n"; name = "node0"; }

This makes it really easy to change the section of our module, which sets up the tinc nodes to accept nix files instead of JSON files:

# from this:
services.tinc.networks = builtins.mapAttrs (network: data: {
  name = data.nodeName;
  hosts = let
    files        = builtins.readDir ("/etc/nixos/vpn/tinc/" + network);
    filenames    = builtins.attrNames files;
    filepaths    = map (x: "/etc/nixos/vpn/tinc/" + network + "/" + x) filenames;
    filecontents = map builtins.readFile filepaths;
    jsondata     = map (x: builtins.fromJSON x) filecontents;
    attrsetdata = builtins.listToAttrs jsondata;
  in attrsetdata;
}) cfg.networks;

# to this:
services.tinc.networks = builtins.mapAttrs (network: data: {
  name = data.nodeName;
  hosts = let
    files = map (x: "/etc/nixos/vpn/tinc/" + network + "/" + x) (builtins.attrNames (builtins.readDir ("/etc/nixos/vpn/tinc/" + network +"/")));
    ## CHECK!! use builtins.readFile before import and check if we notice changes on rebuilds!
    attrsetdata = builtins.listToAttrs (map (x: lib.nameValuePair x.name x.config) (map (x: (import x).tinc) files));
  in attrsetdata;
}) cfg.networks;

(Note the comment, I’m not really sure if NixOS picks up on changed files, when using import as opposed to =builtins.readFile, guess I’ll have to check that out at some point..)

We first create a list called files that contains the absolute path to every file inside our vpn directory, and then create the hosts attrset by importing the tinc section of these files.

Now in order add in node-specific options such as sharing a subnet, we can simply add in additional subsections on a nodes configuration, e.g. something like this in order to support ipv4 routes:

{
  tinc = {
    name = "node0";
    config = ''
      Address = 192.168.0.1
      Subnet = 10.0.0.0
      Subnet = 192.168.0.0/24
      Ed25519PublicKey = ...
      -----BEGIN RSA PUBLIC KEY-----
      ...
      -----END RSA PUBLIC KEY-----
    '';
  };
  routes = {
    ipv4 = [
      {
        address      = "192.168.0.0";
        prefixLength = 24;
        via          = "10.0.0.0";
      }
    ];
    ipv6 = [];
  };
}

Note that also let can be used to make sure these files become somewhat copy-paste-able:

let
  cfg = {
    ipv4 = "10.0.0.0";
    routes = { ipv4 = { subnet = "192.168.0.0"; prefix = 24; }; };
  };
in
{
  tinc = {
    name = "node0";
    config = ''
      Address = 192.168.0.1
      Subnet = ${cfg.ipv4}
      Subnet = ${cfg.routes.ipv4.subnet}/${cfg.routes.ipv4.subnet}
      Ed25519PublicKey = ...
      -----BEGIN RSA PUBLIC KEY-----
      ...
      -----END RSA PUBLIC KEY-----
    '';
  };
  routes = {
    ipv4 = [
      {
        address      = "${cfg.routes.ipv4.subnet}";
        prefixLength = ${cfg.routes.ipv4.prefix};
        via          = "${cfg.ipv4}";
      }
    ];
    ipv6 = [];
  };
}

The new route section can then be added to the module using something like this:

# from this:
networking.interfaces = fold (a: b: a // b) { }
  (flip mapAttrsToList cfg.networks (network: data:
    {
    "tinc.${network}".ipv4.addresses = [
      {
        address      = data.ipv4Address;
        prefixLength = data.ipv4Prefix;
      }
    ];
  }
));

# to this:
networking.interfaces = fold (a: b: a // b) { }
  (flip mapAttrsToList cfg.networks (network: data:
  {
    "tinc.${network}".ipv4 = {
      addresses = [
        {
          address      = data.ipv4Address;
          prefixLength = data.ipv4Prefix;
        }
      ];
      routes = let
        files = map (x: "/etc/nixos/vpn/tinc/" + network + "/" + x) (builtins.attrNames (builtins.readDir ("/etc/nixos/vpn/tinc/" + network +"/")));
        routes = builtins.concatLists (map (x: x.ipv4) (map (x: (import x).routes) files));
      in routes;
    };
  }
));

At this point the whole module - for now only supporting additional IPv4 routes - looks like this:

{ config, lib, pkgs, ... }:

with lib;

let

  cfg = config.services.tincDifferent;

in

{

  options = {

    services.tincDifferent = {

      networks = mkOption {
        default = { };
        type = with types; attrsOf (submodule {
          options = {

            nodeName = mkOption {
              default = null;
              type = types.nullOr types.str;
              description = ''
                Name of the Node in the tinc network.
              '';
            };

            port = mkOption {
              default = 655;
              type = types.int;
              description = ''
                TCP / UDP port used byt the tinc network (The Port
                has to be supplied in the node configuration as well,
                since the original tinc module takes the Port from
                there).
              '';
            };

            ipv4Address = mkOption {
              default = null;
              type = types.nullOr types.str;
              description = ''
                IPv4 Address of the machine on the tinc network.
              '';
              example = "10.0.0.1";
            };

            ipv4Prefix = mkOption {
              default = null;
              type = types.nullOr types.int;
              description = ''
                IPv4 Prefix of the machine on the tinc network.
              '';
              example = 24;
            };

          };
        });

        description = ''
          Defines the tinc networks which will be started.
          Each network invokes a different daemon.
        '';
      };
    };

  };

  config = {

    networking.firewall = fold (a: b: a // b) { }
      (flip mapAttrsToList cfg.networks (network: data:
        {
          allowedTCPPorts = [ data.port ];
          allowedUDPPorts = [ data.port ];
        }
      ));

    services.tinc.networks = builtins.mapAttrs (network: data: {
      name = data.nodeName;
      hosts = let
        files = map (x: "/etc/nixos/vpn/tinc/" + network + "/" + x) (builtins.attrNames (builtins.readDir ("/etc/nixos/vpn/tinc/" + network +"/")));
        ## CHECK!! use builtins.readFile before import and check if we notice changes on rebuilds!
        attrsetdata = builtins.listToAttrs (map (x: lib.nameValuePair x.name x.config) (map (x: (import x).tinc) files));
      in attrsetdata;
    }) cfg.networks;

    networking.interfaces = fold (a: b: a // b) { }
      (flip mapAttrsToList cfg.networks (network: data:
      {
        "tinc.${network}".ipv4 = {
          addresses = [
            {
              address      = data.ipv4Address;
              prefixLength = data.ipv4Prefix;
            }
          ];
          routes = let
            files = map (x: "/etc/nixos/vpn/tinc/" + network + "/" + x) (builtins.attrNames (builtins.readDir ("/etc/nixos/vpn/tinc/" + network +"/")));
            routes = builtins.concatLists (map (x: x.ipv4) (map (x: (import x).routes) files));
          in routes;
        };
      }
    ));

  };
}

And that is basically where I’m at for now.

Conclusion & Outlook

In this post we started out with a bare tinc configuration on NixOS, refined it in order to add some readability and then created a Module in order to enable multiple tinc network configurations using only a few lines of code. Finally we added in the functionality of injecting node specific settings into the configuration.

As of now the module is still a bit of a half-baked affair, but I’m going to add in a bunch of things:

  • the ability to define a path for the network config files
  • IPv6 addresses and routes (not entirely sure if there is the possibility make both IPv4 and IPv6 optional but still require one of both using nix)
  • DNS using either networking.extraHosts or something like dnsmasq
  • most importantly I need to consider Trust: since any node is able to simply inject configuration options it makes sense to:
    • limit the options in some way that makes sense, e.g. using whitelisting for nodes that are allowed to make these kinds of changes
    • only trust nodes which can prove that they are trustworthy, e.g. using gpg or minisign

Also flakes are the new shit apparently and from what I’ve seen so far I really should take a closer look at those and make this module a flake.

For now the tincDifferent module in it’s current state of incompleteness can be found on github, feel free to give it a spin.

Anyways, that’s all for now, I’ll be back at some point in the (hopefully near) future, thanks for having me!