jezahost
  • netbox
  • ipam
  • automation
  • network

Stop using NetBox as a spreadsheet: how we made it the centre of our cloud infrastructure

For us NetBox is not documentation but the place where decisions are made: if it is not in NetBox, it does not exist in the infrastructure. How we set it up and what we learned on the way.

If you are a sysadmin, this may sound familiar. NetBox gets installed, and in the first week everyone keenly enters IPs, VLANs and servers. A few months later the real network lives in one place and NetBox in another. Someone hands out a new IP and forgets to record it in NetBox. In the end NetBox becomes an expensive spreadsheet that nobody trusts.

At JezaHost we took a different route: for us NetBox is not documentation, it is where decisions are made. Here is how we set it up and what we learned along the way.

The core principle: NetBox first, then the network

The rule is simple: if it is not in NetBox, it does not exist in the infrastructure. A new IP block, a new VLAN, a new customer: everything goes into NetBox first and is then applied to the real systems automatically. We don't do "let me quickly assign an IP and write it down later", because "later" never comes.

What we keep in NetBox

  • IP addresses and blocks: IPv4 and IPv6 aggregates, prefixes and every assigned address.
  • VLANs: which VLAN belongs to whom, and what it is for.
  • Tenants: every customer is a separate tenant, and all of a customer's resources sit under their name.
  • Virtual machines: every VM, its IPs and its owner.

Automation: from order to VM

When a customer orders a VPS in the portal, this is what happens:

  1. The portal sends a request to NetBox.
  2. If it is the customer's first VM, a new tenant is created for them in NetBox. Later VMs go into the same tenant.
  3. NetBox allocates a free IP and assigns it to the customer.
  4. The VM is created on the virtualisation platform with exactly this data.

At no step does a person pick an IP or look at a spreadsheet. That makes it fast, and it prevents mistakes: the same IP cannot be given to two customers.

What we learned along the way

  • Dry-run first. Before connecting the automation to production, we ran it in dry-run mode for a long time: the system logged what it would do but changed nothing. Most of the bugs were caught right there.
  • Bulk data does not go in through the UI. Typing hundreds of IPs or VLANs into the web interface one by one is tedious and a source of errors. We do it with scripts: check once, run once.
  • Big changes start in NetBox. When we moved to a new IPv6 block, the first job was updating the aggregates and prefixes in NetBox. Firewall rules and routing came after that. Done this way, you never end up with "an old block still left somewhere".
  • Don't try to store everything. The urge to fill in every detail in NetBox is strong. We keep only the data the automation actually uses. A field nobody uses goes stale quickly.

Conclusion

The value of NetBox is not in how much you put into it but in how far you can trust it. And trust comes only one way: when the real systems are fed from NetBox.

If you want to make NetBox the centre of your infrastructure too, start small: make one process (IP allocation, for example) depend entirely on NetBox, then move on to the next.

JezaHost is a cloud provider based in Baku. Our VPS services run on the infrastructure described in this post.