Comment by sshine
8 days ago
I’d love to see a write-up on that.
Especially in the unlikely event that you used Nix for the deployment.
8 days ago
I’d love to see a write-up on that.
Especially in the unlikely event that you used Nix for the deployment.
I've done exactly that: headscale in production at work, a few hundred client devices, infrastructure mostly powered by nix. What would you want to hear about it?
* Does it work well? * Do you recommend it? * Do your users care? * Is it difficult? Do you have to maintain it or is it basically set it and forget it? * What was memorable about setting it up? * Why did you go for Headscale vs Tailscale or Netbird or some other solution?
I posted a reply to another subthread with some of this: > * Does it work well?
Very well! There are some limitations (see link above), but what's implemented is reliable.
> * Do you recommend it?
Yes, provided your requirements fit headscale's capabilities. If you need things like device trust attestation (e.g. Kandji MDM or Crowdstrike Falcon integration), SCIM provisioning, or various other enterprise features you may find it inadequate. If you can afford to pay for Tailscale, you should just use Tailscale because it's really good.
> * Do your users care?
They like it way better than our previous OpenVPN setup, that's for sure. I don't think they care about Headscale vs commercial Tailscale - the backend implementation is largely invisible to them.
> * Is it difficult? Do you have to maintain it or is it basically set it and forget it?
Not hard at all to set up, and it requires little maintenance attention. I have barely had to touch the control plane (other than version upgrades) since setting it up a year ago.
> * What was memorable about setting it up?
We had to do some custom coding to have automatic user offboarding when employees leave the company, and to emulate app connectors / dynamic routing (this is now OSS!sshine
7 days ago
benley
12 hours ago
avtar
7 days ago
> headscale in production at work
> - How much effort do you put into key management compared to plain WireGuard?
Less effort than plain wireguard; the only key management I do is for non-human clients
> - How automated is the onboarding process; do you generate and hand over keys?
Fully automated. Auth is done via OIDC to my company's SSO provider, so users can enroll their own machines without IT involvement.
> - How do you cope without the commercial Tailscale dashboard?
I don't really miss it. The headscale CLI tool is pretty good, and I use one of the headscale web UI projects (three are several: https://headscale.net/stable/ref/integration/web-ui/?h=web) for quick access to a few features (https://github.com/gurucomputing/headscale-ui)
> - Do you run some kind of dashboard or metrics system?
Yes, I scrape headscale's Prometheus metrics endpoint and have put together a simple Grafana dashboard. The metrics it emits are somewhat limited, but enough to keep an eye on its health.
> - How long did it take to set up?
I had a prototype up and running on Kubernetes with OIDC integration and a web UI in about 1 day of hacking. Going into full production took a few months, but the majority of that time was about planning the migration of all the existing users from OpenVPN.
Come to think of it, maybe I should share my terraform modules for deploying it.
> - Were there any gotchas?
A few, yeah:
- Setting up mobile clients is a bit fiddly, because they hide the "connect to a non-default control plane URL" under a debug menu. The mac and windows apps are similar - it's too easy for users to accidentally try to connect to tailscale.com instead of your headscale instance. If you have the ability to deploy MDM profiles (mac) or windows registry tweaks this is easy to fix, and the headscale server will even generate the configs for you.
- The headscale control plane doesn't support any kind of HA or replication. This doesn't disqualify it since tailscale can handle brief control plane outages without breaking the network, but it's likely to be a concern for serious enterprise users. It's possible to use an external Postgres database, so you can at least replicate data that way, but only one headscale server replica can be active at a time because they don't share runtime state.
- The tailscale API is not fully implemented, so you can't use things like the tailscale Kubernetes operator.
- Some features are missing: tailscale funnel, tailscale serve, app connectors, `autogroup:self` ACLs, SCIM provisioning, SSO group membership sync, and I forget what else. These may or may not be important to you.
For app connectors, I wrote an app to emulate the core functionality: https://github.com/singlestore-labs/tailscale-manager (it's in Haskell, but deployers don't need to care about that)
It's possible to implement group sync with some custom scripting - a python app to scrape your LDAP (or whatever) and generate tailscale ACLs isn't hard to write. But you do have to write it.
`autogroup:self` might be a big deal - you would need this if you want to stop users from seeing or connecting directly to each other's devices. I think there is an implementation of this coming in the next release of headscale.
Summary: headscale is great if you have relatively simple needs and can't afford to pay for Tailscale. You will probably outgrow it if you're running a serious business and need to comply with fancy audit requirements.
> How do you cope without the commercial Tailscale dashboard?
There are a couple open source dashboard options but right now only this one comes to mind: https://github.com/tale/headplane
1 reply →