For people wondering how the hell a user can audit the server is diskless or whatever, the goal appears to be using TPM to provide remote attestation for all code in the boot path. See https://www.system-transparency.org/.
Here are some additional details for those interested. We intend to make use of TPM for remote attestation of the current boot chain, reproducible builds to provide a strong link from source code to build artifacts, and a transparency log for a historical record of previously used boot chains, artifacts, WireGuard server keys, and related signatures.
As dtx1 mentioned elsewhere in this thread, diskless VPN infrastructure is currently in use by many other VPN providers. That is not a novel feature of course. What is novel is user-auditability of running VPN infrastructure. We were the first VPN provider to state our intention to make our infrastructure user-auditable AND provide a realistic roadmap with the specific technologies needed to do so. See the link above.
I believe the technologies we use in System Transparency will ultimately reshape the VPN provider industry into a highly competitive space focused on maximizing the transparency of VPN infrastructure. Or not, but at least OUR users will be able to audit us. :)
Either way we’re looking forward to the future. The opportunity for improvement is immense.
Niiice. I really love the concept of reversing the usual DRM use of remote attestation--forcing customers to prove they're running only software allowed by the megacorps. Instead of DRM, it's proving the corporation/server is trustworthy to the customer.
If you're from Mullvad, let me tell you that I love your service.
I had one technical support question, and I got an immediate response from an actual person who knew this problem, gave me a simple workaround (toggle between wireless connection and not), and told me that a permanent fix was in the works. Likely that fix rolled out because I haven't seen the issue in months.
The idea of an account that doesn't need a password because there's no critical information saved is such a nice one, too.
Keep up the good work - I recommend you to everyone.
This all sounds very exciting. Where will you draw the line between public and private – at the moment your consumer-facing app is on github, but less "server side" stuff (in common with many other VPN providers). I understand that probably you want to keep the database of "active numbers" private, but if I understand you correctly, you want to move to a model where anyone can download your in-memory image, run it in a VM, and audit it independently. I would welcome this. I'm particularly interested in how you maintain access to your bare-metal machines (e.g. do you have ssh / a serial console enabled)
I hate to be this skeptical, but let's say this is 100% possible (I have my doubts, see previous attacks on things like TPMs and SGX, but I digress). You probably could get 90% of the logging capability by putting monitoring in front of and behind the server, and associating connections by traffic/time.
It just seems like this goal of using technology to prove they're trustworthy is unlikely to actually work for a VPN company due to the threat models.
You are correct that System Transparency is not a universal remedy for all threat models. Indeed the word "secure" is undefined until you have a threat model. Most threat models are implied and undocumented assumptions.
At some stage in an R&D project one should shift from exploration to threat model-driven development. Most people, myself included, tend to focus on technical solutions, and argue back and forth how "oh, but it can be broken using X".
System Transparency aims to provide remote auditability assuming (1) the server hardware specification is correct, (2) a correct cryptographic hash of the contents of the SPI flash containing the platform firmware, and (3) a keypair generated on and only accessible to the platform. This is very simplified of course.
An attacker aiming to tap incoming and outgoing network traffic from our servers, who has physical access to the VPN server's Ethernet port, or an upstream router, isn't in the scope of System Transparency to protect against. We need to use other means for that.
Traffic analysis and correlation analysis is indeed a powerful tool, and in general only communicating at a constant bandwidth between all nodes at all times is the only way to completely defeat it (which is what, I understand, some military systems do). That's inherently highly wasteful, however.
To get around this, Mullvad offer very transparent comprehensive multi-hop routing systems [1]; you can bounce your wireguard tunnels around in layered wireguard tunnels (a bit á-la tor) by just choosing a series of ports to tunnel on and to. My understanding is that each one of these adds non-deterministic latency to your connection and probably would help to make such attacks harder at the very least, because from the point of view of an "all seeing" adversary the fact that all of these servers talk to each other all the time makes it very much harder to know where any packet could have gone. Yes, you can see each individual link but the metadata is lost.
I signed up for Mullvad when the UK's Snooper's Charter came into force and the local health inspectors suddenly had the rights to see my DNS record. Since then, I've had it installed on my router and just route everything through a custom wireguard (originally openvpn) tunnel. I've had some issues with my ISP randomly bandwidth limiting traffic on the odd port to 1 MByte/s, but frankly that makes me more inclined to put everything behind an encrypted tunnel. I don't want my ISP to do traffic shaping and I do want them to just leave me alone and let me communicate in peace. I have absolutely nothing to hide, but now have to accept that I partly live in a country where everything is surveilled all the time, and warantless, unaccountable investigation of my (highly personal!) online habits may be happening. I think Mullvad's excellent product, sensible architecture and reasonable price is worth paying. I'm an academic, unlikely to be of interest to three-letter acronyms, and therefore it matches my needs very well.
Damn it, this is a really cool use case for TPMs! So far, every use case I've heard has made me wish they were never invented, but this made me reconsider...
I've been following Mullvad for a long time and my impression (from countless reviews and comments here on HN) has been quite positive. But here's what I don't understand: Why are the servers located in Sweden, a country that's known for online surveillance[0] like no other country in the EU? From the Wikipedia article[1]:
> The law permits the signals intelligence agency, National Defense Radio Establishment, to monitor the content of all cross-border cable-based Internet traffic to combat "external threats" such as terrorism and organized crime.
We have 762 servers spread across 38 countries. Less than 10% of our servers are located in Sweden [0].
> a country that's known for online surveillance
My cofounder and I started Mullvad as a protest against the growing mass surveillance of Sweden as well as other countries. Our intent was direct political action through entrepreneurship. Incorporating the company in Sweden was the obvious choice, since we are both Swedes. We felt that incorporating elsewhere would have been more risky, complicated, and costly.
Mullvad has excellent lawyers who continuously monitor the legal situation in Sweden. Right now there is no Swedish law that can compel Mullvad to start logging [1]. Should the situation change we have contingency plans.
Thank you, this was the response I was looking for!
> Our intent was direct political action through entrepreneurship.
Again, I didn't mean to question your intent – as I said my impression of your company so far has been a very good one! :)
> We have 762 servers spread across 38 countries. Less than 10% of our servers are located in Sweden [0].
My apologies, it's been a few years since I last looked at your server locations (so I didn't remember) and I was probably getting the wrong impression from the fact that your post is only mentioning server locations in Sweden.
> Right now there is no Swedish law that can compel Mullvad to start logging [1].
But at the same time all cross-border traffic in and out of Sweden (so for anyone using Mullvad outside Sweden: virtually all traffic) is being monitored and (probably) logged, isn't it?
I have so much respect for everyone at Mullvad. You are the only VPN provider I trust! I have been a user for years now, and it has always been 5 dollars a month, which is quite generous. It is so cheap that even the poor can afford privacy. You guys have put a ton of effort into making your service as privacy respecting as possible. Not only that, the tech is on the bleeding edge (WireGuard, socks5, etc.) built right in. As a cybersecurity researcher, I could not be happier with the product. I hope you stay true to your mission, thanks again!
OP's been doing a poor job of "following Mullvad for a long time" and apparently has never used Mullvad or visited its website. Heck, it's on their Wikipedia page[1].
If they had, it would be immediately apparent that Mullvad has servers all over the world. They have for many years -- perhaps since it's inception. It takes a bare minimum of effort to learn that.
Your annoyance is understandable, and yet this question and direct response from a founder is a very informative interaction to have recorded in this thread. Much more informative than if GP had simply stated the results of their Wikipedia research.
Sometimes a mediocre question is the landing pad for a great answer. No need to begrudge the question.
I have apologized for the factual incorrectness of my comment and explained how I had arrived at that conclusion / brain glitch: They are only mentioning Swedish servers in their blog post and for a second I had forgotten that, like any other big VPN provider, Mullvad of course has servers in many locations around the world.
What more do you want? Upvotes are not under my control and I also cannot edit my original comment anymore. No reason to get personal.
FWIW, I still think that it's important to raise awareness for the deficiencies of Swedish privacy law (irrespective of Mullvad and where their servers are located). I suspect that at least some of the upvotes were also given because people agreed with that.
Because it's a swedish company?
The location of the servers is kinda irrelevant in that regard. They'd have to provide government access if there is a lawsuit that demands it. If there isn't one than your critique is entirely pointless
I'm not sure what point you're trying to make. If online privacy is as important to them as they say, they could have easily registered their company (or a subsidiary) in a different EU member state.
> They'd have to provide government access if there is a lawsuit that demands it.
IANAL but I am not entirely sure this is true in the EU. Either way, my question was "Why are the servers located in Sweden?" Whether or not this due to the company owning the servers being in Sweden is irrelevant.
A bit tangential to the main post, but I'd to share a recent positive experience with Mullvad:
I am a regular user of Mullvad and recently wanted to try a different VPN, that only provides Wireguard configs (i.e. no native app). I used the default setup.
For some reason, my internet connection was flaky, and when it disconnected and reconnected, my traffic leaked.
That never happened to me with Mullvad as the app comes with an "Always require VPN" option out of the box and it has always worked reliably.
On linux you can create a network namespace exposing only the wireguard network device, so that applications in that namespace cannot leak traffic. Setting this up, however, is quite fiddly in my experience.
In addition it is probably not a bad idea to block all traffic on wlo1 / eth0, except that to the mullvad server ip's, through some ufw rules. If you forget to configure the namespace for some applications then, it is highly unlikely the app has internet access (ie, it would need its own mullvad/vpn implementation included).
It’s easier and more secure to just create a VM that’s bridged to the VPN interface (regardless of protocol) if you don’t use the VPN for everything but the things you do use it for absolutely must go through it.
Any “always require vpn” option is a game of cat and mouse and is going to leak traffic at some point, whether easily detectable or otherwise. As others have said, you need to set up a secure environment that only has that one and only option for accessing the outside world.
Agree, Mullvad provides really good VPN service. I faced almost zero downtimes / speed throttles. It establishes quick connection with server (maybe because it uses wireguard). Anyway, I'm a regular user and I think paying 5E worth it.
> It establishes quick connection with server (maybe because it uses wireguard)
I'm actually kind of curious about what Wireguard does here. I think Wireguard says it's connected almost immediately even when it isn't, presumably holding traffic back locally while it waits for the connection to be active. I was wondering because I spent some time confused by a non-Mullvad Wireguard connection that wasn't working (turns out the server wasn't available at all) that nonetheless appeared as "connected".
Seconded. Very occasionally I'll have to swap servers in a location but that's super infrequent and it's not exactly a primary tier location that I'm using.
One of those things I can generally just setup and leave working.
I love the concept and I even wrote a blog post about how to set up a fully pxe bootable server environment using Alpine Linux [1] (which by default boots from RAM) in 2019. I still use it and it's one of those things that makes recovery or testing so much faster because I don't even need a usb thumb drive
This is awesome, glad that Mullvad is heading in this direction.
For reference, ExpressVPN (which has been audited by PwC) introduced this in 2019 [1].
Unfortunately, ever since ExpressVPN was purchased by Kape Technologies (they also own PIA, Cyberghost, Zenmate all of which do not have reliable histories); Mulvad has been the clear choice for a while now. They're also the backend for Mozilla VPN (mozilla just whitelabels from Mulvad [3])
Some information that could be of interest to those running VPN servers.
I live in Kazakhstan and recently our government decided to shut down the Internet. But apparently there were ways to get out: they did not filter two TCP ports. My guess it was some "backdoor" put by employees who had to obey the orders but wanted to provide people some way to get around those blocks. Those ports were used to run VPN software. I used Outline VPN on my VPS and it allowed me and my friends to have a working Internet.
TLDR: allow specifying port and protocol (TCP/UDP) as some kind of advanced option for those users who need it for some reason.
Right now we've got Internet back and it works fine, but who knows when our government will decide to shut it down again.
PS mullvad.net website apparently is blocked in Kazakhstan as well. I know that they block popular VPN provider websites, so that should not come as a surprise, but still. I have no idea whether actual VPN subnets are blocked or not.
I think it is probably more likely those ports were the backdoor the government or their allies was using to function. Everyone always builds in a backdoor for themselves!
When I hear about things like this it makes me glad I have a simple satellite communicator - it will only do short text messages but that is a hell of a lot better than nothing. Of course one could get a full satellite based mobile internet device or phone but the plans on those get quite expensive.
Disabling USB in BIOS only disables the emulation of classic PS2 keyboards and IDE storage so that old OSes or bootloaders without USB stacks can work with modern equipment. As soon as the OS kernel initializes the PCI bus, USB will work again - however they could go and remove the xHCI modules from the kernel and image.
It's a trade-off. If you have no disk, the disk can't fail, but the network can, and the remote PXE server can, and the remote SAN can. You can get into a state where you have to pray no servers reboot. Intermittent errors can be real annoying when it makes provisioning fail. (used to work a server farm that'd do server rebuilds over PXE, and ran a few diskless cluster projects)
An alternative is you use a RAID array and mount your disks in read-only mode, or use physically read-only disks and when you have to replace a disk, you pre-mirror the replacement disk. In this way the local disks can be replaced as they fail and there's never a point when the server is at risk of not being able to boot.
A network or PXE server can fail regardless, so this are things that always have to be taken into consideration and in those instances then you address those issues. With this type of setup you do not need a remote SAN as it would defeat the purpose of not having external storage that could store logs. Mullvad has servers all over the world, so a temporary failure in one location will not bring down their entire infrastructure.
It's not just a temporary failure, it's potentially the entire AZ going down hard. High Availability network boot without local storage is very difficult/expensive.
They can still use local disks to provision the OS over a network but boot from local storage, and prevent writing to disks from the booted OS (hell, they can completely remove the disk drivers from the kernel!). It just doesn't make sense to ditch the drives from a reliability standpoint. They're going to have a big outage one day just because they didn't want to deal with drives.
if only there was any proof of this actually being the case and there not being some "accidental" debug log enabled, or some other network level component having "accidental" access to the keys.
There's just no good answer to perfect trust-no-one private internet access.
If you need to hide all of your traffic from other users in your local network, you can accomplish that in a trust-no-one fashion by running your own VPN endpoint on a server you control which provides better privacy guarantees compared to a centralised commercial VPN whose business model will eventually involve selling your data (once user growth stops but shareholders demand continued revenue growth).
But if you need to hide your traffic from anybody but your peer on the internet and you need to hide the fact that you talked to that peer, then, I'm afraid, your out of luck.
> If you need to hide all of your traffic from other users in your local network, you can accomplish that in a trust-no-one fashion by running your own VPN endpoint on a server you control which provides better privacy guarantees compared to a centralised commercial VPN whose business model will eventually involve selling your data (once user growth stops but shareholders demand continued revenue growth).
Well not really. There was a great (german) interview with the perfect privacy founders recently [1]. They seem to be decent guys with close ties to the Chaos Computer Club and I strongly suspect they wouldn't want to work like that.
> But if you need to hide your traffic from anybody but your peer on the internet and you need to hide the fact that you talked to that peer, then, I'm afraid, your out of luck.
Nah, that one is easy just use an anonymous sim card or an open wifi and your good to go.
Honestly these discussions often feel pretty asinine to me. I personally use paid VPNs to pirate to my hearts content, work around my ISPs terrible networking and a little bit of geo-unblocking. Of course you can't use these services to protect yourself from three letter agency type surveillance or equally powerful threat actors but if they are "private" enough to block the music industry and their lawyers from suing you that's a pretty high standard of privacy, certainly more than any ISP alone gives you!
>> If you need to hide all of your traffic from other users in your local network, you can accomplish that in a trust-no-one fashion by running your own VPN endpoint on a server you control which provides better privacy guarantees compared to a centralised commercial VPN whose business model will eventually involve selling your data (once user growth stops but shareholders demand continued revenue growth).
the privacy protection for most people using VPNs is required against their ISP and other actors looking to analyse their traffic, not users on the local network. a commercial VPN will be better for privacy due to the crowding effects, ie. large number of users sharing the same IP and protects against correlation attacks - it's much easier to trace the activities on your own VPN endpoint back to you. of course you need to trust the operators, which is as different question.
The server configuration (and therefore customers account numbers) is stored in Server OS images I suppose, right ? It shouldn't be an issue as far as inspection is concerned, should it ?
Also, isn't there a law that enforces logs to be kept for n years ? How is it compatible with diskless setup ?
No, they would not need to be stored in the images. They are deployed as part of provisioning package according to the post. If they were part of the OS image, they would have to update the image anytime a new customer is added, which would be far less optimal than having Ansible or whatever push changes. There is no law in Sweden that requires logs, and the point of a VPN provider is to not store logs. If your VPN provider is keeping logs of your connections then you should look for a new provider.
Isn't this the same as just running a boot cd or PXE server and running all the data out of RAM drives? I mean we've been doing this for years on linux as hobbyists haven't we? Or does this bring somethign new to the table?
Pretty similar to the default (diskless) mode in Alpine, though it lacks the tooling to verify persisted data and the sources apkvols can be applied from at boot aren't that well documented.
For people wondering how the hell a user can audit the server is diskless or whatever, the goal appears to be using TPM to provide remote attestation for all code in the boot path. See https://www.system-transparency.org/.
Correct! Thank you for highlighting that.
Here are some additional details for those interested. We intend to make use of TPM for remote attestation of the current boot chain, reproducible builds to provide a strong link from source code to build artifacts, and a transparency log for a historical record of previously used boot chains, artifacts, WireGuard server keys, and related signatures.
As dtx1 mentioned elsewhere in this thread, diskless VPN infrastructure is currently in use by many other VPN providers. That is not a novel feature of course. What is novel is user-auditability of running VPN infrastructure. We were the first VPN provider to state our intention to make our infrastructure user-auditable AND provide a realistic roadmap with the specific technologies needed to do so. See the link above.
I believe the technologies we use in System Transparency will ultimately reshape the VPN provider industry into a highly competitive space focused on maximizing the transparency of VPN infrastructure. Or not, but at least OUR users will be able to audit us. :)
Either way we’re looking forward to the future. The opportunity for improvement is immense.
Niiice. I really love the concept of reversing the usual DRM use of remote attestation--forcing customers to prove they're running only software allowed by the megacorps. Instead of DRM, it's proving the corporation/server is trustworthy to the customer.
I think I could get behind more of this use!
1 reply →
If you're from Mullvad, let me tell you that I love your service.
I had one technical support question, and I got an immediate response from an actual person who knew this problem, gave me a simple workaround (toggle between wireless connection and not), and told me that a permanent fix was in the works. Likely that fix rolled out because I haven't seen the issue in months.
The idea of an account that doesn't need a password because there's no critical information saved is such a nice one, too.
Keep up the good work - I recommend you to everyone.
1 reply →
This all sounds very exciting. Where will you draw the line between public and private – at the moment your consumer-facing app is on github, but less "server side" stuff (in common with many other VPN providers). I understand that probably you want to keep the database of "active numbers" private, but if I understand you correctly, you want to move to a model where anyone can download your in-memory image, run it in a VM, and audit it independently. I would welcome this. I'm particularly interested in how you maintain access to your bare-metal machines (e.g. do you have ssh / a serial console enabled)
1 reply →
Isn't network monitoring and logging the bigger issue for a VPN service? How can you provide transparency of the network?
1 reply →
I hate to be this skeptical, but let's say this is 100% possible (I have my doubts, see previous attacks on things like TPMs and SGX, but I digress). You probably could get 90% of the logging capability by putting monitoring in front of and behind the server, and associating connections by traffic/time.
It just seems like this goal of using technology to prove they're trustworthy is unlikely to actually work for a VPN company due to the threat models.
You are correct that System Transparency is not a universal remedy for all threat models. Indeed the word "secure" is undefined until you have a threat model. Most threat models are implied and undocumented assumptions.
At some stage in an R&D project one should shift from exploration to threat model-driven development. Most people, myself included, tend to focus on technical solutions, and argue back and forth how "oh, but it can be broken using X".
System Transparency aims to provide remote auditability assuming (1) the server hardware specification is correct, (2) a correct cryptographic hash of the contents of the SPI flash containing the platform firmware, and (3) a keypair generated on and only accessible to the platform. This is very simplified of course.
An attacker aiming to tap incoming and outgoing network traffic from our servers, who has physical access to the VPN server's Ethernet port, or an upstream router, isn't in the scope of System Transparency to protect against. We need to use other means for that.
Traffic analysis and correlation analysis is indeed a powerful tool, and in general only communicating at a constant bandwidth between all nodes at all times is the only way to completely defeat it (which is what, I understand, some military systems do). That's inherently highly wasteful, however.
To get around this, Mullvad offer very transparent comprehensive multi-hop routing systems [1]; you can bounce your wireguard tunnels around in layered wireguard tunnels (a bit á-la tor) by just choosing a series of ports to tunnel on and to. My understanding is that each one of these adds non-deterministic latency to your connection and probably would help to make such attacks harder at the very least, because from the point of view of an "all seeing" adversary the fact that all of these servers talk to each other all the time makes it very much harder to know where any packet could have gone. Yes, you can see each individual link but the metadata is lost.
I signed up for Mullvad when the UK's Snooper's Charter came into force and the local health inspectors suddenly had the rights to see my DNS record. Since then, I've had it installed on my router and just route everything through a custom wireguard (originally openvpn) tunnel. I've had some issues with my ISP randomly bandwidth limiting traffic on the odd port to 1 MByte/s, but frankly that makes me more inclined to put everything behind an encrypted tunnel. I don't want my ISP to do traffic shaping and I do want them to just leave me alone and let me communicate in peace. I have absolutely nothing to hide, but now have to accept that I partly live in a country where everything is surveilled all the time, and warantless, unaccountable investigation of my (highly personal!) online habits may be happening. I think Mullvad's excellent product, sensible architecture and reasonable price is worth paying. I'm an academic, unlikely to be of interest to three-letter acronyms, and therefore it matches my needs very well.
[1] https://mullvad.net/en/help/multihop-wireguard/
1 reply →
How do we audit that the TPM chip is secure? What happened when a bug in the silicone is found later?
We assume it is, just like we assume CPU works as advertised. In other words, TPM is part of TCB.
5 replies →
Damn it, this is a really cool use case for TPMs! So far, every use case I've heard has made me wish they were never invented, but this made me reconsider...
I've been following Mullvad for a long time and my impression (from countless reviews and comments here on HN) has been quite positive. But here's what I don't understand: Why are the servers located in Sweden, a country that's known for online surveillance[0] like no other country in the EU? From the Wikipedia article[1]:
> The law permits the signals intelligence agency, National Defense Radio Establishment, to monitor the content of all cross-border cable-based Internet traffic to combat "external threats" such as terrorism and organized crime.
[0]: https://www.opendemocracy.net/en/can-europe-make-it/didier-b...
[1]: https://en.wikipedia.org/wiki/Internet_in_Sweden#Internet_ce...
Thank you for asking.
> Why are the servers located in Sweden,
We have 762 servers spread across 38 countries. Less than 10% of our servers are located in Sweden [0].
> a country that's known for online surveillance
My cofounder and I started Mullvad as a protest against the growing mass surveillance of Sweden as well as other countries. Our intent was direct political action through entrepreneurship. Incorporating the company in Sweden was the obvious choice, since we are both Swedes. We felt that incorporating elsewhere would have been more risky, complicated, and costly.
Mullvad has excellent lawyers who continuously monitor the legal situation in Sweden. Right now there is no Swedish law that can compel Mullvad to start logging [1]. Should the situation change we have contingency plans.
[0]: https://mullvad.net/en/servers/ [1]: https://mullvad.net/en/help/swedish-legislation/
Thank you, this was the response I was looking for!
> Our intent was direct political action through entrepreneurship.
Again, I didn't mean to question your intent – as I said my impression of your company so far has been a very good one! :)
> We have 762 servers spread across 38 countries. Less than 10% of our servers are located in Sweden [0].
My apologies, it's been a few years since I last looked at your server locations (so I didn't remember) and I was probably getting the wrong impression from the fact that your post is only mentioning server locations in Sweden.
> Right now there is no Swedish law that can compel Mullvad to start logging [1].
But at the same time all cross-border traffic in and out of Sweden (so for anyone using Mullvad outside Sweden: virtually all traffic) is being monitored and (probably) logged, isn't it?
4 replies →
I have so much respect for everyone at Mullvad. You are the only VPN provider I trust! I have been a user for years now, and it has always been 5 dollars a month, which is quite generous. It is so cheap that even the poor can afford privacy. You guys have put a ton of effort into making your service as privacy respecting as possible. Not only that, the tech is on the bleeding edge (WireGuard, socks5, etc.) built right in. As a cybersecurity researcher, I could not be happier with the product. I hope you stay true to your mission, thanks again!
Also, will Monero ever become a supported payment method?
How is this nonsense the top comment?
OP's been doing a poor job of "following Mullvad for a long time" and apparently has never used Mullvad or visited its website. Heck, it's on their Wikipedia page[1].
If they had, it would be immediately apparent that Mullvad has servers all over the world. They have for many years -- perhaps since it's inception. It takes a bare minimum of effort to learn that.
1. https://en.wikipedia.org/wiki/Mullvad#Service
Your annoyance is understandable, and yet this question and direct response from a founder is a very informative interaction to have recorded in this thread. Much more informative than if GP had simply stated the results of their Wikipedia research.
Sometimes a mediocre question is the landing pad for a great answer. No need to begrudge the question.
3 replies →
I have apologized for the factual incorrectness of my comment and explained how I had arrived at that conclusion / brain glitch: They are only mentioning Swedish servers in their blog post and for a second I had forgotten that, like any other big VPN provider, Mullvad of course has servers in many locations around the world.
What more do you want? Upvotes are not under my control and I also cannot edit my original comment anymore. No reason to get personal.
FWIW, I still think that it's important to raise awareness for the deficiencies of Swedish privacy law (irrespective of Mullvad and where their servers are located). I suspect that at least some of the upvotes were also given because people agreed with that.
Because it's a swedish company? The location of the servers is kinda irrelevant in that regard. They'd have to provide government access if there is a lawsuit that demands it. If there isn't one than your critique is entirely pointless
> Because it's a swedish company?
I'm not sure what point you're trying to make. If online privacy is as important to them as they say, they could have easily registered their company (or a subsidiary) in a different EU member state.
> They'd have to provide government access if there is a lawsuit that demands it.
IANAL but I am not entirely sure this is true in the EU. Either way, my question was "Why are the servers located in Sweden?" Whether or not this due to the company owning the servers being in Sweden is irrelevant.
23 replies →
You don't have to use the servers in Sweden if you don't want to.
A bit tangential to the main post, but I'd to share a recent positive experience with Mullvad:
I am a regular user of Mullvad and recently wanted to try a different VPN, that only provides Wireguard configs (i.e. no native app). I used the default setup.
For some reason, my internet connection was flaky, and when it disconnected and reconnected, my traffic leaked.
That never happened to me with Mullvad as the app comes with an "Always require VPN" option out of the box and it has always worked reliably.
On linux you can create a network namespace exposing only the wireguard network device, so that applications in that namespace cannot leak traffic. Setting this up, however, is quite fiddly in my experience.
In addition it is probably not a bad idea to block all traffic on wlo1 / eth0, except that to the mullvad server ip's, through some ufw rules. If you forget to configure the namespace for some applications then, it is highly unlikely the app has internet access (ie, it would need its own mullvad/vpn implementation included).
I have this setup in my homelab, use it to isolate networks by role. The bash script to setup and tear down the namespace is here:
https://github.com/VTimofeenko/wireguard-namespace-service/b...
I would suggest vopono to do this automatically.
It’s easier and more secure to just create a VM that’s bridged to the VPN interface (regardless of protocol) if you don’t use the VPN for everything but the things you do use it for absolutely must go through it.
1 reply →
Any “always require vpn” option is a game of cat and mouse and is going to leak traffic at some point, whether easily detectable or otherwise. As others have said, you need to set up a secure environment that only has that one and only option for accessing the outside world.
Agree, Mullvad provides really good VPN service. I faced almost zero downtimes / speed throttles. It establishes quick connection with server (maybe because it uses wireguard). Anyway, I'm a regular user and I think paying 5E worth it.
> It establishes quick connection with server (maybe because it uses wireguard)
I'm actually kind of curious about what Wireguard does here. I think Wireguard says it's connected almost immediately even when it isn't, presumably holding traffic back locally while it waits for the connection to be active. I was wondering because I spent some time confused by a non-Mullvad Wireguard connection that wasn't working (turns out the server wasn't available at all) that nonetheless appeared as "connected".
1 reply →
Seconded. Very occasionally I'll have to swap servers in a location but that's super infrequent and it's not exactly a primary tier location that I'm using. One of those things I can generally just setup and leave working.
Can you share the WireGuard config? I suspect it is either missing firewall rules, or they're badly set.
I love the concept and I even wrote a blog post about how to set up a fully pxe bootable server environment using Alpine Linux [1] (which by default boots from RAM) in 2019. I still use it and it's one of those things that makes recovery or testing so much faster because I don't even need a usb thumb drive
[1] https://blog.haschek.at/2019/build-your-own-datacenter-with-...
This is awesome, glad that Mullvad is heading in this direction.
For reference, ExpressVPN (which has been audited by PwC) introduced this in 2019 [1].
Unfortunately, ever since ExpressVPN was purchased by Kape Technologies (they also own PIA, Cyberghost, Zenmate all of which do not have reliable histories); Mulvad has been the clear choice for a while now. They're also the backend for Mozilla VPN (mozilla just whitelabels from Mulvad [3])
Some information that could be of interest to those running VPN servers.
I live in Kazakhstan and recently our government decided to shut down the Internet. But apparently there were ways to get out: they did not filter two TCP ports. My guess it was some "backdoor" put by employees who had to obey the orders but wanted to provide people some way to get around those blocks. Those ports were used to run VPN software. I used Outline VPN on my VPS and it allowed me and my friends to have a working Internet.
TLDR: allow specifying port and protocol (TCP/UDP) as some kind of advanced option for those users who need it for some reason.
Right now we've got Internet back and it works fine, but who knows when our government will decide to shut it down again.
PS mullvad.net website apparently is blocked in Kazakhstan as well. I know that they block popular VPN provider websites, so that should not come as a surprise, but still. I have no idea whether actual VPN subnets are blocked or not.
I think it is probably more likely those ports were the backdoor the government or their allies was using to function. Everyone always builds in a backdoor for themselves!
When I hear about things like this it makes me glad I have a simple satellite communicator - it will only do short text messages but that is a hell of a lot better than nothing. Of course one could get a full satellite based mobile internet device or phone but the plans on those get quite expensive.
> If the computer is powered off, moved or confiscated, there is no data to retrieve.
Don't forget to add insta-shutdown when any USB device is connected to the system!
And a tilt sensor that cuts power if the chasis is moved
Or disable usb in bios entirely
Disabling USB in BIOS only disables the emulation of classic PS2 keyboards and IDE storage so that old OSes or bootloaders without USB stacks can work with modern equipment. As soon as the OS kernel initializes the PCI bus, USB will work again - however they could go and remove the xHCI modules from the kernel and image.
2 replies →
The VPN provider I use have had diskless systems for years. Is this the same thing?
https://www.ovpn.com/en/blog/improvement-of-the-physical-sec...
It's a trade-off. If you have no disk, the disk can't fail, but the network can, and the remote PXE server can, and the remote SAN can. You can get into a state where you have to pray no servers reboot. Intermittent errors can be real annoying when it makes provisioning fail. (used to work a server farm that'd do server rebuilds over PXE, and ran a few diskless cluster projects)
An alternative is you use a RAID array and mount your disks in read-only mode, or use physically read-only disks and when you have to replace a disk, you pre-mirror the replacement disk. In this way the local disks can be replaced as they fail and there's never a point when the server is at risk of not being able to boot.
......or they could boot from CDROM :)
A network or PXE server can fail regardless, so this are things that always have to be taken into consideration and in those instances then you address those issues. With this type of setup you do not need a remote SAN as it would defeat the purpose of not having external storage that could store logs. Mullvad has servers all over the world, so a temporary failure in one location will not bring down their entire infrastructure.
It's not just a temporary failure, it's potentially the entire AZ going down hard. High Availability network boot without local storage is very difficult/expensive.
They can still use local disks to provision the OS over a network but boot from local storage, and prevent writing to disks from the booted OS (hell, they can completely remove the disk drivers from the kernel!). It just doesn't make sense to ditch the drives from a reliability standpoint. They're going to have a big outage one day just because they didn't want to deal with drives.
1 reply →
Every year I feel more and more proud to renew my subscription. What a great company.
This is hardly a new thing in VPN providers though. I know that perfect privacy[1] and azire vpn[2] both advertise this feature already.
[1]https://www.perfect-privacy.com/en/features/without-logs [2]https://www.azirevpn.com/docs/environment
if only there was any proof of this actually being the case and there not being some "accidental" debug log enabled, or some other network level component having "accidental" access to the keys.
There's just no good answer to perfect trust-no-one private internet access.
If you need to hide all of your traffic from other users in your local network, you can accomplish that in a trust-no-one fashion by running your own VPN endpoint on a server you control which provides better privacy guarantees compared to a centralised commercial VPN whose business model will eventually involve selling your data (once user growth stops but shareholders demand continued revenue growth).
But if you need to hide your traffic from anybody but your peer on the internet and you need to hide the fact that you talked to that peer, then, I'm afraid, your out of luck.
> If you need to hide all of your traffic from other users in your local network, you can accomplish that in a trust-no-one fashion by running your own VPN endpoint on a server you control which provides better privacy guarantees compared to a centralised commercial VPN whose business model will eventually involve selling your data (once user growth stops but shareholders demand continued revenue growth).
Well not really. There was a great (german) interview with the perfect privacy founders recently [1]. They seem to be decent guys with close ties to the Chaos Computer Club and I strongly suspect they wouldn't want to work like that.
[1] https://www.youtube.com/watch?v=VMr0gJvI-6I
> But if you need to hide your traffic from anybody but your peer on the internet and you need to hide the fact that you talked to that peer, then, I'm afraid, your out of luck.
Nah, that one is easy just use an anonymous sim card or an open wifi and your good to go.
Honestly these discussions often feel pretty asinine to me. I personally use paid VPNs to pirate to my hearts content, work around my ISPs terrible networking and a little bit of geo-unblocking. Of course you can't use these services to protect yourself from three letter agency type surveillance or equally powerful threat actors but if they are "private" enough to block the music industry and their lawyers from suing you that's a pretty high standard of privacy, certainly more than any ISP alone gives you!
5 replies →
>> If you need to hide all of your traffic from other users in your local network, you can accomplish that in a trust-no-one fashion by running your own VPN endpoint on a server you control which provides better privacy guarantees compared to a centralised commercial VPN whose business model will eventually involve selling your data (once user growth stops but shareholders demand continued revenue growth).
the privacy protection for most people using VPNs is required against their ISP and other actors looking to analyse their traffic, not users on the local network. a commercial VPN will be better for privacy due to the crowding effects, ie. large number of users sharing the same IP and protects against correlation attacks - it's much easier to trace the activities on your own VPN endpoint back to you. of course you need to trust the operators, which is as different question.
5 replies →
Is no-trust ever possible? I thought people create their threat models and verify they can trust those they have to trust.
2 replies →
That is the great thing about mullvad. They don't have shareholders except the actual owners.
https://mullvad.net/en/blog/2021/9/16/ownership-and-future-m...
4 replies →
Wouldn't the ownership of the server be easy to trace back to you?
PIA has been promising a fully audited and verifiable infrastructure in the future:
https://www.privateinternetaccess.com/blog/dont-trust-verify...
2 replies →
> There's just no good answer to perfect trust-no-one private internet access.
What about Tor?
4 replies →
https://archive.is/HbjIQ
(it was blocked for me because it's a VPN service provider)
The server configuration (and therefore customers account numbers) is stored in Server OS images I suppose, right ? It shouldn't be an issue as far as inspection is concerned, should it ?
Also, isn't there a law that enforces logs to be kept for n years ? How is it compatible with diskless setup ?
No, they would not need to be stored in the images. They are deployed as part of provisioning package according to the post. If they were part of the OS image, they would have to update the image anytime a new customer is added, which would be far less optimal than having Ansible or whatever push changes. There is no law in Sweden that requires logs, and the point of a VPN provider is to not store logs. If your VPN provider is keeping logs of your connections then you should look for a new provider.
Isn't this the same as just running a boot cd or PXE server and running all the data out of RAM drives? I mean we've been doing this for years on linux as hobbyists haven't we? Or does this bring somethign new to the table?
Pretty similar to the default (diskless) mode in Alpine, though it lacks the tooling to verify persisted data and the sources apkvols can be applied from at boot aren't that well documented.
How do you debug when stuff goes wrong?
The next step: dedicated VPN over FPGAs
*drools*
Wait, Cisco already do this
1 reply →