Comment by holoduke
3 years ago
I am running a cluster of machines at Hetzner for years. Generally i am satisfied, but i like to share some bad experiences which took a month to fix. At some point some of my servers were facing random cpu stalls mostly at occuring midnight. I spend hours and nights to find out the cause. But i wasnt able to find it. Tracing the issue resulting in different causes everytime. After I contacted the Hetzner support team they moved some of the servers to a different host system. Apparently there was a resource issue in their virtualization layer. It fixed all the issues for a week, but then it started again. I contacted support again and received an arrogant email that the issue is related to my software and that they couldnt help me further. I was perplexed. I solved the issue by creating servers in a different zone. Exact copies of the so called faulty ones. The whole stack is now running without issues for about 2 months. But still i am a bit worried.
Perhaps you could give me a ticket number (or the latest ticket number) about this issue, and I can ask a team member to review it for you. Or, you can send the ticket number directly to marketing@hetzner.com with a link to this page, and I can do the same thing. --Katie
High visibility customer support via hacker news is not a feature, it‘s a failure.
And those failed communications between provider and customer are why everyone is frustrated with the big corpos (google, cloudflare, stripe, etc.) which Hetzner is apparently bound to become.
At least you have someone speaking for Hetzner officially here. Contrast that with Google, where Matt Cutts was kind of used as unofficial backchannel for customer complaints, up until he left Google, and now all we have are prayers that some unknown reader will be moved by our plights.
This is not to excuse bad support experiences. I just think we should appreciate it when there is still a channel to a real human being willing to be helpful and engage constructively, because it's getting increasingly rare these days.
Support is a very hard issue to solve when you have volume. It doesn't help that 99.9% companies regard support as a cost center so salaries are low and there's a lot of churn and disregard for supportpeople tools.
Isn't it great that they recognize this failure and are attempting to fix it?
2 replies →
I think it's pretty cool of this person to come and try to help.
We had a similar problem at Hetzner, we were using their consumer grade systems and this was over 6 years ago. To Hetzner's credit though, after we decided it had to be a hardware problem (dmesg would log something about cpu states before every crash) we sent in a ticket and within a reasonable timeframe (I think within a day or a couple, it was long ago so don't know for sure) a Hetzner sysadmin went into the BIOS and changed some feature (I suppose he disabled some sleep mode) and the systems ran perfect after that.
I chalked it up to our decision to run our databases on consumer grade hardware and didn't give it a second thought.
> I contacted support again and received an arrogant email that the issue is related to my software and that they couldnt help me further.
Doesn't surprise me, German companies always know better and will try to prove you wrong, whether you're their customer or supplier. Such is the business culture, I think.
As someone who has customer service experience in both the USA and Germany, I can tell you that there are, indeed, cultural differences. There are certain situations in the USA where Germans and other Europeans don't always have the best experience because of a cultural misunderstanding. I am also the in-house English teacher here at Hetzner. (Most companies have no in-house teacher.) Something that I personally work on with my students in my conversations class and customer service class is intonation, which can cause spoken language to come off as sounding "arrogant" by accident. In addition, written responses may accidentally come of as sounding too direct or "arrogant" for the same reason. We work on these situations in my classes. So if you ever have a ticket that you think might make for good learning material for one of my classes, or that you would like to see escalated because of a serious language/cultural misunderstanding, please write to marketing@hetzner.com and mention my name (and include the relevant ticket number). --Katie
While it might be a cultural issue, I have to admit that this isn't the case with many Germans. I interact daily in open-source projects I'm involved with. While they are direct most of them seem very respectful and welcoming. From time to time some people might seem a bit more arrogant but I think that there might be a cultural bias too at play here.
Kudos for assisting your colleagues with better communication in English, I think this practice could be useful for many organizations with people that speak English as a foreign language.
I think its an issue of inter cultural communication.
My impression is that in Germany you bear the burden of proof that you actually have a problem, that its not your own fault, and that you did your part trying to fix it.
You usually can get good support in Germany ( even at government offices ) , IF you show up with your Leitordner ( legendary German ring binders) with all the receipts, all the possibly relevant account numbers, transaction ids and a detailed analysis of your own problem.
There is nothing that signals to German support staff that you have to be taken seriously like a ring binder, preferably with color coded markers at the margins and lots of punch pockets.
If you think this is satire, try it the next time you have an in person appointment...
Btw, the company making the ring binders is called "Leitz".
The rest is entirely correct, though.
This works really well in the United States, I've found. Showing up with an arsenal of documentation (virtual or otherwise) sets the tone.
Better than support tickets to a country that will ignore you or deny anything is wrong.
> Doesn't surprise me, German companies always know better and will try to prove you wrong
Yep. Its a plight that afflicts almost all engineers around the world, and German companies are even more afflicted by it due to the German engineering culture...
I had a weird performance issues on Hetzner virtual machines where network bandwidth would drop to 50mbit/s like it was being throttled. My application had a tendency to use very little bandwidth most of the time but when updates were available we needed 10-40GB in a day or so and I always suspected we were getting throttled but support never admitted it. On average we were below their resource limits but clearly at times we were above the average. Support was pretty dismissive and arrogant and I don't regret moving off to netcup who in comparison the experience has been flawless.
Something similar happened to me. A server I have was very frequently randomly restarting, and I tried everything to solve it, i.e factory reset, reinstalling my services. I contacted customer support and they offered me to take it offline and perform some tests on it that could take more than 10 hours. After I insisted a lot, they offered me a one time server replacement, which magically fixed the restarting issue.
Thanks for writing about how our team helped you resolve the issue. Have you been happy with the rest of your experiences using the cloud products? --Katie
That's not what we wrote. The support team replaced the server after a lot of cajoling. That experience needs to be fixed.
1 reply →
You thinking this was a positive experience only makes me more concerned about your customer support.
Let's hope I don't need another "one-time" server replacement because of your faulty servers, because I already used mine. 10 hours of downtime with angry customers on my tail sounds fun.
> I contacted support again and received an arrogant email
Yep. The attitude of the engineering support is something that Hetzner needs to improve a LOT on. They tend to treat customers as if they are treating members of their own open source or engineering community - scolding the customers when they think the customer (community member) is being unreasonable etc.
Its a great thing that eng. support is just a ticket and 20-45 minutes away at Hetzner. But the attitude needs a lot of improvement. Imagine that you are a startup facing some quirky issue that affects your business in the middle of the night and having to deal with attitude and scolds from datacenter engineers...
And why are you so sure it's not on your end?
> I solved the issue by creating servers in a different zone. Exact copies of the so called faulty ones.
Considering the issue didn't start happening immediately the previous times I don't see why we can presume it won't start happening again this time.
2 replies →
Are you using their “standard” cloud or dedicated vcpu product?