Ask HN: We just had an actual UUID v4 collision...

> you can never have too many sources of entropy

This is so true. And the beauty is that with algorithms, we don't even need to know much about the entropy to be able to extract it.

There is the Von Neumann method of generating an unbiased coin from a biased coin. Of throwing it twice, and checking if you got HT or TH. And completely discarding all HH or TT results. It doesn't matter if the coin you are using is 20% or 80%, the result will be a true 50/50.

There are more modern algorithms that can be even better (in that they need less coin tosses if you have a very unbalanced coin).

And then there is modern cryptographic hashing. Feed it all the bits you can. Collisions end up only happening in the real world if every single one of those bits is identical. So if you have actual entropy being fed, that cannot be controlled, predicted, or replicated, modern cryptography tells you that the end result is unique.

If I understand it the Lava lamps are 90% PR/fun. They have a lot of other sources for entropy that scales better.

> This is why CloudFlare has done what they did with the lava lamp wall.

Interesting. I wonder how true it actually is that they use it like they claim here: https://www.cloudflare.com/learning/ssl/lava-lamp-encryption.... It's in one of their lobbies, so doesn't that make it susceptible to an attack in some way? I'm not knowledgeable enough to know, but I figured if they actually used that method, they'd have a more controlled environment.

I also don't fully understand it. A large part of that wall is static. And the camera isn't going to pick up on the stochastic properties of the lava as much as exists in the real world. So it feels like their images will be very statistically similar.

The lava lamps are just for show.

You can get entropy just by plugging an oscilloscope into a pile of dirt and cranking the gain up.

Old games are RTA viable to RNG manip: https://m.youtube.com/watch?v=Bgh30BiWG58

In high-reliability systems a criterion for identifier design is easy detection of defective identifiers. This includes buggy systems and adversarial manipulation.

The problem with UUIDs that rely on entropy sources is that it is computationally expensive to detect if the statistical distribution of identifiers is diverging from what you would expect from a random oracle. I've written systems that can detect entropy source anomalies but you'll want to turn it off in production.

It is pretty cheap to sanity check most non-probabilistic identifier schemes. UUIDs that use broken hash algorithms (e.g. UUIDv3/5) or leak state (e.g. UUIDv7) are exposed to adversarial exploitation.

The identifier scheme is dependent on the use case. Does the uniqueness constraint apply to the instance of the object or the contents of the object? Is the generation of identifiers federated across untrusted nodes? How large is the potential universe of identifiers?

The basic scheme I've seen is a 128-bit structured value that has no probabilistic component. These identifiers can be encrypted with AES-128 when exported to the public, guaranteeing uniqueness while leaking no internal state. The benefit of this scheme is that it is usually drop-in compatible with standard UUID even though it is technically not a UUID and the internal structure can carry useful metadata about the identifier if you can decrypt it.

Federated generation across untrusted nodes requires a more complex scheme, particularly if the universe of identifiers is extremely large. These intrinsically have a collision risk regardless of how the identifiers are generated.

All of the standardized UUID really weren't designed with the requirements of scalable high-reliability systems in mind. They were optimized for convenience and expedience which is a perfectly reasonable objective. Most people don't need an identifier system engineered for extreme reliability, even though there is relatively little cost to having one.

The latest UUID (7?) Uses half random gen, half timestamp. This not only makes it sortable by creation, but would also make a collision like this impossible.

UUIDv7 is arguably better, because it is entropy plus time.

Sequences, generally.

I wouldn't say it's "to blame", but it is more susceptible to bad RNG.

If the RNG is bad, you'll get more benefit from adding non-random bits than you would from additional badly RNG'd bits.

The probability of future collisions also rises the more IDs you generate. If you incorporate non-random bits, you can alleviate that:

- timestamps make the collision probability not grow over time as you accumulate more existing UUIDs that could collide

- known-distinct machine IDs make the collision probability not grow as you add more machines

I never blamed UUIDv4 for broken entropy sources. A broken entropy source breaks UUIDv4 even if you are using it correctly.

There is a long history of broken entropy sources showing up in real systems. No matter how hard people try to prevent this it keeps happening. Consequently, a requirement for high-quality entropy sources is correctly viewed as an unnecessary and avoidable foot-gun in high-reliability software systems.

Presumably they mean using randomness as unique IDs.

In these kinds of systems the cryptographic components often aren't even accessible from the software. It isn't a thing you need to worry about.

This makes it easier to audit for use of entropy sources in the software since there really isn't a valid use case for it.

A collision is simple to detect but it requires you to actually check, which is expensive at scale. The entire point of UUIDv4 is that you don't have to check for collisions because it should never happen. But if you don't check and it does happen you are in UB territory which is generally very bad.

A risk of collision before it happens is non-trivial to detect but this is really what you'd want.

In this specific case. In the case of trace IDs (an example of which is [1]) where the equivalent of UUIDs are explicitly used to avoid coordination, it’s hard to imagine how you’d reliably detect and retry.

[1] https://news.ycombinator.com/item?id=48033853

> My early jobs were at startups startups with limited resources. Every decision to build something or hire someone was carefully made after much consideration. This story would have looked like fiction to me at the time.

This was pre-2015

> Later in my career I joined a startup like this where every new concern someone could think up turned into a new microservice with new hires to form a new team. It didn't matter how small it was, everything was a reason to hire new people and form a new team. I sat in meetings where the express goal of the quarter was communicated as growing the engineering team.

This was post-2015

---

Am I right?

You're describing exactly what I've tried to express in various comments. There was a point in the latter half of the 2010s when it became genuinely hard to find tech work where you were building useful stuff. Startups become increasingly absurd and the focuses of their engineering teams even more so.

In 2019 I was working for a company who were so desperate to hire new engineers at one point they decided to just start offering jobs to candidates which failed interviews. It was absolutely insane.

> someone's KPIs to grow the engineering team to a specific number

Sigh!

Specific numbers!

I believe a more common specific number is the yearly EBITDA or ARR (or some other acronyms in this alley I care zero about to memorize) nowadays, for investor's sake. Like in our company. Since we were acquired - and some time before - the only talk in company meetings are EBITDA, ARR, compared to a number dreamed up by someone and to be reached in 5 years time. Specific financial results in specific timeframe. Our goals are specific numbers being above today's numbers by a chosen margin. The company talk are marketing campaigns and reach, campaign efficiency measurements, pricing strategies, subscription centric licensing, sales strategies, churn, and other slang around customer bullying I also do not care about, also organizational streamlining - what a loaded word! -, bla bla bla, all for the specific sacred number put up on the pedestal.

What we have zero talk about? Functionality, engineering.

I seriously do not understand these people. Why are they fiddling around with selling software in a niche sensitive to global economic fluctuations insted of selling ... I don't know. Shoes? Or better yet sugary water ... no, better is vitamin water ... no, the trendiest is protein water. That is something that needs no balanced functionality and engineering that is laborous so it is resource intensive to achieve. And is in the way of reaching the sacred number put up there. Engineers are in the way towards our goals. We are pulling back the cart! We are cost center now!!

I do not stay long.

Define a random 128 bit key that you will never change. Use that key to encrypt 128 bit integers in sequence using AES-128, each one comes out as a, for all practical purposes, unique unpredictable ID.

Twitter snowflakes haven't changed. Most of the bits go to the timestamp, which I guess is a global incrementing counter as you described

> At some point someone optimizes the system to a global company-wide incrementing 128 bit counter.

Some UUID versions include time, so there's a bit of a counter in that.

They should upgrade to Zookeeper II: Zookeepier.

https://www.youtube.com/watch?v=_F-RyuDLR4o

GUIDs are UUIDs are effectively the same thing... the issues often come down the the means of generation and storage... where UUID have versions with specific implementation details that aren't always followed, MS has internal implementations that also aren't always followed. Also worth being aware of are COMB, SequencialIDs (MS-SQL) and other serialization approaches as well as how they affect indexes in practice.

Alternatives include sequencial number generator services, or sequence services that may be entirely sequencial, etc, but may lead to out of order inserts in practice.

Also, generally worth considering UUIDv7 assuming your sotrage and indexing use the time portion at the front of the index process.

You need at least 3 for this. People go on vacation, turnover, can’t risk losing that critical institutional knowledge.

I too have witnessed a "add two numbers" service! Turns out you can be too extreme with rules for isolating out business logic..

No that kind of critical data would be sent to pendo so it could be reported on or shown in a dashboard!

That sort of table can be quite handy when every entity in the business's data stew is identified with a UUID, and there is no way of telling just from looking at an identifier what kind of entity it is. Particularly when the business has disparate databases and/or microservices with their own sets of UUIDs.

In such businesses, inevitably, someone will ask you to run process X for widget 8dbcd950-14c1-4877-a8b0-90c081ce033c, and that particular identifier will actually be an ID of some associated data, not the widget. You can push back and say, "That isn't a widget identifier, can you please look up the widget identifier?" It's better to be able to look that ID up in your ID ⮕ entity type lookup table, and say "the ID you provided is a widget production run ID, which produced a copy of widget a84969be-137a-41ca-97c4-515497184df9. Can you confirm this is the widget you need process X done for?", with a link to the product-facing widget page.

(Also handy for the case where some code was intended to log an ID for one entity, but actually logs the ID for an associated entity with the wrong entity type indicated.)

They thought of that, but they were still working on hiring a team to maintain the hashing microservice.

already laughing from parent comment this is well done

one hash is insufficent, they need k-hashes.

i get the joke, but seriously a bloomfilter would be a good idea.

I will gladly assume that this team was formed after several collisions with UUID's my assumption is that they had tremendous amount of data and enough revenue to justify all of this at least financially. I would have re-evaluated the UUID version used or if adopting Snowflakes would be better at some point.

Damn, I got the paper stack wet with all that ocean water. Guess I'm starting again from scratch...

They have been not at work on doing that.

Tell that to Segment. Hopefully they've fixed that, but they didn't seem to think it was a problem years ago (spoiler: it was a big problem).

Well, this joke dates back to (at least) the dial-up days where {#`%${%&`+'${`%& NO CARRIER

That’s just a result of jounce from localized gravity effects and atmospheric pressure disturbances in the moments before impact.

Think the ultrasonic typing hacking scene in Pantheon combined with the keyboard bouncing due to rumbling.

>The number of systems out there is also astronomical.

Not even close

But defaults should be sane and safe. RNG isn't the sort of thing you want to be messing up. Every JS dev was taught that Math.random is not safe by default, but the crypto package is.

user-generated (as in: on the user's phone) was only at the very early stages of this product, and we've since moved to on-server. It's a cash-register type of app, where the same invoice must not be stored twice. So we used to generate a fresh invoice_id (uuidv4) on the user's device for each new invoice, and a double-send of that would automatically be flagged server-side (same id twice). This has since moved on to a server-only mechanism.

The database flagged it simply by having a UNIQUE key on the invoice_id column. First entry was from 2025, second entry from today.

If it's UUIDv4 and you validate that the UUID is valid and not conflicting I don't really see the issue with user-generated UUIDs. Being able to generate unique keys in an uncoordinated manner is the main selling point of UUIDs

Sure, it's something I'd flag in any design to spend two minutes to talk about potential security implications. But usually there aren't any

Likely a unique index... duplicate insert on a primary or 1:! foreign key. I am currently shimming out a process that will add a trackingid for a job service, and just had my method stub retorn Guid.Empty... second time I ran my local test it blew up on the duplicate key... then I switched it to null, then it blew up again... I neglected to exclude null from the unique index on the foreign key.

In any case, it's easy enough to do. I mostly use UUDv7, COMB or NEWSEQUENTIALID ids myself though.

The smart way would be to check if the id is in use, and generate a new one... Repeat a few times if you're extremely unlucky, and bail out with an error if you have the absolute worst rng. It also works for locally generated ids as well.

The library is using node:crypto, but with a phone target, that's likely shimmed with a JS implementation...

The shim for node:crypto in the browser is likely a weaker implementation in JS than the node implementation... you can cheat and use the browser itself to get a UUIDv4...

    function uuid4() {
      var temp_url = URL.createObjectURL(new Blob());
      var uuid = temp_url.toString();
      URL.revokeObjectURL(temp_url);
      return uuid.split(/[:\/]/g).pop().toLowerCase(); // remove prefixes
   }

Didn't actually want to write a test myself.. but I miss Claudia confirmed it. Pretty concearning.

Synchronous / serial calls:

   import rng from './rng';
   
   const a = rng();
   console.log('a after first call: ', Array.from(a));
   
   const b = rng();
   console.log('a after second call:', Array.from(a));
   console.log('b after second call:', Array.from(b));
   
   console.log('a === b (same reference)?    ', a === b);
   console.log('a equals b (same contents)?  ', a.every((v, i) => v === b[i]));

output:

   a after first call:  [
     101, 193, 125,  19, 142,
     136, 181, 140, 209, 224,
     176, 153, 179, 248, 246,
     166
   ]
   a after second call: [
       4,  29, 48, 215, 162,  60,
      64,  23, 78, 137,   2, 186,
     230, 249, 70, 224
   ]
   b after second call: [
       4,  29, 48, 215, 162,  60,
      64,  23, 78, 137,   2, 186,
     230, 249, 70, 224
   ]
   a === b (same reference)?     true
   a equals b (same contents)?   true
   

and aynchronous calls:

   import rng from './rng';
   
   async function getId() {
      const bytes = rng();
      await new Promise(r => setTimeout(r, 0)); // yield to the event loop
      return Array.from(bytes);
   }
   
   const [id1, id2] = await Promise.all([getId(), getId()]);
   console.log('id1:', id1);
   console.log('id2:', id2);
   console.log('identical?', id1.every((v, i) => v === id2[i]));

output:

   id1 captured:  [
      61, 116, 151,  35, 153,
      75, 105,  15,  59, 235,
     162, 215, 224, 115,  31,
     122
   ]
   id2 captured:  [
      13,  3,  84,  28, 22, 176,
     160, 70,  67, 246,  1,  37,
      38, 61, 171,  23
   ]
   id1 after await: [
      13,  3,  84,  28, 22, 176,
     160, 70,  67, 246,  1,  37,
      38, 61, 171,  23
   ]
   id2 after await: [
      13,  3,  84,  28, 22, 176,
     160, 70,  67, 246,  1,  37,
      38, 61, 171,  23
   ]
   ---
   final id1: [
      13,  3,  84,  28, 22, 176,
     160, 70,  67, 246,  1,  37,
      38, 61, 171,  23
   ]
   final id2: [
      13,  3,  84,  28, 22, 176,
     160, 70,  67, 246,  1,  37,
      38, 61, 171,  23
   ]
   identical? true

https://github.com/uuidjs/uuid/blob/e1f42a354593093ba0479f0b...

became

https://github.com/uuidjs/uuid/blob/f2c235f93059325fa43e1106...

Welp.. time to patch and update everything again. Another day, another npm-package headache. Very odd()

Attack vector: call the rng(), and send the result somewhere. You now have now overwritten someone elses "random number" and know about it. The fun things you can do with those numbers!

But I used nonstandard nonces!

This is probably (ha) a troll thread, but in case anyone here is among today's lucky (ha) 10,000, https://en.wikipedia.org/wiki/Independence_(probability_theo...

The chance of him winning the lottery is identical to before, however the reward if he wins is slightly greater.

He would win the lottery money + he gets to tell people who don’t understand independence this incredible story!

No, the events are independent. If you have a UUID collide, your chance of winning the lottery if you enter it is exactly the same as it was before the UUID collision.

Actually asking ChatGPT this query led it giving me this UUID "550e8400-e29b-41d4-a716-446655440000" which happens to be a very common example UUID

uuid.uuidv4() recently switched to "adaptive entropy" instead of "xmax entropy" in an effort to save costs on non-premium users.

You mean you’re not already entropymaxxing? n00b

You can check https://everyuuid.com/ for collisions.

We should all send our already-generated UUIDs to a shared database, we could just put it on Supabase with a shared username/password posted on HN, so we can all ensure that after generating a UUIDv4 locally, it's not used by anyone else. If it's in the database, we know it's taken.

It's a super simple mechanism, check in common worldwide UUID database, if not in there, you can use it. Perhaps if we use a START TRANSACTION, we could ensure it's not taken as we insert. But that's all easy, I'll ask Claude to wire it up, no problem.

A site previously posted here could be useful: https://everyuuid.com/

But since the randomness is obviously borked, it was much better to use v4 and find out about it after just 15K records instead of X million records later.

UUID v4 consists of only random bits, no timestamp info.

But surely hashing the date still allows for a future collision. Leaving the date as is means it will never collide after that one second has passed.

Assuming they are independent events, OP is not more nor less likely to win the lottery now that before running in the collision. I actually have more question if you claim the events in question are NOT independent!

Not GP, but: not confident. How confident would I be to avoid a (slightly lower entropy) UUID collision while also avoiding a clock desync landing on the exact same logged millisecond? Very, which is how confident I was about not encountering an UUID collision before this thread, so very++ I guess.

I get why sync of mutiple machines matters for ordering and causality, but why is it a problem for uniqueness?

Ironically one of the few comments in this thread that isn’t necessarily the gamblers fallacy!

The chance anyone involved saw or heard about the first one was near zero, now they’ve seen this one the chance they see another is still near zero (I.e unchanged).

> Statistically speaking, does extremely unlikely mean impossible?

No, it means extremely unlikely. Collisions can occur, as op just found out, but the chances are so abysmally small that most people don't care.

Any application I have worked on, I always had a pre-save check to see if the UUID was already present and generate a new one if it was. Don't think it ever triggered unless a bug was introduced somewhere but good practice anyway.

You are replying to an AI bot

No, to be honest. However, as soon as it was pointed out, I checked again and it made sense.

In my opinion, these kind of intuitions have to grow over time. And every time it’s pointed out, you learn. So please, keep pointing it out :).

I did not. Post-conditioning by your comment and the other one,I can see some signs such attempting to be unusually comprehensive. The 'atoms in your liver' could be an awkward human trying to be poetic about scales.

I still don't see idiomatic markers of AI so that's scary if your claim is correct.

I guess not, and I feel dirty now. I'm logging off for the day.

Interesting enough, I skipped it when scrolling through the comments the first time. I think I instinctually do that to most karma whoring comments, no matter if manual or LLM generated.

Only noticed it because I did another pass and saw the replies talking about "AI".

Yes but as a feeling (hunch?) not as something my brain analysed and reached a conclusion.

Weird how I'm already somewhat conditioned to spot it on a intuitive level.

Kind of. It reads a bit too much like tech support you'd get when asking one for help.

when it started going on about all the different cases in the second bullet point... yeah

Yes, stupid comparison with atoms in the liver and a bullet list below? I stopped reading.