Comment by nitwit005
6 days ago
That sounds so expensive it's hard to see it making money. You'd processing a 2fps video stream for each customer. That's a huge amount of data.
And all that is for the chance to occasionally detect that someone's seen an ad in the background of a stream? Do any platforms even let a streamer broadcast an NFL game like the example given?
I used to work for an OTT DSP adtech company i.e. a company that bid on TV ad spots in real time. The bidding platform was handling millions of requests per second, and we were one of the smaller fish in the sea. This system is very real. Your tv is watching what you’re watching. I built the attribution pipeline, which is what this is. If you go buy a product from one of these ads, this is how they track (attribute) it. Not to be alarmist butttt you have zero privacy.
The TV thing isn't a new story, this was public. Everyone should have known about it and no one cared. (I could inset a boilerplate rant about Snowden here)
Those datacenters are not being built so that you can talk to ChatGPT all day, they are being built to generate and optimize ads. People who were not previously very suggestible are going to be. People who are suggestible will have their agency sold off to the highest bidder.
Avoid owning a TV? Your friends will. Maybe you can not have a FB/IG/WhatsApp account, only use cash, not have a mobile phone, but Meta (or Google, or Apple) can still detect your face in the background of photos/videos and know where you shop, travel and when.
This is really interesting. Can you expand on this? What are OTT and DSP in this context?
Do you have a sense for what data is tracked and how it's used? Or if this sort of system is blind in certain cases? (eg: I hook up an N64 to the a/v ports -- will I get retro game ads on the TV?)
OTT = over the top = ads that aren't shown on cable ("linear") DSP = demand-side platform = real-time bidding on ad space on behalf of advertisers
What data is tracked? Don't think we can see what's plugged into the TV if it's not connected to the internet but besides that... all of it... If we have your TV we know where you live. We know what you're watching (hopefully our customers' ads!). We know all the devices that connect to your home network. We know where those devices go when you leave the house. We know you were driving down this stretch of road when you saw that ad on that billboard or on the side of that truck ("out-of-home" advertising). We know if you saw that ad and then bought something ("conversion" + "attribution"). We know what apps you have downloaded. Did you know Candy Crush is spying on you, too? Did you know Grindr sells your IP address? We likely know your age and your race and how much your home cost and where you went to college and how many kids you have ("segmentation"). Privacy laws have gotten in the way a little bit, but not much - it's less "we can't get this data anymore" and more "here's the hoop(s) we now have to jump through but we still get it".
I don't want to freak anyone out. In my time in adtech I never felt like anyone was using this data for anything besides "Please buy more coca-cola..." but you never know. Privacy _can_ exist it's just insanely hard because there's so much money hell-bent on tracking you down.
1 reply →
> you have zero privacy
Is this data linked to me personally in some way (e.g. though an account) or is it anonymous data?
They can definitely work out who you are from your IP address. (or get close enough that the advertisers don't care) Not too many people are putting a VPN on their router and using throwaway accounts for their smart TVs. This might be difficult anyhow if your log into major services such as Amazon, etc, who will know who you are.
I'm not saying this is impossible to avoid, but it ends up being a LOT of work when the alternative is just not connecting the TV to the internet and using a laptop / Apple TV / etc. instead.
Personally identifiable. Most smart TVs force a login to connect to the Internet or even use at all.
>Not to be alarmist butttt you have zero privacy.
Hence why I will never connect my TV to the internet
I understand the perils of a capitalist system but whyyy would you agree to build this
The perils of the capitalist system man. For what’s its worth, I left adtech many moons ago specifically because it is a horrifyingly depressing industry and very very not fun to talk about at parties.
10 replies →
It makes its creator the money they can spend buying the products they see in TV ads.
If someone is going to get paid to build it anyway, I might as well be the one getting paid for it.
25 replies →
Soooo.... Why did you build it for them? You didn't have to further enable it. Despise people who just drop this kind of thing without any hint of repentance or contrition.
Would love to know what are the best things we can do to prevent this sort of tracking in general. PiHole? Don't re-use emails? On a scale of 1 to fucked are we cooked?
I don't think they mean that kinda streamer - the idea is the roku tv can tell you're watching an ad even if it's on amazon prime, apple tv, youtube, twitch, wherever, and associate the ad watching with your roku account to potentially sell that data somehow?
That way they aren't cut out of the loop by you using a different service to watch something and still have a 'cut'.
It'd make sense if they're using streamer in a different sense than I'm used to. I see that's at the bottom of the definitions Google will produce.
Yeah I think they mean "user of a streaming service" here, which would more conventionally be user or watcher or so on.
The actual screenshot isn’t sent, some hash is generated from the screenshot and compared against a library of known screenshots of ads/shows/etc for similarity.
Not super tough to pull off. I was experimenting with FAISS a while back and indexed screenshots of the entire Seinfeld series. I was able take an input screenshot (or Seinfeld meme, etc) and pinpoint the specific episode and approx timestamp it was from.
> The actual screenshot isn’t sent, some hash is generated from the screenshot and compared against a library of known screenshots of ads/shows/etc for similarity.
this is most likely the case, although there's nothing stopping them from uploading the original 4K screengrab in cases where there's no match to something in their database which would allow them to manually ID the content and add a hash or just scrape it for whatever info they can add to your dossier.
I thought that similar inputs do not give similar hashes..but apparently that is cryptographic hashing. Locality-Sensitive Hashing methods (e.g. Perceptual hashing[1]) makes similar inputs have similar hashes.
[1] https://en.wikipedia.org/wiki/Perceptual_hashing
2 replies →
I assume these systems are calculating an on device perceptual hash. So not that much data needs get flown back to the mothership.
That's the thing about scaling; you offload the work to the "client" (the TV in this case) and make it do the work, it need not send back more than a simple identifier or string in an API call (of course they'll send more), so they get to use a little bit of your electricity and your TVs processing power to collect data on you and make money, with relatively little required from them, other than some infra to handle the requests, which they would have had anyway to collect the telemetry that makes them money.
Client side processing like this is legitimate and an excellent way to scale, it just hits a little different when it's being used for something that isn't serving you, the user.
source: backend developer
Confirming how many people actually seen the ad is worth big bucks. No one wants to pay for ads they cannot confirm and publisher can make up impressions - if you can catch publisher making up numbers you might get a huge discount or loads of money back.
Not necessarily, it can be done on-device, the screenshot hashed, and the results deduplicated and accumulated over time, then compressed and sent off in a neat package. It'd still be a huge amount of data when you add it all up, but not too different from the volume that e.g. web analytics produces.
Then server-side the hash is matched to a program or ad and the data accumulated and reduced even further before ending up in someone's analytics dashboard.
Are there video "thumbprints" like exists for audio (used by soundhound/etc) - IE a compressed set of features that can reliably be linked in unique content? I would expect that is possible and a lot faster lookup for 2 frames a second. If this is the case, the "your device is taking a snapshot every 30 seconds" sounds a lot worse (not defending it - it's still something I hope can be legislated away - something can be bad and still exaggerated by media)
There are perceptual hashing algorithms for images/video/audio (dsp and ML based) that could work for that.
Given that the TV is trying to match one digital frame against another digital frame, you could probably get decent results even with something super naive like downsampling to a very low resolution, quantizing the color palette, then looking for a pixel for pixel match.
All this could be done long before any sort of TV-specific image processing, so the only source of "noise" I can think of would be from the various encodings offered by the streaming service (e.g. different resolutions and bitrates). With the right choice of downsample resolution and color quantization I have to imagine you could get acceptable results.
1 reply →
I've been led to believe those video thumbprints exist, but I know the hash of the perceived audio is often all that is needed for a match of what is currently being presented (movie, commercial advert, music-as-music-not-background, ...).
This is why a lot of series uploaded to YouTube will be sped up, slowed down, or have their audio’s pitch changed; if the uploader doesn’t do this, it gets recognized by YouTube as infringing content.
You only need to grab a few pixels or regions of the screen to fingerprint it. They know what the stream is and can process it once centrally if needed.
Is this what these sort of companies are doing?
In a word yes. Here is a starting point.
https://arxiv.org/abs/2409.06203
Attribution is very painful and advertisers will pay lots of money to close that loop.
Is it? I don’t think you need particularly high fidelity to fingerprint ads/programs.
it's hashed on the tv then they compare hashes in aggregate
[dead]