Ask HN: Why does the US Visa application website do a port-scan of my network?
4 days ago
I have recently installed this extension on FF: https://addons.mozilla.org/en-US/firefox/addon/port-authorit... and yesterday I visited this website: https://ceac.state.gov/genniv/ and I got a notification that the website tried to do a port-scan of my private network.
Is this a common thing? I have just recently installed the extension, so I am not sure if there are a lot of other websites who do it.
Since looking into it, I noticed that uBlock Origin already has the default list "Block Outsider Intrusion into LAN" but it wasn't enabled.
Visa application is riddled with scams. From the simple website that charges you twice the price to websites that will tell you that you were rejected and then fake your documents to get in with your name. So they're probably trying to see that you're not one of those web servers, a proxy for them or detect some known C2 channels.
That would be quite clever for an incredibly horrible website. The other day my SO, who is a Turkish citizen, was filling up her visa application and after half an hour of meticulous form filling the system just kick her out. I think the session times out or something. If you haven't created an account or you haven't write down the current application ID everything is lost. In the process she was also directed to a non-.gov website for something during the process, I thought she was getting scammed but no.
It actually makes sense to have a paid service that makes this abomination less painful. Though they work with VFS Global for collecting the applications and relevant documents, the VFS Global itself is an abomination and doesn't help with the handling of the form filling anyway.
Recently EU streamlined the Schengen visa application process for Turkish citizens as those "visa agencies" that are the official agencies and the only way to apply for a visa for many countries don't actually help with anything and are scamming people by selling the "good hours" for the visa appointment on the black market. An agency was dropped for this and the scams by agencies were listed among the reasons to streamline the application process.
Both with US and EU people are losing scholarships etc. due to outrageous wait times that are sometimes are years ahead or there's an issue with the systems handling the applications.
I guess there must be an opportunity there to fix all this together with smaller stuff like handling transliteration and character encodings, I wonder if some of those scam site are not scams and actually help with it. An AI agent can be useful here.
I had to deal with the DS-160 multiple times over the year. I don't think you give justice to how bad this website really is. I have started to notice that these "timeouts" are very random. At the worst times, the session "times out" immediately after login.
These random logouts happens more frequently during certain times of the day and seems to follow a semi-predictable pattern. It is almost certainly tied to system load in some way.
Also, the site's HTML and JavaScript are bloated beyond hope for what should be a fairly simple set of web forms. And itnhas been thisnway since at least 2018 with exactly zero improvements.
5 replies →
You might be making the assumption that the US wants to make the process easier.
55 replies →
The VISA appointment scheduling site rate limits to a ridiculous degree these days. As in refresh your page within 10seconds and get a 429 error.
That's probably because of the fact that the appointments are near impossible to get, they only allow booking a few months out and it's always completely booked. So everyone was refreshing (or if clever botting) to get an appointment slot.
As I wrote elsewhere; they subcontract the bot protection to F5, an external company that I see for some reason a lot on old/horrible banking websites.
1 reply →
Hey, this is actually something I have a keen interest in as I'm fighting my government (as an MP) to drop those scammers where possible. Do you have any media links to send me about them selling the "good hours" on the black market?
Even if the US has a horrible visa system – as I can attest, despite only having to do it every 5 years – the EU countries could benefit from attracting talent by being more welcoming. So that is part of my mission as an MP and tech-entrepreneur. Any help and pointers is welcome.
1 reply →
Whenever I'm filling a long form on an official website, I feel like I'm racing against an invisible clock because of this session time out thing that happened to me countless times.
I had this problem too last year. I found, at the time, it was the website was poorly managing the session in some browsers causing the timeout countdown to not be reset on activity. I had to find a windows computer and use microsoft edge I think (maybe it was chrome). But no browser on my mac would not have that issue.
> In the process she was also directed to a non-.gov website for something during the process, I thought she was getting scammed but no.
No clue if this specific instance if scam but such scams have indeed been done before
https://www.bbc.com/news/articles/cdr56vl410go
> According to Ablakwa, a locally recruited staff member and "collaborators" were allegedly involved in a "fraudulent" scheme whereby they extracted money from visa and passport applicants.
> It is alleged that the scheme consisted of creating an unauthorised link on the embassy's website to redirect visa and passport applicants to a private firm where they were "charged extra for multiple services" without the knowledge of the foreign ministry.
> Ablakwa added that the staff member "kept the entire proceeds" in their private account, and that the scheme had been going on for five years.
> Applicants seeking visas were charged unapproved fees ranging from almost $30 (£22) to $60 by the private firm.
The hard truth of it all is that both the US and (partially) the EU don’t want to make this easier because seeing as wanting “outside” people is now a political liability. You may want to adjust your expectations around that.
12 replies →
>> the system just kick her out
The "waterfall model" is a toxic way of thinking that pervades corporate management. Simplistic minds can't fathom any states other than "done" or "not done". Corporations are determined to crush the human soul. That is why it's not a progressive series of forms, saving your progress all along.
1 reply →
Another data point - 5he Indian visa system is similar. The official website ending in .gov.in, which is hard to find, offers a visa for $10 and minimal hassle. The scam websites, with better SEO sell the same shit for $80. They’re just proxying your application to the real website and pocketing the difference.
It would be good if the Indian government could block the scammers but I guess it’s a lower priority for the moment.
Not sure if this is the case for India, but I've experienced similar situations for other countries, but the 'scam websites' actually provided a real service - if you needed some ultra-urgent processing (like you only realized you needed a visa to this country before boarding a flight, once you were already at the airport check-in...) they were able to provide 30 minute approval, whereas the official site's accelerated processing was 24 hours.
So obviously the only way they could to this is with government contacts meaning the government themselves could already do it, but a lot of immigration stuff everywhere is full of people taking kickbacks.
1 reply →
The scam websites are probably owned by someone who works in the Indian govt.
4 replies →
I found the real website, but the application never went through, always some issue. My boss told me which service to use and everything just worked. (I could expense that service so cost didn't bother me)
1 reply →
[dead]
I'm not too familiar with network side stuff. What would a port scan be able to detect that would indicate that you're a scammer?
Just a guess, but maybe a typical bot has a webserver, ssh server, some other servers running on the same machine, whereas a typical Visa applicant doesn't.
1 reply →
Huh, how do you imagine that would work? This "scan" is happening inside client-side javascript, delivering the file through a proxy wouldn't "detect" anything about the proxy.
I imagine it may not be a proxy in the true sense, but a headless browser that's "proxying" the application process rather than the network traffic itself.
Proxy is being used in the traditional sense here. It’s common for a business (scam or legit) to handle visa applications on behalf of customers.
This is a very clever answer.
If the proxy scams are just a little clever, they'll run the proxy on an another IP.
it's riddled with scams, and thinking any of this will detect any of the things you mention is very foolish, native and show a total lack of understanding of the scams. of you think using a proxy is essential for visa scam, i would even know where to begin to correct you.
it's one hundred per cent clueless privacy invasion. they are probably also opening ports via other means and using that for side channel ID like Facebook does.
just like any other documentation scam, the only weak point is on the "last mile" that's why you will always have a human interviewer.
the visa process is abusive and unpractical because people will work around any hurdle and their kpi will never be affected no matter how crappy they manage to make to whole process. or how many doge kids implement useless privacy invasion tech just because.
It's coming from a F5 script, which is a company that sells anti-bot protection amid other things. (It's coming from obfuscated script at /TSPD, which is a F5 thing.)
https://www.f5.com/
TS seems to be short for TrafficShield (a product of some company F5 acquired in early 2000s) and PD seems to be Proactive Defense (?)
Isn't F5 the company that makes nginx?
That bought nginx, but yes!
I didn't know that! But apparently yes
"Since looking into it, I noticed that uBlock Origin already has the default list "Block Outsider Intrusion into LAN" but it wasn't enabled."
Never knew that this existed. Thank you!
Checking out the initial request on github for this feature I wonder why is this necessary? What access to the local network does the browser provide, or need to provide, and why isn't this something developers are more concerned about? I had a feeling this was possible as I see lots of mdns requests when I connect to certain things running sockets.
https://github.com/uBlockOrigin/uAssets/issues/4318
There are certainly use cases, but whether they’re warranted is a good question.
One popular router maker offers a ‘magic URL’ (domain name) that scans your network for the gateway management page, and redirects. It’s not necessary, but it certainly helps novice users. Having worked in IT support,
I’ve also purchased hardware devices that have a web management UI; which connects directly instead of proxying through a cloud.
Ultimately this is probably one thing that should be behind a permission request (like webcam access), but it’s not a feature without value.
It's being looked at.
https://bugzilla.mozilla.org/show_bug.cgi?id=1481298
2 replies →
I’m flabbergasted that this is even allowed. Who thought it was a good idea to allow any web page you visit to access your local network?
Internal apps on non-private IP addresses occasionally use this. There is a standard called Private Network Access[1] that requires these requests to have preflights like CORS requests. Only Chrome has implemented it so far.
[1]: https://wicg.github.io/private-network-access/
7 replies →
Massively improved my security posture with this. Thanks all!
Likewise I didn't know it existed, but it was enabled on my laptop and mobile browsers.
Is that available in lite version too? Now that the origin js being phased out
… or you can instead phase out those browsers who try to force blocker restrictions i.e. spyware on you (e.g. chrome and such), and use one of the browsers where you can use the full-featured (not "lite") uBlock Origin instead, e.g. Firefox.
5 replies →
It’s only being phased out on Chrome, by Google.
6 replies →
You can't change browser? Or is there something bigger happening?
2 replies →
Just checked, and it seems like it is. Not enabled by default for some reason.
It is not being phased out for Firefox.
I'm using uMatrix and it blocks by default all connections outside the requested site and parent domains. For example, if I request https://mail.yahoo.com, connections to yimg.com are blocked. I need to manually allow each CDN for each website, so this attack/profiling won't work.
Using uMatrix was very annoying at first, most websites are broken without their CDNs, but after a few months or so, the whitelist grew and it contains 90% of websites I visit.
On my system https://ceac.state.gov/genniv/ tries to connect to captcha.com, google-analytics, googletagmanager, 127.0.0.1 and "burp" (a local hostname that doesn't exist in my network). Interestigly, the browser console doesn't list connection attempts to localhost or burp. If I allow 127.0.0.1 and "tcpdump -i lo", I see connections to port 8888, which isn't open.
How does uMatrix handle the Facebook tracking pixel, or the replacement which is the Conversions API Gateway?
This is a container that FB gives you to host that lives under your domain (it can be your main domain) that slurps up user data and sends it to Facebook from the server side. You embed some JS in your website, and they hoover up the data.
It doesn't handle it. Anyway, there's no way to know what a website does on the server site. Even a completely static website could be sending the server logs somewhere.
There are options to not load JS, images, XMLHttpRequests, frames, cookies, for each site, but it doesn't list individual files.
2 replies →
> On my system https://ceac.state.gov/genniv/ tries to connect to captcha.com, google-analytics, googletagmanager, 127.0.0.1 and "burp" (a local hostname that doesn't exist in my network).
That will be this burp: https://portswigger.net/burp/documentation/desktop/tools/pro...
Sounds like they don't want you to analyze their site.
uMatrix is archived and I think uBlockOrigin is now advised to use(which incorporate uMatrix by enabling advanced settings)
For those who want to try blocking more stuff you can enable hard mode and bind relax blocking mode keyboard shortcut
I'd recommend also enabling filter lists(I advice yokoffing/filterlists and your region/language)
https://github.com/gorhill/uBlock/wiki/Blocking-mode:-hard-m...
But uBlockOrigin UI is so much worse...
Besides, uMatrix works fine. It's that kind of program that doesn't need any updates.
4 replies →
Until uBO has an even remotely usable interface for this use case people (including myself) will continue to use uMaxtrix or forks of it instead.
1 reply →
I reluctantly switched to only uBo because of uM bugs. But the UI/UX is just a huge step backwards to enable mobile usability.
uBO advanced settings still isn't as flexible as uMatrix was though, fwiw. (I did give in and switch in the end though.)
With uBO I can't block cookies by domain.
It seems to try to check if you are using the Burp Suite on their web application.
How does it manage to hide the requests to 127.0.0.1 from the network tab?
The requests are not made, because some operating systems prevent this.
If you're on OSX, the permission to "discover on the local network" prevents it from happening ( System Settings -> Privacy & Security -> Local Network -> yourbrowser )
Could also be 'network' permissions on firefox ( Go to Settings > Privacy & Security > Permissions ) which is on a per site level, but iirc that could be set site-wide at some point.
The other browsers likely have similar configs, but this is what I have found.
1 reply →
I have no ideea. Possibly that's a limitation of Chrome+Firefox developer tools (I get the feeling it's the same code)?
But I found what "burp" is: https://portswigger.net/burp/communitydownload
3 replies →
Whitelisting seems to be the way to go. With IPv6 and OS generated IPs (up to what the ISP domestic router allows) could be very efficient.
How and why do browsers allow this? Why wouldn't the browser ask for permission in the same way that it does for Microphone access?
It's insane to allow any random website to port scan my LAN. If this wasn't a "feature", I would have considered this a high severity vulnerability
Chrome doesn't allow it - local network services have to opt-in to being fetchable from public sites (https://github.com/WICG/private-network-access), although they're replacing it with a user-permission-based approach (https://github.com/WICG/local-network-access).
(There is some language online suggesting PNA has not actually shipped, but I experienced it myself in stable Chrome several years ago, so I am unsure of the current state).
Firefox doesn't implement either approach -- I assume this is indicative of their lack of development resources.
> Firefox doesn't implement either approach -- I assume this is indicative of their lack of development resources.
Since ublock had this as a feature for a long time, I'm sure they are aware of it. Unlike other non funded oss projects, Firefox can't and shouldn't shield themselves with this lack of development resource excuse. They have millions.
4 replies →
That extension has "Access your data for all websites" ... I really don't get how anyone can give that permission to anyone that isn't well known (a company with a lot on the line) or a person famous for their work (the uBO dev) who has stated he will never sell to anyone or do bad things.
"Hacks and Hops" doesn't even have a valid home page. The extension links to https://g666gle.me/ which does not exist. The domain name itself does not want to make me give access to all my data for all websites to them.
As nice as this extension seems, I would ever in a million years install it.
Unfortunately this level of incoherence is almost universal on HN and similar forums. You'd have to be completely out of your mind to install this extension, but people for some reason believe they can install privacy. They got whipped into fearing nebulous online actors so much that they'll download FSB rootkits dressed as VPNs. The minimal set of actions a rational person would take after realizing they've been tricked into installing this extension is setting their entire PC on fire and then running it over with their car, while moving all of their bank accounts to new accounts, in person, and changing all of their passwords using a brand new device.
Many sites do it .Included in many standard device fingerprinting / anti anonymity SAAS. Ebay facebook etc all do this ! But it looks this is first party to prevent the adblocking of them
1MB of obfuscated fingerprinting + portscan + Webgl . But oddity this one is trying to find burp suite specific route's.
Madness! How do I harden my network against that?
Chrome is already in the process of killing it https://developer.chrome.com/blog/local-network-access
4 replies →
Enable "Block Outsider Intrusion into LAN" filter list in uBlock Origin.
1 reply →
You should actually harden your browser or PC... to block any unwanted requests. Apparently some browser extensions can do that.
It would be the job of the operating system to give or take away the ability of your browser to access your local network. But you can run your browser in a container/vm and disable localhost. (And use a separate browser for localhost only if you need it.)
my bank did this on the site they sent me to in order to activate my new card.
The "port scan" just seems to be a local connection to 127.0.0.1:8888. I don't know what purpose it serves on this page, but our government websites often use this technique to communicate with native software for digitally signing documents.
Are you seeing connection attempts to other IPs?
Might also be card readers, debug servers, etc.
Could also be incompetence :D until I fixed it, deploying from my local machine rather than CD resulted in one of the baked in URLs being localhost rather than the public host on the project I'm working on now. Their local development server might just be at port 8888. Wouldn't surprise me.
I looked at the website again and noticed that the request paths looked odd, one of them being `/400_random_url_with_numbers_403`. I googled that and it looks like it's part of a client-side bot detection script that's testing something, the explanation isn't very informative.
https://my.f5.com/manage/s/article/K000138794
> These requests are caused by the bot profile to test the different browser capabilities.
> 'http://127.0.0.1:xxxx' request is a call to the localhost/client machine, which is normal when trying to protect assets like end-server using ant-bot defense. It does not have any impact regarding application page load.
This is most likely an attempt to connect to a webserver on your own device to collect data and/or do tracking.
Remember back in June when Facebook/meta got caught tracking users trough a webserver on Android phone thought Messenger and Instagram? Same thing.
See: https://news.ycombinator.com/item?id=44175940
Why do you say that’s most likely?
This is a common pattern for connecting to smart cards / hardware security devices. Probably a service or hardware that’s run on official CBP machines that should be disabled for prod, but forgot.
1 reply →
How are you so sure?
Just a little side note - in this context, it makes sense if the website tries to connect to a local port because you might be running a card reader(ie. terminal). This is how it works with some(all?) EU countries that have a chip in their ID cards, or even vehicle registration cards, which you can use to access sensitive information or perform certain administrative tasks on government websites.
Although, from personal experience, it used to require java and it worked only on internet explorer and since it has been retired and replaced with chromium, i am not sure what is the way to make it work nowadays, as i have not been able to figure out to use it when i needed the last time.
It requires installing a local service that bridges between the browser and the smartcard driver (what Java applets did in earlier years). The web app then communicates with the service via requests on localhost. The card-specific driver and bridge service are often bundled together for installation.
I've had it before where it asked me to use an iPhone/Android app which can read the passport's NFC chip. I guess that's the modern replacement for IE/Java.
Embarrassed to say that I wasn't aware of this practice. Are there malicious uses for this beyond fingerprinting?
Yes. Facebook was using this trick on Android. Meta's android apps would host a server on localhost, and their sites would communicate with this local server to pass tracking information that would otherwise be blocked by all browser protection methods on Android. I guess it is still fingerprinting, but at the most extreme end.
https://news.ycombinator.com/item?id=44169115
Mostly it's great for tracking although I'm sure it could also be used to exfiltrate data (e.g. if the user is running something sensitive on localhost).
https://www.digitalsamba.com/blog/metas-localhost-spyware-ho...
Routers with vulnerable URLs. You can search for: "router" "authentication bypass".
Isn't CORS supposed to prevent this?
1 reply →
https://files.catbox.moe/g1bejn.png
When I visit the site from Safari on macOS I see this in the console. Are there any particular services that use port 8888 for the website to do this?
https://my.f5.com/manage/s/article/K000138794
It seems to be part of some "bot defense" product by these F5 people, to "test the different browser capabilities". I doubt it's intended to hit a real endpoint on any system.
Capturing forensic artifacts of the local network allows a building a bridge strategy for identifying fraudulent networks without requiring knowledge of the path taken from destination to recipient. Other local devices do this and send the network map during a phone home, allowing comparison to a source of truth that is tied almost directly to the person, or group of people.
There is also a lot of fingerprintable material within such a port scan from clock skew, TCP ISN, and a few other areas.
You can sieve this quite easily with this available, thanks to Roku's, Phone's, and other things doing this while just sitting locally in a shared collision domain (a digital soldier quartered in every home).
The metadata node graph of devices locally acts as a unique fingerprint once in RFC1918 space, technically not unique but close enough.
> Blocks malicious websites from port-scanning your computer/network
How does that work? A browser extension can't influence how your router and other machines in your network react to incoming requests.
As far as I understand it, it is supposed to be a scan done by the browser on the user's computer, not an external scan, which a browser extension wouldn't be able to detect.
Hopefully should soon be a thing of the past with https://developer.chrome.com/blog/local-network-access
I see. So the website would try to access private IP adresses (RFC 1918) by having elements like <iframe src="http://10.0.0.1"> in the web site and then the web site would check if the iframe was loaded successfully?
1 reply →
Judging just from the screenshots, it seems it blocks websites from accessing 127.0.0.1 get requests. Not a port scan to the outside, more of what do you have running on the local machine inside your network.
but it can hook javascript methods before that scan can happen.
Very interesting. Having looked at NoScript it seems like you can disable LAN as a default value under the allow tab.
Looking further
* uBlock Origin and Lite have it as an option under Filter List > Privacy > Block Outsider Intrusion into LAN
* Brave prevents it, tested with Aggressively block Trackers and Ads.
Why do you need a heavyweight extension to block sites from scanning your local network? Ridiculous.
Also I wonder if this protection is available only with old extension manifest version or new network request hooks API also supports it.
Data my friend, data. Ports scanning? Well, tell us about the hosts and the port numbers. Add some logs if you got.
If you did not go into the details, chances are that when you will, this will turned out to be a false positive case.
If you did, where are the evidence?
Perhaps to avoid people using misconfigured open proxies https://en.wikipedia.org/wiki/Open_proxy
Like a less sophisticated Tor/VPN that is easily detected by port scans
If would be interesting to see what happens on OpenBSD. With pledge(2) and unveil(2) in Firefox, I wonder what it would see. I expect it would see nothing.
I will give it a try and see what happens and if I see anything I will add it here.
I saw nothing of note on OpenBSD. I added the plugin and it prompted me an attempt was made to scan the network, it said it blocked the scan
SO, I guess that is going to be used on all my firefox runs.
Pushing the burden of network permissions management outside the browser, to the OS? Heresy!
To be serious, this has introduced me to sandboxing on BSD via pledge [0] and comparisons against Linux seccomp [1] - thank you!
[0] https://news.ycombinator.com/item?id=44264021)
Have you double-checked whether the IP isn't shared among multiple website domains? That's quite a classic with IP based filtering with hosters like GCP...
They’d likely block you if they detected something like RDP open, cause that would likely indicate you’re hiding your real IP address.
Checking if you are sharing torrents, run a tor node, mine coins?
It's most likely smartcard authentication code.
My biggest grief with that site is that it's like something from the 90s.
Yeah it should have a fixed header and footer along with a pop-up consent drawer so you can only see 10% of the actual site content.
So much better.
Modern web design is a joke.
t-shirt worthy quote - "modern web design is a joke" ;)
>like something from the 90s
It looks useful and looks good, there's minimal unneeded whitespace and I'm glad it looks as it does. We'd be better off if the entire web switched to a style like this.
As something from the 90s myself, I find this rude.
It's also inaccurate, as this style of page (relating to layout and specific graphic style) didn't appear until 2006ish.
The 1990s web was actually good
Looking like something from the 90s would be a feature, not a bug.
In the 90s and early 00s, we did tons of user-testing and feedback collection. We threw all that research away to create UX's that are minimal and "sleek". Tons of unnecessary whitespace and the concept of "Discovery" just thrown into the dumpster. Skeuomorphism was one of the greatest features of 90s-00s software, ironically thrown away as computers got faster and were able to handle the graphics better.
I think you are confusing something from the 90 with something from the gov
These guys need to look at Gov.uk, this site is a total horror show.
I wish gov.uk was even a smidgen as "outdated" looking as that page.
Most likely some "antivirus" bs. Probably harmless. Fun fact. Most browsers allow by default GET access to web resources on localhost and LAN. Been used for exploits since last century.
For another example, studentaid.gov doesn’t work in private browsing.
I can one better (worse): A state-run website that my sister frequents for her job requires Internet Explorer. Seriously. I installed a Chrome extension that modifies her user-agent header to IE, and it works fine. Easy work-around, but totally lame.
I just tried opening it in a private window and the page loaded and rendered. What part doesn't work?
Isn't it sort of contradictory to try to use private browsing with a service that requires your identity?
Not necessarily.
I might create a login for a porn site so that I can have some favorite videos bookmarked and it can figure out the type of material I like. That doesn't mean I want my history saved locally.
Not contradictory at all. These days private browsing for most people just means (1) don't save the browsing history and (2) log me out of all websites temporarily.
1 reply →
[dead]
[flagged]
[flagged]
[flagged]
this is awesome
Be careful your security tool isn't producing false positives.
I remember years back when people would run these firewalls and we'd get complaints from home users about normal traffic.
Thinks like complaints our mail servers was scanning them on port 25 when they sent email.