{"id":44866516,"user":"whimsicalism","time":1754931283,"time_ago":"a day ago","type":"comment","content":"

reviewers?","comments":[{"id":44866559,"user":"layer8","time":1754931457,"time_ago":"a day ago","type":"comment","content":"

Auditors?","comments":[{"id":44866654,"user":"dathinab","time":1754931975,"time_ago":"a day ago","type":"comment","content":"

yes auditors from a security audit","comments":[{"id":44867062,"user":"whimsicalism","time":1754934014,"time_ago":"a day ago","type":"comment","content":"

you could barely convince your auditors that using github was okay? well, my opinion of security audits is reaffirmed","comments":[{"id":44867251,"user":"anileated","time":1754934896,"time_ago":"21 hours ago","type":"comment","content":"

Security audits are just theater. If they were not, you could not ever convince them that using a platform feeding unlicensed source (including apparently from private repositories) to their commercial LLM is ever a pass.","comments":[{"id":44869959,"user":"dathinab","time":1754949709,"time_ago":"17 hours ago","type":"comment","content":"

> Security audits are just theater.

It really depends on you auditor, audit approach and goals.

There are many audit companies which have a "under the hand" reputation of not properly looking and being easy to convince that you are secure, naturally at a above average audit cost (same but worse btw. for certificates showing compatibility with industry standards).

So if the audit was paid for by the company themself you can't trust it at all (which doesn't mean the company wanted to hide anything, this "bad" audit companies also tend finish the audit fast. So sometimes companies go for it, even if they don't have anything to hide).

Similar sometimes audit companies ask if they can audit you, this is for boosting their publicity using your name. This can easily turn into a "one hand washes the other" situation where they won't overlook massive issues, but still judge issues leniently.

Lastly there are some automated partial audit services which scan you public APIs/websites etc. Realistically they tend to be kinda dump, and might tell you they find a medium issue because (no joke) your REST API allows PUT and DELETE (1). Still I now take them a bit more serious after they pointed out, that there was a configuration error of a web gateway leading to some missing security headers.

(1: There is some history behind that, it's still dump for 90% of REST APIs)

Anyway, the situations so far are security audits which are at least 50% theater. BUT if a huge customers fully pays a audit company with a good/strict reputation then it often really isn't a security theater and can be quite a bad surprise if you company isn't prepared (because you have to fix so much). Like such reviews tend to not only be focused at your deployment or code but the whole software live cycle, including fun questions like "what measurements have you taken in case one of your developers tries to inject a supply chain attack" (which to be clear don't need to have perfect answers, just good enough, and most importantly clear and well documented).","comments":[],"comments_count":0,"level":4,"url":"item?id=44869959"},{"id":44868446,"user":"marcosdumay","time":1754940613,"time_ago":"20 hours ago","type":"comment","content":"

From a company with a long history of leaking private data... That AFAIK never even claimed to have fixed their side of the Solar Winds issue...","comments":[],"comments_count":0,"level":4,"url":"item?id=44868446"},{"id":44868508,"user":"whimsicalism","time":1754940937,"time_ago":"20 hours ago","type":"comment","content":"

from private repos? they explicitly say they do not

https://www.copilot.live/blog/does-github-copilot-use-your-c...","comments":[],"comments_count":0,"level":4,"url":"item?id=44868508"},{"id":44868204,"user":"shortrounddev2","time":1754939415,"time_ago":"20 hours ago","type":"comment","content":"

Absolute theater. They do nothing to validate that you are compliant with whatever ISO cert you're pursuing. They make you install a root cert on your macbook and they say that's good enough to ensure compliance. You just attest that you don't do stupid shit like committing directly to master or testing in production and they believe you","comments":[{"id":44870021,"user":"dathinab","time":1754950089,"time_ago":"17 hours ago","type":"comment","content":"

> compliant with whatever ISO cert you're pursuing

ISO cert compatibility audits are very different from a proper security audit.

And weather they do anything to check if depends on which you high, many of the slightly more expensive ones have the reputation to be "fast" and "overlook most issues".

But that doesn't apply to all security audits (but most audits for ISO compatibility, like really it's bad).

Anyway see my way to long answer about the on a sibling comment.","comments":[{"id":44874870,"user":"shortrounddev2","time":1754997753,"time_ago":"4 hours ago","type":"comment","content":"

I'm certain there are good firms out there which will actually give you a legit audit and make recommendations. But if the client is not actually interested in security, there will always be unscrupulous firms who will essentially sell you an ISO cert for no effort required. In my experience, most medium to small sized companies place little value in security","comments":[],"comments_count":0,"level":6,"url":"item?id=44874870"}],"comments_count":1,"level":5,"url":"item?id=44870021"},{"id":44869851,"user":"UK-AL","time":1754948871,"time_ago":"17 hours ago","type":"comment","content":"

People test in production in all the time via Canary releases.","comments":[],"comments_count":0,"level":5,"url":"item?id=44869851"}],"comments_count":3,"level":4,"url":"item?id=44868204"}],"comments_count":7,"level":3,"url":"item?id=44867251"},{"id":44869685,"user":"dathinab","time":1754947783,"time_ago":"18 hours ago","type":"comment","content":"

we are EU based and have besides other attorney customers.

Cloud Act and more then just one or two cases of the US engaging in industry espionage against their allies(1) makes it a high legal liability to use more or less any service from a US company even if it's in the EU and a EU daughter company

On GitHub we only have some code, which always anyway goes through additional testing and analysis before hitting production, this is why it's barely okay. No code from GitHub directly goes to production.

The only reason we ever where on GitHub is because we didn't always had sensitive customers and switching CI over is always a pain.

So I don't know if imply them being incompetent for allowing GitHub or for wanting to not allow it, but both point have very good reasons.

(1): And I mean cases before Trump, the US (as in top government, not people) was always a highly egoistic, egocentric ally which never hesitated to screw over their allays when it came to economical benefits. The main difference is that in the past the US cared (quite a bit) about upholding a image of "traditional" values like honesty, integrity and reliability. Especially when it would affect their trade routes.","comments":[],"comments_count":0,"level":3,"url":"item?id=44869685"}],"comments_count":9,"level":2,"url":"item?id=44867062"}],"comments_count":10,"level":1,"url":"item?id=44866654"}],"comments_count":11,"level":0,"url":"item?id=44866559"}],"comments_count":12,"url":"item?id=44866516"}

Comment by whimsicalism