Comment by bredren
15 hours ago
We saw yesterday that expert orchestration around small, publicly available models can produce results on the level of the unreleased model.
I take a contra view and instead see this as fuel on the fire for tinkering to squeeze advanced functionality out of more available things.
It has always been like this, the amateur improvising tooling and equipment to outdo companies with comparably infinite resources.
>> We saw yesterday that expert orchestration around small, publicly available models can produce results on the level of the unreleased model.
This is false. Yesterday's article did not actually show this, and there are many comments in the discussion from actual security people (like tptacek) pointing that out.
From what I can tell, this was not clearly settled.
Your example author, actually corrected themselves saying LLMs “possibly” could perform successfully: https://news.ycombinator.com/item?id=47732696
>> We already know this is not true, because small models found the same vulnerability.
>> No, they didn't. They distinguished it, when presented with it. Wildly different problem.
https://news.ycombinator.com/item?id=47733343