Comment by catlifeonmars
3 months ago
This looks like it is a really cool toy.
It does not strike me as particularly useful from a scientific research perspective. There does not appear to be much thought put into experimental design and really no clear objectives. Is the bar really this low for academic research these days?
Keep in mind anyone can publish on Arxiv and it's not at the top of HN on the merit of its research contributions.
it looks like a group consisted largely of ex-academics using aspects of the academic form but they stop short of framing it as a research paper as-such. they call it a technical report, where it's generally more okay to be like 'here's a thing that we did', along with detailed reporting on the thing, without necessarily having definite research questions. this one does seem to be pretty diffuse though. the sections on Specialization and Cultural Transmission were both interesting, but lacked precise experimental design details to the point where i wish they had just focused on one or the other.
one disappointment for me was the lack of focus on external metrics in the multi-agent case. their single-agent benchmark focusses on an external metric (time to block type), but all the multi-agent analyses seems to be internal measures (role specialization, meme spread) without looking at (AFAICT?) whether or not the collective multi-agent systems could achieve more than the single agents on some measure of economic productivity/complexity. this is clearly related to the specialization section but without consideration of the whether said emergent role division had economic consequences/antecedents it makes me wonder to what degree the whole thing is a pantomime.
wouldn't surprise me if in a few weeks/months we see this repo packaged up as a for-sale product for the games industry
The scientific method has utility, but it's not a pre-requisite for utility.
Some people prefer speed and the uncertainty that comes with it.