Comment by randomImmigrant

12 hours ago

I think bearish on LLMs for automation, and bullish for LLM+human experts in specific fields, is about the right expectation for current architectures.

Apart from issues with task generalization, or perhaps related to it, is the fact that LLMs have real trouble with timekeeping, and cannot estimate the real world time it will take them to do things very well. This plus the memory issues make dreams of long horizon agents, that could plausibly handle changing specifications, quite implausible with current architectures.

In narrow domains with more deterministic outputs though, this is less of an issue, and we see multiple agents succeed much better.

The fusion of that capacity, with humans in the loop able to better direct such agents and act as their temporal tethers, is where I think the real action will be for a while at least.

7 comments

randomImmigrant

Reply
oscillonoscope  42 minutes ago

My belief is that LLMs will fundamentally change how we approach domain expertise. From what I've seen, SDEs tend to be over-specialized compared to what the company actually needs to implement due to the need to understand enough of the domain to pick a best path. If an LLM can see the domain enough so that someone in an adjacent field can be confident in their approach and quickly change course then you don't need as many niche SDEs

antupis  2 hours ago

I think automation is coming but it will be way more gnarly than frontier labs want public to believe. Value is just too big, when you can automate most of eg customer support it will create huge savings and same time customer satisfaction will get better.

handfuloflight  10 hours ago

> This plus the memory issues make dreams of long horizon agents, that could plausibly handle changing specifications, quite implausible with current architectures.

Any reason why that can't be solved through context management and keep-forward scaffolding?