Slacker News Slacker News logo featuring a lazy sloth with a folded newspaper hat
  • top
  • new
  • show
  • ask
  • jobs
Library
← Back to context

Comment by killerstorm

7 hours ago

No, layer looping increases effective depth, but it still has to go through decode. So it's more like they increased number of layers from 100 to 200 without increasing number of parameters.

"Latent reasoning" is rather trivial - you can just replace unembed-embed step with a MLP. But labs don't do that largely because they want to read the output of unembed.

0 comments

killerstorm

Reply

No comments yet

Contribute on Hacker News ↗

Slacker News

Product

  • API Reference
  • Hacker News RSS
  • Source on GitHub

Community

  • Support Ukraine
  • Equal Justice Initiative
  • GiveWell Charities