Comment by ACCount37

What are you even talking about? If there was a "baseline" for how to decode activations into text we wouldn't be having this conversation.

The choice is between "this reconstruction sucks" and "no reconstruction", and we're only now beginning to learn how to make those reconstructions suck less.