Comment by k__
9 hours ago
Half-OT:
Is Jev a decoder (e.g., BERT) or is it some kind of encoder (e.g., GPT) that just happens to be trimmed down to only outputting a handful of tokens for the answers and their probability?
9 hours ago
Half-OT:
Is Jev a decoder (e.g., BERT) or is it some kind of encoder (e.g., GPT) that just happens to be trimmed down to only outputting a handful of tokens for the answers and their probability?
You've got encoder and decoder reversed. BERTs and other models that primarily convert text (or other input) into latent representations are encoders. Models that convert their internal representations back into outputs are the decoders (in the case of GPT et al., autoregressive decoders because they perform this decoding based on past tokens).
You got it reversed. Bert is encoder only and gpt is decoder only.