Comment by gallerdude

Local models you can get just the pretrained versions of, no RLHF. IIRC both Llama and Gemma make them available.