Comment by teravor

there have been people who took existing LLM's and conducted an algoritmic search to find out which group of layers they can duplicate in order to improve performance, and it worked.