Comment by freehorse

Let’s say TTFT needed the most improvement. At some point, loading the model with enough context size may take tens of seconds in some macs.