Comment by sgt101 5 months ago 31B won't run in 48GB for me - it needs 54. 2 comments sgt101 Reply yassa9 5 months ago what quantization did u try ? u can use Q4 quantization, im pretty sure that 48GB would be enough sgt101 4 months ago 8bits is fine.... I was talking full bore.
yassa9 5 months ago what quantization did u try ? u can use Q4 quantization, im pretty sure that 48GB would be enough sgt101 4 months ago 8bits is fine.... I was talking full bore.
what quantization did u try ? u can use Q4 quantization, im pretty sure that 48GB would be enough
8bits is fine.... I was talking full bore.