GJLさんがSDXLモデルの高精度FP8量子化の方法を公開されました。 HSWQ法という手法です。 内容については何回かにわたってGJLさんのNote記事に詳細に記載されていますので、興味がある方は是非。 ざっくりというと、オーダーメイド型のFP8変換をすることで ...

Generally, a more complex model would achieve better performance in the task, but it also leads to some redundancy in the model. Quantization is a technique that reduces this redundancy by reducing ...

Model quantization converts the high-precision floating-point weights in a neural network (32-bit or 16-bit) into compact lower-precision representations (8-bit, 4-bit, or even 2-bit integers). This ...

Reducing the precision of model weights can make deep neural networks run faster in less GPU memory, while preserving model accuracy. If ever there were a salient example of a counter-intuitive ...