GSQ-RCO: Better Non-Uniform GGUF Quantization

Name: GSQ-RCO: Better Non-Uniform GGUF Quantization
Uploaded: 2026-10-16T19:05:09.000Z
Duration: 57 min 13 s
Description: Exploring GSQ and RCO, two advanced methods for accurate low-bit LLM quantization, this presentation delves into Gumbel-Softmax sampling and Riemannian…

Michael Helcig; Rush Tabesh

ETH Zürich·Michael Helcig & Rush Tabesh

57:13

Notebook

Type below to lock the video timestamp, or press S to capture a slide.

GSQ-RCO: Better Non-Uniform GGUF Quantization

Michael Helcig Rush Tabesh ETH Zürich at GPU MODE

57:13·2026·research·Research Seminar

Exploring GSQ and RCO, two advanced methods for accurate low-bit LLM quantization, this presentation delves into Gumbel-Softmax sampling and Riemannian optimization for precision allocation.

Michael Helcig (student at ETH, collaborating with ISTA DASLab) and Rush Tabesh (PhD candidate at ISTA DASLab) will talk about GSQ and RCO, two complementary methods for accurate low-bit LLM quantization. GSQ uses Gumbel-Softmax sampling to improve quantization, while RCO uses Riemannian optimization to allocate precision across a model under a fixed size budget. The talk will cover the ideas behind both methods, why they work so well together, and practical lessons from building the GSQ-RCO models released on Hugging Face.

Systems & CS AI & ML Computer Architecture Language Models & NLP Large Language Models (LLMs)Neural Networks & Backpropagation Quantization Gumbel-Softmax

Lecturers

2

Speaker directory →

Michael Helcig

Student at ETH, PhD candidate at ISTA DASLab

Michael Helcig is a PhD candidate at ISTA DASLab and a student at ETH Zürich, focusing on advanced neural network applications and low-power high-performance VLSI design.

1 lecture in archiveView profile

Rush Tabesh

Student at ETH, PhD candidate at ISTA DASLab

Rush Tabesh is a PhD candidate at ISTA DASLab and a student at ETH Zürich, focusing on advanced neural network applications and parallel computing.

1 lecture in archiveView profile