Positional Encodings and Group Theory in Transformers
Grant Sanderson and Alok Puranik explore the theoretical foundations of positional encodings in transformer networks, applying group theory to analyze methods like RoPE and ALiBi.
Self-contained public lectures, keynote addresses, and research seminars.
Grant Sanderson and Alok Puranik explore the theoretical foundations of positional encodings in transformer networks, applying group theory to analyze methods like RoPE and ALiBi.