Positional Encodings and Group Theory in Transformers
Grant Sanderson and Alok Puranik explore the theoretical foundations of positional encodings in transformer networks, applying group theory to analyze methods like RoPE and ALiBi.
Self-contained seminars, public lectures, and invited talks.
Grant Sanderson and Alok Puranik explore the theoretical foundations of positional encodings in transformer networks, applying group theory to analyze methods like RoPE and ALiBi.
Origin of the famous software engineering maxim 'make illegal states unrepresentable': practical algebraic data types, phantom types, and module abstraction from trading systems engineering.
This talk presents a vision for a live type-theoretic computational commons, detailing foundational work on incremental type checking, evaluation with typed holes, and an object-capability architecture for bounding AI agent behavior based on the principle of least authority.
Professor Nate Foster discusses applying programming language theory and formal methods to network engineering, exploring software-defined networks, hyperscaler hardware, and the role of BGP.
This session features a discussion with Jane Street's Aaron Bauer on the evolving landscape of developer education, particularly addressing how large language models impact teaching methodologies and curriculum design within a financial technology context.
This talk explores the practical implications of large language models on software development workflows, detailing the creation of agentic coding platforms and the evolving social contract of code review. Key topics include adversarial models, reward hacking, and the challenges of integrating AI agents into human-centric programming environments.