Part 1/7:
Scaling Monos Semanticity: Insights from the May 2024 Paper
One of the key breakthroughs in the field of large language models came with the scaling monos semanticity paper published in May 2024. This work, led by a team that included Tom Henighan, explored the scaling laws for interoperability - how the scaling of models impacts their ability to learn meaningful, interpretable representations.
Scaling Sparse Autoencoders