While this was a strong theoretical result, its practical implications weren’t clear, because modern LLMs are so much more complex. “It’s not easy to extend our proof,” Peng said. So his team used a different approach to study the abilities of more complicated transformers: They turned to computational complexity theory, which studies problems in terms of the resources, such as time and memory, needed to solve them.
You are viewing a single comment's thread from: