Part 7/10:
To effectively gauge problem difficulty, Epoch AI is utilizing a tri-axis scale that accounts for prerequisite knowledge, creativity in problem-solving, and computational effort. However, this system can be subjective, and disparities among reviewers have prompted the search for a more uniform difficulty rating system.
Ultimately, Epoch AI seeks to predict not only how long problems will endure before AI can solve them but also which challenges will present the most resistance to AI capabilities. This foresight allows for a structured approach to monitoring AI progress in mathematical reasoning.