While Orion getting closer to GPT-4 at 20% of its training might sound impressive to some, it is important to note that the early stages of AI training typically deliver the most dramatic improvements, with subsequent phases yielding smaller gains.
So, the remaining 80% of training time isn’t likely to produce the same magnitude of advancement seen in previous generational leaps, sources said.