Part 7/10:
The successful reproduction of the deep seek algorithm involved using a base language model, supplemented with clearly defined prompts, ground truth rewards, and reinforcement learning applied directly to the countdown game. Throughout the training process, the models were initially generating nonsensical outputs but gradually learned to formulate solutions through mechanisms such as searching, revising, and self-verification.
The key findings suggested that the quality of the base model has a significant impact on the learning process. Ja experimented with various model sizes, discovering that models with at least 1.5 billion parameters showcased a notable increase in reasoning abilities as they learned to reassess their methods.