Part 3/7:
- Uploading Documents into the Context Window
Retraining a Model
Dave likens the retraining process to sending a student back to school. The model learns new material, fine-tuning its embedded knowledge, which becomes permanent. However, retraining requires significant resources—time, data, and computational power, often making it impractical for most users.
Retrieval Augmented Generation (RAG)
In contrast, RAG allows the model to dynamically access external resources rather than undergo extensive retraining. This method is compared to a student quickly consulting a library when posed a question. It is faster and enables the model to adapt and provide up-to-date information without needing permanent changes to its knowledge base.