Factors Affecting Compute Requirements
Several factors influence the amount of inference compute needed for test-time training:
Task Difficulty: The complexity of the task or question being addressed affects the compute requirements. Easier tasks may require less additional compute, while more challenging problems might necessitate more intensive computation
Model Size: The base size of the language model impacts the overall compute needs. Smaller models adapted with TTT might require less total compute than much larger pre-trained models for certain tasks2