Overall Recommendation
If speed and efficiency are your top priorities, GPT-2 (117M or 345M) is likely better suited. However, if you prioritize nuanced text generation and can manage some performance overhead, GPT-Neo 125M (quantized) might give you better quality without straining your APU too much.