GPT3 existed 5 years ago, and the trajectory was set with the transformers paper. Everything from the transformer paper to GPT3 was pretty much speculated in the paper, it just took people spending the effort and compute to make it reality. The only real surprise was how fast openai producterized an LLM into a chat interface with chatgpt, before then we had finetuned GPT3 models doing specific tasks (translation, summarization, etc.)