Google AI Strategy
Google is a large company and it's difficult to pinpoint a single "AI strategy".
Google was one of the first to take deep learning seriously.
If we take scaling hypothesis seriously, who would be in the best position to emerge victorious?
For scaling modern machine learning models, we need data and compute. Google is in an excellent position in both regards.
Google, with its vision “to provide access to the world's information in one click,” has been hoarding information for decades.
Our company mission is to organize the world's information and make it universally accessible and useful.
It’s not very difficult to imagine Google incorporating a training loop to handle every single page they crawl. Not that it’s an easy task, but in principle, it should be possible. And Google has had multiple experiences with scaling such systems already. Whenever the system fetches a page, a model gets updated as well.
GPT-3 used hundreds of GBs worth text data. Imagine training a large language model with Google scale amount of text data. No one can compete with Google in terms of the sheer number of documents they can get their hands on.
Google introduced TPUs to the public in 2016. It seems that they realized the potential for creating dedicated hardware for neural networks will have a big payoff in the end.
Indeed, another component of training a large neural network is the compute. Equipped with TPUs, Google is in a better position than any other large tech companies except for Nvidia.
In terms of research capabilities, Google is second to none. Google came up with the transformer architecture and created BERT. All of the language models that came after basically adopted transformers.