A month ago, Jeff Dean, lead of Google AI, announced at TED that Google is developing a "general purpose intelligence system". To give a bit of context, last January, Google already published a paper on a 1.6-trillion parameter model based on the architecture of switch transformers, which improves by on order of magnitude upon GPT-3. Yet, I've heard that Google usually only publishes a weaker version of the algorithms they actually develop and deploy at scale.

From a technical viewpoint, and given my understanding of switch transformers and of the challenges of scaling neural networks, I'm guessing that Pathways' model will now be decentralized on multiple machines (as it does not fit on a single machine), and its computation (both in forward pass and backprop) must optimize its "pathway" by only leveraging some of the machines Pathways is deployed on. Google seems to want to leverage this to build models with 100-trillion+ parameters (note that models have been roughly x10 per year since Bert).

As a comparison, the human brain is estimated to have hundreds of trillions of synapses connecting its 100 billion neurons. In terms of numbers, Google's algorithms may soon match the human brain. However, as opposed to a human brain, these algorithms crunch billions of data points per second. Moreover, these data are extremely informative, as they track the daily habits, behaviors and beliefs of billions of humans on earth, through their activities on Google Search, YouTube, Google Mail, Google Drive, Google Maps or even Google Smart Keyboard. Perhaps more importantly still, Google's algorithms leverage this "intelligence" to have a massive-scale impact, by choosing which information will be shown to which human on earth, especially through the YouTube recommendation algorithm and Google Adsense.

I'm curious to know how this news updates your beliefs on the following questions:

Elicit Prediction (forecast.elicit.org/binary/questions/ICy5uv219)

Elicit Prediction (forecast.elicit.org/binary/questions/XpuhHCKuR)

Elicit Prediction (forecast.elicit.org/binary/questions/mRtRf22m-)

EDIT: I see that many of you doubt that a recommendation algorithm can be an AGI. Does this mean that you reject the orthogonality thesis?

Elicit Prediction (forecast.elicit.org/binary/questions/xCBK7XM8X)

(if you believe tha...

Posts

Wikitag Contributions

Comments