BEGIN ARTICLE PREVIEW:
Simon Knowles, chief technology officer of the AI computing startup Graphcore, told a supercomputing conference on Wednesday most of the world won’t have the dollars required to train neural network models, which are trending toward a trillion parameters, or weights, apiece. Instead, such huge models will be distilled down by users with less compute power, to make something purposeful and manageable.
Simon Knowles, chief technologist for Bristol, England-based AI computing startup Graphcore, on Wednesday told an audience of supercomputing professionals that the large mass of AI work in years to come will be done by people distilling large deep learning models down to something useable that is more task-specific. “There are three user scales for people who might want to buy and own, or rent, an AI computer,” said Knowles in the talk, a video of which is posted online. “There’ll be a very small number of people who will train enormous tera-scale models from scratch,” said Knowles, referring to deep learning models that have a trillion parameters, or weights. “From those universal models, other companies will just rent those and distill from them task-specific models to do valuable things,” said Knowles. A third category of users won’t train …
END ARTICLE PREVIEW