LinkedIn Introduces Pensieve: An Embedding Feature Platform Using Supervised Deep Learning Techniques | MarkTechPost

Advertisement

BEGIN ARTICLE PREVIEW:

Source: https://engineering.linkedin.com/blog/2020/pensieve

The LinkedIn Talent Solutions and Careers team aims to build an efficient marketplace for job-seeking members and employers. LinkedIn achieves this by matching members to the open job postings. There are a diverse set of products and recommendations modules built for LinkedIn Jobs and LinkedIn Recruiter. AI models are used in tandem with all these products to produce the final results. 

Significant features are created by leveraging supervised deep learning techniques so that models are trained to produce entity embeddings. ‘Representation learning’ or ‘Feature learning’ (through deep learning algorithms) has built a state-of-the-art performance on the LinkedIn platform. Its use in computation is accelerating because the network architectures have begun to use hundreds of millions of parameters. Thus the burden of entity embedding inference is pushed from the request time computation to nearline (or stream) pre-computation with no strict SLA.

Leveraging this technology, LinkedIn introduces Pensieve, an embedding feature platform that pre-computes and publishes the entity embeddings with supervised deep learning techniques. Across Talent Solutions and Careers, the embeddings are used by the ranking models for latency-sensitive applications.

The Pensieve platform can be divided into three pillars.

Offline Training Pipeline: The infrastructure streamlines …

END ARTICLE PREVIEW

READ MORE FROM SOURCE ARTICLE