The text search support in Pinot was directly contributed to Apache Pinot open source project. In this blog post, we will discuss our journey to add text search support in Pinot and how it unlocked keyword search functionality for LTI. Given that we wanted keyword search functionality for LTI somewhat similar to the one in LinkedIn Recruiter, one initial thought that came to mind was: “Should we leverage Galene for powering LTIs keyword search functionality?” We decided against that approach, since supporting keyword search through Pinot instead would provide the most seamless integration because Pinot already powers all the analytics for the LTI platform. This use case is different from LTI, which focuses on returning accurate counts for understanding macro trends across talent pools and companies. On the other hand, LinkedIn Recruiter is built on top of Galene (a Lucene-based index), because it has a heavy focus on search relevance/ranking so that a recruiter can find the best candidates to reach out to. As discussed in a previous blog post, Apache Pinot OLAP datastore is the foundation for the LTI platform, because it computes all the talent metrics displayed to the user. However, the two products are built on two entirely different tech stacks. The two products are integrated to enable recruiters to define their hiring strategy in LTI and then source candidates in Recruiter. Many LTI users are also users of LinkedIn Recruiter which already had support for keyword search functionality. Adding this functionality to LTI empowers users to construct searches using specific keywords and generate a wider variety of talent pool metrics to better understand the talent landscape. Keyword search unlocks the ability for a user to search for any text present on LinkedIn profiles. To solve this problem, we decided to add keyword search functionality to LTI. A skill is added to the taxonomy once LinkedIn detects sufficient growth in the number of members adding the skill to their profile as a result, there’s a delay between when a skill first appears on profiles and when it is added to the taxonomy, which was limiting users’ ability to search for an emerging skill or technology with LTI. Historically, a user’s search was constrained by the taxonomy values available. For example, the title “software engineer” is represented with title ID 9. Users can generate customized talent metrics using the values present in LinkedIn’s taxonomy, with each entity in the taxonomy represented with a unique identifier. Users of LTI have the flexibility to construct searches using the various facets of the LinkedIn Economic Graph (skills, titles, location, company, etc.). LinkedIn Talent Insights (LTI) is a platform that helps organizations understand the external labor market and their internal workforce, and enables the long term success of their employees.

Co-authors: Siddharth Teotia and Tim Santos Introduction

Linkedin talent insights