The hardest AI problems are rarely AI problems alone. They're systems problems. In this 'In the Tech' CTO of Engineering Erran Berger, digs in to a great example of the engineering required to turn LLM potential into production reality.
The most interesting AI stories aren’t about the model, they’re about constraints. At LinkedIn, we had a hypotheses that LLMs can be applied to traditional ranking and recommendations problems by better understanding what members find relevant. But using them for a domain like ads is far from straightforward. Decisions need to happen at enormous scale and in a fraction of a second, which means the usual approach quickly becomes too slow and too expensive. That’s the engineering challenge Xīan Xíng Zhāng, Distinguished Software Engineer for Ads AI, and his team took on. They designed a way to use LLMs to better match what members are looking for with what advertisers are offering, while maintaining the efficiency and performance needed to operate at the scale of LinkedIn.