article thumbnail

Supporting Diverse ML Systems at Netflix

Netflix Tech

For ETL and other heavy lifting of data, we mainly rely on Apache Spark. In addition to Spark, we want to support last-mile data processing in Python, addressing use cases such as feature transformations, batch inference, and training. A key challenge in creating a knowledge graph is entity resolution.

System 90