Vancouver, BC, Canada
August 27 & 28 - Co-Located Events, Tutorials, Labs & Lightning Talks
August 29-31 - Conference
Click Here For Information & Registration
Back To Schedule
Friday, August 31 • 4:00pm - 4:40pm
Pinball - Open Source Workflow System from Pinterest - Jack Ye & Shuo Xiang, Pinterest

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
Pinterest is a personalized data driven product that serves hundreds millions users. To this end, engineers and analytics need to collect data, extract features, build signals, train machine learning models, do ETLs, generate a/b dashboard and compute business metrics. Job orchestration and scheduling system is a fundamental need: there needs a way for connecting each small dots into necklaces.

In this talk, Shuo and Jack will present how Pinterest solved this problem using their open source workflow management system called Pinball (https://github.com/pinterest/pinball). More specifically, they will share the pain points, why and how they built Pinball and make it a scalable/flexible ecosystem to cover use cases from dashboard generating and index building to complicated machine learning model training.

avatar for Shuo Xiang

Shuo Xiang

Data Engineer, Pinterest
Shuo Xiang is a software engineer in Big Data team at Pinterest, where he focuses on workflow system, machine learning platforms, analytics/experiment engines and data processing technologies. Before joining Pinterest, he received a Ph.D in machine learning from Arizona State University... Read More →

Jack Ye

Engineering Manager, Pinterest

Friday August 31, 2018 4:00pm - 4:40pm PDT
Room 116/117