Friday, August 31 • 11:00am - 11:40am
Scaling Big Data Interactive Workloads across Kubernetes Cluster - Luciano Resende, IBM

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
The Jupyter Notebook Stack has become the "de facto" platform used by data scientists to interactively work on big data problems. With the popularity of deep learning, there is also an increasing need for resources to make deep learning effective. In this session, we will discuss how we brought support for Kubernetes into Jupyter Enterprise Gateway and touch on some best practices on how to scale an interactive big data workloads across a Kubernets managed cluster.

In this session, we will discuss the limitations we have found when running interactive workloads on a Kubernetes environment and how we overcome some of these limitations by enabling distributed containers managed by Kubernetes using Jupyter Enterprise Gateway. We will also describe the roadblocks found during the implementation and how we overcome them. We also plan to discuss how these can be leveraged in different platforms to enhance scalability.

avatar for Luciano Resende

Luciano Resende

Data Science Platform Architect, IBM
Luciano Resende is a Data Science Platform Architect at IBM Spark Technology Center. He has been contributing to open source at The ASF for over 10 years, he is a member of ASF and is currently contributing to various big data related Apache projects around the Apache Spark ecosystem... Read More →

Friday August 31, 2018 11:00am - 11:40am PDT
Room 116/117