Vancouver, BC, Canada
August 27 & 28 - Co-Located Events, Tutorials, Labs & Lightning Talks
August 29-31 - Conference
Click Here For Information & Registration
Back To Schedule
Friday, August 31 • 11:00am - 11:40am
Accelerating I/O in Big Data – A Data Driven Approach and Case Studies - Yingqi (Lucy) Lu, Intel Corporation

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
The I/O infrastructure is key to Big Data ecosystem. New networking and storage hardware technologies are continuously being developed while software I/O stack remains relatively slow. In order to ensure applications are able to take full advantage of modern devices, deep understanding of I/O subsystems and optimizations to Java libraries and Big Data frameworks are required. In this presentation, a data driven approach is used to identify software I/O bottlenecks inside four Big Data frameworks - Apache Cassandra, HBase, Spark and HDFS. To fix the bottlenecks, new Java library APIs Intel contributes to OpenJDK are introduced. Corresponding software changes to the target Big Data frameworks are also discussed in the presentation as examples of how to use the new Java APIs. At the end of each case study, performance analysis is used to demonstrate throughput and latency improvements from the software optimizations.

avatar for Yingqi (Lucy) Lu

Yingqi (Lucy) Lu

Software Development Engineer, Intel Corporation
Yingqi (Lucy) Lu is a Senior Software Performance Engineer in the Software Solution Group. She has been at Intel for over 10 years working on performance optimizations of Virtualization, Power Efficiency, Webservers and Java Virtual Machine. She is currently focusing on enabling new... Read More →

Friday August 31, 2018 11:00am - 11:40am PDT
Room 119/120
  AI & Data Analytics