Open Source Summit North America 2018

As more applications move to a DevOps model with CI/CD pipelines, the testing required for this development model to work inevitably generates lots of data. There are valuable insights hidden in this data that ML can help extract with minimal human intervention. Using open source tools like TensorFlow and Pandas we trained ML algorithms with real-life data from the OpenStack community's CI system. We built a Kubernetes application that sets up a prediction pipeline to automate the analysis of CI jobs in near real time. It uses the trained model to classify new inputs and predict insights like test results or hosting cloud provider. In this talk, we present our experience training different ML models with the large dataset from OpenStack's CI and how this can be leveraged for automated failure identification and analysis. We also discuss how these techniques can be used with any CI system