The rapidly growing size of data has broadened the role of data scientists. Instead of creating models for one-off analysis, it’s becoming common to use prototypes in real-world applications. Spotify Luigi helps to enable such scenarios, without having to learn a whole bunch of new technologies. We will define what production-readyness for data centric applications means and how Luigi’s concepts and modules help us to stride step by step closer to a production ready application.
Goals to achieve in this talk:
* Learn the nitty-gritty details of operating a data-centric application in production
* Build a robust ML-Pipeline from scratch with Spotify Luigi
* Discover and learn how to utilize some of Luigis modules to avoid boilerplate code
* Learn how easy it is to transition prototyped models to production ready code