Building Systems that Automate and Scale Machine Learning and Machine Learning Engineering Workflows
Over the past couple of years, several professionals and teams have realized the need to design and build scalable, low cost, and maintainable machine learning workflows and pipelines. Understanding the concepts alone will not guarantee success especially when dealing with modern complex requirements involving Machine Learning and Data Engineering. In this talk, we will talk about how to use different tools and services to perform machine learning experiments ranging from fully abstracted to fully customized solutions. These include performing automated machine learning, automated hyperparameter optimization, and automated ML bias detection when dealing with intermediate requirements and objectives. We will also show how these are done with different ML libraries and frameworks such as scikit-learn, PyTorch, TensorFlow, Keras, MXNet, and more. In addition to these, I will also share some of the risks and common mistakes Machine Learning Engineers must avoid to help bridge the gap between reality and expectations.