Big Data and Machine Learning on OpenStack Backed by Nova-LXD

Hadoop was built with bare metal in mind – get your commodity hardware, stick hadoop on it and let YARN do all the hard work managing resources. However, Big Data software deployments on other substrates such as AWS (ec2 and EMR), AZURE, GCE are gaining popularity. We look at the challenges and a relevant solution related to deploying big data software in an OpenStack cloud. Perhaps most interestingly, we discuss and demonstrate what it looks like to run a machine learning job with Nova-LXD in