Are Big Data systems ready for the cloud? Aren't they just pet applications?
Big Data systems are among the most demanding applications that can be run on a IaaS Cloud platform. They have special requirements for storage and network in particular and react sensitive on latency and bandwidth variations.
A possible solution would be to consume Big Data as a Service, but this lacks security and flexibility, making it necessary for tenants to setup and operate their big data processing systems on their own. Running it within the cloud as a virtualized workload seems like the way to go, but it poses real challenges to cloud orchestration systems, because of it's special needs. This can result in subpar performance, because the placement of resources may not be optimal according to storage characteristics.
This talk shows possible optimizations for OpenStack clouds and in particular how to optimize the orchestration of storage to fulfill the needed characteristics.
New ways of thinking how big data systems can be managed and operated.