We share the harrowing tale of a team of operators, devops engineers, and developers as they pursue their goal of deploying a scalable, high-performance chemistry search engine based on the Lambda architecture in Openstack. This is not a story for the faint of heart, dear listener.
Tragedies in this talk include:
- Laggard networks! (VXLAN/VLAN trade-offs)
- Murderous kernels! (CPU pinning and pre-allocated hugepages)
- Recalcitrant storage! (Cinder backed with flash; Problems found in os-brick and multipathd)
This is the story of our team’s efforts to deploy a Lambda architecture search engine on Openstack, showcasing the trials and tribulations we encountered on our journey. We share both what worked for us and what didn’t. How taking advantage of all that Nova Compute offers made this deployment a success. We also highlight the importance of the Openstack operators working closely with developers to ensure maximize performance and stability. Openstack is excellent for all things, but best when both devs and operators tailor the environment to match the expected workloads.
