Here at Scalr, we’re building an open source Cloud Management Platform (CMP). For those that aren’t familiar with what a CMP is, it’s an higher-level interface to your cloud resources than the APIs or UIs provided by your Cloud Platform (think OpenStack Horizon or the AWS Console, and even more advanced tools such as Heat and CloudFormation).
If you’d appreciate an in-depth look at what a CMP does, and the problems it solves, we suggest you take a look at our whitepaper “Architecting the Right Stack for Your Enterprise”. If you don’t have the time, here’s a quick rundown. A CMP usually allows you to:
Configure, provision, and visualize entire infrastructure clusters (e.g. a set of instances, their associated volumes, elastic IPs, and load balancers), instead of individual resources.
Manage the lifecycle of the infrastructure. This includes monitoring and restarting failed servers, autoscaling, and coordinating different services.
Track resources and enforce security, cost-control, and compliance policies.
Numerous enterprises and smaller companies have taken up the challenge of building their own CMP. As an example, HubSpot showcased theirs at the 2013 Portland OpenStack summit.
The process usually starts out with a DevOps team implementing a subset of the CMP’s features to scratch a particular itch they have (this is usually provisioning and autoscaling). Then, to ease management, they’ll put up a web-based GUI, or even an API.
This first iteration of the in-house CMP is generally really only a few open source components glued together with a bit of custom code. But as the company’s cloud usage increases, new requirements arise.
At that point, we’ve seen one of two things happen:
The original developers press on and deliver more custom code. If cloud is valued by the company as a core competency, they may succeed; this was notably the case for Netflix Asgard. However, when it isn’t, success is infrequent. More on why below.
The company adopts an external CMP. Sometimes it’s an open-source one like Scalr, and sometimes it’s not. This post however isn’t about choosing the right CMP for your business, so I won’t say more about that.
In this post, we wanted to give a few pointers — and maybe a few words of caution — to the adventurous types that set out to build their own CMP against the odds. Mind you, this isn’t going to be a list of programming tips. Even if you’re the business type, you can read on.
This isn’t a build vs. buy discussion. All of the usual discussion points for both sides of that argument apply here such as cost of customization, time to market, ongoing support and bug fixes.
There are indeed some additional technical concerns that those choosing to build their own CMP should be aware of. We’d like to cover those in this post. We’re here to help you on your journey. After all, we’ve been building a CMP for almost as long as there has been a cloud to manage.