AWS Massively Rebooting Instances - What you need to know

by Thomas Orozco on Sep 24, 2014 12:53:00 AM

Earlier today, AWS announced that it would begin rebooting instances across its EC2 service over the coming days, likely in order to patch its underlying (Xen) hypervisors. This maintenance event is happening on very short notice; here’s what you need to know as an AWS (and Scalr) customer.

What You Need To Know

With earlier AWS maintenance events, it was possible to stop and restart an instance in order to have it migrate to a patched host (and avoid an uncontrolled reboot). This time, however, AWS is not guaranteeing that restarting an instance will have that effect.

Here’s why. AWS hosts can be split into two groups: hosts that have already been patched or don’t need the patch (let’s call these “good” hosts), and hosts that need the patch (let’s call these “bad hosts”).

If one of your instances is on a “bad host”, you’ll want to move it to a “good host” by stopping it and restarting it. Unfortunately, so does every other AWS customer. When the “good host” capacity runs out (and it may already have), instances you restart will land on “bad hosts” again, and will still have to go through a reboot.

Are Your Instances Affected?

You can view which (if any) of your instances are affected by logging in to the EC2 Console and opening the “Events” Tab. Be mindful that this may not be updated in real-time (though AWS is reportedly working on this).

Read More

Topics: Technical, AWS, Tips, Amazon

Deciphering VMware's OpenStack Play

by Thomas Orozco on Sep 23, 2014 8:00:00 AM

A couple weeks ago at VMworld, VMware announced that it was introducing “VMware OpenStack”, an integrated OpenStack release that is specifically designed (or configured?) to be deployed on a VMware virtualization layer (note: if you’re unfamiliar with what a “resource layer” is in this context, we encourage you to review this paper).

Is This New Software?

Not really. VMware OpenStack is essentially a repackaging of existing functionality and software. Indeed, OpenStack support for VMware virtualization is not new:

  • For compute virtualization, OpenStack Nova already supports VMware vSphere

  • For network virtualization, OpenStack Neutron already supports VMware NSX

  • For storage virtualization, OpenStack Cinder and Glance already support VMware VSAN and vSphere storage

So, in other words, OpenStack already had support for VMware virtualization software (and all of that is open-source), and therefore VMware OpenStack is not actually much more than an OpenStack distribution that is pre-configured to integrate with VMware virtualization software.

Then What is VMware’s Value Add?

The central value proposition of VMware OpenStack is to easily deploy OpenStack on an existing VMware “Software Defined Data Center” (SDDC); a VMware virtualized resource layer.

To that end, VMware provides an OpenStack installer (which ships as an OVF package). VMware argues it will let you trivially deploy OpenStack to an existing VMware infrastructure, configure it, and manage the controller services (to learn more, the SDDC2198 VMworld session includes a demo, and can be viewed online).

Note that, functionally, this is somewhat similar to what Mirantis provides with Mirantis Fuel.

Is It Still OpenStack When It’s VMware OpenStack?

Fundamentally, OpenStack’s functionality and core value proposition is to abstract away your virtualization infrastructure, and present standardized developer-friendly APIs.

Now, the APIs exposed by VMware OpenStack are the actual OpenStack APIs. So, yes, VMware OpenStack is in fact OpenStack. As such, it will be compatible with the ecosystem of tools that have been developed around OpenStack itself — including Cloud Management Platforms like Scalr.

Read More

Topics: OpenStack, Opinion, Cloud Platform, vCloud, Private Cloud, enterprise cloud, VMware

Announcing Scalr Cloud Management Platform 5.0

by Sebastian Stadil on Sep 16, 2014 3:00:00 AM

The hardworking team at Scalr is proud to announce the release of Scalr 5.0. This release furthers Scalr’s focus on the enterprise, with an emphasis on IT management and control, and new integration capabilities.

As with previous releases, Scalr 5.0 is licensed under the Apache 2.0 License. If you are a new user, you can download this new release now. If you’re already using Scalr, then head for the upgrade instructions on the Scalr Wiki.

Highlights of this release include:

Enhanced Enterprise Capabilities

Cost Analytics

First and foremost, Scalr 5.0 introduces Cost Analytics. Cost Analytics leverages the substantial amounts of infrastructure metadata generated by Scalr (such as “Who launched this instance?” or “What is this machine used for?”), and enables IT departments and their finance counterparts to use that metadata to better understand their costs across public and private clouds.

Cost Analytics is the result of close and intense collaboration with Scalr enterprise customers, and we’re delighted to be able to bring this new feature to our community of open-source users.

Policy Enforcement with Global Orchestration

When we introduced Scalr 4.5 in December 2013, two highlights of the release were Governance and Role-Based Access Control (RBAC). Governance enabled IT to control what cloud resources should be exposed to end-users (to e.g. prevent dev workloads from being deployed to a production network), and RBAC enabled IT to control permissions on a user basis (to e.g. restrict root access to instances to a specific group of users).

With Scalr 5.0, we’re adding a third layer of IT control: Global Orchestration. Using Global Orchestration, IT departments can centrally define IT policies that will be enforced at the instance level. Example use cases include deploying a standard firewall or authentication policy across all of the organization’s cloud resources, or enforcing the presence of auditing software.

Compliant Agent Upgrade Schedule with Custom Scalarizr Repositories

Scalr relies on an agent, Scalarizr, to remotely perform automation tasks on managed instances. Those tasks include deploying applications, enforcing policies using Global Orchestration, and more.

Up until Scalr 5.0, the Scalarizr agent was deployed through Scalr-managed repositories, whose upgrade schedule was sometimes incompatible with enterprise IT change management policies. Starting with Scalr 5.0, IT departments can manage their own Scalarizr repositories, and therefore control their organization’s agent upgrade schedule.

Facilitated Integration with Webhooks

As a Cloud Management Platform, Scalr is central to the provisioning and management of the cloud infrastructure at organizations that deploy it. It is therefore natural that these organizations need to integrate Scalr with other systems, such as change management databases, audit systems, and more.

With Scalr 5.0, we’re introducing a compelling and flexible solution: Webhooks.

Webhooks are outbound notifications that are delivered by Scalr to external systems whenever infrastructure events are triggered, such as when an instance is launched or decommissioned. Webhooks are dispatched as standard HTTP JSON requests, so that integration developers feel right at home when using them.

DevOps Enhancements

Of course, we didn’t forget about our audience of DevOps end-users with Scalr 5.0. Besides a ton bug fixes and enhancements (which were detailed on the Scalr Product Blog throughout the Scalr 5.0 development cycle), we’re introducing two major new features in this new release:

Read More

Topics: Announcements, Strategy, Features, Technical, Cost, Cloud Management, DevOps, Cost Analytics

Eucalyptus Acquired by HP — How Did That Happen?

by Sebastian Stadil on Sep 12, 2014 2:24:45 PM

Thursday, HP and Eucalyptus surprised everyone in cloud as they announced that HP was acquiring Eucalyptus Systems for an undisclosed amount, rumoured to be somewhere below the 100-million-dollar mark.

Now, while that amount is nothing to be sneezed at (comparatively, Eucalyptus had raised $55 million in venture capital), it is well below the typical return of a successful startup. And for good reason: Eucalyptus wasn’t exactly a runaway success. In fact, it was definitely one of the underdogs (along with Apache CloudStack) in the open-source private cloud platform market.

Now, what makes this acquisition surprising is the fact that the leader in this market happens to be OpenStack, to which HP has so far committed significant resources, and which it supports through its HP Helion OpenStack distribution. In fact, HP was one of the most significant contributors to OpenStack’s upcoming “Juno” release (1st by commit count; other metrics likewise show HP in a solid position).

So, why would HP acquire Eucalyptus Systems, a small competitor that had so far failed to make a dent? (though it wasn’t totally unsuccessful either)

Broadly speaking, there are three main reasons why acquisitions happen:

  1. Acquiring market share, or killing a competitor

  2. Acquiring a product

  3. Acquiring a team

Considering Eucalyptus’ position in the market, option 1 seems unlikely. What about the others

Did HP Acquire Eucalyptus For Its Product?

HP’s commitment to OpenStack makes it unlikely that it is acquiring Eucalyptus Systems to integrate the Eucalyptus private cloud platform into its own product portfolio.

Indeed, OpenStack is a mature and well-architected platform that not only significantly overlaps with Eucalyptus’ functionality (they are both private cloud platforms), but is also largely incompatible with it (Eucalyptus is a largely monolithic Java codebase, whereas OpenStack is a modular Python one).

So Was It For The Team?

The Eucalyptus team doesn’t seem like an obvious fit to work on HP’s product (if only because of the programming language differences), but would presumably bring significant experience in terms of AWS compatibility, which HP could use (AWS compatibility is Eucalyptus’ key differentiator in the private cloud market).

However, there are OpenStack startups that have built their business on enabling AWS compatibility for OpenStack (CloudScaling is one such example). If HP’s goal was to acquire AWS compatibility experience, one of those might have been a better buy than Eucalyptus Systems.

What About The Leadership?

With the acquisition, Eucalyptus Systems CEO Mårten Mickos goes on to head HP’s cloud business.

As a friend, I know from firsthand experience that Mårten is a world-class executive. He’s a good speaker, an inspiring leader, and a visionary. Plus, he does have a proven track record.

Considering that HP’s cloud business has been without a leader since the departure of Biri Singh, bringing Mårten on board could be just what the company needs to finally turn its cloud division around and become a solid contender in the space.

What’s Next?

At this point, HP’s leadership hasn’t revealed its plans, but it is doubtful that those plans even exist. Indeed, in the wake of the acquisition, HP VP of product Bill Hilf stated that:

"There’s going to be a lot of strategic discussion we have to have about [the acquisition] — how and what makes the most sense over time".

So, perhaps HP’s strategy is to cross fingers and hope that Mårten will know what to do with its cloud business. Will the acquisition be worth it for HP? Surely, the price tag seems steep for a single executive, even for one of Mårten’s caliber.

Finally what will the acquisition entail for the Eucalyptus product? If the product wasn’t the motivator for the acquisition — which seems likely —, then it’s both probable and unfortunate that it might eventually be discontinued. Time will tell.

Read More

Topics: OpenStack, Opinion, Eucalyptus, Cloud News, Private Cloud

A Taxonomy of the Cloud Management Ecosystem

by Sebastian Stadil on Aug 20, 2014 8:30:00 AM

Cloud management tools form a constantly evolving landscape.

Just a few weeks ago, Hashicorp introduced Terraform. In June, Google was introducing Kubernetes. In October of last year, Airbnb was open-sourcing SmartStack. Heat was introduced in OpenStack Havana in 2013. Even older tools like Chef, Salt, and Scalr (that’s us!) aren’t that old, and they are constantly changing as new releases get introduced.

And all in all, these tools make the same promise: manage your cloud (better). But does that mean they are one and the same? Not really. There definitely is some overlap, but these tools have strongly diverging focuses, and this post hopes to shed some light as to what those are.

To make this more concrete, we’ll look at an example. Let’s assume you are a DevOps engineer, and have your favorite N-tier web application running in your cloud of choice, backed by your favorite database. How did you get there?

Cluster Discovery

If your application is to be backed by a database, it definitely needs to know how to connect to it (e.g. what is the IP of your database server?!). There are tons of options here (DNS is one), but if you are dealing with a reasonably complex (or redundant) deployment, the right choice is usually cluster discovery software.

Cluster discovery systems provide a centralized repository of configuration information. In our example, application servers would connect to the cluster discovery system in order to retrieve the IP of the database servers they should use, but this is of course applicable to many other use cases.

Now, of course, you still need to somehow hardcode connection details to access your cluster discovery system! For this reason, cluster discovery systems are expected to be highly-available and redundant, so that discovering the cluster discovery system doesn’t become a problem. This is of course something you should bear in mind when choosing one!

Popular cluster discovery systems include: ZooKeeper, Etcd, Consul, and SmartStack. It’s important to note that there are other pieces of software that can play that role too, like Chef Server.

Read More

Welcome to the Scalr blog!

We build a Cloud Management tool that helps businesses efficiently design and manage infrastructure across multiple clouds.

Here, we post about our experience working with the Cloud, and building Scalr. On average, we do that twice a week.

Sometimes, we'll also cover Cloud-related news.

Subscribe to Email Updates