Continuous Delivery/Release – a Basic Howto with Examples

When we first went to build a setup with Continuous Delivery (CD) as the goal, we found plenty of excellent theoretical fodder but little in the way of specifics. As with all the hubbub about the cloud, we wondered how to actually get there let alone how best to get there. Here is a rundown of the basics. Continue reading

Chef Node Deregistration For Autoscaling Groups

Cloudreach in the UK has a great article on Chef node de-registration. With autoscaling groups bringing machines up and down all over the place, Chef needs a way to know that they’ve been terminated and remove them from the configs lest things become a cluttered mess. Luckily, AWS provides some great shortcuts. We implemented the solution, then bulletproofed it. Here’s what it takes. Continue reading

SeeDub: Universal Custom Metrics for AWS CloudWatch

When AWS supported an API call to push whatever metric we want, we got very excited. Once a metric is in CloudWatch, we can key a scaling policy to it, have it trigger an action, an alarm, basically perform any action we very well please. Finally, ultimate flexibility in what factors determine what actions, and we get pretty graphs to boot! For example, we could scale a processing group up and down like an accordion based on the number of entries in a DB table, or maybe the amount of free system memory. The hurdles proved disheartening, but we created a solution.

Continue reading

Chef and CloudFormation

The ephemeral nature that comes along with cloudy virtual machines means that we need to be able to go down and come up at any time without flinching. What good is hardware scriptability if the app itself doesn’t lend itself to automated management? For years, we’ve been using custom sets of homebrew scripts, checked-in alongside both the app code and any virtual hardware scripting. It was cumbersome, but it worked beautifully. Chef promised to make the cumbersome elegant. Did it? Continue reading

AWS Architecture for Startup Looking to Scale Quickly

We were asked to prepare a generic document for a LAMP-based startup, running one on machine in the corner of an office, looking to cope with massive scale, disaster recovery, and self-healing resilience, all within three months. Here’s what we came up with given only the above in the context of Amazon Web Services offerings. Most of their services are just shortcuts for doing the same thing yourself. Continue reading

GSLB Failover for AWS ELBs

We just launched the first phase of a low-latency ad app in three AWS regions, us-east-1, us-west-1, and eu-west-1. One DNS record distributes load across all three via GSLB. In case you don’t know, this just means that when a client asks the authoritative nameserver where it is, the nameserver asks where the client is. If they’re in California, they go to us-west-1. If they’re in europe, they go to eu-west-1. Asia, it depends. You can configure what goes where, even if it doesn’t make sense. Now that we’re load balancing everywhere, what about failover? Continue reading