Sunday, October 5

Rails on EC2: A Tale of Perseverance (Part One?)

One of my pet projects has been getting a lot of "yeah but can it really scale?" naysaying of late so I decided to find out if my on-paper theory translated to a real-world infrastructure. Essentially I need to take my all-on-one-server Rails application and split it up into several segregated modules across several servers: the main web site, main database, slave databases, web services layer, caches, queues, workers, etc. I'm not mister moneybags, so I opted to use Amazon's Elastic Computing Cloud (EC2) service to virtualize all the servers.

Network Architecture
Uploaded with plasq's Skitch!
Up until Friday morning the only thing I knew about EC2 was what I'd read in blog posts. The first thing I had to do was RTFM. I can't stress enough how phenomenal is Amazon's documentation. Everything I needed to know to get started was spelled out step by step in the Getting Started Guide. It walks you through setting up your account, configuring your local environment for communicating with the cloud, then setting up your first virtual server. It all worked as advertised and I didn't hit any of those "oh crap I'm going to have to ask on IRC or the Google Groups WTF this means" speed bumps.

Once I was proficient with the basics of EC2, I wanted to get Rails on it, and I was pretty sure somebody out there had already done all the hard work for me. I was right, of course, and that somebody was the EC2 on Rails project. It was a bit confusing at first as there seems to be some stale information over on RubyForge but apparently the project moved to GitHub; at least that's where the most recent activity and updated documentation can be found.

EC2 on Rails is simply a Ruby gem - accompanied with an EC2 image pre-configured for hosting Rails applications - and a toolbox of Capistrano recipes for deploying your application to the cloud. Like Amazon's documentation, EC2 on Rails walks you through the process of installation and configuration pretty well. However, I did run into a plethora of thorns. The first and most painful of which was the Capistrano incompatibility - it insists on version 2.4.3. I tried at first to hack it to work with the latest, but after falling down far too many rabbit holes I gave up and conceded to downgrade my copy of Capistrano.

Once the Capistrano conflict was resolved I ran into problems with the eyecap gem and some of my app's recipes (too many definitions of the "symlink" task), then configuring the virtual server to talk to GitHub was an outside-of-the-box somewhat-convoluted experience (which I might detail in a follow-up post), and I wasn't able to initialize the database due to an issue with migrations and a bad model name conflicting with some native libraries, so I ended up scp'ing a dump from another host and importing it. But most, if not all, of those issues were related to the quirks of my Rails application and not necessarily issues with the EC2 on Rails package. After everything was ironed out, I could deploy to the cloud with a single command.

Although it spanned three calendar days I only clocked in six hours on the project; from EC2 noob to a live Rails application. Not too shabby for a weekend.

But, I've thus far only replicated what I already have: an all-on-one-server environment. My next step is to start up a half-dozen other virtual servers and start breaking off pieces of the application to see just how well it can scale up and out.

1 comment:

W2MDW said...

How much did the 6 hours of EC2 cost you in the end for the learning experience? I've been wanting to play with EC2 but haven't wanted to toss money down the drain without much result. But things look pretty painless and to be able to spin up an instance very quickly for whatever use could be pretty handy.