Disabling third-party services when they stop performing (in Rails)

Uploaded with plasq's Skitch!

One of my clients uses the hosted version of CompanyX (not their real name) to serve ads on their site. A couple weeks back, CompanyX applied some "upgrades" and things didn't go as planned, so for nearly a week their service was up and down like a yo-yo. That resulted in me getting calls along the lines of, "Hey our site is loading slow because of the CompanyX ads, please take them all off," followed a few hours later with another call, "Hey CompanyX seems to be OK now please turn their ads back on," and a little while later the cycle repeats itself. That got real old, real quick.

So, I decided to whip up a little automated solution. I needed two core components:

1. A way to programatically turn the ads on and off.

2. A way to periodically test the third-party service, and enable or disable the ads based on its response time.

For disabling and enabling the serving of ads, I created a new model called CompanyXStatus which is essentially a toggle switch, it's either on or off, and for auditing purposes I have it store the date and time whenever it's flipped. The database table looks like this:

create_table "companyx_statuses" do |t|
  t.column "enabled",    :boolean
  t.column "created_at", :datetime
end

And the app-facing API of the model looks like this:

class CompanyxStatus < ActiveRecord::Base
  
  class << self
    
    def enabled?
      latest.enabled
    end
    
    def disabled?
      !enabled?
    end
    
    def disable!
      CompanyxStatus.create!(:enabled => false) if enabled?
    end
    
    def enable!
      CompanyxStatus.create!(:enabled => true) if disabled?
    end

    private

    # I'm not using a named scope because this client is on an OLD version of Rails...
    def latest
      CompanyxStatus.find(:first, :order => 'created_at desc') || CompanyxStatus.new(:enabled => true)
    end

  end

end

So in the views when I'm building a page I just have to check if CompanyxStatus.enabled? before rendering an ad tags.

Now, for the actual toggling logic, I'm using the Benchmark module to call out to the service and measure the response time. If if exceeds the threshold (2.5 seconds in this case) the service is disabled, otherwise enabled. Here's the rest of the model:

  def test
    if 2.5 > latency
      CompanyxStatus.enable
    else
      CompanyxStatus.disable
    end
  end

  private

  def latency
    Benchmark::measure{ connect }.real
  end

  def connect
    socket = Socket.new( AF_INET, SOCK_STREAM, 0 )
    sockaddr = Socket.pack_sockaddr_in( 80, 'blah.companyx.org' )
    socket.connect( sockaddr )
    socket.write( "GET /blah.php HTTP/1.0\r\n\r\n" )
    results = socket.read
  end

Finally I need a way for the system to periodically make these checks and toggles so my client and I don't have to worry about babysitting the site. For this I wrote a simple rake task:

namespace :companyx do
  desc "Ping CompanyX and disable it if too slow"
  task :ping => :environment do
    CompanyxStatus.test
  end
end

And scheduled it as a cron job to run every minute:

* * * * * cd /home/client/apps/production/site/current && RAILS_ENV=production rake companyx:ping

That's it! Now I can get a good night's sleep knowing that the next time CompanyX has a burp in their service, my client's site is going to automatically shut them off until they get their act back together again.

Tracking AJAX calls with Google Analytics (and Rails)

With my recent pet project I used AJAX to provide a majority of the site functionality on the main page with it never having to reload itself or load another page. This resulted in a drastic drop of "page" views in my Google Analytics reports, because the only "page" being loaded on each visit was the sole main page; every click after that was an AJAX call to update only a portion of the already loaded page, hence not triggering any calls back to the Google Analytics tracking server. So I set out to see if I could remedy that, and I did.

I started out with a Google search for assistance. Surprisingly, the results were grim. The highest ranked result required a "donation" to read the answer, and I opted not to. Other results were vague or dated or focused on the same issue with Flash. I had to crack open the code and get my hands dirty.

When you set up a profile to track with Google Analytics, they give you a little snipped of JavaScript code to paste into your website pages. It looks something like this:

<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
</script>
<script type="text/javascript">
try {
var pageTracker = _gat._getTracker("UA-1377203-6");
pageTracker._trackPageview();
} catch(err) {}</script>

Seems cryptic enough, right? If you look closely you'll see it's actually two scripts, not one. The first script imports the Google Analytics tracking library:

<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
</script>

And the second script calls the library to track the page.

<script type="text/javascript">
try {
var pageTracker = _gat._getTracker("UA-1377203-6");
pageTracker._trackPageview();
} catch(err) {}</script>

In order to track a page load, including an AJAX call, you merely need to call the tracking library, but you need to import the library only once. That's where things get a little tricky.

For starters, I separated the two scripts into their own respective partials. I render both partials on the main page, so the library is loaded and then called to track the page load. However, from the AJAX calls, I only render the second script, the one that calls the library, since it will have already been loaded by the main page.

Here's the first curve ball. The first time I render and serve the main page, the AJAX-y portions of the page are rendered in-line. Since each of them in turn renders the call to the tracking library, the loading of the main page could erroneously track several hits. In order to prevent this, I wrapped the script with a global flag check and set, so no matter how many times it's rendered for a single page, it only injects the script once:

<% unless @already_tracked %>
<% @already_tracked = true %>
<script type="text/javascript">
try {
var pageTracker = _gat._getTracker("UA-1377203-6");
pageTracker._trackPageview();
} catch(err) {}</script>
<% end %>

The second curve ball is that you need to render the first partial - the one that loads the Google Analytics library - at the top of your page, rather than the bottom as Google recommends in its documentation. I put mine immediately inside the body tag. Why? Because the first partial, with an embedded tracker call, to render on the page is going to try to call the tracking library when the browser processes it, and if the tracking library isn't yet loaded, the call will silently fail (thanks to the try/catch block).

Now that the main page loads the tracking library, and the HTML snippets returned from the AJAX calls in turn call the already-loaded tracking library, each AJAX call is tracked as a page view on your Google Analytics report. Note that this solution is specific to AJAX calls that inject pre-rendered HTML into the page. If you need to track AJAX calls that deal with behind-the-scenes processing you should be able to simply make the same JavaScript calls you see in the second script; wrap them up in a helper function for convenience.