1 is a Beautiful Number

1 Machine
1 Person
1 Day
1 Document

That's all it takes to deploy a large commercial web-based application I designed seven and a half years ago which today is still running and serving millions of pages a day for a dozen of clients so large and well-known I'm contractually prohibited from disclosing their names.

Every time I hire a new developer, their first day on the job is spent building the system from scratch on their workstation: 1 machine, 1 person, 1 day, 1 document. The most recent hire set a new record and had his system operational by lunchtime.

I attribute this accomplishment to mainly two mantras: "no fringe tools", and "the build document must stand alone".

No fringe tools means we don't use the latest and greatest one-offs or widgets just because we can. The old cliche of "the best tool for the job" breaks down when you have thirty jobs and thirty tools to master and maintain. If one tool does five jobs "good enough" then you've saved yourself a headache in staffing and training and documentation and licensing and on and on and on.

The build document is the bible in my organization. My colleagues know that I'm not one for formal documentation in most cases, but the build document is the exception. That sucker is kept up to date (the responsibility of each new hire that has to follow it) and it's kept simple. It doesn't gloss over any details and it doesn't defer to other documents. When it starts getting too long, too verbose, including too many diagrams and appendices, it's time to simplify.

So what inspired this burst of braggadocio?

By contrast, a friend's organization is currently deploying, or attempting to deploy, a web-based application with a nearly identical feature set which thus far requires nineteen separate physical servers for operation. Configuring this beast has required the consultation of three separate agencies (including the vendor). And, the grunt work of actually installing it, setting it up, and getting all the pieces to communicate and play well together has required the coordination of a half-dozen people from five different departments.

As I type up this rant, it's late into the evening, and the e-mails are still flying, and the system is not yet churning, and all I can think of in the back of my mind is, "1 is a beautiful number."

Heroku: Rails development in a web browser, with hosting.

Heroku is too cool for school. I just got my beta invite this morning and I'm having a blast playing with it. It's a Rails development interface (basically a webified text editor with some shell wrappers) that runs inside your browser with AJAXy goodness, letting you create and edit Rails applications in real time. It's rather impractical from a real-world point of view, lacking features like version control, and IDE niceties like code completion, and I haven't figured out how to use gems without "vendoring" them, but it's still insanely fun to dabble with and show off to your colleagues. The migration management is rather clever, and the drill-down logging is nice. They are still in beta, but perhaps by the time they go public it might be a decent little platform for very simple deployments like Grandma's Flickr and Twitter mash-up. But putting all practicality and business potential aside, it's an impressive feat of programming. Kudos to them.

Partitioning, Shards, or Federating with ActiveRecord?

Dear Lazyweb,

My Google-fu has proven to be weak sauce and I've not been able to find a nice clean solution to the following dilemma.

From Wikipedia's database partitioning definition...

"Horizontal partitioning involves putting different rows into different tables."

"Range partitioning selects a partition by determining if the partitioning key is inside a certain range."

It seems that acts_as_partitioned might be able to do this for a single table across multiple databases, but I need to partition data across multiple tables in a single database, and it gets worse...

For my particular needs, I want to get even a little bit crazier than the standard partitioning strategies. I want new records to be inserted into tables specific to the date on which they are created. For example, if I create a new Address record on November 12th, 2007, I want that record inserted into a table named ADDRESSES_20071112. OK, that's admittedly a bit screwy, but there's a big gotcha there: that table might not exist, which means I need the ActiveRecord to create tables on an as-needed basis!

Am I completely nuts here? Wait, don't answer that, but do answer me this: can it be done? Based on the amazing things I've seen done with Ruby thus far, I'm pretty confident this is possible, but my Ruby-fu hasn't yet reached black belt status. I'll try building it myself if I must, but I'm hoping to avoid reinventing any wheels.

Nested ActiveRecord Relationships and Validation Conundrum

Originally posted to another blog.

I ran into a bit of a pickle earlier this week with some models and their relationships.

The Situation

* A partner has an account.

* An account has an address.

* Accounts do not require an address; it's optional.

* Partners do require an address.

The Problem

So how do I enforce the presence of an address for the account assigned to the partner?

My First Attempt

My first thought was to break the encapsulation and create a direct partner-to-address relationship. Then I came up with the clever hack of keeping the partner-address in sync with the account-address during assignment, so when you set the address for the partner, it would automatically set the account to the same address. But it just didn't sit right with me; it wasn't the typical Ruby on Rails elegance I had come to expect. So I reached out to my partner in crime (Jack Danger) and asked for his opinion.

A More Elegant Idea

Jack suggested a more elegant simplification of just validating the presence of the address from the partner level like so:

validates_presence_of :address

def address
  account && account.address
end

Now that's the simplicity and elegance I've come to expect from a Ruby on Rails application.

A Possible Third

But Jack also took it a step further and suggested subclassing account and making address required, so we'd have two types of accounts, one with optional addresses and one with required addresses. That might be even less code and arguably more elegant, but my gut just tells me it's a little less of a hack than my original attempt and not quite as pure as his first suggestion.

Wrapping Up

I went with Jack's first suggestion and I'm satisfied with the results, but I'm curious to hear if anybody else has tackled the same, or similar, problem, and how they solved it.

Mail.app and Safari... sadly not up to snuff for my needs

Mail

When I saw the IMAP feature enabled in my GMail account last week, I thought I'd hit the jackpot. I set it up immediately and started using the built-in OS X mail client I'd heard so many raves about. I gave it a good solid week of faithful use but in the end I just had to go back to GMail's web interface. There were two features I just couldn't live without:

1. Labels. I'm the kind of guy that puts multiple labels on some messages. This message might be an order status from Amazon for my daughter's X-mas gift, so it gets the "expenses" label and the "daughter" label. You just can't do that in OS X Mail; it translates labels to folders and flattens it to a one-to-one relationship; you can only put the message into one folder.

2. Searching. I didn't realize, until after my switch, how often I search my massive archive of e-mail for absolutely ancient pieces of correspondence. Just this morning somebody asked me when I had purchased my first Mac. I typed the model name into GMail's search field and in less than a second up came the order confirmation e-mail.

Safari

Even though Mail.app didn't float my boat, I decided to continue the experiment this week and switch cold-turkey to Safari from Firefox. It's been two hours now and I'm already feeling the pain.

1. Tabs. Praise be to all things holy that Firefox eventually got tabs right. I want every pop-up link from every site and every embedded link in every application to open up in a new tab, never a new window. One window to rule them all! This simply isn't possible with Safari. Granted, there is a commercial plug-in that supposedly resolves this, but I'm not paying a dime to make a built-in application work as well as the free market leader.

2. Plug-ins. I try to keep my plugs-ins to a minimum. I hate bloat and waste of memory and CPU cycles. It's my OCD nature. But there's a few plug-ins that I just can't live without.

- Firebug simply can't be lived without when you need to debug Javascript and CSS debacles.

- Linky let's me select a group of links on a page and open them all in tabs. This is priceless for doing change-set code reviews in Trac.

- AdBlock Plus blocks ads. 'Nuff said.

- Flashblock blocks Flash applets, presenting them with a nice little "play" button so that if I really (for some insane reason) want to punch the monkey I have the option.

I don't think I'm going to make it the entire week on Safari. I'm certainly going to have to pull up Firefox for some tasks I simply can't do in Safari (re: Firebug and Linky) and it doesn't make much sense to switch between two browsers when one of them already satisfies all your needs. But I'm going to try...

Update #1: I do have one huge gripe with Firefox (on the Mac). When you open a new tab, the page up/down and arrow keys do not work until you click somewhere on the page. Safari doesn't have this annoyance.

Update #2: Game over! I just tried to load up Google Spreadsheets in Safari and it doesn't work and it's not supported. That's one too many strikes. Sorry Safari.

Update #3: I gotta admit that the eye candy in Safari (native OS X form widgets, buttons, fields, font smoothing, etc.) looks a lot better than in Firefox, but looks don't compensate for lack of functionality, usability, and productivity.

Worthy Technology and Business Podcasts

(Update: I finally got around to hyperlinking the podcasts.)

An old colleague just wrote me...

"I'm driving to work now and need listening material for the trip. What podcasts do you get/recommend?"

So I figured I'd respond here for posterity...

With a 2-hour daily commute, I've got a veritable butt-load of podcasts in my iTunes. Here's about half of my favorites (I'm omitting the video podcasts):

Technology:

- Drunk and Retired Podcast (two grizzled veterans rant about programming)

- FLOSS Weekly (really comes out quarterly, interviews with big OSS guys)

- Java Posse (if you're still into Java, good news and discussions with occasional interviews)

- Google Code Blog (mostly same guys as Java Posse, they all work for Google, discussing Google projects, tools, APIs, processes, etc.)

- Rails Envy (two local [Orlando] guys' weekly news blurbs about Rails; the same guys that do the Rails vs. Framework X spoof videos)

- Ruby on Rails (the PeepCode guy does occasional interviews with Rails pundits, usually at conferences)

Business:

- Entrepreneurial Thought Leaders (interviews with successful business people)

- Startup Studio (interviews with successful business people)

- Venture Voice (interviews with successful business people; long defunct but the back episodes are worth cherry picking)

- Web 2.0 Show (occasional interviews with the people/teams of popular/rising Web 2.0 sites)

- Vitamin (short interviews with successful people)

Humor:

[These come out pretty rarely, like once a month if you're lucky, but since you're just getting into it you'll have a gold mine of back-episodes to devour...]

- Downloadable Content (the Penny Arcade guys record some of their comic brainstorming sessions)

- Maniacal Rage (f.k.a. Garret Murray Podcast, just short comedic skits)

News, Reviews, Debates:

[When I've exhausted all of my favorites, these are decent time fillers...]

- This Week in Tech (Leo Laporte and a constantly rotating panel of industry journalists discuss current events in the tech world)

- MacBreak Weekly (Leo Laporte and a somewhat regular crew of Apple fanboys discuss current events in the Apple world)

- Security Now (in-depth discussions on network/software security issues. This used to be a really good podcast, but eventually it became one-half advertising and every other episode is just listener e-mail now. It's my bottom of the barrel podcast that I listen to after I've caught up on everything else)

- LUG Radio (a half-dozen thick-accented foul-mouthed guys discuss Linux and OSS and occasionally interview some interesting people)

The Circle of Life

I do not work at the same place today as I did eight years ago... or do I?


Eight years ago I had my own office with a beautiful view.

Today I have my own office with a beautiful view.


Eight years ago that office view included the corner of the building where the smokers hung out.

Today my office view includes the corner of the building where the smokers congregate.


Eight years ago I was the last remaining founder in the company.

Today I am the last remaining founder in the company (which is now a mere "division" of a bigger company due to acquisition).


Eight years ago I had a prestigious title (Chief Architect) that was more a symbol of tenure than any real authority.

Today I have a prestigious title (Vice President of Development) that is more a symbol of tenure than any real authority.


Eight years ago I was considered by my peers to be the most reliable, productive, and knowledgeable technical resource in the organization, due mainly to the fact that I had been there the longest and understood the system more thoroughly than anyone else.

Ditto today.


Eight years ago I spent all day every day in back to back meetings.

Guess what I do today? Same.


Eight years ago I was the guy everybody came to for answers... but oddly enough those answers were disregarded because they "didn't have time to do it right", and so I sat there helplessly watching the ship sink.

Today? You guessed it.


Eight years ago I decided that the salary and stock simply weren't worth the frustration of being a disconnected cog in a broken machine, so I resigned and helped found a new company.

Today... I'm seeing the circle of life come back around.

Stupid Dot Net Tricks

(In homage to my colleague's series of the same name... although in this case it might be me who's stupid, not Dot Net.)

Disclaimer: I am not a real Dot Net developer (at least not a good one).

Long story short: I was young and needed the money, so about six years ago I ported a very large Java web application to Visual Basic Dot Net for one special client, and to this day I still maintain it, only touching it two or three times a year.

As part of this port, I created a little logging utility. It's a teeny tiny class whose only job is to accept a string and write it to a file.

No big deal, right? That's what I thought.

My first attempt simply opened a file handle, kept it open, and wrote strings to the file as they came in. What's the problem with that? Windows locks the file and you can't open or read it from another application. So I couldn't load a web page then open the file with a text editor to read the logged messages... without killing the web server. Screw that.

My second pass opened and closed the file for every logged message. Gah! Yeah, I'm not proud of it. But you know what? It fricken' worked... for a few years. Note that this is only for development; this mechanism doesn't log in production, so don't soil yourselves in disgust... yet.

I don't know if it's Windows itself or the file system or faster processors or what is to blame, but now for some reason the operating system (or file system) can't seem to free the lock on the file in time for the next request to open it, resulting in an IO exception when you try to log a bunch of messages consecutively. Even though the code closes the file before attempting to re-open it for the next message, Windows says "nope, sorry, somebody is still using that file." Even more perplexing is that the exception doesn't occur when you open the file, it fails on the first write attempt. It smells like a classic race condition, and it stinks.

For some silly reason I thought I could solve this by synchronizing access to the logging utility... but that didn't work, and then it dawned on me that the application was essentially single threaded. I'm not loading multiple web pages simultaneously when I'm testing, so a race condition in the code was a red herring.

So here's my dilemma: I can't keep the file handle open because that prevents me from reading the log messages as they are being written, and I can't open and close the file in rapid succession as that causes Windows to asplode.

What's my solution? I hope you're sitting down...

A queue.

This is how it works:

1. You send a message to the utility to be logged.
   
2. It puts the message in a queue (at the back).
   
3. It opens the file (this always succeeds).
   
4. It pulls the first message off the queue.
   
5. It attempts to write the message to the file (this is where Windows barfs).
   
6. It the write fails, it puts the message back on the queue (at the front).
   
7. It the write succeeds, it keeps looping until the queue is empty.
   
8. It closes the file.
   

What's the method to this madness? If I pass a message to the logger and it fails to write it to the file, the message stays on the queue and the next time I write another message to the log, if it succeeds, it writes all of the failed-so-far messages to the log, clearing out the queue.

This whole block is synchronized on the queue just in case I decide to get crazy and load two web pages at the same time.

Since I have to put things back on to the front of the queue, I can't use a real Queue object, so I have to settle for an ArrayList. Why not a LinkedList? Oh yeah, did I forget to mention that I'm stuck in version 1.1!?

Here's the code (please don't burn me at the stake):

Private Shared ReadOnly queue As ArrayList = New ArrayList

Public Shared Sub WriteLine(ByVal msg As String)
    If LoggingEnabled Then
        SyncLock (queue)
            queue.Add(msg)
            Dim out As StreamWriter
            Try
                out = New StreamWriter(File.Open(LogFile, FileMode.Append))
                While (queue.Count > 0)
                    msg = queue.Item(0)
                    out.WriteLine(msg)
                    out.Flush()
                    queue.RemoveAt(0)
                End While
            Catch e As IOException
                ' put it back on the queue
                queue.Insert(0, msg)
            Finally
                If out Is Nothing Then
                    ' noop
                Else
                    out.Close()
                End If
            End Try
        End SyncLock
    End If
End Sub

Where are the design agencies that "get" usability?

My day job is looking to do some site redesign and we've been running the gamut of RFPs from so-called "design" agencies and it's got me on the brink of tears... and not tears of joy, we're talking tears of sheer agony. One after another I sit there and look at these proposals that are simply glorified brochures with inane and convoluted interfaces.

Our site is a service. Our customers need to be productive! The tools need to be intuitive and helpful. Clever and "innovative" are all fine and good but two things I really don't need are a) customers that leave because they can't figure it out, and b) customers that drive up support center costs because they can't figure it out.

Seriously, how hard is it to understand that a hover-scrolling list with no scroll-bar doesn't convey to the user that there's more items on the list than they can currently see? How hard is it to understand that when the entire set of content is embedded in a flash applet you can't use the browser's "find" feature to search the page, and you can't bookmark certain articles to access later or send to your friends, and it wont get indexed or cached in a search engine, and it's much harder to brand or use a translation proxy?

It pains me to read some of the design discussions on the 37signals blog, because those guys get it, and I want them and their talent working on my project. There's got to be some other agencies out there that "get" it. Where are they? Seriously. If you know of a good one, please leave me a comment. Thanks!