Ted's Has Many Helpers is on Ruby Forge

This is my first endeavor into a publicly available chunk of code (if you don't count my old JavaWorld articles -- eight years after the fact I still get e-mails about them), so it's probably a good thing that I'm starting with a script that's currently about a dozen lines long ;-) I haven't quite grokked all of the Ruby Forge goodness so bear with me as I try to figure out how to get it all set up and make the code easily available. If you have any experience publishing Rails plug-ins as gems or svn:externals (which is what I'm doing at the moment) please feel free to share your wealth of knowledge. Thanks, enjoy, and please be merciful.

http://rubyforge.org/projects/tedshasmanyhelp/

has_many :continuing sagas

A few quick updates this morning.

First, I fixed a little boo boo in the implementation from my last post related to my deferral of the "find_by_widget_id" call. I was calling it on the wrong model, but returning the correct value oddly enough to fool my unit tests. Once I started using the tool in real code I discovered the error.

Secondly, Josh Susser has posted a write-up (a lot more eloquently) of exactly what I'm dealing with on his blog. He concludes his explanation with the claim, "I do actually have some ideas about how to add some more magic ... I'll have to save that for next time." Here's to hoping he doesn't wait too long to share.

Finally, I had to deploy a self-referential cross-reference model last night, one that links to itself. Essentially, a "friends" mapping that maps Collectors to Collectors. This, of course, broke my little hack as the "guessed" foreign key no longer worked. However, I think I might be able to dig it up since it is passed in to the "belongs_to" declaration of the mapping model. If I figure it out, I will of course post the new code.

So without further ado, here's the latest.

module ActiveRecord
  module Associations
    class AssociationCollection < AssociationProxy
      
      def teds_guess_foreign_key( widget )
        widget.class.to_s.downcase + '_id'
      end
      
      def teds_find_relationship_wrapper( widget )
        conditions = "#{ teds_guess_foreign_key( widget ) } = #{ widget.id }"
        find( :first, :conditions => conditions )
      end
      
      def include?( widget )
        unless widget.instance_of?( @reflection.klass )
          widget = teds_find_relationship_wrapper( widget )
        end
        super( widget ) # ! index( widget ).nil?
      end
      
      alias_method :old_delete, :delete
      
      def delete( widget )
        unless widget.instance_of?( @reflection.klass )
          widget = teds_find_relationship_wrapper( widget )
        end
        unless widget.nil? # I should probably raise an error
          old_delete( widget )
        end
      end
      
      alias_method :old_push, '<<'
      
      def <<( widget )
        if widget.instance_of?( @reflection.klass )
          old_push( widget )
        else
          create( { teds_guess_foreign_key( widget ) => widget.id } )
        end
      end
      
    end
  end
end

Google Calendar - Oh so close!

It took me a few years to switch from Yahoo! Mail to Gmail as I patiently waited for Google to get their act together an add the feature allowing you to define the "From:" address as something other than your Gmail account. Yahoo! Mail had this for as long as I can remember, and it kept me as a loyal customer.

Sadly, it turns out that Google Calendar is history repeating itself. The calendar feature is the last and only thing I still rely on Yahoo! for and I'd glady switch to Google in a second if it supplied the features I need. But, alas, it does not. It has two major deficiencies:

1. It doesn't support the recurring schedule of "every other week". Most of the regular events on my calendar repeat in this manner, such as weekends with my daughter. A rather disappointing oversight on their part.

2. Per-event notification configuration. You can have all of your notifications via e-mail, or all via SMS, or both, but you can't have one come by e-mail (like a week early warning of an upcoming birthday) and another come by SMS (like 10 minutes before a meeting). Again, a frustrating limitation.

Slightly related to the latter, but not necessarily a deal killer, is the fact that you can only set notifications for events on your primary calendar. I set up a separate calendar for my company-related events, and another for just birthdays and anniversaries, then realized to my dismay that I couldn't set up notifications for any of them. Thankfully, it's easy to move events from one calendar to another.

Oddly enough, you can set a notification on an event in the primary calendar then move the event to another calendar and the notification icon remains, but I've not tested to see if the notification actually gets sent. That's an ugly hack.

My colleague assures me that they will fix these problems quickly, but based on how incredibly long I had to wait for them to match Yahoo! Mail's feature set, I'm not going to hold my breath.

Oh so close, Google. Oh so close.

has_many :polishing

As I noted in the update to my last post, I had a little bug in my extension of the has_many feature that broke the old vanilla usage. The bug was in the way that "super" called from the "<<" override wasn't correctly/consistently deferring to the former implementation. I couldn't figure out why it didn't work, but I did figure out how to work around it. I just aliased the method name and deferred to the alias. Slick.

Oh, I also moved the extension into it's own file "lib/has_many_extensions.rb":

module ActiveRecord
  module Associations
    class AssociationCollection 
      
      def foreign_key( widget )
        widget.class.to_s.downcase + '_id'
      end
      
      def find_relationship_wrapper( widget )
        instance_eval "#{ @reflection.klass }.find_by_#{ foreign_key( widget ) }( #{ widget.id } )"
      end
      
      def include?( widget )
        unless widget.instance_of?( @reflection.klass )
          widget = find_relationship_wrapper( widget )
        end
        super( widget )
      end
      
      alias_method :old_delete, :delete
      
      def delete( widget )
        unless widget.instance_of?( @reflection.klass )
          widget = find_relationship_wrapper( widget )
        end
        old_delete( widget )
      end
      
      alias_method :old_push, '<<'
      
      def <<( widget )
        if widget.instance_of?( @reflection.klass )
          old_push( widget )
        else
          create( { foreign_key( widget ) => widget.id } )
        end
      end
      
    end
  end
end

Then I added the necessary "require" to the end of my "config/environment.rb" file:

require 'has_many_extensions'

Enjoy!

has_many :gets even better

Update: My redefinition of << below is breaking normal has_many relationships. I haven't yet figured out why, but when I do I'll post a fix.

As I alluded to in the update to my last post, after it had stewed in my noggin for a couple hours, I realized that I had grossly overcomplicated two issues.

First of all, I didn't need to "cheat" the system by overriding the has_many signature and passing in the "hint" of the model being mapped to (Collectible). I can very easily check it at run time. I know the model of the relationship mapping (Possession), so I can check to see if the passed in object is an instance of the mapping (Possession) and if not, assume it's the model to be mapped to (Collectible).

That leads me to the second drastic improvement. I don't need to define new methods! I can override the existing has_many helper methods, and defer to their overridden counterparts when the mapping model (Possession) is passed in, or apply my new magic when the model to be mapped (Collectible) is passed in.

So first of all I beefed up the unit test with lots of commenty goodness and followed each test of the new overridden methods with a sanity check using the old [ugly] way:

  def test_has_many_extensions
    collector_id = 1
    collectible_id = 3
    collector = Collector.find( collector_id )
    collectible = Collectible.find( collectible_id )
    # ensure we are starting with a clean slate
    assert ! collector.possessions.include?( collectible )
    # sanity check -- compare it with the old-fashioned way
    assert collector.possessions.find_by_collectible_id( collectible.id ).nil?
    # add the collectible to the collector's possessions
    collector.possessions << collectible 
    # ensure that the collectible was added to the collector's possessions
    assert collector.possessions.include?( collectible )
    # sanity check -- compare it with the old-fashioned way
    assert ! collector.possessions.find_by_collectible_id( collectible.id ).nil?
    # try to re-add it -- should this break?
    collector.possessions << collectible 
    # reinitialize the collector so we can...
    collector = Collector.find( collector_id )
    # ...ensure that the collectible possession mapping was saved to the database
    assert collector.possessions.include?( collectible )
    # sanity check; compare it with the old-fashioned way
    assert ! collector.possessions.find_by_collectible_id( collectible.id ).nil?
    # remove the collectible from the collector's possessions
    collector.possessions.delete( collectible )
    # ensure that it was removed
    assert ! collector.possessions.include?( collectible )    
    # sanity check; compare it with the old-fashioned way
    assert collector.possessions.find_by_collectible_id( collectible.id ).nil?
    # re-remove it -- should this break?
    collector.possessions.delete( collectible ) 
    # reinitialize the collector so we can...
    collector = Collector.find( collector_id )
    # ...ensure that the collectible possession mapping was removed from the database
    assert ! collector.possessions.include?( collectible )
    # sanity check; compare it with the old-fashioned way
    assert collector.possessions.find_by_collectible_id( collectible.id ).nil?
  end

Then I revered the Model declaration back to good old vanilla Rails syntax:

class Collector < ActiveRecord::Base
  
  has_many :possessions
  
end

Then I completely scrapped the hack to override the has_many declaration and beefed up the AssociationCollection hack by overriding the actual old-school methods and deferring to their overridden counterparts when appropriate:

module ActiveRecord
  module Associations
    class AssociationCollection 
      
      def foreign_key( widget )
        widget.class.to_s.downcase + '_id'
      end
      
      def find_relationship_wrapper( widget )
        instance_eval "#{ @reflection.klass }.find_by_#{ foreign_key( widget ) }( #{ widget.id } )"
      end
      
      def include?( widget )
        unless widget.instance_of?( @reflection.klass )
          widget = find_relationship_wrapper( widget )
        end
        super( widget )
      end
      
      alias_method :old_delete, :delete
      
      def delete( widget )
        unless widget.instance_of?( @reflection.klass )
          widget = find_relationship_wrapper( widget )
        end
        old_delete( widget )
      end
      
      def <<( widget )
        if widget.instance_of?( @reflection.klass )
          super( widget )
        else
          create( { foreign_key( widget ) => widget.id } )
        end
      end
      
    end
  end
end

Now this is a piece of code I can be proud of! Feast your eyes on that puppy and try not to drool on your keyboard ;-) Ha! I'm high on Ruby on Rails, and I ain't never comin' down!

has_many :revelations

Update: On the drive home tonight, it dawned on me that there's two ways to drastically improve this little hack. I'm trying to make them work right now and I'll post a new entry revealing them as soon as possible.

It wasn't easy, and it's not pretty, but I did it! I managed to achieve my Holy Grail of has_many functionality. (Read my last two posts if you're not familiar with the problem I was having.) Here's how I did it.

In true TDD fashion, I decided on how I wanted to call the API first and wrote a test:

def test_has_many_extensions
    collector = Collector.find(1)
    collectible = Collectible.find(3)
    assert ! collector.possessions._include?( collectible )

    collector.possessions._add( collectible )
    assert collector.possessions._include?( collectible )

    collector = Collector.find(1)
    assert collector.possessions._include?( collectible )

    collector.possessions._delete( collectible )
    assert ! collector.possessions._include?( collectible )

    collector = Collector.find(1)
    assert ! collector.possessions._include?( collectible )
end

As you can see, I simply took the existing common has_many helper method names (which normally take instances of the joining Model as an argument) and prefixed them with underscores then instead passed in instances of the Model to be mapped to/from.

This, in my mind, seems like a more logical and readable way of "conversing" with the API. I don't want to "add a Possession to a Collector's Possessions." I want to "add a Collectible to a Collector's Possessions." Get it?

So the first thing I had to do was delve into the guts of Rails and start trying to reverse-engineer how everything worked. That was painful. It was excruciating. I'm used to working with Eclipse at the office and navigating Java projects. Eclipse makes it easy to trace up and down call stacks, bookmark points of interest, etc. No such luck with Rails. Even though I was using RadRails (which is built on the Eclipse platform), it doesn't yet offer such niceties [and might not ever due to the complications of parsing and pre-processing a vibrantly dynamic scripting language like Ruby, but I'll keep my toes out of that flame pool for now].

I'll save you the gory details, not so much out of mercy but more from the fact that I couldn't retain it all. My head was swimming trying to keep it all together, and I never really grok'd it completely. But, what's important is that I figured out enough of it to extend it with the functionality I desired.

My new underscore-prefixed methods needed to be added to ActiveRecord::Associations::AssociationCollection, so I placed the following in my "config/environment.rb" file:

module ActiveRecord
  module Associations
    class AssociationCollection 
      
      def _add( widget )
        create( { teds_foreign_key => widget.id } )
      end
      
      def mapping( widget )
        instance_eval "#{ @reflection.klass }.find_by_#{ teds_foreign_key }( #{ widget.id } )"
      end
      
      def _delete( widget )
        delete( mapping( widget ) )
      end
      
      def _include?( widget )
        include?( mapping( widget ) )
      end
      
    end
  end
end

If you're actually reading the code, you'll notice some scary stuff in there. First of note is the eval statement. I needed to make a call to a method that was defined at runtime by the framework, so I don't know beforehand what the name of that method is, and that method's name is based on the model being referenced, so it differs with context. Such is the paradoxical beauty and horror of Ruby. So I formatted the call I needed to make in a string using variables that would be populated at run-time to fill in the holes. This is how I get instances of the mapping Model necessary for passing in to the default has_many helper methods.

The second scary thing is the mysterious "teds_foreign_key" references. I tried, truly I did, but could not find any way in the existing has_many code to extract the foreign key column name. I know it's in there. It has to be in order for it do generate the proper database queries. But I couldn't find it. Undoubtedly it's buried in some abstraction of a library that creates the code for another library on the fly and eval's it all ;-) Perhaps some enlightened reader will set me straight. But, in the meantime, I had to cheat the system. I am passing in a clue to the foreign key from the Model declaration:

class Collector < ActiveRecord::Base

  has_many :possessions, :referenced_class =>  Collectible
  
  ...

That "referenced_class" option is what I added to the has_many signature. I use it to deduce the foreign key (and not very elegantly, admittedly) so the aforementioned methods can use it. Here's how I accomplished that (also in my "config/environment.rb" file):

module ActiveRecord
  module Associations
    module ClassMethods
      
      alias_method :old_has_many, :has_many
      
      def has_many( association_id, options = {}, &extension )
        if options[:referenced_class]
          @@teds_foreign_key = options[:referenced_class].to_s.downcase + '_id'
        end
        old_has_many( association_id, options = {}, &extension )
      end
      
      def teds_foreign_key
        @@teds_foreign_key
      end
      
    end
  end
end

And that's it! I tested it from the console and everything worked fine; I queried the database after every step as a sanity check. And, of course, the unit test at the top of this post ran without error. I'm pretty fricken psyched about this. If you like it and it helps you out, please let me know. If you see some sort of problem with it, please also speak up. I'm certainly no Ruby or Rails expert [yet] and I'd love to hear the advice of others.

has_many :headaches

Continuing my rant from earlier, I decided to delve into the source and see if I could figure out exactly what's going on, and maybe by some miracle add the functionality I need and expected.

First stop was to find out where the "<<" operator was being overloaded. Yes, I know that in Ruby it's not an operator, it's a message, and it's not overloaded. But I'm an old fart, and those are the terms I was weaned on, so deal with it. But I digress.

I found the magic in this file:

rails/activerecord/lib/active_record/associations/association_collection.rb

And the code looks like this:

module ActiveRecord
  module Associations
    class AssociationCollection < AssociationProxy
    
      ...
    
      def <<(*records)
        result = true
        load_target

        @owner.transaction do
          flatten_deeper(records).each do |record|
            raise_on_type_mismatch(record)
            callback(:before_add, record)
            result &&= insert_record(record) unless @owner.new_record?
            @target << record
            callback(:after_add, record)
          end
        end
    
        result && self
      end

Pretty straight forward. It looks like it's defering to a method named "insert_record" to handle the databaseness, so that was my next stop, and I found it in this file:

rails/activerecord/lib/active_record/associations/has_many_association.rb

And here's the code:

module ActiveRecord
  module Associations
    class HasManyAssociation < AssociationCollection

        ... 
 
        def insert_record(record)
          set_belongs_to_association_for(record)
          record.save
        end

Now some of the pieces are starting to fall into place. What it's doing here is only going to (and only does) work for a one-to-many relationship. What is does is takes the record passed in to the collection, sets its foreign key value, then saves the record.

What I need it to do is recognize that in some cases it's dealing with a many-to-many relationship and rather than saving the record that is passed into the collection it needs to create the record for the cross-reference.

Now the code above should have all of the information it needs to do just that. After all, it built the collection by querying the right tables, so it knows all of the tables names and their important columns, so it should be able to write that relationship back into the database. I need to dig a little more to figure out how I can leverage that.

I'm a bit surprised it hasn't already been done, which leads me to fear that it's rather ugly or nigh-impossible to accomplish, but I'm holding out hope. I'm hoping that somebody in the know stumbles across these rants and offers a helping hand in either making it work or explaining why it wasn't built to do so.

has_many: apparently doesn't have enough

These last couple weeks I've been making blazing progress on my pet Project Basement. I've fallen even more in love with Ruby on Rails, as if that was possible. You know the kind of sick infatuation when you start thinking to yourself, "I wonder how long it would take me to rewrite my company's six-year-old one-hudred-seventy-five-thousand-line Java application in Rails?" That's how sick it is. But I'm already off track before I've even begun. So on to my latest rant.

As the title implies, I've been a little disappointed with "has_many" as of late, specifically with the management of the "through" cross-reference records. For example, I've got the following models:

class Collectible < ActiveRecord::Base
end

class Collector < ActiveRecord::Base
  has_many :possessions
  has_many :collectibles, :through => :possessions
end

class Possession < ActiveRecord::Base
  belongs_to :collector
  belongs_to :collectible
end

Simple enough, right? If I want a list of all the Collectibles possessed by a Collector, I just call "collector.collectibles" and the back-end does all the magic for me. It's a thing of beauty.

But what if I want to add a new Collectible to a Collector? This is where things get ugly. I'm hoping that this is simply a misunderstanding on my part and somebody will set me straight, but I've been banging my head against this for the last couple night and it's not adding up.

According to the documentation for has_many:

collection<<(object, …) - adds one or more objects to the collection by setting their foreign keys to the collection’s primary key.

That seems to imply that I should be able to call "collector.collectibles << some_collectible" and the new collectible will automatically be added to the list and the appropriate database record created, but that's not what's happening. Here's one of my test cases from "test/unit/possession_test.rb":

def test_collector_collectibles
    ted = Collector.find(1)
    nickel = Collectible.find(2)

    assert ! ted.collectibles.include?( nickel )
    # add a collectible to a collector
    ted.collectibles << nickel

    # sanity check 
    assert ted.collectibles.include?( nickel ) 

    ted = nil
    ted = Collector.find(1)
    assert ted.collectibles.include?( nickel ) 
end

That last assertion fails! When I reinitialize the Collector, forcing it to reload all it's guts from the database, the Collectible I just added to his Possessions isn't there. It apparently wasn't committed to the database.

I considered perhaps I needed to explicitly save the change, so I tried "ted.save" but that did nothing and "ted.collectibles.save" threw an error as the method isn't implemented.

Now, if I deal directly with the "possessions" list I can make it work, but I don't like the syntax at all:

def test_collector_possessions
    nickel = Collectible.find(1)
    ted = Collector.find(3)
    assert ted.possessions.find_by_collectible_id( nickel.id ).nil?

    ted.possessions.create( { :collectible_id => nickel.id } ) 

    ted = nil
    ted = Collector.find(3)
    assert ! ted.possessions.find_by_collectible_id( nickel.id ).nil?
end

That, in my humble opinion, is butt-ugly by Ruby and Rails standards. I contend that my first test, the one with the simple and elegant syntax, should work!

I've been in this situation before, where I've found myself frustrated with something in Ruby on Rails, but I've almost always eventually hit the "A ha!" moment where it all made sense. But, this time, it ain't happening. Can somebody out there explain and/or justify this disconnect?

In-place select-and-submit for Ruby on Rails

If you'll recall from my last post (on my now deprecated blog), the built-in in-place-editting tools that come with Rails didn't provide exactly what I needed, so I had to roll my own. Well, it happened again; this time with a drop-down menu. I wanted to provide the user with a drop-down list that automatically updated the back-end, without refreshing the entire page, when an item was chosen from it.

Just like with my text in-place-editting solution, this one hides the drop-down element once an option is chosen so that the user can't quickly re-select another option causing a possible race condition. In its place, it shows the user a progress indicator, which for me is a little animated GIF of a spinning circular arrow and the phrase, "Saving..." Once the server has confirmed the value was saved, the element is restored back to a drop-down menu with the saved value pre-selected.

I ran into an odd little behavior while implementing this utility. I had originally tried to call "form.submit();" when the value of the selection was changed, but this resulted in the browser posting the form and completely ignoring the "onsubmit" directive declared in the form tag. This was frustrating, and Googling the issue didn't turn up anything useful. But, I discovered through trial and error that I could call "form.onsubmit();" directly and all was good.

So without furher ado, here's the method from my application_helper.rb file:

  def select_and_submit( controller, action, id, element_for_results, options = {}, selected = nil, extra_hidden_values = {} )
    label = "#{controller}_#{action}_#{id}"
    extra_hidden_tags = ""
    extra_hidden_values.each_pair do |key, value|
      extra_hidden_tags << "<input id=\"#{key}\" name=\"#{key}\" type=\"hidden\" value=\"#{value}\" />"
    end
    options_rendered = ""
    for value in options
      if value.eql? selected then
        selected_rendered = ' selected="selected"'
      else
        selected_rendered = ''
      end
      options_rendered << <<EOF
                     <option value="#{value.id}"#{selected_rendered}>#{value.name}</option>
EOF
    end
    <<EOF
<div id="#{label}_select">
<form id="#{label}_form" action="/#{controller}/#{action}" method="post" onsubmit="Element.hide( '#{label}_select' ); Element.show( '#{label}_saving' ); new Ajax.Updater('#{element_for_results}', '/#{controller}/#{action}', {asynchronous:true, evalScripts:true, parameters:Form.serialize(this)}); return false;">
<input id="select_id" name="select_id" type="hidden" value="#{id}" />
#{extra_hidden_tags}
<select id="value_id" name="value_id" onchange="this.form.onsubmit(); return false;">
<option value=""></option>
#{options_rendered}
</select>
</form>
</div>
<div id="#{label}_saving">
<img src="/images/indicator_arrows_circle.gif" alt="*">
Saving...
</div>
<script>
Element.hide( '#{label}_saving' );
</script>
EOF
  end

And here's how I call it:

<%= select_and_submit 'collectibles', 'set_value_for_attribute', attribute.id, "attribute_row_#{attribute.id}", attribute.values, collectible.get_value_for_attribute( attribute ), { :collectible_id => collectible.id } %>

How do I delete this post?

How do I delete this post?