So I decided to go ahead and work up the frameworks comparison by porting a small application to several different frameworks. I'll be using the latest versions of any frameworks I'm porting.
Here's my list:
I've contemplated also using ColdSpring or Lightwire to manage the business components with a single service layer across all the frameworks. I decided against doing this not only because I'm not a big fan in the first place but also because I decided not to write my own application to port to each framework. Instead I decided to port another open source application from another author and settled quickly on Ray Camden's Galleon forums. Why? Several reasons - it's popular, it's reasonably small, it's a "real world" application (as opposed to cfpetmarket.com) and there was already a port of it for ColdBox. Galleon already had its business objects implemented, so while some folks might like to see me add Lightwire or ColdSpring it's extra work for me that would be outside the scope of what I wanted to accomplish.
So I downloaded a copy of Galleon and a separate copy of each framework and where needed, the "app skeleton" for a couple frameworks. I installed the Galleon 2.2 database tables (I'm using SQL Server 2005 Express), configured my ColdFusion DSN in the administrator and installed the existing port of Galleon for ColdBox.
Here's where I hit my first snag. I'd just installed the Galleon 2.2 database, which I planned to use across all the frameworks. Well unfortunately the ColdBox port was done with Galleon 1.7 and the database wasn't entirely compatible. So I copied the updated CFCs from Galleon 2.2 into the ColdBox port and spent the next several hours weeding out a number of updates that needed to be made in the views (event handlers were mostly okay). I'm sure there are still problems with the ColdBox port, but I got it to a point where you can log in with it, etc. and I wasn't going for perfection, I just want to show generally speaking how these different frameworks handle the logic, application flow and separation of concerns. So even if the ColdBox example (or any of them) is still "buggy" when I'm done, it should be enough for people to see the fundamental differences (or mostly similarities) between these frameworks.
Once I got the ColdBox sample properly installed I then started copying the original Galleon 2.2 code into a clean copy of the latest version of the onTap framework. Several hours later I think I'm mostly done with it. One thing I did notice however is that the Galleon business objects use a string variable to identify which database platform is being used. The Galleon project came with install scripts for MySQL, SQL Server, Oracle and included an Access database file, all of which are supported natively by the onTap framework's SQL abstraction library. In the galleon business objects code, it uses that string to determine how to handle things like reserved words, so for example in the user.cfc you find this query:
</cfswitch> = <cfqueryparam value="#arguments.group#" cfsqltype="CF_SQL_VARCHAR" maxlength="255">
I'm giving serious consideration to porting the business objects separately to use the framework's SQL Abstraction layer, even though I won't be doing this with any of the other frameworks because none of them have any equivalent features. The reason being that I think this is one of the nicer features of the onTap framework and really shows how it saves time and headaches during development, since the above query can be reduced to:
var ds = getDatasource();
var thetable = variables.tableprefix & "groups";
var checkgroup = ds.getSelect("id",thetable).filter("group",arguments.group).execute();
Notice how that gets rid of that big nasty switch statement? Well that also means something else - it means that you could in theory run the application on another database platform (PostgreSQL for example) without modifying the business objects, simply by creating a new SQL Agent (CFC) for your preferred database platform. This is why I disagree with Ben Forta who says this kind of emulation is "not doable and not worth doing" (paraphrased).
Anyway, whether I port the business objects or not, once I'm done with the other three ports, I plan to write up a comparative analysis showing some of the strengths and weaknesses of the various architectures (as I see them, so yes, quite biased). I welcome both critique and criticism. :) The writeup will be included along with all the code in a zip archive that I'll upload here, including the core versions of all the frameworks I used (to ensure they're installed using a compatible version). To make sure the various versions use the same core framework versions I installed them on, I'll also be using the new application-specific mappings feature (this.mappings) in Application.cfc for each port. This means my distribution will require ColdFusion 8. I'll also include a summary of the writeup in my blog here and I'll cross-post to the riaforge forum for the Galleon project and each of the individual framework projects and probably the CF-Talk list.
And if there's anything you'd like to see me mention specifically in the writeup, just let me know either in the comments here or you can email me.
UPDATE: The first draft is available here.