I'd like to say that every upgrade is as seamless as the upgrade from ColdFusion 7 to 8, where I didn't have to do anything (that I remember). But of course they don't always turn out that way. Back when Macromedia released ColdFusion MX, myself and a number of other developers had to do a fair amount of work to remove references to structure variables named "this". It turns out that they've added a "local" scope in ColdFusion 9 and wouldn't you know, about the time I stopped using "this", I started using "local" instead. ;)

Ryan McIlmoyl gave me the heads up on the issue on the DataFaucet list. About a day and a half later, I've just finished up the basic modifications for CF9. (There were 3.5-thousand references to "local" in the core code, plus another 500 or so in the Members onTap plugin, with plenty of false-positives.) I don't actually know if the onTap framework is still working on CF9 yet, because I haven't installed it to test with yet. If you give it a shot, let me know how it goes for you.

One thing I'm glad of though, is the way I designed the ontap.cfc. This is a component with some basic functionality that was created before onMissingMethod() was added to the language, otherwise it might have looked a bit different. But in essence, the lack of onMissingMethod() is why the ontap.cfc has generic getters and setters (or accessors and mutators for you lingophiles out there). These are two private methods name getProperty() and setProperty() and two public methods named getValue() and setValue() instead of having loads of individual functions named getBlahBlah() and setYaddaYadda(). What's cool about this arrangement is that in all my CFCs, nowhere to I directly reference the structure that holds property values, which was named (you guessed it), "variables.local". Well, nowhere except in about 3 private methods in ontap.cfc. And because all my other CFCs faithfully relied on getProperty() and setProperty() instead of simply referencing local.x or local.y, that means I only had to change "variables.local" in about 3 places. This saved me HUGE headache. If I'd gone the other way and used all hard-coded (or more to the point hand-coded) getters and setters, I'd have had easily another couple thousand references to replace and another day's work on this update for CF9. I love it when a plan comes together! :)

I was actually thinking about this the other day too, because I was looking at a small application someone else wrote (I won't mention any names), where they used all hand-coded getters and setters in their code and I was trying to figure out how to use it for configuration. After about an hour or two of looking through the code for documentation of one of the features I realized that said feature was never completed. There were just 23 references to it hanging out in the code, not doing anything at all (other than passing a useless variable around). All those 23 references were mostly hand-coded getters and setters, plus a couple of form fields and param tags. And I was thinking to myself, how nice it is to know that, for the most part, that sort of thing doesn't happen in my applications. Sure I make mistakes, everyone does, but the point here is, if I'm writing config for an app, there's not going to be 23-lines of code hanging around for a single config value, there's probably going to be about 3 or 5 and that means not only less maintenance, but less work up-front as well. :)

So new zip archives of the core framework and of the Members onTap plugin are up on RIAForge.

I just realized that there was a small issue with the Plugin Manager that was causing a problem for an upgrade install for CacheBox. When you go into the plugin manager and you select a new plugin to install, it was reporting the information for the installed plugin instead of the new plugin version waiting to be installed. So for example, if you already had CacheBox 0.9.1 installed and then you downloaded the plugin to install 0.9.8, it would say "Installing CacheBox 0.9.1" despite the fact that you were actually installing 0.9.8. And because of that also when it finished installing, it wasn't recording the new version in the plugin manifest, so it would still show as the upgrade having not yet been installed.

So I fixed this in the Plugin Manager and uploaded a new copy of the framework core.

This version of the core also includes a new version of the CacheBox Agent that was required to fix a small bug with the CacheBox service. Unless you have the CacheBox service already installed, it shouldn't affect you. If you've installed it, you might get an error from the onTap framework until you update your core files. So I would update the onTap framework first and then CacheBox.

So a week or so ago I published a new version of the framework core with a minor enhancement to improve on the workaround for web-services/onRequest that may resolve the current issues with running the framework on BlueDragon/Railo. At the time I didn't realize that I had introduced a new bug (albeit small) that prevented the onMissingTemplate() feature from working, so any page request would require the base template in addition to the process code in the /_tap/ directory. This is true for ColdFusion 7 anyway, but on CF8 you're supposed to be able to omit the base template and let the server automate it (except for index.cfm).

Anyway in the process of working on some integration for a project for Eric Jones, I discovered this bug as well as a couple of other minor bugs in the Members onTap plugin that were introduced since the recent upgrades for new DataFaucet features. The member plugin bugs only occurred if an administrator attempted to edit their own account via the admin area (instead of the member area). Admins aren't supposed to be able to edit their own roles (because they can't add roles they don't already have themselves and shouldn't be allowed to de-permission themselves), so there's no really powerful reason for an admin to use the admin area to edit their own profile anyway. But it ought to work, so I've fixed those bugs as well... one of which resulted in the admin de-permissioning themselves when they saved the form. D'oh! ;)

Also the integration for Jonese's project is going pretty darned well! I've been able to bypass the IoC for the member plugin to introduce a custom member object which extends the default member object and adds a 2nd table with some additional profile information to the member objects for this project. That in itself is something I'm rather happy with because of how seamless it is. :) I'll be posting more blogs later with additional details about how the integration is done, as well as more information about the project itself as an example of the onTap framework and DataFaucet in the wild. ;)

On a more personal note, I honestly don't think that "who's using this framework?" is ever really a good question to ask. I mean... it's not that asking it is a bad thing, but rather that it's simply unhelpful and especially using it to make decisions can be counter-productive. And I'll share briefly the reason why I feel this way.

The Fusebox framework has been around for a good long while now and there are a variety of mixed feelings about it in the community from the die-hard Fusebox enthusiast to the extreme opposite end of people who really just can't stand it. Each of them has their reasons... but what I'm really getting at here is that Fusebox projects are much the same way. I've never been a big fan of Fusebox myself, otherwise I wouldn't have created the onTap framework. ;) But despite my own preferences I'm reasonable enough to acknowledge that of the large number of Fusebox applications there are in the wild, some of them are well made and some of them are poorly made. Whether a given Fusebox application is good or bad depends to a much greater extent on who managed the project and what resources they had available than it does on the fact that they used Fusebox as their foundation.

I expect the same will be true of just about any framework. There are going to be good and bad Model-Glue apps and good and bad ColdBox apps and I'm sure there will be good and bad onTap framework apps. The fact that the framework isn't being used at Wells-Fargo or on the website for the Metropolitan Opera says absolutely NOTHING about the quality of the framework. The only thing it says is that I've done a poor job of marketing it thus far so it hasn't gained traction (and this is shown to be the case in our frameworks survey also, which I'll talk more about later).

That's the larger reason why I'm now focusing a lot of my time and energy toward being more vocal in the community and improving the marketing. And looking for an evangelist to take the reigns some in the marketing department. :)

Oops! It looks like when I made some recent changes to the member management plugin to take advantage of some of the new features in DataFaucet, I somehow accidentally got the addManyToManyRelationship() call into the member.cfc twice. Which in my case caused an error when I tried to install with a clean database. I guess I also neglected to test the installation into a fresh database, so that's my bad... There should probably be mxUnit or some other testing framework for these things, but TDD has been on my growing "when I have time" list for a while. In any event it was easy enough to fix, so I fixed it and uploaded a new build both to RIAForge and to the web service. So if you've been unable to install the Members onTap plugin recently, you can follow these steps to download the new build and try the installation again.

1. open up your plugin manager (http://[ontap]/admin/plugins)
2. select the "more" tab
3. hit "search"
4. find the Members onTap plugin in the list and hit "install"

Well it seems I committed what I've described in the past as a "cardinal sin" when using the onTap framework. That is, I created a distribution that included a file that would be modified during or after installation.

The reason why this is a cardinal sin when using the onTap framework is precisely because of what I ran into today. I had made a couple of what I thought were minor changes to some code, but was having a problem with some other code and even though I didn't think they were related, I decided to "roll back" the changes just in case. It turns out they actually weren't related, so the small changes I made stayed in the code. However because my method of rollback involved unzipping the framework core archive (ontap32.zip) into my working directory, it overwrote the new XML file that the Plugin Manager uses to store version information for the previously installed plugins... And of course, like magic that meant it was suddenly interpreting two plugins as uninstalled when they were in fact installed.

Thing is, it's supposed to be safe to extract a newer version of the framework into your working directory. Because one of the framework's guiding principals is "never edit someone else's file", that should apply to the framework core equally as much as to those of us working with is. So just as I should never edit one of your files (and you should never edit one of mine), the framework should never overwrite your data files either.

So long story short there's a new copy of the framework core archive up on RIAForge that's now minus the Plugin Manager's new XML file so that when you update it won't be overwritten. However there's no real urgency for anyone to update the core at this point. Unless your copy is already tragically old. ;)

Rob Parkhill alerted me a few days ago to difficulties installing the Members onTap plugin with MySQL. There turn out to have been a couple of issues with MySQL, one of them required an update of DataFaucet and the other required an update of the member plugin. I was finally able to get around to installing a new copy of MySQL on my notebook to test and fix these yesterday. I blogged about these issues on the DataFaucet blog and uploaded a new build of both DataFaucet and the member plugin. The DataFaucet update is a download from the RIAforge project. To get the member plugin, just use the webservice in your plugin manager at http://yoursite/admin/plugins/.

Rob Parkhill recently reported to me that having the ColdFusion debugging enabled in his ColdFusion administrator caused the navigation pane of the documentation to timeout.

The navigation pane renders fine with debugging disabled on that machine.

We're still not entirely certain what the problem is, this doesn't occur in every environment and we suspect it may be related to a specific debugging configuration, but have yet to isolate the particular switches in question.

Once we have a better idea of the specific cause, we'll add some information about it to the blog, wiki and release notes.

Thanks!

I'm reminded that when ColdFusion 7 was released, it shipped with a small bug in the XForms implementation which caused multiple-select input elements to display with none selected if a user had previously selected multiple options. It had turned out to be something fairly simple, an oversight in their default XSL sheet and at the time I'd provided the fix for Jeff Small (although at the moment I can't find his blog entry where he posted it for everyone else -- send me the url if you have it and I'll post it here). I'm sure it was something the folks on the CF development team just hadn't antiscipated, after all how often do you use a multiple-select? (Of course, I say this expecting a quick response from the couple of folks who use them frequently.) :)

It's sort of come back to haunt me... In the previous versions of the onTap framework, I'd built in query-driven select input elements. The first iteration of these were unfortunately painfully slow. The problem at the time was that I was making it create a whole new html element structure for each option, instead of simply holding on to the query until display time. It turned out to be far too much overhead, and just as well most of the data in those structures was going unused anyway. So I then moved to the more logical approach of storing the query in the element structure for the select input and looping over them to create the options at time of display. This worked pretty well and even allowed me an easier way of fetching the current value of the select input by using the native ValueList() function. However, when setting the current value, I was still looping over each record in the query, like this:

You can see from this code how I can then use ValueList(input.query.inputvalue) to fetch the list of currently selected options. Well this was great, and then shortly after I started working on version 3.0 of the framework I realized that there may be an even more efficient way. Instead of looping over the query one record at a time, I could set the entire query all at once and it could potentially be much more efficient. Something like this:

So in this case the loop only processes one iteration for each selected option, instead of for each record in the query. In some cases where there are a lot of options in the list this can be much more efficient. I tested this after I implemented it and it is much more efficient when the select list is excessively large: hundreds of options or more, which may not be the best idea in the first place, but that's another blog. For most uses though, the efficiency probably won't change much from the last version.

The problem lies in that getSelectedIndexes() function (which is a simplification for the sake of the blog -- suffice to say that it creates a list of query indexes that should be selected without looping over the query). The algorithm I had for determining which query indexes were selected turned out to have a minor flaw in it and so where I had a query containing the values "en,en_CA,en_US" (and a few others), it was selecting all options beginning with "en", so the options for CA and US were being additionally selected when only "en" should have been. Of course I fixed this yesterday and uploaded a new build of the framework core code a few minutes ago.

I will give one caveat and that's the form tools do assume that you won't have any linefeed characters (ASCII char(10)) in your option values. So if for some reason you need that particular character in your data, you'll have to replace them with something else before displaying the form and after submitting it. Probably most people won't have any issues with that, since the inputvalue column will usually contain either integers, UUID strings or some other equivalent. Speaking of which, this is probably why I didn't catch this sooner, because I tend not to use autonumber / identity columns (though they're better supported now), and since most of my select elements contain a UUID equivalent, the values generally don't offer that kind of overlap.

I know that verity includes a lot of advanced search features, but I honestly just haven't worked with it much. So I'm not sure if verity offers an "and/or" keyword feature, though I wouldn't be surprised. Then again verity requires you to create and maintain collections and that means double-maintenance if the data is stored in your database and you don't always need the other features verity has. So a while back I decided I wanted the ability to include and/or keywords in searches, (similar to google I suppose) and rather than copying and pasting that code every time, the framework's SQL abstraction layer gave me the ability to simplify the process of creating those and/or filters. The framework code looks something like this:

This code then takes whatever you've entered in the search form, breaks it apart into phrases separated by the words "and" and "or"1 and spreads those phrases across the title, keywords and contentText columns in the tbl_content table. Being able to do this with just one line of code is awfully convenient. The resultant query is something like this:

searched for "fish and chicken"

So with this sql output, you can see that the query will return any record that contains both words, but it also returns results where the words are in different portions of the content, for example the word "fish" in the title and "chicken" in the contentText, so if there's an article titled "let's go shark fishing" with the phrase "are you chicken?" in the contentText, this search will return that record. What makes this really nice however is that as the number of columns increase and/or the complexity of the search string increases, the SQL statement becomes very large and very complicated rather quickly, however, the andOrFilter facade which handles the thankless task of creating the SQL syntax never becomes any more complicated than the one simple line above2.

Today I realized that the facade was limited to only accessing columns in a single table, so although it worked it was still not as flexible as I wanted it to be. I was working on a contact manager and needing to include two columns from joined tables in the and/or column list. So I updated the facade to allow that and now you can also do this:

It was a really good thing for me, because the query I was working on was massive... Check this out: 3

SELECT DISTINCT [ONTAP].[DBO].[TAP_CONTACT].[contactid],[ONTAP].[DBO].[TAP_CONTACT].[authorityid],[ONTAP].[DBO].[TAP_CONTACT].[contactauthorid],[ONTAP].[DBO].[TAP_CONTACT].[contactmemberid],[ONTAP].[DBO].[TAP_CONTACT].[namelast],[ONTAP].[DBO].[TAP_CONTACT].[namefirst],[ONTAP].[DBO].[TAP_CONTACT].[namemiddle],[ONTAP].[DBO].[TAP_CONTACT].[namepreferred],[ONTAP].[DBO].[TAP_CONTACT].[nametitle],[ONTAP].[DBO].[TAP_CONTACT].[namesuffix],[ONTAP].[DBO].[TAP_CONTACT].[contactlocale],[ONTAP].[DBO].[TAP_CONTACT].[contactregionid],[ONTAP].[DBO].[TAP_CONTACT].[contacttimezoneid],[ONTAP].[DBO].[TAP_CONTACT].[contactcreated],[ONTAP].[DBO].[TAP_CONTACT].[contactmodified],[ONTAP].[DBO].[TAP_CONTACT].[contactbirthdate],[ONTAP].[DBO].[TAP_CONTACT].[contactgender],[ONTAP].[DBO].[TAP_CONTACT].[contactphotouri],[ONTAP].[DBO].[TAP_CONTACT].[contactlogouri],[CR].[contactrelevance],[PHONE].[methoddata] AS [phone],[EMAIL].[methoddata] AS [email],[NOTE].[notetext] FROM [ONTAP].[DBO].[TAP_CONTACT] INNER JOIN [ONTAP].[DBO].[TAP_CONTACTLANGUAGE] ON ([ONTAP].[DBO].[TAP_CONTACTLANGUAGE].[contactid] = [ONTAP].[DBO].[TAP_CONTACT].[contactid]) LEFT JOIN [ONTAP].[DBO].[TAP_CONTACTRELEVANCE] [CR] ON ([CR].[contactid] = [ONTAP].[DBO].[TAP_CONTACT].[contactid] AND UPPER([CR].[memberid]) = 'MBRADMIN') LEFT JOIN [ONTAP].[DBO].[TAP_CONTACTMETHODXREF] [PHONEXREF] ON ([PHONEXREF].[contactid] = [ONTAP].[DBO].[TAP_CONTACT].[contactid] AND [PHONEXREF].[methodisprimary] = 1 AND UPPER([PHONEXREF].[methodmedium]) = 'PHONE') LEFT JOIN [ONTAP].[DBO].[TAP_CONTACTMETHOD] [PHONE] ON ([PHONE].[methodid] = [PHONEXREF].[methodid]) LEFT JOIN [ONTAP].[DBO].[TAP_CONTACTMETHODXREF] [EMAILXREF] ON ([EMAILXREF].[contactid] = [ONTAP].[DBO].[TAP_CONTACT].[contactid] AND [EMAILXREF].[methodisprimary] = 1 AND UPPER([EMAILXREF].[methodmedium]) = 'EMAIL') LEFT JOIN [ONTAP].[DBO].[TAP_CONTACTMETHOD] [EMAIL] ON ([EMAIL].[methodid] = [EMAILXREF].[methodid]) LEFT JOIN [ONTAP].[DBO].[TAP_CONTACTORGMEMBER] ON ([ONTAP].[DBO].[TAP_CONTACTORGMEMBER].[contactid] = [ONTAP].[DBO].[TAP_CONTACT].[contactid]) LEFT JOIN [ONTAP].[DBO].[TAP_CONTACTORGTREE] ON ([ONTAP].[DBO].[TAP_CONTACTORGTREE].[orgid] = [ONTAP].[DBO].[TAP_CONTACTORGMEMBER].[orgid]) LEFT JOIN [ONTAP].[DBO].[TAP_CONTACTORG] [ORG] ON ([ORG].[orgid] = [ONTAP].[DBO].[TAP_CONTACTORGTREE].[orgparentid]) LEFT JOIN [ONTAP].[DBO].[TAP_CONTACTNOTE] [NOTE] ON ([NOTE].[contactid] = [ONTAP].[DBO].[TAP_CONTACT].[contactid] AND UPPER([NOTE].[memberid]) = 'MBRADMIN') WHERE ([ONTAP].[DBO].[TAP_CONTACT].[authorityid] IS NOT NULL OR (UPPER([ONTAP].[DBO].[TAP_CONTACT].[contactauthorid]) = 'MBRADMIN' AND [ONTAP].[DBO].[TAP_CONTACT].[authorityid] IS NULL)) AND (((UPPER([ONTAP].[DBO].[TAP_CONTACT].[namelast]) LIKE '%NYGMA%' OR UPPER([ONTAP].[DBO].[TAP_CONTACT].[namefirst]) LIKE '%NYGMA%' OR UPPER([ONTAP].[DBO].[TAP_CONTACT].[namemiddle]) LIKE '%NYGMA%' OR UPPER([ONTAP].[DBO].[TAP_CONTACT].[namepreferred]) LIKE '%NYGMA%' OR UPPER([ONTAP].[DBO].[TAP_CONTACT].[nametitle]) LIKE '%NYGMA%' OR UPPER([ONTAP].[DBO].[TAP_CONTACT].[namesuffix]) LIKE '%NYGMA%' OR UPPER([ORG].[orgname]) LIKE '%NYGMA%' OR UPPER([NOTE].[notetext]) LIKE '%NYGMA%'))) AND UPPER([ONTAP].[DBO].[TAP_CONTACTLANGUAGE].[contactlanguage]) IN ('EN') ORDER BY [cr].[contactrelevance] DESC,[ONTAP].[DBO].[TAP_CONTACT].[namelast] ASC,[ONTAP].[DBO].[TAP_CONTACT].[namefirst] ASC,[ONTAP].[DBO].[TAP_CONTACT].[namemiddle] ASC

Aside from the fact that the sql abstraction layer escapes all the column names for me and ensures case insensitivity (even with a case-sensitive database collation), creating this query required me to write a remarkably small amount of actual code, much of which is even reused in other queries.

So anyway, I uploaded a new archive with the enhancement for multi-table and/or filters4.

1. Actually it's a bit more sophisticated than that even because it uses localized strings for the and/or keywords for international use, so for example, german users can use the keywords "und" and "oder" in their search.
2. And really, without a facade like this, and/or keyword searching is one of those features that just never gets implemented because, as nice as it may be for the users, it's tedious and never makes it into the list of priorities we can spend time working on.
3. Thanks to search.getSyntax() for the sql syntax - this lets you output the syntax instead of executing the query for use in generating scripts to be executed elsewhere, as in my case it's easier to debug a query sometimes by copying and pasting the SQL syntax into an IDE provided by the database vendor like Query Analyzer. This way I don't have to sit through the tedium of copying all the query-param values, sifting through the query and replacing question-marks, because getSyntax() performs the merge for me.
4. I also discovered that when using filterGroups (they allow you to wrap parenthesis around groups of filters like in the query above) if you misspelled the name of a table alias, it was producing an unusual error -- it's supposed to throw a custom "column not found" error, which it wasn't doing, so I fixed that also.

Apparently I somehow managed to get a copy of an archive called "tapmce.zip" uploaded to the RIAForge server containing all the sourcecode for the Members onTap plugin project. Someone let me know about it, so I recreated the archive for the project & uploaded it and made sure it's the correct source this time. I'm sure this is related to the recent switch from my own server to RIAForge and having had to change all my batch files for distribution.

For a long time now the onTap framework has included features for managing timezones, something for which ColdFusion's support is traditionally somewhat weak. But timezones are a messy issue and notorious for creating unexpected challenges in software development. For example, it's generally considered a best practice to store dates in UTC (formerly GMT) timezone because that time zone is neutral, having no offest and no daylight savings adjustments, meaning that 5-o-clock is 5-o-clock is 5-o-clock. Really what this means is that it's a good baseline from which times for other timezones can be extrapolated.

That's actually the intention of UTC in the first place and why it is that many platforms like ColdFusion and SQL Server have native functions for getting the current time in UTC or converting times to and from UTC. SQL Server has a GetUTCDate() function for example although I'm not aware that it has any native functions for converting to or from. So the onTap framework by default will assume that you want to store dates in UTC and when you enter dates into forms, if you've added a time, it will convert that date/time value to UTC before it's stored. If there's no time, it leaves the date alone.

(Of course if you don't like the fact that the framework handles this time-zone converting for you and you'd rather do it all yourself, you can disable it by setting the timezone for the request to UTC. If there's no offset and no DST adjustment for the current timezone, then it will only be adding or subtracting zeros from your dates, effectively canceling out the feature.)

A while back I implemented a feature in the SQL Abstraction tools that allows you to use "now" as an alias for the current time in UTC when performing inserts or updates. I did that because I was finding that I wanted to insert the current UTC time frequently (you might say routinely) and at the time it was a bit of a challenge because the tools' default behaviors were converting the date from the current timezone of the request (which belongs either to the user or to the server). So in order to get the "now" UTC I was having to first convert the UTC date to the local time zone so that it would be the current time in UTC when it was converted back. So the fix that made it much easier to handle that common task was allowing me to assign it via "now", i.e. myobject.update(SomeDateColumn="now"). That's pseudocode -- the real thing is a bit more involved but you get the idea.

I was just noticing a similar issue with the forms this evening. It wasn't quite as simple as I would like to create a date input in a form with a default value of the current time (or date). So I decided to fix that by adding a feature to the input validator that will interpret values of "now" or "today" as the current date, with or without the time respectively. So in my case, the input element is

And that gives the input element a default value of today's date plus time. If everything is working properly, it should match the clock in my system tray. :) It's important that the input be validated as a date, otherwise the framework doesn't even try any date massaging. They also get massaged a second time when you use the framework tools to validate the form on the server side, so it's important also make sure you are performing the server-side validation unless you're doing your own date massaging. On the other hand if you use tap:dbtable in your form to automate validation for the form based on your database metadata, then you won't need to include the tap:validate attribute you see in this snippet since the automation will add that for you.

Part of the upshot of this is that if javascript is disabled in their browser, users might be able to type "now" or "today" into the form to get those values. (If they have javascript enabled, the default javascript validation will stop them, although if you really like the idea of using "now" and "today" in forms, you can change the regular expression the framework uses for javascript validation.

Oddly, while I was in there adding this, I noticed also that the localization directories were being included in the wrong order... So /_application/l10n/en/us/100_dateformat.cfm that overwrites the date format for the united states to be mm/dd/yyyy was being included before instead of after /_application/100_i18n.cfm where the default format of dd/mm/yyyy for the rest of the world is set. Basically it wasn't using US date formatting in the US. So I fixed that. It seems odd that I hadn't seen that earlier, since I know it was working before. Anyway it's fixed now (and for most folks wouldn't have been a deal-breaker anyway, since most of us in the US could just set the value application-wide to use US dates).

p.s. I updated the zip archive as usual.