Trace function improperly handled by ColdFusion 9

While the function probably works exactly the way you would expect, the code analyzer for CF9 doesn't produce an error (or even an "information") flag if you have a function in your code named "trace".

You can even create the function and the server won't throw an error like it's supposed to when it encounters this:

<cffunction name="trace" ...>...</cffunction>

Instead it will compile fine and then when you try to use the function you may or may not get an error message like "Attribute validation error for TRACE. The value of the VAR attribute is invalid. Cannot set variable with name [blah blah]".

This is about the 3rd or 4th time I've discovered unexpected and mostly undocumented changes in behavior in CF9 or 9.0.1. Does anyone else feel like the release of this version of CF was a bit sloppy or is it just me?

(not so) massive problem with Query of Query in ColdFusion 9.0.1

Sorry for this... I've had a closer look at all the code and calmed down a bit and realized actually that the core CacheBox framework service had already been rewritten (years ago) to store content in a structure. So this problem turned out to be isolated to the agent CFC and only if the service wasn't installed. So updating the code for the agent to work the same way as the service seems to have resolved the issue.

I'm going to try not to be so jumpy from now on. ;P

----------

OH CRAP! THIS IS A GIANT PROBLEM!

ColdFusion 9.0.1 changed the behavior of Query of Queries and appears to have eliminated the ability to select from ANY query containing non-simple values... like we did just fine in ColdFusion 8.

Here's some code...

<cfset test = QueryNew("") />
<cfset QueryAddColumn(test,"columna",['a','b','c']) />
<cfset QueryAddColumn(test,"columnb",[1,2,3]) />
<cfdump var="#test#" />

<cfquery name="test2" dbtype="query">
   select * from test
   where columna = <cfqueryparam value="b" cfsqltype="cf_sql_varchar">
</cfquery>
<cfdump var="#test2#" />

This works fine on both CF8 and CF9.

Now we make one small tweak and add a complex value in columnb and it continues to work on CF8, but fails completely on CF9 with the message "complex objects cannot be converted to simple values".

<cfset myStruct = {a=1, b=2} />

<cfset test = QueryNew("") />
<cfset QueryAddColumn(test,"columna",['a','b','c']) />
<cfset QueryAddColumn(test,"columnb",[1,myStruct,3]) />
<cfdump var="#test#" />

<cfquery name="test2" dbtype="query">
   select * from test
   where columna = <cfqueryparam value="b" cfsqltype="cf_sql_varchar">
</cfquery>
<cfdump var="#test2#" />

Building the query isn't the problem -- that works fine in both cases -- there's nothing wrong with HAVING complex values in queries... and in CF8 as long as you didn't try to perform comparison operations against them, you could use a query-of-query to select those rows.

But on CF9, the presence of any non-simple value (it seems to be anywhere) in a query-of-query causes it to fail completely... even if I'm not selecting the rows where the structure is located, it still fails. And WHY WOULD IT? What's special about selecting a value in columnA that FORCES it to check the entire columnB column to make sure that every value is simple?! Doesn't that even run contrary to the logic that the query-of-query shouldn't constraint columnB unless it needs to?

This is HUGE because the entire CacheBox framework completely DEPENDS on query-of-query functionality that we had in CF8 and appear to have lost in CF9!

Damnit! I have no idea how to move forward from here...

If you've got a non-updated copy of CF9 (before the 9.0.1 update) it would be nice if you could run this code and see if it works there. I'd like to know if this happened in 9 or 9.0.1. Thanks.

OnRequestEnd behavior changed in CF9 (was Request Variable Bug in ColdFusion 9)

Here's a weird one. I haven't had any luck finding any information about this on Google, so I'm wondering if any of you have seen this before?

I've got a CFC in the request scope and then in the onRequestEnd event I grab that CFC out of the request scope and do some end-of-request stuff with it. The problem is I can't reference the variable in my onRequestEnd event because it produces an error that says it's not defined in the scope... but here's where it gets really weird and why I KNOW this is a bug (it's not just a suspicion)... If I DUMP the variable, the cfdump tag successfully displays the CFC and all its stuff... and then produces the same "is undefined in scope" error. Here's a screen-capture.

So... anybody seen this before? 'Cause I'm totally stumped. I've already installed the 9.0.1 updater and both of the cumulative hot fixes.

p.s. Yeah, I know it says OnRequestEnd.cfm, but this is actually inside the Application.cfc onRequestEnd method -- it's a legacy from the framework dating back originally to CF5, just go with it. ;P


Okay, it just got weirder... I tried using evaluate() to set it to a local variable, which apparently works because then I dump the local variable and the dump still works even though it's now on line 2 AFTER line 1 where the error is still occurring.


EDIT: Here's the code from the Application.cfc that includes the file where the error occurs:

<cffunction name="onRequestEnd" access="public" output="true">
<cfinclude template="OnRequestEnd.cfm">
</cffunction>

It appears to have something to do with the combination of the method and the include file.

It still fails if I execute the method in the onRequestStart like this:

<cffunction name="onRequestStart" access="public" output="true">
<cfset onRequestEnd() />
</cffunction>

But it works fine if I include the file in onRequestStart like this.

<cffunction name="onRequestStart" access="public" output="true">
<cfinclude template="OnRequestEnd.cfm">
</cffunction>

HOWEVER! There's obviously more to this because I can't create a simple test case. If I create a new project with a very simple application.cfc in it and replicate all these details, it works fine. So there's something else in the framework that's contributing to it beyond just the method names and file names.

And the file name doesn't seem to actually contribute, since it still fails in the same way if I change the name of the file like this:

<cffunction name="onRequestEnd" access="public" output="true">
<cfinclude template="reqend.cfm">
</cffunction>

Okay it has nothing to do with the file... well it does, but not with the file name... At the bottom of the onRequestEnd.cfm is this code

<!--- this seems to help resolve a leaky-memory issue in CF/JRun --->
<cfset structClear(variables) />
<cfset structClear(request) />
<cfabort />

If I comment out those StructClear statements, then the error goes away, which told me that it was executing the OnRequestEnd.cfm twice... and I THINK that means that CF9 changed the behavior of the CFABORT tag and it now executes the onRequestEnd event when the tag is used... it didn't in previous versions of cf...

I didn't find documentation of this, but I did find this blog from Ben Nadel that mentions it also occurring with CFLOCATION, so it seems to be more generally about the onRequestEnd event. It seems now in CF9, no matter how a page finishes executing, the onRequestEnd event is executed at the end... That's a change from all the previous versions and so it mucks with code I had created in order to actually cause this to happen on previous versions. Since I was causing execution of the onRequestEnd and then aborting the page early, CF now executes the onRequestEnd, aborts and then executes the onRequestEnd again because of the abort. (I tested to see if this happens with an OnRequestEnd.cfm and it doesn't -- it's just the Application.cfc method.)

Luckily, this fairly simple bit of code at the top of the template seems to fix the problem:

<cfif structIsEmpty(request)>
   <cfexit method="exittemplate" />
</cfif>

ColdFusion 9 and the onTap Framework

I'd like to say that every upgrade is as seamless as the upgrade from ColdFusion 7 to 8, where I didn't have to do anything (that I remember). But of course they don't always turn out that way. Back when Macromedia released ColdFusion MX, myself and a number of other developers had to do a fair amount of work to remove references to structure variables named "this". It turns out that they've added a "local" scope in ColdFusion 9 and wouldn't you know, about the time I stopped using "this", I started using "local" instead. ;)

Ryan McIlmoyl gave me the heads up on the issue on the DataFaucet list. About a day and a half later, I've just finished up the basic modifications for CF9. (There were 3.5-thousand references to "local" in the core code, plus another 500 or so in the Members onTap plugin, with plenty of false-positives.) I don't actually know if the onTap framework is still working on CF9 yet, because I haven't installed it to test with yet. If you give it a shot, let me know how it goes for you.

One thing I'm glad of though, is the way I designed the ontap.cfc. This is a component with some basic functionality that was created before onMissingMethod() was added to the language, otherwise it might have looked a bit different. But in essence, the lack of onMissingMethod() is why the ontap.cfc has generic getters and setters (or accessors and mutators for you lingophiles out there). These are two private methods name getProperty() and setProperty() and two public methods named getValue() and setValue() instead of having loads of individual functions named getBlahBlah() and setYaddaYadda(). What's cool about this arrangement is that in all my CFCs, nowhere to I directly reference the structure that holds property values, which was named (you guessed it), "variables.local". Well, nowhere except in about 3 private methods in ontap.cfc. And because all my other CFCs faithfully relied on getProperty() and setProperty() instead of simply referencing local.x or local.y, that means I only had to change "variables.local" in about 3 places. This saved me HUGE headache. If I'd gone the other way and used all hard-coded (or more to the point hand-coded) getters and setters, I'd have had easily another couple thousand references to replace and another day's work on this update for CF9. I love it when a plan comes together! :)

I was actually thinking about this the other day too, because I was looking at a small application someone else wrote (I won't mention any names), where they used all hand-coded getters and setters in their code and I was trying to figure out how to use it for configuration. After about an hour or two of looking through the code for documentation of one of the features I realized that said feature was never completed. There were just 23 references to it hanging out in the code, not doing anything at all (other than passing a useless variable around). All those 23 references were mostly hand-coded getters and setters, plus a couple of form fields and param tags. And I was thinking to myself, how nice it is to know that, for the most part, that sort of thing doesn't happen in my applications. Sure I make mistakes, everyone does, but the point here is, if I'm writing config for an app, there's not going to be 23-lines of code hanging around for a single config value, there's probably going to be about 3 or 5 and that means not only less maintenance, but less work up-front as well. :)

So new zip archives of the core framework and of the Members onTap plugin are up on RIAForge.

Plugin Manager Fix and CacheBox Update

I just realized that there was a small issue with the Plugin Manager that was causing a problem for an upgrade install for CacheBox. When you go into the plugin manager and you select a new plugin to install, it was reporting the information for the installed plugin instead of the new plugin version waiting to be installed. So for example, if you already had CacheBox 0.9.1 installed and then you downloaded the plugin to install 0.9.8, it would say "Installing CacheBox 0.9.1" despite the fact that you were actually installing 0.9.8. And because of that also when it finished installing, it wasn't recording the new version in the plugin manifest, so it would still show as the upgrade having not yet been installed.

So I fixed this in the Plugin Manager and uploaded a new copy of the framework core.

This version of the core also includes a new version of the CacheBox Agent that was required to fix a small bug with the CacheBox service. Unless you have the CacheBox service already installed, it shouldn't affect you. If you've installed it, you might get an error from the onTap framework until you update your core files. So I would update the onTap framework first and then CacheBox.

Minor Updates and a small Announcement

So a week or so ago I published a new version of the framework core with a minor enhancement to improve on the workaround for web-services/onRequest that may resolve the current issues with running the framework on BlueDragon/Railo. At the time I didn't realize that I had introduced a new bug (albeit small) that prevented the onMissingTemplate() feature from working, so any page request would require the base template in addition to the process code in the /_tap/ directory. This is true for ColdFusion 7 anyway, but on CF8 you're supposed to be able to omit the base template and let the server automate it (except for index.cfm).

Anyway in the process of working on some integration for a project for Eric Jones, I discovered this bug as well as a couple of other minor bugs in the Members onTap plugin that were introduced since the recent upgrades for new DataFaucet features. The member plugin bugs only occurred if an administrator attempted to edit their own account via the admin area (instead of the member area). Admins aren't supposed to be able to edit their own roles (because they can't add roles they don't already have themselves and shouldn't be allowed to de-permission themselves), so there's no really powerful reason for an admin to use the admin area to edit their own profile anyway. But it ought to work, so I've fixed those bugs as well... one of which resulted in the admin de-permissioning themselves when they saved the form. D'oh! ;)

Also the integration for Jonese's project is going pretty darned well! I've been able to bypass the IoC for the member plugin to introduce a custom member object which extends the default member object and adds a 2nd table with some additional profile information to the member objects for this project. That in itself is something I'm rather happy with because of how seamless it is. :) I'll be posting more blogs later with additional details about how the integration is done, as well as more information about the project itself as an example of the onTap framework and DataFaucet in the wild. ;)

On a more personal note, I honestly don't think that "who's using this framework?" is ever really a good question to ask. I mean... it's not that asking it is a bad thing, but rather that it's simply unhelpful and especially using it to make decisions can be counter-productive. And I'll share briefly the reason why I feel this way.

The Fusebox framework has been around for a good long while now and there are a variety of mixed feelings about it in the community from the die-hard Fusebox enthusiast to the extreme opposite end of people who really just can't stand it. Each of them has their reasons... but what I'm really getting at here is that Fusebox projects are much the same way. I've never been a big fan of Fusebox myself, otherwise I wouldn't have created the onTap framework. ;) But despite my own preferences I'm reasonable enough to acknowledge that of the large number of Fusebox applications there are in the wild, some of them are well made and some of them are poorly made. Whether a given Fusebox application is good or bad depends to a much greater extent on who managed the project and what resources they had available than it does on the fact that they used Fusebox as their foundation.

I expect the same will be true of just about any framework. There are going to be good and bad Model-Glue apps and good and bad ColdBox apps and I'm sure there will be good and bad onTap framework apps. The fact that the framework isn't being used at Wells-Fargo or on the website for the Metropolitan Opera says absolutely NOTHING about the quality of the framework. The only thing it says is that I've done a poor job of marketing it thus far so it hasn't gained traction (and this is shown to be the case in our frameworks survey also, which I'll talk more about later).

That's the larger reason why I'm now focusing a lot of my time and energy toward being more vocal in the community and improving the marketing. And looking for an evangelist to take the reigns some in the marketing department. :)

New Members onTap Plugin Build

Oops! It looks like when I made some recent changes to the member management plugin to take advantage of some of the new features in DataFaucet, I somehow accidentally got the addManyToManyRelationship() call into the member.cfc twice. Which in my case caused an error when I tried to install with a clean database. I guess I also neglected to test the installation into a fresh database, so that's my bad... There should probably be mxUnit or some other testing framework for these things, but TDD has been on my growing "when I have time" list for a while. In any event it was easy enough to fix, so I fixed it and uploaded a new build both to RIAForge and to the web service. So if you've been unable to install the Members onTap plugin recently, you can follow these steps to download the new build and try the installation again.

  1. open up your plugin manager (http://[ontap]/admin/plugins)
  2. select the "more" tab
  3. hit "search"
  4. find the Members onTap plugin in the list and hit "install"

Fix for new Plugin Manager Enhancement

Well it seems I committed what I've described in the past as a "cardinal sin" when using the onTap framework. That is, I created a distribution that included a file that would be modified during or after installation.

The reason why this is a cardinal sin when using the onTap framework is precisely because of what I ran into today. I had made a couple of what I thought were minor changes to some code, but was having a problem with some other code and even though I didn't think they were related, I decided to "roll back" the changes just in case. It turns out they actually weren't related, so the small changes I made stayed in the code. However because my method of rollback involved unzipping the framework core archive (ontap32.zip) into my working directory, it overwrote the new XML file that the Plugin Manager uses to store version information for the previously installed plugins... And of course, like magic that meant it was suddenly interpreting two plugins as uninstalled when they were in fact installed.

Thing is, it's supposed to be safe to extract a newer version of the framework into your working directory. Because one of the framework's guiding principals is "never edit someone else's file", that should apply to the framework core equally as much as to those of us working with is. So just as I should never edit one of your files (and you should never edit one of mine), the framework should never overwrite your data files either.

So long story short there's a new copy of the framework core archive up on RIAForge that's now minus the Plugin Manager's new XML file so that when you update it won't be overwritten. However there's no real urgency for anyone to update the core at this point. Unless your copy is already tragically old. ;)

MySQL and the Member Plugin

Rob Parkhill alerted me a few days ago to difficulties installing the Members onTap plugin with MySQL. There turn out to have been a couple of issues with MySQL, one of them required an update of DataFaucet and the other required an update of the member plugin. I was finally able to get around to installing a new copy of MySQL on my notebook to test and fix these yesterday. I blogged about these issues on the DataFaucet blog and uploaded a new build of both DataFaucet and the member plugin. The DataFaucet update is a download from the RIAforge project. To get the member plugin, just use the webservice in your plugin manager at http://yoursite/admin/plugins/.

ColdFusion Debugging

Rob Parkhill recently reported to me that having the ColdFusion debugging enabled in his ColdFusion administrator caused the navigation pane of the documentation to timeout.

The navigation pane renders fine with debugging disabled on that machine.

We're still not entirely certain what the problem is, this doesn't occur in every environment and we suspect it may be related to a specific debugging configuration, but have yet to isolate the particular switches in question.

Once we have a better idea of the specific cause, we'll add some information about it to the blog, wiki and release notes.

Thanks!

Small Select Bug Fix

I'm reminded that when ColdFusion 7 was released, it shipped with a small bug in the XForms implementation which caused multiple-select input elements to display with none selected if a user had previously selected multiple options. It had turned out to be something fairly simple, an oversight in their default XSL sheet and at the time I'd provided the fix for Jeff Small (although at the moment I can't find his blog entry where he posted it for everyone else -- send me the url if you have it and I'll post it here). I'm sure it was something the folks on the CF development team just hadn't antiscipated, after all how often do you use a multiple-select? (Of course, I say this expecting a quick response from the couple of folks who use them frequently.) :)

It's sort of come back to haunt me... In the previous versions of the onTap framework, I'd built in query-driven select input elements. The first iteration of these were unfortunately painfully slow. The problem at the time was that I was making it create a whole new html element structure for each option, instead of simply holding on to the query until display time. It turned out to be far too much overhead, and just as well most of the data in those structures was going unused anyway. So I then moved to the more logical approach of storing the query in the element structure for the select input and looping over them to create the options at time of display. This worked pretty well and even allowed me an easier way of fetching the current value of the select input by using the native ValueList() function. However, when setting the current value, I was still looping over each record in the query, like this:

<cfloop query="input.query">
<!--- get the value of the current option --->
<cfset thisvalue = input.query.inputvalue[currentrow] />
<!--- check to see if it's selected --->
<cfset isselected = ListFindNoCase(input.selected,thisvalue) />
<!--- reset the option selection --->
<cfset input.query.selected[currentrow] = iif(isselected,"thisvalue",de("")) />
</cfloop>

You can see from this code how I can then use ValueList(input.query.inputvalue) to fetch the list of currently selected options. Well this was great, and then shortly after I started working on version 3.0 of the framework I realized that there may be an even more efficient way. Instead of looping over the query one record at a time, I could set the entire query all at once and it could potentially be much more efficient. Something like this:

<!--- reset all the options --->
<cfset ArraySet(input.query["selected"],"",1,input.query.recordcount) />
<!--- get the indexes of the selected options --->
<cfset selected = getSelectedIndexes(input.query,input.selected) />
<!--- set the selected flag for selected options --->
<cfloop index="i" list="#selected#">
<cfset input.query.selected[i] = input.query.inputvalue[i] />
</cfloop>

So in this case the loop only processes one iteration for each selected option, instead of for each record in the query. In some cases where there are a lot of options in the list this can be much more efficient. I tested this after I implemented it and it is much more efficient when the select list is excessively large: hundreds of options or more, which may not be the best idea in the first place, but that's another blog. For most uses though, the efficiency probably won't change much from the last version.

The problem lies in that getSelectedIndexes() function (which is a simplification for the sake of the blog -- suffice to say that it creates a list of query indexes that should be selected without looping over the query). The algorithm I had for determining which query indexes were selected turned out to have a minor flaw in it and so where I had a query containing the values "en,en_CA,en_US" (and a few others), it was selecting all options beginning with "en", so the options for CA and US were being additionally selected when only "en" should have been. Of course I fixed this yesterday and uploaded a new build of the framework core code a few minutes ago.

I will give one caveat and that's the form tools do assume that you won't have any linefeed characters (ASCII char(10)) in your option values. So if for some reason you need that particular character in your data, you'll have to replace them with something else before displaying the form and after submitting it. Probably most people won't have any issues with that, since the inputvalue column will usually contain either integers, UUID strings or some other equivalent. Speaking of which, this is probably why I didn't catch this sooner, because I tend not to use autonumber / identity columns (though they're better supported now), and since most of my select elements contain a UUID equivalent, the values generally don't offer that kind of overlap.

More Entries

BlogCFC was created by Raymond Camden. This blog is running version 5.5.006. | Protected by Akismet | Blog with WordPress