ColdFusion Unit Testing Frameworks

April 19, 2008

I am starting from scratch, putting in a unit testing framework, and working within the existing organization’s culture to introduce enough of the tool’s benefits without burdening the organization with too many of the downsides. Everything has a downside, a cost, or whatever you wish to call it. There is no free lunch.
The first steps were to gage the organization’s desire for the possible benefits of a unit testing framework, and their willingness to commit to moving forward, and accepting the cost. Having been in an “extreme” programming environment, which was only “extreme” in the sense that projects were “extremely” haphazard and “extremely” mismanaged, making sure that people are clear on what can be expected from implementing some test automation for unit testing and regression testing. Automated does not mean that you do nothing, and the machine does everything. It is difficult to know what people have been sold by the trade journals, and blogs, and common terms are often misappropriated.
The current environment is Flex/Actionscript and HTML, with a ColdFusion backend, with a ColdSpring framework. The goals are:
1) Automated testing with Ant, with performance metrics logged in a database, and pass/fail results e-mailed nightly to interested parties. (Thorough regression testing and reporting is key).
2) Eclipse Integration or reasonably nice Unit Runner, with easy ability to select individual tests, or test suites. (Easy sell to the engineers is key)
3) Mock Objects, automated test generation, anything to minimize the time impact on engineering, while maximizing the benefit, and keeping the program moving.

Sean Corfield had mentioned MXUnit fairly recently, so it seemed worth a look. Given the higher volume of activity on MXUnit, when compared to CFCUnit and CFUnit, it seemed like a reasonable way to go, for the best of all worlds for future itegrated add-ons. The first downside I will mention is that *MXUnit‘s Eclipse Integration is Not Flex Builder 2.01 Compatible, or rather, not compatible with the version of Eclipse (3.11) that ships with Flex Builder 2.01*  This wasn’t super clear from the  MXUnit website, so it’s possible for an engineer to go through the installation instructions, and end up with a failing plug in.  This might not be a problem for some organizations, but may be for others.  Most of the team has not yet upgraded to FB 3, so it could be an issue at the moment.

  MXUnit’s Eclipse integration isn’t quite finished, or at least that’s how it felt for me.  It “worked”, but was lacking a few things that I thought were important.  An html or Flex based unit runner will do the trick though, so it’s not a huge problem.  The things which make the integrated unit runner a little less than usable, for me, are: 1) It seems to have no memory of past tests, so I would have to go through the selection process for the unit test every time; some way to save the available test cases/test suites would be desireable/required, 2) The test case selection process was awkward; I found myself unable to select a Test Case which I knew was present on more than one occasion;  This could be forgiven more easily, if it would remember the test cases I had selected in the past, but I was required to select/find the test case with each restart of Eclipse, or any time I wanted to switch tests

On the plus side, the ant unit runner does a great job, and it is clear how to modify the ant task.  The availability of the different test result formats, allows me to feed the results into a variety of destinations with ease.  The key item was logging test results and performance metrics, and not just logging into a text file, but logging into a database table, so it could be consulted for history.  I was able to hack this item in really easily using the cfquery output format for the testResults.

I should at least mention that CFUnit and CFCUnit are both seemingly capable frameworks.  I was able to set up integrated Eclipse (with Flex Builder 2.01!) unit tests, and test suites.  But the availability of the test Result Formats, and the volume of project activity, were enough to convince me.  That does not mean that I am right, nor does it mean it is the best choice for everyone.  But the software selection is but one small piece of the puzzlement.

ColdFusion Unit Testing Frameworks, and related info:

Next steps, automating test generation, designing effective unit tests, and Flex Unit testing frameworks.  I’ll post if anything interesting comes up.

 

Advertisements

ColdFusion 8 ValueObjects vs Typed Structs for Flex RemoteObjects

March 28, 2008

ColdFusion Objects transfer nicely over to Flex, making the remote data connection for Flex run like a dream.  It can get out of hand, however, when there are a large number of Objects to return from ColdFusion, for example, an array of ValueObjects, which might have other value objects as members.  In researching this, I ran across a few very helpful postings.  Basically, CF8 has a workaround wherein you use typed Structs instead of full blown Objects to pass VO’s to Flex:

http://www.briankotek.com/blog/index.cfm/2008/1/28/Returning-Typed-Structs-vs-CFCs-to-Flex

http://groups.google.com/group/transfer-dev/browse_thread/thread/0abdb2191dcfd226#2cbf827754e75376


So, I decided to run a few very simple tests to see what the impact of CreateObject was on a large dataset.  I put together a CF script with a loop to create 5000 objects using CreateObject (with init), another for 5000 with CreateObject (without init), another to simply duplicate a template Object and another to create 5000 structs.  The breakdown and the code follow:
 

Test

Iterations Elapsed Time (ms) ms/Instance
structTest 5000 157 0.0314
FullCreateTest 5000 17468 3.4936
createTest 5000 16610 3.322
dupeTest 5000 12469 2.4938
structTest 5000 15 0.003


This was run on a CF8 Box with JRun running on Java 1.5 I had similar results on a CFMX 7 Box with JRun running on Java 1.42 The option of rolling query results into typed structs is really a lifesaver given the performance numbers.       

<cfset t1 = GetTickCount() />

<cfset myArray=ArrayNew(1)>
<cfloop from=”1″ to=”5000″ index=”i”>
<cfset newStruct=StructNew()>
<cfset newStruct[“iter”] = #i# />
<cfset ArrayAppend(myArray,newStruct) />
</cfloop>
<cfset t2 = GetTickCount() />
<cfset elapse1 = t2 – t1 />
<cfset myArray2=ArrayNew(1)>

<cfloop from=”1″ to=”5000″ index=”i”>
<cfset newObj=CreateObject(“component”,”com.test.TestIter”) />
<cfset newObj[“iter”] = #i# />
<cfset ArrayAppend(myArray2,newObj) />
</cfloop>
<cfset t3 = GetTickCount() />
<cfset elapse2 = t3 – t2 />
<cfset myArray3=ArrayNew(1)>

<cfloop from=”1″ to=”5000″ index=”i”>
<cfset newObj2=CreateObject(“component”,”com.test.TestIter”).init() />
<cfset newObj2[“iter”] = #i# />
<cfset ArrayAppend(myArray3,newObj2) />
</cfloop>
<cfset t4 = GetTickCount() />
<cfset elapse3 = t4 – t3 />
<cfoutput>Results<br/>Structs : #elapse1# ms for 5000</br>

CreateObject (no init) #elapse2# ms for 5000<br/>CreateObject (with init) #elapse3# ms for 5000<br/></cfoutput>

<cfcomponent name="TestIter"> 
      <cfproperty name="iter" type="numeric"> 
      <cffunction name="init"> 
            <cfreturn this /> 
      </cffunction> 
</cfcomponent>

CFSCRIPT – The little language that …

November 18, 2007

To quote Ben Forta in 2004: “<CFSCRIPT> today is in a strange place. ” (http://www.forta.com/blog/index.cfm?mode=e&entry=1416) I don’t think much has changed in that regard. The fact is, Macromedia ColdFusion stopped partway through the metamorphosis of becoming a scripting language. Kind of like stopping part way through a sex change; the important parts are there, sometimes both sets, and it can be ugly to watch. Now it’s in Adobe’s care, and there are more than just PHP and ASP wandering by and looking at CFSCRIPT, with a mild sense of relief (glad that’s not me) and perhaps a bit of Schadenfreude.

I’ve read articles about Coldfusion programmers jumping ship and running to Ruby on Rails, I wish everyone the best voyage possible, whichever boat they sail. I spent years coding C++ as a plugin to Netscape, to serve up web pages. Then Apache Server. Coldfusion looked like a treat from that vantage point. Despite it’s imperfections, it is a powerful way to make many web applications happen very quickly. That doesn’t change the fact that CFSCRIPT could never run on it’s own. It’s incomplete. Like Pinnoccio, always needing Gepetto around to keep working on him, and no Blue Fairie coming to turn him into a real boy. Yes, CFSCRIPT is in a strange place, and it will likely see out its days in the place that it is.

There is an almost audible sigh of relief that escapes when entering a block of logic which won’t be needing html tags or CFML to make it do it’s job. But eventually the lines of code free of the greater than and less than signs is brought to a screeching halt by one of those things that cfscript just cannot do properly, or requires the esoteric approaches of the guru to get through.

I noticed Sean Corfield’s scripting plugin post, basically allowing a Coldfusion 8 developer to plop in some PHP here, some Ruby there. (a moody person could write alternative verses of PHP and Lua, bringing a Wagnerian feeling to the code). A colleague has been telling me about Groovy, and its possibilities and I stand intrigued. Something will have to yield, one way or another. For things CF could not handle, I used to opt to extend CF with C++ CFX tags. Now, with CF being Java based, it is cleaner to extend with Java (my opinion). Can Groovy solve some problems that CFSCRIPT could not? Is it compelling enough to justify a significant effort in that direction? Groovy and Hibernate play well together (supposedly). I may do a little with Groovy’s Gant (to get past the Ant XML build structure), but beyond that I don’t know. Groovy is too new for risking on my customers, for the time being. But what the future holds, I cannot tell.

As for the rest, I am grateful for having had the chance to work with all of the languages I have. I wish every programmer could have a wide exposure to languages in that way. I feel the same way about natural languages (written and spoken and signed-eg. American Sign Language). The more one can learn about other languages, and their inherent approaches to solving problems, and their implications, the better one can understand and utilize their home language. And though I make fun of different computer languages, I enjoy working with them a great deal.

After doing some work with Lex/Yacc for preprocessing of some code, I did some experimentation with the problem of carrying the cfscript language the distance, by adding “real” classes, to replace the tag based cfcomponent artifact.    Then, adding to that some special system classes that could yield database connectivity and scaffolded views, and cfscript is more complete, operationally.  None of this changes the fact that  CF is locked behind a webserver, and there is no past nor any future for CF based console apps or desktop apps.  It is a backend database application server for the web, when it is doing what it does best.

It is difficult *not* to conclude that Adobe (and perhaps Macromedia) purposefully prevented the completion of the ColdFusion CFSCRIPT “language”.  And given that, it would be interesting to understand exactly why, though one could guess that they don’t want to be a big player in that marketspace, to distract their focus from other more interesting areas.  For the developer who continues to hitch his career on ColdFusion, I would admonish them to add a few additional tools into their toolboxes.


Get Outta my namespace (revisited)

November 18, 2007

Earlier, I discussed some of the problems/issues with Flex Builder’s default configuration of outputting the swf into the “bin” folder. (see Get Outta My Namespace). Operationally, the dot Net server’s claim to the “bin” folder in server root context was one of the roadblocks to use of the “bin” folder. It’s easy enough to use another name, to avoid conditions like this one.

Well, I was recently concerned with the performance of a web application, running on IIS on the JRun4 J2EE server distributed with ColdFusion. I decided to try running the application on a JBoss server, with ColdFusion 8, and do some performance benchmarking, and analysis, to see if the problem’s root was in JRun’s management of the memory objects, or in the tuning of the JRun server, or if the change in platform would make a difference at all. The performance problem is likely rooted in the application’s architecture, and the database configuration, but it was worthwhile and easy enough to see if there was any difference in changing the platform this way.

I have a JBoss 4.05 Server running Tomcat and ColdFusion 8, basically following the easy instructions at Steve Brownlee’s blog when I had first set it up with ColdFusion7, then updating it with ColdFusion 8, using the advice from Adobe. When I attempted to use the Flex application deployed in a “bin” folder which was *not* in the root, that is it was several subfolders away from the root, the server acted as if the html container and the swf file were simply not there. I tried several things before eventually simply renaming the deploy folder to “out”. Immediately, the server recognized the deployed html and swf. I haven’t yet determined what exactly is going on, but I know the bin folder has various application specific uses for JBoss and Tomcat. It seems as though this is interefering, once again, with the default “bin” folder output for Flex deployment. I’ve updated the project, locally, to build to “out” and mapped the application to “out” and it is all working just fine.

As for benchmarking performance, I have started. I’ll try to post anything interesting I might find.


CF8 in Notepad++ (Continued)

October 13, 2007

I’ve uploaded a new package to:

http://labs.soundrage.com/CF8NPP.zip

It includes a chm for CF8 help, the bulk of the size. It’s servicable, but the conversion from html to chm was not perfect.

It includes a script for Inno Setup, which I won’t be able to get back to for a while, so if someone is interested in picking that up, feel free. Otherwise, I’ll get to it when I get to it.  I have it kind of working but for a setup package, that’s really not good enough, and there is a lot more work to be done on it.

The key items of interest are:

1) The shortcuts.xml has a line for launching keyword sensitive help right from the editor to Adobe’s LiveDoc’s or to the local help file. I had to bump the php help off of the F1 key for my own setup, but feel free to do whatever you wish. CTRL-F1 and ALT-F1 launch the help, and highlight the keyword. The shortcuts.xml file needs to be in the folder with your Notepad++.exe, default is C:\Program Files\Notepad++\

2) The keywords themselves are identified in the a\apis\coldfusion8.api file.

3)  The (User App)\Notepad++\InsertExt.ini needs to contain regular expressions to mark the beginning and ending of a function, and how to pick out the function name.  These will show up on the function list, so you can click on the function name and jump straight to it.

[User Languages]
Count User Languages=2
Name 1=ColdFusion8
KeyWBodyBeg 1=
KeyWBodyEnd 1=
MatchCase 1=0
Count Comment 1=3
Comment P1.1.1=
Comment P1.2.1=
Comment P2.2.1=
Count Syntax 1=2
Syntax P1.1.1= Syntax P2.1.1=\"\t.*\>
Syntax P3.1.1=[A-Za-z]*
Syntax P4.1.1=
Syntax P5.1.1=
Syntax P6.1.1=
Syntax P1.2.1=cffunction[\t ]*name[\t ]*\=[\t ]*\"
Syntax P2.2.1=
Syntax P3.2.1=[A-Za-z_]*
Syntax P4.2.1=
Syntax P5.2.1=
Syntax P6.2.1=

4)  Finally, our old friend, UserDefinedlanguages.xml, needs ColdFusion8 set up inside of it.  Basically, you take the contents of UserDefinedLanguages_Coldfusion8.xml, and insert it into the appropriate area inside of the real/active UserDefinedLanguages.xml.  This also sits in the (User App) folder, which is, for me, C:\Documents and Settings\Howard\Application Data\Notepad++\, so yours will be something like that.

I’m posting this over with the Notepad++ people, and will followup there.

For InnoSetup, information can be found at:

http://www.istool.org/    – The extension to InnoSetup, which adds some good stuff.

http://www.innosetup.org/    – The base InnoSetup package.

Hopefully, it will be helpful to someone.  I just enjoyed the diversion for a few hours.


Value Object Bustifiers

July 2, 2007

  I’ve been fashioning Brian Renaldi’s the <a href=”https://howardscholz.wordpress.com/wp-admin/code.google.com/p/cfcgenerator/&#8221; target=”_blank”>Illudium PU-36 Code Generator</a>  package to produce scaffolding for the Cairngorm framework.

People tend to have opinions about code generation, scaffolding being a euphemism for a specific portion of code generation.  I really like the dynamic scaffolding employed by Groovy/Grails and Ruby/Rails, wherin the functionality of the scaffolding is generated, but the code is not.  That seems pretty cool.  It would be Uber Cool to generate the entire application that way, and have no code except for that very special case stuff.  But even generated scaffolding serves its purpose, and helps a group development effort have some consistency.
Typically, what I do in my VO’s is have them accept a parameter that is of type object, and let the constructor act as a sort of copy constructor, if it is fed an object, otherwise leaving the default values.
A further item that is handy in scaffolding, is making the assignment case insensitive.  Coldfusion is case insenstive, and Actionscript, is case sensitive.  So those assignments can be troublespots sometimes.  Also, sometimes there may be a mismatch between the VO properties and the properties returned from the persistent store, so any mismatches are observed and traced to the console output.

To handle these items, I’ve added the following:

/**
* Default constructor.
*/
public function CategoryVO(data : Object = null) : void {
if (data == null) return;
var dataDesc:Object = ObjectUtil.getClassInfo(data);
var myDesc:Object = ObjectUtil.getClassInfo(this);
var props:Array = dataDesc.properties;
var myProps:Array = myDesc.properties;
var found:Boolean = false;
for each (var name:String in props) {
found = false;
for each (var myName:String in myProps) {
if (!found &amp;&amp; myName.toUpperCase() == name.toUpperCase() ) {
this[myName.toUpperCase()] = data[name.toUpperCase()];
found = true;
break;
}
}
if (!found)
trace("CategoryVO - Input Object has extra property " + name);
}

The business delegate layer I’ve seen done in a pretty boilerplate manner, where it acts as kind of a pass through layer to the Commands.  More correctly, any data conversion that may be platform specific, like Coldfusion or PHP related Object constructs, should be handled in the business delegate layer, isolating the platform details from the remainder of the Actionscript code.  That makes this layer a little more interesting for matters of code generation.


ColdFusion 7.02 profiling with Netbeans 5.5

June 26, 2007

I am in the position of needing to check out what is going on in the memory of the JVM instance running a particular process. I need to see if it’s being release properly (no memory leaks), and get an idea of when the process will top out (how many requests). So, I tried out FusionReactor, and that gave me a decent picture of things (http://www.fusion-reactor.com). And I decided to give JFluid a whirl. There are lots of good blogs, and pages by Sun on this, so have a look around, if you are planning to set this up. Here are a couple of links that were helpful to me:

I had some difficulty before getting JFluid to run (links later) and didn’t have a pressing need. But now I really wanted to have a good look. And when I did look, I found that JFluid had been "promoted" – the once experimental package had been packaged for NetBeans, the old pages for JFluid forwarding to the new pages. The package is now officially known as NetBeans Profiler (http://profiler.netbeans.org/index.html) It supports JVM 1.42, which is what we are running CF 7.02 on, so it will work well. It also supports JVM 1.41, 1.5, but each flavor with a different manner of support. In addition to the JVM version support having nuanced differences, the Netbeans platforms each have their own specific packages (Netbeans 4.1, 5.0, 5.5, 5.51 ). I run Netbeans 5.5 right now, so that’s what I got, in three downloads, the three parts to the package. One part is a modified jvm.dll for JVM 1.42 (Windoze) or .so for Unix platforms, amidst caveats that Sun does not support the 1.42 build, that it is presented "as is." Another part is the profileattach.dll’s (or .so) for attaching the profiler to the jvm. And lastly there is the runtime. Read Sun’s page carefully, as it’s easy to miss something. I did not get dynamic attachment working, but rather, Direct Attachment, which is actually better for what I am doing. I have to launch the Coldfusion server from the command line, but I see the system out messages. Netbeans has "wizard" for setting up the profiling, which presents these options. At the end I had these instructions:

Step 1. Add the profiler native libraries to the Path, e.g. using   

SET Path=C:Program Filesnetbeans-5.5profiler1libdeployedjdk142windows;%Path%   

Step 2. Setup the server to run on modified Profiler 1.4.2 JDK, which can be found in   

C:Documents and Settingshoward.netbeans5.5modulesprofiler-ea-vm   

This typically means changing system/environment variable JAVA_HOME as follows:   

SET JAVA_HOME=C:Documents and Settingshoward.netbeans5.5modulesprofiler-ea-vm   

Step 3. When starting the server, provide extra startup option to the java command:   

-Xrunprofilerinterface:"’"C:Program Filesnetbeans-5.5profiler1lib’"",5140   

Step 4. The JVM will start, but will not proceed with server execution until you connect the profiler to it.

Note: I had to remove the -XX:UseParallelGC setting before it would run. Once these changes have been made, I can launch the CF Server with "jrun -start cfusion" and it will pause, waiting for the profiler to attach. In netbeans, I attach the profiler, and I start seeing info. Netbeans Profiler Screenshot Sure looks pretty, but it doesn’t do much for me unless I can interpret the data, and explain what it means. So, the first step is getting some baseline measures on processing and proceed from there. (continued) Netbeans Profiler Screenshot