Thursday, November 29, 2012

Quick ColdFusion Tip: Use ClearParams() When Doing SQL Inserts in a Loop

Today I had the need to write code that would loop over a recordset, parse the data in each iteration, and then insert the transformed data into another table.  Nothing I haven't done countless times, but this time I was using the query functions in cfscript to do my inserts.  I knew the resulting code was going to be a bit slow (as inserting thousands of rows takes time), but when I ran it it was a LOT slower than I expected.

After adding some code to time the various parts of my routine, I discovered that the time it took for each insert transaction was steadily growing with each iteration of the loop, to the point where it was taking 500+ milliseconds per insert.  But why?

Then I saw the problem.  I had forgotten to invoke the query object's clearParams() function at either the beginning or end of my loop.  Apparently ColdFusion will let you create a query parameter with the same name attribute using addParam() - as was happening in my loop - and not throw an error (which is what I would have expected to happen), but it leads to a performance issue with the SQL execution.

In the few times where I've reused a query object with different parameters, I've been careful to use clearParams(), but I simply overlooked it this time.  Lesson learned.

 

Thursday, October 25, 2012

Adding a Time Selector to the jQuery UI Datepicker Widget

One of the functional requirements for the voter registration application I blogged about recently was that the application should not allow further registrations between the registration deadline (October 16 at 9pm) and a date after the election specified by the state Board of Elections.  For the initial run of the application, I simply hard-coded the deadline and restart date into the application logic, knowing full well I couldn't leave it that way unless I wanted to personally change the code year after year....which I don't.

So this week I set out to write a tool within the administrative interface of the application that would allow a non-programmer to update the deadline and restart date every year.  The jQuery UI Datepicker widget is my tool of choice when it comes to having users enter or edit a date, but I've used a few different approaches to having users enter a time of day.  This time around, I decided to see I if could find something comparable to the Datepicker widget for setting the time.

What I found was a rather sweet plugin called the Timepicker Addon that adds a set of time controls to the jQuery UI Datepicker.  If you customize your jQuery UI download to include both the Slider and Datepicker widgets, you can present the time controls as sliders, like so (without the Slider widget, you get select boxes):

The plugin comes with a number of configuration settings so you can do things like adjust the time increments, change how the time is displayed, and allow the user to denote the time zone associated with the time value.  Once I had the plugin configured the way I wanted, I simply had to write some code to validate the date and time string submitted from the form field, and I was done.  Very cool.

Tuesday, October 23, 2012

Simple Triangles with CSS

Recently, I had to create JavaScript-based replacements for 2 Java applets in an application I inherited.  One of the applets was a collapsible hierarchy tree of organizational units that used the conventional triangle icons (pointing right for closed, pointing down for open) to indicate state.

In past applications, I'd simply created actual triangle icons of the appropriate color for this sort of thing, but this time I decided I wanted to do it in pure CSS. 

After a quick search on the web (The Shapes of CSS is a nice resource on this topic) and a bit of trial-and-error, I came up with some CSS triangles that offset nicely from their tree items (and coded a blank triangle for items without children).  They work in all the common modern browsers and in IE as far back as IE 7:

.arrowNone {
  border-left: 5px solid transparent;
  border-right: 5px solid transparent;
  border-top: 10px solid transparent;
  float:left;
  height: 0; 
  margin-right:5px;
  width: 0; 	
}

.arrowOpen {
  border-left: 5px solid transparent;
  border-right: 5px solid transparent;
  border-top: 10px solid #666;
  float:left;
  height: 0; 
  margin-right:5px;
  margin-top:6px;
  width: 0; 
}

.arrowClosed {
  border-bottom: 5px solid transparent;
  border-left: 10px solid #666;
  border-top: 5px solid transparent;
  float:left;
  height: 0; 
  margin-right:5px;
  margin-top:4px;
  width: 0; 
}

Granted, they're not the smoothest of triangles, but they worked well enough for this purpose.

Friday, October 12, 2012

Getting Out the Vote, ColdFusion-Style

Normally I don't blog about the actual ColdFusion web applications I create at the University of Maryland College Park (UMCP).  Most of them are extranet applications inaccessible to the public and computerize business processes that would take too long to explain.  But my most recent application doesn't fit the normal mold and it's received some attention from the local media because of what it does.

The application is an online application form for registering to vote in the State of Maryland in November designed specifically for our students at the university.  It was commissioned by our undergraduate Student Government Association (SGA) as part of a larger, multi-faceted effort to get out the vote this year, and supported by the university administration and the State Board of Elections (who provided the guidelines for how the data would be collected and sent to them).

The application runs on Adobe ColdFusion 9 and uses FW/1 as its application framework.  The model is comprised of record and service objects, all written in script syntax, with the service objects instantiated via ColdSpring.  Most of the client and server-side validation is handled by ValidateThis, and the application is styled with Bootstrap as well as some custom CSS and a bit of jQuery.

After reading the introduction page, students log into the application using our university's single-sign on solution.  As soon as they're logged in, the system pulls information about the student from our LDAP system.  That information is used to pre-populate certain form fields (name, date of birth, etc.) and it also affects the behavior of the application.  For example, if the LDAP information indicates that the student has a home ("permanent") address in Maryland that is different from their current on-campus / near-campus address, it will show both addresses to the student and let them choose which one they want to use as their residential/voting address. 

But if the student only has one Maryland address on file, they're only presented with one set of address form fields.  And in both cases, the addresses pulled from LDAP are formatted to conform to the way the state recognizes voter street addresses, making it more likely that the submitted address exactly matches an address on file with the state (that part took awhile: the state parses street addresses in a rather unique manner).

In short:  if the LDAP information is accurate and up-to-date, most students can complete the online registration without ever typing a key on the keyboard.

Perhaps the coolest part of the application is the last step prior to the review page.  In that step, the student has to acknowledge that they understand the legal requirements to vote.  They acknowledge this by clicking on a checkbox next to an image of their electronic signature.  When students enroll at UMCP and get their student photo ID, they write their signature on an electronic pad, and the signature is stored in binary format in a database table.  It was the fact that we had these signatures on file that made this project viable, because the new Maryland law that permits online voter registration made the submission of a signature a mandatory requirement.  And thanks to ColdFusion's image manipulation functions, rendering that binary data as an image takes but one line of code.

Admittedly, we did run into one issue with the signatures:  some of the older ones were stored in a proprietary format that couldn't be read outside of the ID card system.  But we found an option in that software that let us export large batches of those signatures as JPEG files, and from there I used ColdFusion's image functions to reduce each image to a reasonable size and write them to a new database table in binary format.

Once the student reaches the review page and submits their voter registration application, the form data is written to a table record and their electronic signature is output as a file to a directory outside the web root.  And every night a scheduled task runs that writes out all the new registration records to a data file in the format specified by the state and uploads that file and the related signatures files to a server run by the state.

And, like most of the applications I build, there is also an administrative interface that certain individuals can log into to view some anonymous registration statistics and to update the list of politicial parties currently recognized by the state (as they change from year to year).

The system went live October 1st, and as of right now over 1650 students have used it to submit voter registration applications.  The SGA is promoting the application at every opportunity and they're hoping to get to 2,000 registrations before the October 16th deadline.  While that might not seem like a huge number, I believe it would be the highest number of on-campus registrations since the SGA started doing voter registration drives several years ago.

Once we hit the October 16th deadline, we'll shut down the application until after the election has past, then reopen it.  The goal is to let the system accept registrations year-round, and I already have a few small improvements planned based on user and stakeholder feedback.  I also need to add a tool or two to the administrative interface so the stakeholders can run the system without my help. 

But otherwise, I'm pretty happy with how it turned out, and so are the SGA and the other stakeholders.

Monday, August 13, 2012

Tip on Combining Column Filtering and Column Hiding with jQuery DataTables Plugin

I'm a big fan of the jQuery plugin DataTables.  I use it in a number of projects to enhance HTML tables, making them sortable and searchable.

DataTables provides a number of advanced options and functions that let you customize the table's functionality.  One of these functions is fnFilter(), which you can use to filter the table contents based on the presence of a value in a particular column.  The most common use case for this function is to add text inputs to the header or footer of each column, and bind the use of the function to the keyup event, as in the official DataTables example:

$("tfoot input").keyup( function () {
    /* Filter on the column (the index) of this element */
    oTable.fnFilter( this.value, $("tfoot input").index(this) );
} );

 

...The "oTable" is a reference to the DataTables-enhanced table, the first parameter is the value of the input box in the footer, and the second parameter is the index value (position) of the column in the table.

DataTables also provides the fnSetColumnVis() function for showing/hiding columns, which is helpful if you have a table with a lot of columns and want to let the user get rid of the columns they don't need at that moment.  It takes two parameters, the index position of the column you want to hide and true or false to set the visibility state of the column:

oTable.fnSetColumnVis( 1, false );

 

Both are very useful functions, but you have to be careful when you enable both column filtering and column hiding on a single table. Hiding a column takes it out of the DOM, and therefore it changes the index position of every column to the right of the hidden one. While that makes sense, the problem is that DataTables keeps internal track of the original DOM positions and will execute the filter against the column with that original index value.

So say you have a table with columns A, B, and C going left to right.  DOM index positions start with 0, so column A's position is 0, column B's is 1, etc..  If you hide column A, then column B now has index position 0 and column C has position 1.  When you try and filter column C, fnFilter gets the current position of column C (1) and runs the filter against the column that originally had index position 1...which is column B.  So the table gets filtered based on column B even though the user entered their filter term in the footer of column C, which is obviously not what you want.

You can work around this issue by recording the initial index position of each column in an attribute in the input tag, and use that value as the second parameter for fnFilter().  So if you used an attribute like "colPos":

...
    <tfoot>
        <tr>
            <th><input type="text" value=" colPos="0"/></th>
            <th><input type="text" value=" colPos="1"/></th>
            ...
         </tr>
    </tfoot>
...

 

...then you'd rewrite your keyup bind function like so:

$("tfoot input").keyup( function () {
    /* Filter on the column (the index) of this element */
    oTable.fnFilter( this.value, $(this).attr("colPos") );
} );

 

That will keep the column filters tied to their original columns.

Sunday, July 29, 2012

Technique For Adding Basic CSRF Protection in a FW/1 Application

Just over two years ago, I wrote a blog post about how I protected my web applications from cross-site request forgery (CSRF) attacks using Model-Glue's event types feature.  Recently, I've started building ColdFusion applications using the Framework One (FW/1) MVC framework, so I needed to come up with a new approach to try and block CSRF attacks. Ideally, I wanted to do it in such a way that once I had my CSRF protection set up, I didn't have to think about it anymore, that I wouldn't have to remember to add one or more lines of code to each action I wanted to safeguard (and risk overlooking an action or two).

First, a refresher on what CSRF is: it's an attack where the hacker is counting on the user still being logged in/authenticated to the web application he's targeting.  The hacker creates a web page that posts data to the target application using a known URL or form submission destination and lures the user to that page.  When the user triggers that page, whatever data that page sends to the target application is accepted because the target application sees it as a valid request from a valid user.  It's a serious vulnerability, which is why ColdFusion 10 comes with new functions designed specifically to help block CSRF and why those functions were included in the CFBackport project (an open-source project that makes some ColdFusion 10 functions available to ColdFusion 8 and 9).

The basic way to prevent CSRF attacks (or at least make them far less effective) is to generate a unique value that is stored in the user's current session, include that value in the URL or form submissions you want to protect, and then match the value in the URL or form submission to the value stored in session and make sure they match before allowing the action to proceed.  So unless the hacker can figure out what that unique value currently is for a particular user, they cannot mimic it in their fake web page and data from that page won't be accepted.

So first I added code to my controller function in charge of validating user authorization that would create a token variable in the session:

...
session.user= arguments.rc.user;
session.token= CreateUUID();
....


At this point, I could have manually started adding the token variable to URL strings or placed it in hidden form fields in my forms, but again I wanted my CSRF to be as "automatic" as possible.

In FW/1, the best practice for creating URLs for hyperlinks and form submissions within the application is to use the buildUrl() function built into the framework.  With buildUrl(), the following line of code:

<form name="addItem" method="post" action="#buildUrl(action='item.add',queryString='foo=2')#" >

 

...gets rendered as (assuming you've stuck with the FW/1 defaults)...

<form name="addItem" method="post" action="index.cfm?action=item.add&foo=2" >


The buildUrl() function lives inside the framework.cfc file that is the heart of FW/1.  Implementing the FW/1 framework in your application is as simple as changing your Application.cfc to extend that framework.cfc file.

I wanted buildUrl() to automatically incorporate my session.token (if it existed) in the resulting URL. So to do that, I created a subclass of framework.cfc called frameworkExt.cfc with the following code:

component extends="framework" {
    public string function buildURL( string action = '', string path = variables.magicBaseURL, any queryString = '', string tokenKey= "token") {

        //check for presence of tokenKey in session
        if(StructKeyExists(session,tokenKey)) {
            if(isStruct(arguments.queryString)) {
                arguments.queryString[arguments.tokenKey]= session[tokenKey];

            } else {

                if(arguments.queryString== "") {

                     //If the action has the url query elements hard-coded (which means queryString should be empty), append to that
                     if(ListLen(arguments.action,"?") GT 1) {
                         arguments.action &= "&#arguments.tokenKey#=#session[tokenKey]#";
                     } else {
                         arguments.queryString= "#arguments.tokenKey#=#session[tokenKey]#";
                     }

                } else if(ListLen(arguments.queryString,"?") EQ 1) {
                    arguments.queryString &= "?#arguments.tokenKey#=#session[tokenKey]#";

                } else {
                    arguments.queryString &= "&#arguments.tokenKey#=#session[tokenKey]#";
                }

            }
        }

        return super.buildUrl(arguments.action,arguments.path,arguments.queryString);
    }
}


Basically, I created a new version of buildURL() that acts as a kind of pre-processor to the original buildURL() function. If session.token exists, it will get incorporated into the URL just as if I had explicitly submitted it to buildURL() in either the action or queryString parameter, then transformed into the final URL by the original function in the super class.

(Some caveats here...I don't know for certain if this technique would work for every use case of buildURL(). It's a complex function capable of creating URLs in a number of different formats in order to produce SES-style links and point to subsystems. But I would think that since I'm simply adding in another URL variable that it ought to work in every case. Also, any time you write a function that modifies a framework function (even via extension) you run the risk of that framework function changing in a future version, potentially breaking your modification).

Once I changed my Application.cfc to extend my frameworkExt.cfc instead of framework.cfc and reloaded my application, the session token became part of all of the URLs I visited or invoked as an authenticated user.

The final step was to add code that would check the token value submitted in the URL or form against the value of session.token, and if it didn't redirect the request to an error or login page.  In Model-Glue, I could add that check as part of a message broadcast in an event type, and then apply that event type to any request where I wanted to enforce CSRF protection.  FW/1 doesn't have event types, but it has something similar:  in an FW/1 controller, you can create before() and after() functions that execute code (you guessed it) before and after the processing of each function in that controller.  So if I wanted to check the submitted token against the session token before every page request that invoked a function in my "widgets" controller, I could put that code in the before() function.

That would have been the conventional way to go about it, but I decided to do it a bit differently in my application.  Since I wanted to add CSRF security to any action undertaken by an authenticated user, and all of my unauthenticated requests were handled by a small set of controllers, I decided to do a conditional check on the session token in the setupRequest() function in Application.cfc (setupRequest() being a function inherited from framework.cfc that runs at the start of each request):

function setupRequest() {

    if(!ListFind("main,auth",getSection())) {
        controller("auth.checkSessionSecurity");
    }

    if(StructKeyExists(session,"user")) {

        if(!StructKeyExists(rc,"token") OR rc.token NEQ session.token) {
            controller("main.notAuthorized");
        }
        rc.user= session.user;
    }
    ...
}


The FW/1 getSection() function returns the section value of the request (in FW/1, the section equates to / matches up with a controller CFC of the same name), so the first if statement basically states that unless the request is invoking the main or auth controller, the action should redirected to the checkSessionSecurity() function of my auth.cfc controller, which in this application makes sure the user has authenticated and a user object exists in session.

If a user object does exist in session, that means session.token should exist (since it's added to the session at the same time the user object is). So the nested if statement verifies that a token variable was submitted via URL (and hence appears in FW/1's rc struct) and that its value equals that of session.token, and if either of those conditions fails then the user is kicked out to an error page and prevented from executing the original request.

So with this setup, I get protection from CSRF attacks simply by using buildURL() as normal and putting all of my sensitive controller functions in controllers other than main or auth.

Some Additional Notes

  • The drawback to having URLs in your application that change with every user session is that it can interfere with behavior testing using automation tools like Selenium IDE, because any token value captured during the creation of your testing script will be invalid the next time you run them.  One way you can get around this is to add conditional logic to the code that that determines if the application is running in your local development environment (your own machine or a test server), and if so sets the session token value to the same value each and every time (regardless of user session), giving you an unchanging URL in your test environment that you can record in automation scripts.

  • If your application has access to very sensitive personal or financial information, you may want to take your CSRF security a step further.  Some applications rewrite the session token value every few minutes instead of leaving it the same during the user's authenticated session.  You should be able to modify the technique described in this post to do that if you feel it's warranted.

Friday, May 25, 2012

Converting Letters in Phone Numbers to Their Corresponding Digits

In a recent post, I mentioned that the feature that allows a user to tap on a phone number in their smartphone web browser and dial that number doesn't work if the phone number contains letters.  Well, some of the phone numbers in the database tables I'm working with contain letters (usually to spell out building abbreviations) so I needed to convert those letters into the corresponding digits on American phones (apparently it's not an international standard).

Before writing my own function to do this, I did some Googling and was a bit surprised to find that while folks have written routines in CFML to strip out dashes and such, no one had really written (and then published) a routine for this task.  So here's mine:

(Shrug) Nothing fancy, but it works. If you need to do additional adjustments to the phone number (like I did), you can just add them to this function or put them in separate functions (which is probably better).

Sunday, May 20, 2012

Post-cf.Objective() Thoughts

Just got back home from cf.Objective().  During the plane trip home, I wrote down a few paragraphs about the sessions I attended that I'll share now before kicking back for the rest of the day:

  • Marc Esher's session on threads and concurrency was excellent.  If you implement or plan to implement concurrent code execution in your CFML applications, you need to look at the CFConcurrent library he developed (available on RIAForge and GitHub).  The library utilizes the Java Concurrency Framework (JCF) and makes it possible to return results/data from with a thread (something you can't do with <cfthread>), schedule/defer threaded actions, and set up "threading within threading."

  • Nathan Strutz provided so much information and so many examples of using LESS to manage and generate CSS that I missed a few things trying to take notes.  No worries, though: he put up everything from his presentation on GitHub, including a number of examples and demos that show you some best practices on how to add LESS compilation into your development and production workflow.  And not long after the session, Matt Gifford made his own contribution to the LESS toolset:  a CF Builder extension that lets you do the compiling within the IDE.

  • Curt Gratz's session on mocking covered the reasons why you should mock data and objects when you unit testing, then showed attendees how MockBox makes it easy to mock function calls and their results on the fly.  As someone still finding his way regarding unit testing, I found his one walkthrough on how to make sure you're always testing a single function, and not a parent or dependent function, very helpful.  And even though MockBox is part of the ColdBox family of projects, it can be run as a standalone product, so you can make it part of your unit testing strategy regardless of what framework you use.

  • Billy Cravens also did an excellent job with his session on authenticating users via social networks.  Adding Google, Facebook, and Twitter-based authentication to your site is really now just a matter of getting an API and private key for your site from those services and making use of the available.  His demo code, available on GitHub, shows examples for all three service providers within a simple application built with the FW/1 framework.

  • I attended Denard Springle's multi-factor authentication workflow session mostly out of curiosity, as it's not something I can really implement with my particular user base.  But he showed us his technique of securing URL and form variables submissions, which involves hashing the variable names and encypting the variable values.  That's something I've never seen anyone else do.  Slides from his demo that illustrate that technique are available from his collection of presentation slide decks (look for the "Multi-Factor Authentication" one).

  • A room for conducting ad-hoc discussions or coding sessions was provided, and I ended up hosting a half-hour session on CFML applications in higher education.  The seven or so of us described some of the applications we've worked on, and as I suspected there was quite a diversity of applications, mostly built in support of the administrative processes at our schools rather than the educational process itself.  But I did learn that a few of us use (or are in the process of implementing) the Jasig CAS single sign-on tool, and that there would be interest in having some kind of robust form/survey tool (possibly with plugins like one to handle online payments) that we could set up such that non-programmers could create what they needed from it.  The question was raised as to whether it would make sense to augment Mura CMS to create such a thing.  We didn't discuss or make any further plans beyond that, so whether the discussion will continue in some form (move further forward) online remains to be seen.

As my conclusion, I will say that the conference was excellent as always (hardly a unique opinion, I know).  I've been to cf.Objective() before (in 2008), but this was my first time attending at the current venue, which is the Nicolett Mall area of Minneapolis.  It's a very pedestrian-friendly area of the city, and with the near-perfect weather we had it was quite lively.  It gave us a lot of options for dining and socializing after the sessions, and being able to hang out with other members of the community is one of the great benefits of the conference.

Thursday, May 3, 2012

Simple Technique For Creating Multiple Twitter Bootstrap Popovers

One of the JavaScript/jQuery plugins that comes with Twitter Bootstrap is the Popover plugin, which lets you create windows of content that appear when you hover over or focus on a DOM element.  They behave like tool tips but are larger and better suited to support styled content.

As noted in the documentation, the title and content of an individual popover can be coded in one of three ways:  via HTML attributes within the DOM element that triggers the popover, by assiging text values to the title and content properties when applying popover functionality to the DOM element, or by assigning functions that return markup to those title and content properties.  Here's an example of the last of those options:

$("#pop1".popover(
    {
        title: getPopTitle(),
        content: getPopContent()
    }
);

function getPopTitle() {
    return "Title 1";
};
		
function getPopContent() {
    return "Content 1";
}

The page I was working on needed to have several popovers, and the popover content included styled text: paragraphs, bolded text, etc. I didn't want to have to put that content into HTML attributes or long string values, nor did I want to code several different invocations of the popover function.

So what I decided to do was to create hidden blocks of HTML to hold the popover content and associate each set of "content" (title and content) with a unique id:

<style>
    .popSourceBlock {
        display:none;
    }
</style>

<div id="pop1_content" class="popSourceBlock">
    <div class="popTitle">
        Title 1
    </div>
    <div class="popContent">
        <p>This is the content for the <strong>first</strong> popover.</p>
    </div>
</div>

<div id="pop2_content" class="popSourceBlock">
    <div class="popTitle">
        Title 2
    </div>
    <div class="popContent">
        <p>This is the content for the <strong>second</strong> popover.</p>
    </div>
</div>


Then I assigned ids to the DOM objects that would trigger the popups when hovered over and gave them a CSS class of "pop". I opted to use one of the icons provide by Bootstrap to trigger the various popovers:

<i id="pop1" class="icon-question-sign pop"></i>
...
<i id="pop2" class="icon-question-sign pop"></i>


Finally, I used jQuery's each() function to loop through all the DOM elements with a class of "pop", grab the id value and use that to locate the matching content to associate with each popover:

$(".pop").each(function() {
    var $pElem= $(this);
    $pElem.popover(
        {
          title: getPopTitle($pElem.attr("id")),
          content: getPopContent($pElem.attr("id"))
        }
    );
});
				
function getPopTitle(target) {
    return $("#" + target + "_content > div.popTitle").html();
};
		
function getPopContent(target) {
    return $("#" + target + "_content > div.popContent").html();
};

With this code in place, I can add new popovers by creating additional hidden HTML blocks and DOM elements with the "pop" CSS class and an id that follows the pattern I've established.

Another nice aspect to this is that, down the road, I could put the page with the HTML blocks in a directory where a web designer or even a client with some basic HTML skills could edit the content, and I could call that file into my page programmatically. So long as they didn't add new content blocks, they could tweak the wording of the content without having to call me to do it (assuming I can trust them not to mess up the HTML formatting).

Wednesday, April 4, 2012

Letters In Place of Numbers in Phone Number Links on Mobile Phones

Part of an upcoming project involves displaying office phone numbers on a web page such that, if the page is being viewed on a smartphone, tapping on the number pulls up the phone's dialing program with the number filled in and the user just needs to start the call.

Some of the phone numbers in question use letters in place of numbers (like "1-800-555-CFML" as an example) and I wondered if such numbers would work.

The answer is "probably not," as it's not something supported in the spec (the spec being RFC3966).  Tried it on my Galaxy Nexus anyway: no dice.  So I'll have to convert any such letters to their numerical equivalent

Some other things I learned today:

  • Per the W3C recommendation regarding "click-to-call" links, you should always include the "+" sign in front of the number to support international callers ("+1-800-555-CFML").
  • You can also use "." as a separator instead of a "-", and you can get away with using both types of separators in the same number.