Thursday, October 23, 2014

Task-based web browsing: Grunt and the grunt-open plugin

Lately I've been playing around with Grunt, which is a JavaScript-based task runner similar to Ant and Gradle. Generally, these tools are used to automate software builds, but they can be utilized in other ways.

While browsing through the large collection of Grunt plugins, I came across one called grunt-open. The grunt-open plugin lets you create a task that opens a URL in your web browser. It looks like it was created with the idea of opening a web page at the end of a chain of Grunt tasks to check the result of a software build. But I thought it might be useful in automating everyday web browsing tasks.

I like to start off my work day with a bit of Internet humor and I have certains sites I visit for that. Dilbert has a new strip every day, while JoyOfTech and XKCD get published every Monday, Wednesday, and Friday, and the XKCD "What-If" article is published once a week, usually by Thursday. Rather than manually opening the appropriate bookmarks for the appropriate day of the week, I figured I'd program a task to take care of opening the appropriate sites.

I won't reinvent the wheel here trying to explain the basics of Grunt: the Grunt "Getting Started" guide does a good job of explaining how to install Grunt and generate the files needed (the package.json and the Gruntfile.js files). Here's what my Gruntfile looks like:

So in the directory where this Gruntfile.js lives, I just have to type "grunt webhumor' in the command, and Grunt will open the appropriate sites. Not a huge timesaver, but kinda cool.

Wednesday, January 15, 2014

Determining if Data Has Changed in Your AngularJS-Powered Form

So here's the scenario:  you've got a form enabled with AngularJS.  The form is populated with data from a data model object retrieved from a REST call.  You need to know at a certain point (perhaps at the end of every user action, or perhaps at the moment of submission) whether the form data is different from when it was originally retrieved.  How would you do that?

If you wrap your form elements within a set of HTML form tags and name the form, Angular automatically (via the ngFormController) monitors the overall state of the form and provides some status flags, one of which is the $dirty property.  Problem solved, right?  Well, not quite.

Take a look at this example form I've set up with AngularJS 1.2.6 and Bootstrap 3:

https://bcswartz.github.io/AngularJS-formDataChangeDetection-demo/index.html#/demo1/1

I threw a bit of a twist into my form.  The tasks at the end of the form (stored as an array of objects in the main project object model) can be reordered by dragging:  that functionality is accomplished with the addition of jQuery, jQuery UI, and the Angular UI Sortable project.  As you might guess, if you did nothing to the form but reorder the tasks, the $dirty status of the form (displayed in the diagnostic area beneath the form) would remain "false":  that status evaluates the data in the form fields but doesn't recognize other changes to the underlying data model.

But even if you didn't have a UI element like that in your form, $dirty isn't going to tell you what you need to know.  Go into the form and change "Form" in the Name field to "Forms".  Note the $dirty status changes to "true".  Now change it back to "Form."  $dirty still evaluates to "true".  That's because $dirty isn't evaluating differences between the original data and the new data, it's just tracking interactions.  Any form field the user interacts with is considered to be dirty, and that propagates up to the form.

So what you need then is a means of recording the original state of the data when the page/form is loaded and a means to compare that original data with the current model data.

Fortunately, Angular provides two convenience functions that can answer those needs.  The angular.copy() function can create a clone of an object, while angular.equals() will do a deep comparison between two objects (comparing all property names and values, rather than telling you if one object is a reference/pointer to the other).

Here's another version of that same form (open up the page but don't do anything with it yet):

https://bcswartz.github.io/AngularJS-formDataChangeDetection-demo/index.html#/demo1/2

With this version of the form, I added code to the controller function to create an "original" clone object of the starting "project" model object as soon as the data was made available, then used angular.equals() to verify that both objects were identical.  I also added a $watch function that would monitor changes in the "project" model object and re-compare the "original" and "project" objects.

projectResource.get().$promise.then(function(project) {
  $scope.project= project;
  $scope.original= angular.copy(project);
			
  $scope.initialComparison= angular.equals($scope.project,$scope.original);
  $scope.dataHasChanged= angular.copy($scope.initialComparison);
});

...

$scope.$watch('project',function(newValue, oldValue) {
  if(newValue != oldValue) {
    $scope.dataHasChanged= angular.equals($scope.project,$scope.original);
  }
},true);

Now go to the form and reorder the tasks, and you'll see that the first block of diagnostic data indicates that the data has changed, even though $dirty is still false.  Go into the form and change "Form" in the Name field to "Forms," and then back again, and you'll see that the change status updates accordingly.

There's still a catch, though.  Note the second second of diagnotic information, where (in both the "original" and "project" objects) it denotes the result of adding 1 to the estimated hours, and whether the Meeting Notes property is a null value or an empty string.  They start off the same, but you can probably guess where this is going.  Change the hours from 40 to 41, and then back to 40.  Type a character or two in the Meeting Notes field, and then delete them.

The form inputs set the corresponding model data as strings, so once you edit the data in the inputs, even if you put it back to the way it was, anything that was not a string before (like a number or a null value) is now a string, and now the "property" object doesn't equal the "original" object anymore.

The workaround is straightforward enough:  before you create the clone of the original data, parse it and "stringify" the appropriate properties.

Here's the final example:

https://bcswartz.github.io/AngularJS-formDataChangeDetection-demo/index.html#/demo1/3

This time, before using angular.copy(), I processed the "project" data model via a service function, which in turn executed a "stringification" function on each of the relevant properties:

Controller code:

$scope.project= service.stringifyProjectData(project);

 

Service code:

.factory("demo1_service", [function() {
  var stringifyProperty= function(property) {
    if(property== null) {
      return "";
    } else {
      return property.toString();
    }
  };
		
  return {
    stringifyProjectData: function(project) {
      var simpleProperties= ["projectName","description","hours","meetingNotes"];
      for(var p= 0; p < simpleProperties.length; p++) {
        project[simpleProperties[p]]= stringifyProperty(project[simpleProperties[p]]);
      }

      for(var t= 0; t < project.tasks.length; t++) {
        project.tasks[t].description= stringifyProperty(project.tasks[t].description);
      }

      return project;
    }
  };
		
}]);

 

...the code could probably be more elegant, but it gets the job done. So now if you change the Estimated Hours and Meeting Notes fields in the final example but then change them back, the code will correctly denote that "original" and "project" are back in sync after you remove your changes.

Since the demos consists solely of HTML, CSS, and JavaScript, the entirety of the code is there in the GitHub repo that powers the demo for you to look at if you wish.  I created separate controller functions for each demo to make clear what was happening in each one.

Thursday, September 12, 2013

Introducing validORM: A Development Tool for Creating ORM CFC And ValidateThis Files

In my previous post about relating different data constructs in AngularJS to each other via property names, I mentioned that I figured out that technique while working on an internal development tool.  This blog post is about that tool.

The AngularJS-powered validORM tool lets you design the ID and column properties of your ColdFusion/CFML ORM CFCs while at the same time creating a matching set of validation rules using the ValidateThis (VT) library (a great library for handling both client and server-side validation).

The first view of the tool is a menu view.  You can choose to generate an updated version of your CFC and VT rules file based on a previous session or create a new set of files from scratch.  Either choice takes you to the generator view.

The first two sections of the in the generator view are pretty obvious:  that's where you choose the name and database table for your ORM object and configue the ID property.

The third section, where you define your column properties and create your validation rules, is where AngularJS really comes into play.  The property drop-down selection, the form controls and the hover hints for each property attribute you add, and the validation rule options and parameters:  those are all AngularJS manifestations of JavaScript data constructs.  Using the "Add Attribute" button to add a property attibute simply adds a JavaScript object of that name to the dataset; the creation of the form elements on the page for that attribute is handled automatically by AngularJS based on the Angular directives in the HTML.  That's Angular's strength: driving page behavior purely through data manipulation. 

The data constructs created by your choice of property attributes and validation rules are separate from the JavaScript objects and arrays of objects that represent all of the attribute and rule options, but they reference each other via the attribute and rule key names.

The last two sections allow you to further define any ValidateThis conditions and contexts referenced in your column property rules.  When you're done, clicking the submission button will trigger the file generation process.  Your ORM and VT configuration data will be parsed by a CFC which will then generate three files in the output folder of the tool:

  • An ORM CFC written in script format.
  • A ValidateThis rules definition file matching that CFC written in JSON format.
  • A JSON-formatted snapshot of the configuration data, with a filename reflecting the name of the object and the timestamp when the file was generated.  Said file will then appear in the menu view of the tool.

Some caveats:  the tool only allows for one ID property, doesn't include absolutely every type of property attribute (just the majority of them), and it doesn't let you define ORM relationship properties.  And as my first foray with using AngularJS, I'm sure it violates one or two Angular best practices (if you're an Angular guru, feel free to chime in with any suggestions for improvement).

But it works and can provide a kickstart to creating any ORM files and ValidateThis rules for your projects.  It comes with an example "Employee" configuration set that you can play with and then delete once you start using the tool.

The tool is available for download (and modification) via GitHub:  https://github.com/bcswartz/validORM. I also created a short video of the tool in action and posted it on YouTube at: http://www.youtube.com/watch?v=-fnt_n65NWg

Monday, August 12, 2013

Quick AngularJS Tip: Relating User Input Data to Metadata

I started learning AngularJS a few weeks back because I thought it would work well for an internal tool I was building (more on that in a later post).

Many AngularJS examples, including those in the current AngularJS tutorial (like this one), illustrate how to use the ng-repeat and ng-options directives using a typical Javascript array of objects.  So given such an array in the model:

$scope.plans= [
  {"name": "Basic", "cost": "$250.00"},
  {"name": "Deluxe","cost": "$325.00"},
  {"name": "Premium","cost": "$335.00"}    	
];

...you can output a list of the data like so:

<ul ng-repeat="plan in plans">
  <li>
    {{plan.name}}: {{plan.cost}}
  </li>
</ul>

But, as noted in the AngularJS API, you can also use ng-repeat and ng-options to loop through a set of properties/keys in an object, and that works even if each property references another object.  So you can accomplish the same thing with this model object:

$scope.plans= {
  "Basic": {"cost": "$250.00"},
  "Deluxe": {"cost":"$325.00"},
  "Premium": {"cost": "$335.00"}    	
};

...and this HTML:

<ul ng-repeat="(planKey,planObject) in plans">
  <li>
    {{planKey}}: {{planObject.cost}}
  </li>
</ul>

The advantage to using the latter technique comes into play when you want to relate something in the part of the model you want to preserve / process with a part of the model that provides metadata.

Taking the above examples a step further, say you wanted to create a short conference registration form. The user has to select a conference package, and you want to provide information about each package (what it entails and the cost) but you don't want that information to be part of the form submission. You can do this by having the user's conference selection pull the related data from the plan metadata.

So with the following AngularJS controller function:

function ConfReg($scope, $http) {
  $scope.reg= {};
  $scope.reg.plan= "Basic";
  
  $scope.plans= {
    "Basic": {
      "description": "Access to all first-run conference sessions on Thursday and Friday.",
      "cost": "$250.00"
    },
    "Deluxe": {
      "description": "Access to all first-run conference sessions on Thursday and Friday, access to repeat sessions on Saturday.  Complimentary breakfast included all three days.",
      "cost": "$325.00"
    },
    "Premium": {
      "description": "Access to all first-run conference sessions on Thursday and Friday, access to repeat sessions on Saturday.  Complimentary breakfast included all three days.  Also includes conference t-shirt and kazoo.",
      "cost": "$335.00"
    }

  };
}

...and the corresponding HTML:

<h3>Conference Registration</h3>
  <form name="regForm" id="regForm" ng-controller="ConfReg">
    <p>
      <label>First Name:</label>
      <input type="text" name="firstName" id="firstName" ng-model="reg.firstName" />
    </p>
			
    <p>
      <label class="control-label" for="lastName" id="lblLastName">Last Name:</label>   
      <input type="text" name="lastName" id="lastName" ng-model="reg.lastName" />
    </p>
				
    <p>
      <label class="control-label" for="plan" id="lblPlan">Conference plan:</label>   
      <select name="plan" id="plan" ng-model="reg.plan" ng-options="planName as planName for (planName,planObject) in plans"></select>
    </p>
    <div>
      <p>{{plans[reg.plan].description}}</p>
      <strong>Price:</strong> {{plans[reg.plan].cost}}
    </div>
				
</form>

...The user's selection of plan determines which plan object data is pulled from the "plans" part of the model for display in that div block, yet the data sent as the form submission (the "reg" object) only includes the plan name.

Monday, April 29, 2013

Quick CFML Tips: Script-based Queries And Tag/Script Hybrid CFCs

While refactoring a particular process in a ColdFusion project at work, I learned two things I thought worth passing on:

1.  There are certain scenarios where doing queries in cfscript, instead of using the <cfquery> tag, does come with a significant performance penalty.  In my case, the scenario was one where I was taking two recordsets created by querying the database and running subqueries against those recordsets in a recursive fashion.  My guess is that instantiating the query object needed to perform query functions in cfscript (which I was doing with each recursion) was the main issue.  When I converted the queries and functions involved in the recursive process to use tag-based syntax, the amount of time it took the process to run dropped by over 33%.

Not saying that this means you should avoid doing script-style queries, just that if there are a large number of such queries in a long-running process, you might want to experiment with refactoring them.

2.  I already knew that I could have one or more cfscript blocks inside the body of a CFC function, but I didn't know that I could combine tag-syntax functions and script-syntax functions in the same CFC like so:

<cfcomponent accessors="true">
    <cffunction name="doX" output="false" returntype="void">
        ...
    </cffunction>

    <cfscript>
        public void function doY () {
            ....
        }
    </cfscript>
</cfcomponent>

...it had never crossed my mind to try it that way. Certainly useful when you need to run a CFML function that has no cfscript equivalent in whatever version of CFML engine you're using (I should note that I was using ColdFusion 9.0.1 in both of the above cases).

Wednesday, April 10, 2013

Overcoming the Cookie Size Limit When Using jQuery DataTables and Saved State

One of the features of the jQuery DataTables plugin is that you can use the configuration option "bStateSave" to store the current view of the table (the sort order, the number of rows displayed, the filter term, etc.) in a cookie, so if the user navigates away from the page then comes back, the table view is the same as how they left it.

However, if your website or web application stores the state of several DataTables, and a user hits all those tables faster than those cookies can expire (their default lifespan is 2 hours but can be customized with the "iCookieDuration" option), the user could hit the browser cookie size limit and start seeing errors on your site.  I ran into this problem today with an application I've been working on.

Fortunately, starting with version 1.9, DataTables provides functions that let developers intercept the process of saving the table state and reloading that state, and the plugin author provides a documentation page on how to use those functions to store the state data in localStorage, which in most browsers allows you to store a few MBs of data per site (far more than the 4K limit per site for cookies):

http://datatables.net/blog/localStorage_for_state_saving

In my case, I decided that I would store the state data in sessionStorage rather than localStorage, but the principle was the same.

Wednesday, March 13, 2013

Adding Oracle Support to BugLogHQ: How the Application Design Made it Easy

As I mentioned in my previous blog post, in order to try out BugLogHQ in my work environment, I knew I would have to update the code so it could run against Oracle database tables.  So I started looking through the code to see how difficult of a task that would be, to see if it was worth the trouble.  Changing the code to support another database would mean at least looking at (if not updating) every block of code that performed a SQL operation:  if there were a lot of such blocks, it could be a time-consuming task.

Fortunately for me, BugLogHQ was designed to abtract most of the SQL operations behind a DAO layer.  Each database table used by BugLogHQ is mapped to a DAO CFC in the components/db folder of the BugLogHQ application, and those CFCs spell out the name and cfqueryparam data type for each table field and inherit getter and setter-type functions from a base DAO class.  When it comes time to persist data from instantiated DAO objects into the database, the DAOs are processed by the dbDataProvider.cfc in the components/lib/dao/ folder, and that CFC contains the SQL code for performing all of the CRUD operations.

(I should point out that this is a somewhat simplified description of all of the "moving parts" involved in data persistence in BugLogHQ, but after following the code through the process of adding and removing records I determined that I didn't need to make changes to other objects such as the DAO factory).

The main issue I needed to address was the fact that the current code for inserting a new record took advantage of the autoincrement feature in the other databases supported by BugLogHQ (MySQL, MS SQL, etc.).  Oracle doesn't come with an autoincrement field option.  Instead, Oracle has what are called sequences, which are essentially autoincrementing numbers that exist separately from the database tables (meaning you could, if you wanted to, pull the next integer from a particular sequence into any table).  So in the SQL install script I wrote to create the needed tables in Oracle (mirroring the install scripts for the other database engines present in the BugLogHQ install folder), I added lines to create sequences for each table, with each sequence name being the table name with "_seq" appended to the end of it.  I then added code to the _insert() function in the dbDataProvider.cfc to use the sequences for the primary key value when inserting a record:

  
<cffunction name="_insert" access="private" returntype="any">
  <cfargument name="columns" required="true" type="struct">
  <cfargument name="_mapTableInfo" type="struct" required="true">
  ...  
  <cfset var dbtype = variables.oConfigBean.getDBType()>
  <cfset var lstFields = structKeyList(arguments.columns)>
  <cfset var tableName = arguments._mapTableInfo.tableName>
  <cfset var seqName= tableName & "_seq">
  <cfset var pkeyName= arguments._mapTableInfo.PKName>
  ...
  <cfif dbtype eq "oracle">
    <cftransaction>	
      <cfquery name="qry" datasource="#DSN#" username="#username#" password="#password#">
        INSERT INTO #getSafeTableName(tableName)# (#pkeyName#,#lstFields#)
        VALUES (
        #seqName#.nextVal,
        <cfloop list="#lstFields#" index="col">
          <cfqueryparam cfsqltype="#arguments.columns[col].cfsqltype#" value="#arguments.columns[col].value#" null="#arguments.columns[col].isNull#">
          <cfif i neq listLen(lstFields)>,</cfif>
            <cfset i = i + 1>
          </cfloop>
        )
      </cfquery>
				
      <cfquery name="qry" datasource="#DSN#" username="#username#" password="#password#" result="qryInfo">
        select #seqName#.currVal as lastId from dual
      </cfquery>
    </cftransaction>
    <cfset newID = qry.lastID>
  <cfelse>
  ...

So if the dbtype value (which comes from the config/buglog-config.xml.cfm file) is "oracle", the Oracle insert code block will execute. The first cfquery block will insert the record, using the nextVal() function of the sequence associated with the table to get the next value of the sequence, while the second cfquery block retrieves the current/latest value of the sequence via Oracle's DUAL "table" (a construct designed to perform pretty much any utility operation that returns a single record or value).

The other issue I needed to address was adjusting some of the field data types.  Oracle's "varchar2" data type can only hold 4,000 characters of text; to store larger amounts of text, you need to use fields with the CLOB data type.  And in order to use cfqueryparam with such fields, you need to set the cfsqltype parameter of cfqueryparam to "cf_sql_clob".  So I updated the code in the entryDAO and extensionDAO CFCs to handle that (with the entryDAO.cfc code shown below):

<cfif variables.oDataProvider.getConfig().getDBType() EQ "oracle">
  <cfset addColumn("exceptionDetails", "cf_sql_clob")>
  <cfset addColumn("HTMLReport", "cf_sql_clob")>
<cfelse>
  <cfset addColumn("exceptionDetails", "cf_sql_varchar")>
  <cfset addColumn("HTMLReport", "cf_sql_varchar")>
</cfif>

After that, all that I had left to do was to add Oracle-specific conditional blocks to two other queries (ones that involved using database functions specific to each database platform) in two other files (components/hq/appService.cfc and components/lib/entryFinder.cfc), and I was done.

Had I done all of the changes I just described perfectly on the first try - which admittedly I didn't  :) - it would have taken me less time than it did to write this blog post.  And that's a mark of tight, well-designed application architecture, where certain key aspects of application behavior are centralized in a manner that lets you add functionality (in this case database engine support) by updating just a few files responsible for handling that particular task.