Sunday, December 4, 2011

Announcing tableSnapshots: A ColdFusion Tool For Preserving and Reverting Database Table Data

Several months ago, I was doing some functional testing on a web application that was taking forever. Each run of the test resulted in changes to the data in several tables, and in order to reset the test I had to go into the tables and undo the changes. It occurred to me that it would be nice to have a tool or script that could preserve the current table data and then later write that version of the data back to the tables.

That marked the beginning of the tableSnapshots tool.

Creating the initial version of the tool just to suit my own needs was quick and easy compared to the work I've put into the tool since then to make it worth sharing, but I'm pretty happy with the result.  It's essentially a small web application that you can drop into any folder on your ColdFusion-enabled (ColdFusion 8 or 9) web server. Once you update the configuration settings in its Application.cfc file with the name of your datasource, the tool enables you to:

  • Take a "snapshot" of one or more of your database tables, storing all of the current table data as JSON data in a text file (NOTE: this tool will not work on tables with binary data fields).

  • Replace the current data in a table with the data stored in a snapshot, creating a backup snapshot of the data being replaced in the process.

  • View the data in a snapshot or backup snapshot in a jQuery-enhanced table that allows you to sort and filter the data.

  • Delete older snapshot and backup snapshot files.

All of this functionality is available via a web interface.  Creating a snapshot is as easy as clicking on the "Create" menu option, selecting the tables from your datasource that you want to generate snapshots from an HTML table (enhanced with the jQuery Datatables plugin), and clicking a button.  You reload the data from one or more snapshot files in the same fashion (though of course reloading the data takes longer), and you can view the data stored in any snapshot or backup file in tabular format.  The tool even makes it easy to delete snapshot and backup files you no longer need so you don't have to manually delete them from your file system.

The tool serializes the table data using the functions provided in the JSONUtil project because the "strictMapping" option provided by JSONUtil avoids the data conversion issues that can occur with the native ColdFusion JSON functions (such as strings being converted to numbers and "true"/"false" string values being converted to "yes/no").  However, it uses the native JSON functions for deserializing the data because they seem to perform better when working with large data sets.

This tool was built and tested on ColdFusion 9.0.1. It should work on Adobe ColdFusion 8, and perhaps other CFML engines as well, but it has not been tested in those environments. It relies on the <cfdbinfo> tag to acquire the data types of the table fields it processes, so access to that tag is a requirement.

The current version supports Apache Derby, MySQL, and Oracle:  there is one .cfc file in the tool for each of these databases that tells tableSnapshots how to map the data stored in JSON format to the relevant table fields during the snapshot reload process. So adding support for additional databases simply means writing similar .cfc files and placing them in the main tool directory. Further details about how tableSnapshots works and what configuration options are available are described in the "Technical Details" page within the application itself.

As I said at the beginning, the original purpose of tableSnapshots was to make it easy to rollback data in multiples tables during testing.  But there are a couple of other potential uses as well:

  • You could use it to simply track changes to data over time (since each snapshot file is timestamped).

  • If for some reason you don't have an IDE or database client program that lets you browse table data, you can create snapshots and view the data with the snapshot view option (which gives you the ability to sort and filter the data).

  • You can use it to copy table data from one type of database server to another.  I used it for this purpose myself:  I made a snapshot of the data, created a similar table with a different name on the other datasource, renamed the snapshot file to match that different name, reconfigured and reset tableSnapshots to interact with the new datasource, and "reloaded" the snapshot data into the new table.

  • If your development database isn't being backed up as well or as often as you'd like, you could use the snapshots as your own backup.

  • You can create snapshots as data for use while offline or in mock objects. tableSnapshots comes with a mini-tool in the "tools/jsonasquery" subfolder that can be used to pull snapshot data into a query object.

I do want to be clear that tableSnapshots is meant for use as a development tool.  While there's no risk of data loss when taking a snapshot, reloading snapshots involves deleting data and no software is perfect, so make sure you take additional steps to safeguard any data that you're afraid to lose.

tableSnapshots is available for download from both RIAForge and GitHub, so have at it!