SCCTextX for Data

Over the years, there have been debates about whether it is best to use source control integrated with the VFP Project Manager or to keep it separate.  I’ve always preferred to have the integrated experience.  Regardless of which side you fall on, it is very useful to have textual representations of VFP’s binary source files (SCX, VCX, etc.).  These text files enable diffs, so a developer can compare different versions of a source file and see what changes were made.  VFP includes SCCText.prg in the box, which has improved over time, but leaves a lot to be desired.  The SCCTextX project on VFPX is a major improvement and makes the resulting text files much more usable.

However, one thing I’ve always wanted was the ability to generate text files for DBCs and DBFs that we include in our project and source control.  The DBCs contain valuable information about the data structures, as well as local/remote view definitions.  The DBFs are primarily metadata for things like Stonefield Database Toolkit and the framework we use, Visual ProMatrix.  Before I checked in my latest changes to these files, I decided to crack open SCCTextX.prg and take a look at what could be done.  Lo and behold! There is already code to deal with DBCs and the beginnings of code for DBFs, which by default had been disabled.  I thought to myself, “I could have something working within a couple of hours”, so I dug in. Three days later… I finally had a solution, but with caveats.

There was a reason the code for DBCs was disabled.  The text file it produced was useless for diffs.  After some trial, error, and experimentation, I ended up with modified versions of SCCTextX.prg and FoxPro’s GenDBC.prg .  SCCTextX_Data.prg now calls GenDBC_SCCTextX.prg to generate a text file for DBCs.  It expects GenDBC_SCCTextX.prg to be in the same directory as SCCTextX_Data.prg.  I made two modifications to the GenDBC program.  The first was to sort the entries, so they are created in a consistent order.  The second was to parse CREATE SQL VIEW commands into multiple lines, which otherwise appear in GenDBC on one line, making it very difficult to see what has changed.  I’ll tell you up front that the parsing is not very good, and definitely not as good as I have seen in other VFP products/projects, but I needed something simple and lightweight, and I find it good enough for diff purposes.  Also, GenDBC is a little slow compared to other text file generation, but it wasn’t a showstopper for me.  NOTE: GenDBC_SCCTextX.prg is only intended for source control diff purposes, and I do not recommend it as a replacement of the standard GenDBC.prg for creating databases.

Aside: If you look in SCCTextX.prg, you may notice the developers tried to change the extension for DBC text files from “DBA” to “DCA”.  I agree with this change.  Unfortunately, the VFP Project Manager forces and expects the DBA extension.  If some aspiring developer were to create a fully functional replacement for the Project Manager on VFPX (hint, hint), this (and other limitations with source control integration) could be overcome.  But as it stands, we have no control over it.

That takes care of DBCs, how about DBFs?  Well, it turns out that the code included in SCCTextX.prg for DBCs is actually pretty good for DBFs.  So, easy right? Wrong.  The first problem has to do with the Project Manager integration.  VFP doesn’t even call SCCTextX for files in the Free Tables section.  That explains why we only have the “beginnings” of DBF support.  However, we can trick VFP into calling SCCTextX by putting the DBF into the Databases section of the project.  There are three ways to do this:

1. Add the DBF manually in the Databases section.  VFP will complain, but the file will still be there.
2. Hack the PJX (USE MyProject.PJX) and change the type from “D” to “d” on the applicable files.
3. If the project is open: _VFP.ActiveProject.Files(“MyTable.dbf”).Type = “d”

Once the DBF is in this section, VFP will call SCCTextX and otherwise integrate properly with source control.  SCCTextX_Data.prg is smart enough not to run GenDBC for files that don’t have a “DBC” extension.  The text file extension for both DBCs and DBFs will be “DBA”, so you can’t have a table and database of the same name, but that wasn’t a problem for me.

So far, so good, but there are other issues with DBFs.  You might want to exclude certain fields like ID fields or timestamps that change often and clutter the diffs.  Or you might want set the order for the table to get consistent results.  For this purpose, SCCTextX_Data.prg will call SCCTextX_Custom.prg if it exists in the same directory, giving you an opportunity to specify these settings.  See SCCTextX_Custom – Example.prg in the download.

So now we’ve got text files for DBCs and DBFs, integrated with the Project Manager.  Time for a quick build and… FAIL.  Ugh! VFP doesn’t like the DBFs in the Databases section.  Nothing a little project hook (included in the download) can’t fix though.  It moves all non-DBCs to the Free Tables section before the build and puts them back afterwards.

With all of these caveats, I think it is obvious why I won’t be submitting my changes to the SCCTextX project manager at VFPX.  That said, I’ll tell you it is VERY nice to finally be able to run diffs on these files.  Definitely worth the caveats and overall effort for me.  If you want to try it yourself, feel free to download SccTextX_data.zip.