Double Trouble with COM Interop

I have been calling FoxPro code from C# via COM Interop (using DotNet2Fox) for several years, but I ran across an interesting issue last week. I have some FoxPro code that performs calculations and returns the results to .NET. I expected one of those values to be zero, but according to C#, it was -3.5527136788005e-15. Huh? After repeated tests, I got the same result.

I was assigning the FoxPro results to C# variables of type “decimal”. The C# compiler had already informed me that FoxPro was returning values with type “double” over COM. I have heard about the tension between binary and floating point math in computers before, but in practice, I have never run into an issue. I initially thought that the double-to-decimal conversion in C# was exposing this tension. As a test, I changed my C# variables to type double, but the problem persisted. I stepped through my code in FoxPro, and according to the debugger, the value was indeed zero. What the heck was going on?

I came up with a simpler calculation to reproduce the issue: 1.23 + 4.56. The result should be 5.79, right? Enter ? 1.23 + 4.56 in the FoxPro command window, and that’s what you’ll get. However, run the code below in C# and the result will be 5.790000000000001.

        dynamic vfp = Activator.CreateInstance(Type.GetTypeFromProgID("VisualFoxPro.Application", true));
        var result = vfp.Eval("1.23 + 4.56");
        vfp.Quit();
        return result;

Had I run across some glitch with COM interop? I needed to be certain that calculations done in FoxPro would come over to C# accurately. This was disconcerting.

Upon further thought… someone would have surely caught an issue like this with general COM interop, right? Maybe this was more on the FoxPro side. Then, I remembered a blog entry Christof wrote years ago on Significant digits in Visual FoxPro that mentions some oddities with 15-digit floating point calculations in VFP. Hmmmm… 15 digits. That’s the same precision of double types in C#. And look how many digits are after the decimal place in 5.790000000000001. So, I tried another test in the FoxPro command window:

SET DECIMALS TO 15
? 1.25 + 4.56

There it is! Now, FoxPro shows 5.790000000000001! Clearly, this is either some kind of binary-to-floating-point issue or something to do with that 15th digit, but SET DECIMALS masks that most of the time. SET DECIMALS is a mysterious command. It specifies the number of decimal places that will be displayed from calculations. It apparently also affects comparisons: ? 1.25 + 4.56 = 5.79 returns .T. when SET DECIMALS TO 2. But SET DECIMALS TO 15 and the same expression returns .F. Weird! SET DECIMALS appears to have no effect on calculated values returned over COM interop and always returns the full 15-digit precision of a double.

So, that explains why I had the issue when using COM interop. How do we fix it? Simple… round the calculated result to the desired number of decimals in FoxPro before returning the result, or in C# after the result is returned from FoxPro. In fact, I was able to resolve by rounding down to 14 digits (eliminating the troublesome 15th digit), but I can’t guarantee that would always work. By the way…if you want C# rounding to have the same behavior as FoxPro rounding, specify MidpointRounding.AwayFromZero:

    public static double FixDouble(double number, int roundDecimals = 14)
    {
        return Math.Round(number, roundDecimals, MidpointRounding.AwayFromZero);
    }

BindWinEvent and FoxTabs 1.3

Years ago, Greg Green reported that FoxTabs was conflicting with his custom VFP editors, due to VFP’s limitations binding to Win Msg events. Greg wrote this framework to deal with the limitation, and I made some modifications. I wrote about this on my blog, and BindWinEvent has been included with FoxTabs ever since.

More recently, I found that FoxTabs was conflicting with Doug Hennig’s ProjectExplorer for the same reason, so I implemented BindWinEvent in ProjectExplorer. Doug reported back with some fixes to issues he found. I then realized that BindWinEvent needed a separate repository of its own, so it is now a separate project and part of VFPX. If you have an IDE utility that binds to Win Msg events, I recommend using BindWinEvents to avoid conflicts with other utilities.

FoxTabs hasn’t had any love since 2014, so while I was in there, I figured I would investigate a long-standing issue where a label would disappear on some of the tabs. This turned out to be a compatibility issue with PEM Editor, because it sometimes changes the window title to include more information. Once I accounted for that, no more disappearing labels!

While working on that issue, I ran across some old code from Jim Nelson that used _EdGetEnv() from FoxTools.fll to determine the type of each window. I had never learned about that function before. I used it along with some other tricks to improve window type detection and the icons displayed for each type. Instead of a bunch of Fox heads, the tab icons are now a lot more informative and colorful.

You can get all these improvements in FoxTabs 1.3, available on GitHub or Thor Updater (coming soon if it’s not there already).

DotNet2Fox is now part of VFPX!

I’m happy to announce that DotNet2Fox has been accepted as part of the VFPX project.  “What is DotNet2Fox?”, you say?  Good question!  DotNet2Fox was first presented and released at Virtual Fox Fest 2021 (October), and I somehow failed to mention that here on my blog!  That is being rectified right now…

DotNet2Fox is an open-source library that simplifies calling Visual FoxPro code from .NET desktop and web applications. Have you tried your hand at COM interop only to be met with limitation after limitation? Did you find that exposing your existing FoxPro code over COM would require a major refactoring effort? DotNet2Fox provides a simplified interface for calling into existing real-world FoxPro code, and without all those limitations.

Over the past several years, I have been working to surface features from my desktop applications and make them available on the web via ASP.NET.  I found that the classic approach to COM interop recommended by Microsoft a couple of decades ago did not fit my needs. Over time, a library emerged to meet those needs: DotNet2Fox.

Geek Gatherings has graciously made recordings of Virtual Fox Fest sessions available to the entire FoxPro community on YouTube.

You can also get the whitepaper (documentation) and examples at the GitHub Wiki. Yes, full source code is on GitHub as well.  If you need help with DotNet2Fox, please start a Discussion or report an Issue. We’ll even accept Pull Requests.  I look forward to working with more folks  on this project.

ParallelFox 1.3 Released

The last release of ParallelFox was over 8 years ago in January 2012.  Also, 3 years have somehow elapsed since CodePlex was shut down and VFPX moved to GitHub. Time most definitely flies!  I have been using Doug Hennig’s Project Explorer on some projects. I decided to implement it with my VFPX projects, as well as do some housekeeping on GitHub. In the process, I noticed there were several tweaks/fixes (things checked in years ago on CodePlex) that never made their way into an official release.  So, I packaged those changes up into a new release: ParallelFox 1.3.

POTENTIAL BREAKING CHANGE: This release includes and requires ParallelFox.exe version 2.0 dated March 5, 2012. If you are using a previous version, this version needs to be installed and registered. This release is otherwise backwards compatible with previous versions, and it should not require changes in your code.

NOTE: If you previously cloned/downloaded the GitHub repository, you already have the changes in this release, including the latest ParallelFox.exe.

Welcome to JoelLeach.net!

Nearly a decade ago, I started this blog on the Foxite.com Community Weblog, a free service made available to the FoxPro community by Eric den Doop.  I wanted more control on the admin side — especially for dealing with spam – so I have moved the blog to my own site: joelleach.net.

All posts/comments have been retained, and links to the old site should automatically redirect to the new site.  Please update your RSS readers to point to the new feed.

Special thanks to Eric for making this service available to the community and for hosting my blog all these years.

Setting Subclass Properties in TypeScript

TypeScript brings full class-based inheritance to JavaScript projects.  This works very well, but you may run into some unexpected behavior when setting properties in a subclass and using those properties in the parent constructor. 

Here is a simple example that demonstrates the problem:

  • MyBaseClass is defined as an abstract class. Like other OOP languages, this means that the class cannot be instantiated directly.
  • MySubClass is a concrete subclass of MyBaseClass that can be instantiated. 
  • New in TypeScript 2.0 is the ability to define properties as abstract, which I have done with string1 and string2.  These properties must be set in the subclass, or the transpiler will generate an error. 
  • The parent class constructor sets the string3 property based on the values of string1 and string2 set in the subclass.  Imagine that string3 is a property that will be used by other methods in the base class (not shown in the example code), so it is a valid design choice to set that property in the constructor.
  • Finally, the last two lines of code instantiate the class and display string3.
// Abstract base class
abstract class MyBaseClass {

    // Abstract properties to be set in subclass
    protected abstract string1: string;
    protected abstract string2: string;

    public string3: string;

    constructor() {
        // Set string3 so it is available for rest of class
        this.string3 = this.buildString3();
    }

    protected buildString3(): string {
        return this.string1 + " " + this.string2;
    }
}

// Concrete subclass
class MySubClass extends MyBaseClass {

    // Abstract properties must be set or 
    //  transpiler error will occur
    protected string1 = "Hello";
    protected string2 = "World!";
}

// Create object and display string3
var myObject = new MySubClass();
alert(myObject.string3);

 

Of course, I expected this code to display “Hello World!”, but in fact it displays “undefined undefined”.  Why is that?  A look at the transpiled Javascript of the subclass constructor will give us a clue.

    function MySubClass() {
        _super.apply(this, arguments);
        // Abstract properties must be set or 
        //  transpiler error will occur
        this.string1 = "Hello";
        this.string2 = "World!";
    }

 

As you can see, the subclass properties aren’t set until AFTER the base constructor is called. Ryan Cavanaugh from Microsoft explains:

This is the intended behavior.

The order of initialization is:

1. The base class initialized properties are initialized

2. The base class constructor runs

3. The derived class initialized properties are initialized

4. The derived class constructor runs

Follow the link for more details on the reasons, but it comes down to the fact that property initialization is inextricably intertwined with the constructor. Alternative approaches have been suggested, but besides breaking existing code, the order above is likely to become part of the EcmaScript (JavaScript) standard. 

As an OOP veteran of other languages, I find this behavior unfortunate. By defining a class as abstract, you are in effect saying it is “incomplete“, and it will be completed by its concrete subclasses. These technical restrictions on property initialization and constructors get in the way, but there are things we can do to work around the problem.

Constructor Parameters

Rather than setting properties in the subclass, you can pass values to the base class constructor. 

// Abstract base class
abstract class MyBaseClass {

    // Changed abstract properties to private properties
    //  available in base class only 
    private string1: string;
    private string2: string;

    public string3: string;

    constructor(string1: string, string2: string) {
        // Set private properties in constructor
        this.string1 = string1;
        this.string2 = string2;
        // Set string3 so it is available for rest of class
        this.string3 = this.buildString3();
    }

    protected buildString3(): string {
        return this.string1 + " " + this.string2;
    }
}

// Concrete subclass
class MySubClass extends MyBaseClass {

    constructor() {
        // Pass values to base constructor
        var string1 = "Hello";
        var string2 = "World!";
        super(string1, string2);
    }
}

// Create object and display string3
var myObject = new MySubClass();
alert(myObject.string3);

 

This works, but stylistically, I don’t like it for an inheritance-based approach.  I’d rather have the ability to simply set properties in the subclass, but call it personal preference.  There is nothing wrong with this solution.

Constructor Hook Method

Here, I’ve added an initialize() hook method to the constructor that runs before the buildString3() method. This gives the subclass an opportunity to set properties the base class needs at the appropriate time.  I’ve declared the initalize() method as abstract, so that it must be implemented in the subclass.

// Abstract base class
abstract class MyBaseClass {

    // Properties to be set in subclass
    protected string1: string;
    protected string2: string;

    public string3: string;

    constructor() {
        // Call subclass initialize() before string3 is set
        this.initialize();
        // Set string3 so it is available for rest of class
        this.string3 = this.buildString3();
    }

    abstract initialize(): void;

    protected buildString3(): string {
        return this.string1 + " " + this.string2;
    }
}

// Concrete subclass
class MySubClass extends MyBaseClass {

    // Set properties
    initialize(): void {
        this.string1 = "Hello";
        this.string2 = "World!";
    }
}

// Create object and display string3
var myObject = new MySubClass();
alert(myObject.string3);

 

This also works, but it leaves much to be desired.  Even though I have declared the initialize() method as abstract, nothing forces the string1 and string2 properties to be set.  Notice that I had to remove the abstract keyword from those properties for this to transpile without error. In general, I like the idea of adding hook methods for subclasses to use, but they should be optional.  The base class should not depend on them, nor should it be ambiguous about which properties need to be set.

Getters/Setters

As you may have gathered from the above, methods do not suffer from the same constructor timing issues as properties.  The base class constructor called into the subclass initialize() method, and it functioned as expected.  Likewise, using getter/setter syntax for properties is an option:

// Abstract base class
abstract class MyBaseClass {

    // Abstract properties to be set in subclass
    // Use getter syntax
    protected abstract get string1(): string;
    protected abstract get string2(): string;

    public string3: string;

    constructor() {
        // Set string3 so it is available for rest of class
        this.string3 = this.buildString3();
    }

    protected buildString3(): string {
        return this.string1 + " " + this.string2;
    }
}

// Concrete subclass
class MySubClass extends MyBaseClass {

    // Abstract properties must be set or 
    //  transpiler error will occur
    // Getter syntax must be used in subclass as well
    protected get string1() { return "Hello"; }
    protected get string2() { return "World!"; }
}

// Create object and display string3
var myObject = new MySubClass();
alert(myObject.string3);

 

This is closer to the original vision.  Having to use getter syntax is a little wordy for my taste, when all you want to do is return a simple value.  You may not mind if you are used to this from other languages.

Move the Code

Finally, my favorite solution is to move the code out of the constructor, which is where the timing issue is.  I moved the code into the string3 property with getter syntax.  It won’t run until the property is accessed after the object has been constructed, so the timing issue is avoided.  I also added a private _string3 property for improved performance, but of course, that is optional.

// Abstract base class
abstract class MyBaseClass {

    // Abstract properties to be set in subclass
    protected abstract string1: string;
    protected abstract string2: string;

    // Private backing property (optional)
    private _string3: string;
    // Public property using getter syntax
    public get string3(): string {
        // Set private backing property if not already set
        if (!this._string3) {
            this._string3 = this.buildString3();
        }
        return this._string3;
    }

    protected buildString3(): string {
        return this.string1 + " " + this.string2;
    }
}

// Concrete subclass
class MySubClass extends MyBaseClass {

    // Abstract properties must be set or 
    //  transpiler error will occur
    protected string1 = "Hello";
    protected string2 = "World!";
}

// Create object and display string3
var myObject = new MySubClass();
alert(myObject.string3);

 

This solution is the closest the original code.  I also like the idea of doing more in the base class, so you can do less in subclasses.

Your mileage may vary depending on your specific scenario, so choose the workaround that works best for you.

References

Report Metadata Error

A few of our clients were getting an obscure error when running reports.  I couldn’t find any information on the error online, so I thought I would post a potential workaround here to help the next person that runs into it.

While the report is running and being prepared for preview, a dialog pops up prompting the user to find a table/DBF. When the user presses Cancel, the dialog may or may not pop up again multiple times. Eventually, the user receives this error:

File ‘m_4nl0nroh6.dbf’ does not exist.
The metadata for some report definition rows could not be loaded.
Some dynamic report features may be missing, or a report could not conclude successfully.

The error occurs in frxcursor.UnpackFrxMemberData() while attempting to ALTER TABLE on a cursor that was created only a few lines earlier. For some reason, the underlying DBF is missing, but I can’t explain why.

What I did find is that some of our reports contained generic MemberData in the STYLE field of the FRX that was not being used. Once removed, the code producing the error will not need to run. I suspect the data inadvertently got in there in the first place using the Report Designer. If you go to Field Properties->Other and click on the Run-time extensions->Edit Settings button, it will automatically add code to the STYLE field, which will be saved if you press OK.

NOTE: There are legitimate uses of MemberData in the STYLE field, such as rotated labels. In those cases, the STYLE field needs to remain populated, so be sure before deleting the data.

SCCTextX for Data

Over the years, there have been debates about whether it is best to use source control integrated with the VFP Project Manager or to keep it separate.  I’ve always preferred to have the integrated experience.  Regardless of which side you fall on, it is very useful to have textual representations of VFP’s binary source files (SCX, VCX, etc.).  These text files enable diffs, so a developer can compare different versions of a source file and see what changes were made.  VFP includes SCCText.prg in the box, which has improved over time, but leaves a lot to be desired.  The SCCTextX project on VFPX is a major improvement and makes the resulting text files much more usable.

However, one thing I’ve always wanted was the ability to generate text files for DBCs and DBFs that we include in our project and source control.  The DBCs contain valuable information about the data structures, as well as local/remote view definitions.  The DBFs are primarily metadata for things like Stonefield Database Toolkit and the framework we use, Visual ProMatrix.  Before I checked in my latest changes to these files, I decided to crack open SCCTextX.prg and take a look at what could be done.  Lo and behold! There is already code to deal with DBCs and the beginnings of code for DBFs, which by default had been disabled.  I thought to myself, “I could have something working within a couple of hours”, so I dug in. Three days later… I finally had a solution, but with caveats.

There was a reason the code for DBCs was disabled.  The text file it produced was useless for diffs.  After some trial, error, and experimentation, I ended up with modified versions of SCCTextX.prg and FoxPro’s GenDBC.prg .  SCCTextX_Data.prg now calls GenDBC_SCCTextX.prg to generate a text file for DBCs.  It expects GenDBC_SCCTextX.prg to be in the same directory as SCCTextX_Data.prg.  I made two modifications to the GenDBC program.  The first was to sort the entries, so they are created in a consistent order.  The second was to parse CREATE SQL VIEW commands into multiple lines, which otherwise appear in GenDBC on one line, making it very difficult to see what has changed.  I’ll tell you up front that the parsing is not very good, and definitely not as good as I have seen in other VFP products/projects, but I needed something simple and lightweight, and I find it good enough for diff purposes.  Also, GenDBC is a little slow compared to other text file generation, but it wasn’t a showstopper for me.  NOTE: GenDBC_SCCTextX.prg is only intended for source control diff purposes, and I do not recommend it as a replacement of the standard GenDBC.prg for creating databases.

Aside: If you look in SCCTextX.prg, you may notice the developers tried to change the extension for DBC text files from “DBA” to “DCA”.  I agree with this change.  Unfortunately, the VFP Project Manager forces and expects the DBA extension.  If some aspiring developer were to create a fully functional replacement for the Project Manager on VFPX (hint, hint), this (and other limitations with source control integration) could be overcome.  But as it stands, we have no control over it.

That takes care of DBCs, how about DBFs?  Well, it turns out that the code included in SCCTextX.prg for DBCs is actually pretty good for DBFs.  So, easy right? Wrong.  The first problem has to do with the Project Manager integration.  VFP doesn’t even call SCCTextX for files in the Free Tables section.  That explains why we only have the “beginnings” of DBF support.  However, we can trick VFP into calling SCCTextX by putting the DBF into the Databases section of the project.  There are three ways to do this:

  1. Add the DBF manually in the Databases section.  VFP will complain, but the file will still be there.
  2. Hack the PJX (USE MyProject.PJX) and change the type from “D” to “d” on the applicable files.
  3. If the project is open: _VFP.ActiveProject.Files(“MyTable.dbf”).Type = “d”

Once the DBF is in this section, VFP will call SCCTextX and otherwise integrate properly with source control.  SCCTextX_Data.prg is smart enough not to run GenDBC for files that don’t have a “DBC” extension.  The text file extension for both DBCs and DBFs will be “DBA”, so you can’t have a table and database of the same name, but that wasn’t a problem for me.

So far, so good, but there are other issues with DBFs.  You might want to exclude certain fields like ID fields or timestamps that change often and clutter the diffs.  Or you might want set the order for the table to get consistent results.  For this purpose, SCCTextX_Data.prg will call SCCTextX_Custom.prg if it exists in the same directory, giving you an opportunity to specify these settings.  See SCCTextX_Custom – Example.prg in the download.

So now we’ve got text files for DBCs and DBFs, integrated with the Project Manager.  Time for a quick build and… FAIL.  Ugh! VFP doesn’t like the DBFs in the Databases section.  Nothing a little project hook (included in the download) can’t fix though.  It moves all non-DBCs to the Free Tables section before the build and puts them back afterwards.

With all of these caveats, I think it is obvious why I won’t be submitting my changes to the SCCTextX project manager at VFPX.  That said, I’ll tell you it is VERY nice to finally be able to run diffs on these files.  Definitely worth the caveats and overall effort for me.  If you want to try it yourself, feel free to download SccTextX_data.zip.

Why Microsoft Cancelled Visual FoxPro

For a while now, I have debated whether or not to post this entry.  About six months ago, I wrote most of this article, then decided not to post it.  So yeah, I’m wishy-washy on this one.  Since then, Microsoft has put a lot of emphasis on HTML5 and introduced Windows 8 and the Metro UI, leaving a lot of existing MS developers wondering about the future of their previous technology choices.  With that in mind, I think it’s good to look at Microsoft’s treatment of VFP and how their decision processes work in regard to development tools.

Can you believe it has been over four years since Microsoft posted A Message to the Community and announced that Microsoft would cease development on Visual FoxPro?  It has been over three years since Microsoft released VFP9 SP2 and Sedna, and over six years since VFP 9.0 was released!  While I was saddened that Microsoft chose to cancel VFP, I appreciated the sensitivity the Fox Team, particularly YAG, showed to the Fox community.  However, at the time, I wished Microsoft would have been more transparent and given a more thorough explanation of their reasons for making that decision.  YAG may have been constrained in what he could/should say in his position, and I imagine there was some disagreement with the decision within the Fox Team, but those are just guesses.  Regardless, we didn’t get an official statement from Microsoft, other than it was happening, and the Fox community was left to piece together the reasons.  That led to comments like “writing on the wall”, “head in the sand” and conjecture on lack of sales vs. lack of marketing, etc.  Ultimately, it comes down to the fact that VFP was not a “strategic product” for Microsoft, but why was that and what does it mean?  Answers lead to more questions, but I think that is worth exploring.

Note that this blog entry is just more conjecture/opinion.  I don’t have any more facts than you do.  I am just putting all the pieces I have on the table and building a picture.  Over the past couple of years, I have debated whether or not to write this, because it is a negative subject and obvious flame-bait.  But I think enough time has passed now and it could be a good thing.  To move forward, you have to let go of the past, and this has helped me do that.  I still use VFP as my primary tool, but this has helped me “get over” Microsoft’s decision.  It may also be helpful in deciding where you want to go in the future.  Now, I am in no way defending Microsoft or saying I agree with their decision.  I am just trying to understand why they would make a business decision to discontinue FoxPro.  Statements here may be “obvious” or “old news”, but I think it is helpful to pull it all together.

“Not a Strategic Product”

Enough disclaimers, let’s get to the subject at hand.  Microsoft has stated for some time that FoxPro was not a strategic product for them.  What does that mean?  To my mind, a strategic product is one that Microsoft would invest in heavily and recommend as the primary path for their customers.  MS would build upon the technology and form an entire “strategy” around it.  VB was strategic.  COM was strategic. .NET is strategic.  Fox was not.  Why not?  To answer that, I think you have to look at why Microsoft bought Fox Software in the first place.

Why Buy Fox?

So, why did Microsoft buy Fox Software.  I will quote an article Jordan Powell wrote in FoxTalk when Access 1.0 and FoxPro 2.5 were about to ship:

Microsoft was working on its Access DBMS which uses a modern variant of the BASIC language. It had to have been embarrassing for Microsoft to have such a glaring hole in its product lineup. They had no DBMS, and their partnership with Ashton-Tate failed to get Microsoft SQL Server off the ground. Some of the marketing types at MS realized that FoxPro was the best version of X-Base out there, and had been trying to talk Bill Gates into doing something about it. They knew that the X-Base language commanded a huge segment of the market and that a product which used the X-Base language would get them into the DBMS market in a big way. They had the marketing resources to put behind FoxPro, and Fox had some interesting and useful technology — not to mention some very talented people, the kind Microsoft likes.

More recently, Ken Levy wrote the following in his blog:

The purchase of Fox Software for $173 million in 1992 was very strategic for Microsoft, and was the biggest corporate purchase Microsoft had ever made up until that time. Borland had purchased Aston-Tate’s which included dBase III and IV, and had Paradox. And growing in popularity at the time was PowerBuilder as the king of client/server tools, with Sybase releasing PowerBuilder 12 last year ironically based on the free Visual Studio Shell runtime. Microsoft needed three things from the Fox Software deal – the Fox developer team, the Fox technology, and the customer market share of FoxPro/FoxBase. Microsoft was just starting work on Access and it was more targeting power users, but there was still some overlap. Visual Basic was still in its early days.

Basically, Microsoft wanted a stronger presence in the database market and they were in severe need of database products, people, and technology.  Fox Software was a perfect fit for them.  To give you a little more context, in 1992 the xBase market was still booming, Access (Cirrus) was still in development, Visual Basic 1.0 had been released and VB 2.0 was in development, and the first release of SQL Server for Windows was not until 1993.

I think if you asked anyone “in the know” at Microsoft, they would tell you that the Fox acquisition was a resounding success (unlike other much more expensive acquisitions that Microsoft has recently dumped).  They got a solid product, key technology that made its way into several other products, and valuable people that went on to take major roles in the company.  Why then did FoxPro not share that level of success?  I do not believe Microsoft had malicious plans to kill FoxPro from the beginning, but the landscape had changed, as it tends to do in technology.  The xBase market declined.

xBase Market Decline

FoxPro was first and foremost a competitor in the xBase market.  As that market declined, so did the value of FoxPro as a strategic product to Microsoft.  What led to the decline of a technology that had been so popular in the 80’s and early 90’s?  Technology trends are constantly changing, but here are few key things that in my opinion diminished the xBase market:

dBASE IV: dBASE IV was a buggy disaster, and it was two years before they released version 1.1.  Borland bought Ashton-Tate, but could not undo the damage.  dBASE for Windows was not released until 1994.  This was good for FoxPro, which became the biggest fish in the xBase pond, but the pond itself began to shrink.

Lawsuit: Ashton-Tate sued Fox Software for cloning dBASE.  The suit was dropped when Borland bought Ashton-Tate, but it could not have inspired confidence in the xBase market.

Client-Server: By the early 90’s, client-server technology picked up in popularity and developers were beginning to flock towards database servers and client-server development tools like PowerBuilder and VB.  At the same time, Microsoft was trying to enter the server market with Windows NT and SQL Server, so I’m sure there was strong emphasis on this style of development from them.  I believe there was talk of a “FoxServer” product at Fox Software, but it never saw the light of day before the Microsoft acquisition.

Those are reasons that the xBase market declined, but about now you’re thinking that VFP is so much more than an xBase tool.  I couldn’t agree more.  VFP can go toe-to-toe with VB, PowerBuilder, Delphi, .NET, and others.  If FoxPro was supposed to “go quietly into the night”, someone forgot to tell the VFP 3.0 team, because they transformed the Fox into a full-fledged OOP development platform ready for the 32-bit world and beyond.  So, why wasn’t the emphasis there from Microsoft? 

An important point to make about Microsoft is that they are a follower of development trends, not a leader.  With a few exceptions (the VB GUI designer comes to mind), Microsoft has not been the one to create a development trend.  “Embrace and extend” was their motto, and they have done well with that.  Windows was Microsoft’s answer to the Mac.  .NET is Microsoft chasing Java into the enterprise.  They follow current trends and they do so mercilessly.  Even now, Microsoft is emphasizing HTML5, leaving Silverlight developers thinking “Wait, I thought we were on the cutting edge?”  It would be out of character for Microsoft to promote and strategize around a product built for a market that was trending downwards.  It’s nothing personal against the Fox, it’s just not in their DNA. 

FoxPro Market Decline

Even with the xBase decline, if FoxPro revenue had continued upward, I wouldn’t be writing this article.  Sales declined, and there are several reasons for that:

Power Users: Going all the way back to dBASE you could question whether it was a platform for power users with development capabilities or a platform for developers that power users could use.  It was both.  Visual FoxPro put it squarely in the developer category, and Access took over as the preferred database for power users.  The result: much fewer licenses sold.

VB, SQL Server, .NET: VFP faced a lot of competition from other products within Microsoft.  With the emphasis always on the latest trends, many developers felt compelled to move to other technologies. 

Visual FoxPro: That’s right, VFP itself.  While VFP 3.0 was a massive improvement in development capabilities (and most of us are happy with that decision), it was also a big leap from FoxPro 2.x in terms of learning curve.  It took some developers quite a while to make the jump, and some never did. 

Not Invented Here Syndrome:  Microsoft took a great product and made it even better, which makes their treatment of FoxPro all the more frustrating.  But Fox was still the stepchild and it was never going to supersede other products developed internally.  By the time Microsoft purchased Fox, they had already made significant investments in VB, Access, and SQL Server.  Those would be Microsoft’s strategic products while Fox would continue serving the declining xBase market and otherwise fit between the lines.

Why 2007?

People had been foretelling the death of FoxPro since Microsoft bought it in 1992.  What made 2007 the year when Microsoft finally decided to cancel it?  Had sales declined to the point that Microsoft could no longer justify Fox development?  Did they want to use the Fox Team in other parts of Microsoft?  Did big customers move to something else?  Were the people that cared gone or no longer in a position to do anything about it?  Your guess is as good as mine.  We will never know. 

There are a couple of ways to look at this: 1) Microsoft always wanted to cancel Fox and they finally got their way, or 2) in spite of Fox not being a strategic product, Microsoft continue to create new versions for Fox developers.  I tend to think of it as the latter.  While there was always a question of Microsoft’s commitment to FoxPro, by the release of VFP 5, it had become clear that it would not be a strategic product.  Per Ken Levy’s blog:

In the initial years after the Fox Software merger, Microsoft put a huge effort and lots of resources into creating VFP 3.0. There were about 50 people on the Fox team with a big marketing budget. In the following years, both Access and VB grew in market share and also competed in ways with the VFP market (and messaging), and by the time VFP 5.0 was released, many upper managers wanted Microsoft to just end VFP there. In fact, they did for a short time. I was there, in a meeting with 40 people, and the formal announcement was made to the Fox team that VFP was dead. It was very early 1996, and that meeting lead to the Gartner Group releasing their report that VFP was dead, which had a major impact on future VFP sales.

Most of Microsoft’s competitors would have ended it right there and VFP 5 would have been the last version.  So, the real question isn’t “Why 2007?”, it’s “Why not 1996?”.  Ken Levy continues:

But the Fox team members along with the community helped convince the developer tools management to keep VFP evolving while decreasing the resources. In fact, the primary reason VFP lasted another decade with 4 more versions released was more about Windows sales than VFP sales. There are many Windows machines running VFP apps. When Steve Ballmer jumps around like monkey boy and yells “developers, developers, developers”, he’s thinking about selling Windows and Office more than sales of developer tools.

If VFP 5 had been the last version, then I may have never had the joy of working with Visual FoxPro, because I really didn’t make the jump from FoxPro 2.x until version 6.0.  In fact, I’m not sure where I’d be today, as I took my current job back in 2000 to upgrade a Fox 2.x app to VFP.  So, I’m definitely thankful Microsoft saw fit to continue development.

That said, Microsoft’s handling of VFP support since the announcement has been appalling.  VFP 9 SP2 introduced several bugs.  After months of begging, we were able to get them to fix one key bug, but others remain that will never be fixed and must be worked around.  Microsoft claims that VFP is supported until 2015, but I’m sorry, that’s not support.  To be clear, I’m not blaming the Fox Team for this.  I’m blaming Microsoft for the fact that there was no Fox Team and management was unwilling to provide resources to fix these problems.  Real support ended when the Fox Team was disbanded and assigned to other projects.

So, what now?  That’s the big question Fox developers are asking themselves or have already answered.  I don’t know about you, but I continue to be extremely busy with Visual FoxPro as my primary development tool.  I also keep tabs on new technologies as they are introduced with an eye towards how they could benefit me.  Maybe that will be the subject of a future post.

ParallelFox and HyperThreading

Years ago, Intel added a HyperThreading feature to their CPUs before dual-core processors were available. More recently, Intel reintroduced the technology into their “Core i” series of processors. What is HyperThreading and how does it affect ParallelFox? Let’s start with the Wikipedia description:

Hyper-threading works by duplicating certain sections of the processor—those that store the architectural state—but not duplicating the main execution resources. This allows a hyper-threading processor to appear as two “logical” processors to the host operating system, allowing the operating system to schedule two threads or processes simultaneously. When execution resources would not be used by the current task in a processor without hyper-threading, and especially when the processor is stalled, a hyper-threading equipped processor can use those execution resources to execute another scheduled task. (The processor may stall due to a cache miss, branch misprediction, or data dependency.)

What this boils down to is that a single thread/process will not utilize all of the execution “slots” or “units” in a CPU core. This is especially true when the processor is “stalled”, meaning that the processor is waiting on something before it can continue. This may be due to the inherent design of the CPU, or because it is waiting for data to be accessed from main memory. HyperThreading allows a second thread/process to utilize the unused execution slots. Generally, this is a good thing and can provide a 15-30% performance boost to parallel processing.

However, in cases where there is heavy competition among threads for the same execution slots and other resources, HyperThreading can be slower than running a single thread on each core. The examples that ship with ParallelFox exploit this weakness. On a single-core HyperThreading CPU, the “after” examples are actually slower than the “before” examples. Of course, this was not intentional. The reason is that the examples simulate work rather than resemble real-world code. Here is the SimulateWork() function:

Procedure SimulateWork
   Local i
   For i = 1 to 1000000
      * Peg CPU
   EndFor
EndProc

While this code does a good job of pegging a CPU core at 100%, it also causes the same few instructions to be executed millions of times. With HyperThreading enabled, competition between the two threads for the same CPU resources is extreme. In a real-world scenario, there would likely not be this much competition for resources and HyperThreading would be beneficial.

As with most things, your mileage may vary. If you find that your code runs slower with HyperThreading, you can tell ParallelFox to use only half of the “logical” processors and start only one worker per physical core. Here is example code for that:

* Use only physical cores
If Parallel.DetectHyperThreading()
   Parallel.SetWorkerCount(Parallel.CPUCount / 2)
EndIf
Parallel.StartWorkers("MyApp.EXE")

ParallelFox uses WMI (Windows Management Instrumentation) to detect if HyperThreading is enabled. WMI has shipped with windows since Windows 2000. However, WMI can only detect HyperThreading on Windows XP SP3, Windows Server 2003, and later versions, because that is when Microsoft introduced the required APIs. On previous versions of Windows, Parallel.DetectHyperThreading() will always return .f. even if HyperThreading is enabled.

There are several other features I want to add to ParallelFox, but at this point, I think it is feature complete for version 1.0.  Also, very few issues have been reported from previous versions, so I am moving this release up to release candidate status.

Download ParallelFox at VFPX.