Reg-Free COM with Out-of-Process Servers

Generally speaking, a COM object must be registered in the Windows registry to be used by applications. In the past, I have not had an issue with this, as it was just another step in the installation package. In recent years, we have been offering cloud services to our clients. On a single server, we now have multiple clients and dozens of users all relying on the same COM objects. This makes it more difficult to deploy updates to a COM object. It also prohibits having multiple versions of a COM object, because only one version can be registered at a time (AKA “DLL Hell”). These days, registration-free COM is looking a lot more attractive, as it enables multiple “side-by-side” versions of a COM object that can be managed independently. It also allows deployment without the admin elevation that is normally required to register a component.

For a while now, we have had the ability to use custom manifests that allow applications to make use of COM objects without registering them. Some Visual FoxPro devs might make use of tools by Craig Boyd to assist with this. Visual Studio provides tools for automating manifests as well. There’s one catch… this technique only works with in-process DLL COM servers. I often use out-of-process EXE COM servers for Visual FoxPro because they have fewer limitations than DLLs. Is there a way to use an EXE COM server without registering it first?

A cursory Google search might lead you to answer “No”, but Larry Osterman of Microsoft fame tells us otherwise. In his blog post from 2005, he points out a “clever trick” to accomplish this: “When COM tries to activate a COM object, before it looks in the registry, it looks to see if there’s a server already registered for that class.” In more detail… if the COM server EXE has already been launched and CoRegisterClassObject() was called BEFORE instantiating the COM object in a client application, the COM object can be used even though it is not in the Windows registry. Clear as mud?

First of all, we are talking about two different types of registration here. There’s the standard registration of a COM object into the Windows registry that we are all familiar with. Then there’s CoRegisterClassObject(), which registers out-of-process COM objects in memory with the Windows COM/OLE system (not the Windows registry). This is done from inside the COM server, and it is what allows the COM objects to be found by client applications. Okay… so how do we call CoRegisterClassObject()?

Good news! No Win32 API calls are necessary. Your out-of-process COM server already knows how to do this, you just have to tell it to do so. There is not a mandatory standard for all COM servers, but Microsoft convention is to include the following parameters when launching the EXE: “/automation -Embedding”. This convention is used for VFP COM EXEs, MS Office, and probably many others. Here’s some VFP code for launching the COM EXE:

Local lcRun
lcRun = [Run /n "] + FullPath("MyCOMServer.exe") + [" /automation -Embedding]
&lcRun

I have used FoxPro’s RUN command to launch the executable. Win32 API ShellExecute() could be used as well, although I should mention I did run into some timing issues that I didn’t bother to troubleshoot. CreateProcess() is another option.

If your COM server doesn’t use this convention, register and instantiate it normally on your dev machine. Then open Windows Task Manager, go to the Details tab, and enable the Command Line column. That will show you what parameters were included when the COM EXE was launched.

At this point, the COM EXE server is running and ready to be used by a client application. You should be able to use VFP’s CreateObject() to instantiate the object, right? Unfortunately, no. CoRegisterClassObject() has registered the object in memory using a “Class ID” (CLSID), which is a GUID. CreateObject() expects the more friendly “Program ID” (ProgID) that we are familiar with. Behind the scenes, CreateObject() calls CLSIDFromProgID() to lookup the ProgID in the Windows registry and return the corresponding CLSID, then it calls CoCreateInstance() using that CLSID. We want to avoid using the Windows registry for deployment, so job one is to find the CLSID for our COM object. To do that, you can look in the Windows registry on your development machine and find your ProgID under HKEY_CLASSES_ROOT. For VFP, there is also a VBR file in the project folder that you can open in a text editor and find an entry like this:

HKEY_CLASSES_ROOT\MyCOMServer.MyObject\CLSID = {12345678-90AB-CDEF-1234-567890ABCDEF}

Now that we have the CLSID, we are ready to instantiate the object. Again, VFP’s CreateObject() will not accept a CLSID, but CreateObjectEx() will!

#DEFINE MYCOMOBJECT_CLSID = "{12345678-90AB-CDEF-1234-567890ABCDEF}"
loMyComObject = CreateObjectEx(MYCOMOBJECT_CLSID, GetEnv("COMPUTERNAME"))

I should point out that this calls CoCreateInstanceEx() rather than CoCreateInstance() behind the scenes. This is typically used to instantiate a remote DCOM object on a different machine, but I have designated that it is on the current computer. I have not encountered any repercussions from doing this (performance or otherwise), but your mileage may vary. You could call CoCreateInstance() directly to instantiate the object as a “Local” out-of-process server instead.

We now have everything we need to instantiate an out-of-process COM object without registering it, but one caveat remains. FoxPro’s RUN command and ShellExecute() cause the client application to lose focus when the COM server EXE is launched. This doesn’t happen when instantiating a COM object that has been registered the normal way. You could probably prevent this by using CreateProcess() to launch the server, but it has a complex API. Using LockSetForegroundWindow() seems to avoid the unwanted behavior. Here’s the complete code:

#DEFINE MYCOMOBJECT_CLSID = "{12345678-90AB-CDEF-1234-567890ABCDEF}"
Local lcRun, loMyComObject 
DECLARE INTEGER LockSetForegroundWindow IN user32 INTEGER uLockCode
LockSetForegroundWindow(1) && lock	
lcRun = [Run /n "] + FullPath("MyCOMServer.exe") + [" /automation -Embedding]
&lcRun
loMyComObject = CreateObjectEx(MYCOMOBJECT_CLSID, GetEnv("COMPUTERNAME"))
LockSetForegroundWindow(2) && unlock

Double Trouble with COM Interop

I have been calling FoxPro code from C# via COM Interop (using DotNet2Fox) for several years, but I ran across an interesting issue last week. I have some FoxPro code that performs calculations and returns the results to .NET. I expected one of those values to be zero, but according to C#, it was -3.5527136788005e-15. Huh? After repeated tests, I got the same result.

I was assigning the FoxPro results to C# variables of type “decimal”. The C# compiler had already informed me that FoxPro was returning values with type “double” over COM. I have heard about the tension between binary and floating point math in computers before, but in practice, I have never run into an issue. I initially thought that the double-to-decimal conversion in C# was exposing this tension. As a test, I changed my C# variables to type double, but the problem persisted. I stepped through my code in FoxPro, and according to the debugger, the value was indeed zero. What the heck was going on?

I came up with a simpler calculation to reproduce the issue: 1.23 + 4.56. The result should be 5.79, right? Enter ? 1.23 + 4.56 in the FoxPro command window, and that’s what you’ll get. However, run the code below in C# and the result will be 5.790000000000001.

        dynamic vfp = Activator.CreateInstance(Type.GetTypeFromProgID("VisualFoxPro.Application", true));
        var result = vfp.Eval("1.23 + 4.56");
        vfp.Quit();
        return result;

Had I run across some glitch with COM interop? I needed to be certain that calculations done in FoxPro would come over to C# accurately. This was disconcerting.

Upon further thought… someone would have surely caught an issue like this with general COM interop, right? Maybe this was more on the FoxPro side. Then, I remembered a blog entry Christof wrote years ago on Significant digits in Visual FoxPro that mentions some oddities with 15-digit floating point calculations in VFP. Hmmmm… 15 digits. That’s the same precision of double types in C#. And look how many digits are after the decimal place in 5.790000000000001. So, I tried another test in the FoxPro command window:

SET DECIMALS TO 15
? 1.25 + 4.56

There it is! Now, FoxPro shows 5.790000000000001! Clearly, this is either some kind of binary-to-floating-point issue or something to do with that 15th digit, but SET DECIMALS masks that most of the time. SET DECIMALS is a mysterious command. It specifies the number of decimal places that will be displayed from calculations. It apparently also affects comparisons: ? 1.25 + 4.56 = 5.79 returns .T. when SET DECIMALS TO 2. But SET DECIMALS TO 15 and the same expression returns .F. Weird! SET DECIMALS appears to have no effect on calculated values returned over COM interop and always returns the full 15-digit precision of a double.

So, that explains why I had the issue when using COM interop. How do we fix it? Simple… round the calculated result to the desired number of decimals in FoxPro before returning the result, or in C# after the result is returned from FoxPro. In fact, I was able to resolve by rounding down to 14 digits (eliminating the troublesome 15th digit), but I can’t guarantee that would always work. By the way…if you want C# rounding to have the same behavior as FoxPro rounding, specify MidpointRounding.AwayFromZero:

    public static double FixDouble(double number, int roundDecimals = 14)
    {
        return Math.Round(number, roundDecimals, MidpointRounding.AwayFromZero);
    }

DotNet2Fox is now part of VFPX!

I’m happy to announce that DotNet2Fox has been accepted as part of the VFPX project.  “What is DotNet2Fox?”, you say?  Good question!  DotNet2Fox was first presented and released at Virtual Fox Fest 2021 (October), and I somehow failed to mention that here on my blog!  That is being rectified right now…

DotNet2Fox is an open-source library that simplifies calling Visual FoxPro code from .NET desktop and web applications. Have you tried your hand at COM interop only to be met with limitation after limitation? Did you find that exposing your existing FoxPro code over COM would require a major refactoring effort? DotNet2Fox provides a simplified interface for calling into existing real-world FoxPro code, and without all those limitations.

Over the past several years, I have been working to surface features from my desktop applications and make them available on the web via ASP.NET.  I found that the classic approach to COM interop recommended by Microsoft a couple of decades ago did not fit my needs. Over time, a library emerged to meet those needs: DotNet2Fox.

Geek Gatherings has graciously made recordings of Virtual Fox Fest sessions available to the entire FoxPro community on YouTube.

You can also get the whitepaper (documentation) and examples at the GitHub Wiki. Yes, full source code is on GitHub as well.  If you need help with DotNet2Fox, please start a Discussion or report an Issue. We’ll even accept Pull Requests.  I look forward to working with more folks  on this project.

COM Interop with .NET Core 3.x and .NET 5.0

I previously wrote about COM Interop with .NET Core 1.0 and .NET Core 2.0. I haven’t had a chance to write it about it since, so this discusses both .NET Core 3.x and .NET 5.0.  If it wasn’t obvious from the previous articles, this is about calling COM components from .NET.  I believe work was done using COM interop in the other direction (calling .NET from COM components), but it is not discussed here. If you need to call .NET from Visual FoxPro, wwDotNetBridge is much simpler than standard COM Interop and highly recommended.

.NET Core 3.x

.NET Core 3.0 was a big release.  Unlike ASP.NET Core 2.0, ASP.NET Core 3.0 removed the option to run on .NET Full Framework 4.x.  It only runs on .NET Core, meaning you can no longer fall back to 4.x for COM Interop support.  WinForms and WPF desktop applications were also brought into the Core fold for the first time, and some of those apps might rely on COM Interop.

Some progress was made in 3.0, but the overall COM Interop story is the same as .NET Core 2.0.  That is, COM Interop works, but late binding using the C# dynamic keyword still doesn’t function.

dynamic excel = Activator.CreateInstance(Type.GetTypeFromProgID("Excel.Application", true));
excel.Visible = true;
Console.WriteLine("Press Enter to close Excel.");
Console.ReadLine();
excel.Quit();

As with .NET Core 2.0, the above code will function in .NET Framework 4.x, but in .NET Core 3.x, an exception occurs:

‘System.__ComObject’ does not contain a definition for ‘Visible’

Microsoft had planned to bring full support for the dynamic keyword to .NET Core 3.0, but the job proved to be too large, so it was deferred. However, there are a couple of improvements.

In .NET Core 2.0, you had to jump through hoops to create an interop assembly for COM components.  In .NET 3.x projects with Visual Studio 2019, you can simply right-click on the project and select “Add COM Reference”.

In addition, some folks put together a workaround using a DynamicObject wrapper to access properties/methods on COM objects using the dynamic keyword, so you don’t have to use ugly reflection calls.  This may not work in every scenario, but you may find it meets your needs on .NET Core 3.x.

.NET 5.0

.NET 5.0 was just launched at .NET Conf on November 10, 2020. It represents the reunification of .NET Core and .NET Full Framework, hence the dropping of “Core” in the product name.  .NET 4.x remains a part of Windows and will continue to be supported indefinitely.  Some things are not coming over to .NET 5, such as Web Forms, WCF, and WWF… at least not by Microsoft. There are open-source projects to port WCF and WWF.  But what about COM Interop?  In my testing, it just works.  Indeed, Microsoft finished the work to support using the dynamic keyword with COM objects.  There’s not much more to say than that.  If incomplete COM Interop support has been keeping you on .NET Full Framework 4.x, now may be the time to take a look at .NET 5.0.

COM Interop with .NET Core 2.0

We previously discussed that COM Interop does not function in .NET Core 1.0.  What about .NET Core 2.0?  The answer is YES… with limitations.

The first and most obvious is that COM Interop only works with Windows, not other platforms.

The second limitation is that the .NET Core implementation does not include IDispatch, which means that late binding is not supported.  Almost 10 years ago, Microsoft introduced the dynamic keyword in C# 4.0, which made late binding with COM and other platforms much easier.

dynamic excel = Activator.CreateInstance(Type.GetTypeFromProgID("Excel.Application", true));
excel.Visible = true;
Console.WriteLine("Press Enter to close Excel.");
Console.ReadLine();
excel.Quit();

The above code will function in .NET Framework, but in .NET Core 2.0, an exception occurs:

‘System.__ComObject’ does not contain a definition for ‘Visible’

If you look in Task Manager, you’ll see that Excel.exe has indeed started, but the object members cannot be access directly without IDispatch.  To work around this, you can use interop techniques that pre-date the dynamic keyword.

Interop assemblies are wrappers that enable .NET Core to interact with COM objects using early binding.  Microsoft provides interop assemblies for Office automation on NuGet and elsewhere.  Once the assemblies are installed, this code will work:

using Excel = Microsoft.Office.Interop.Excel;
...
var excel = new Excel.Application();
excel.Visible = true;
Console.WriteLine("Press Enter to close Excel.");
Console.ReadLine();
excel.Quit();

Aside: For web applications, Office automation is not recommended.  Check out the Open XML SDK.

What about your own COM objects? .NET Core projects do not provide a means to reference them directly.  However, you can create a .NET Framework 4.x project and add a reference to the COM object.  This will create an Interop.MyCOMObject.dll assembly in the obj/Debug folder, which you can then reference directly in the .NET Core project.

A COM object may return other objects that are not part of the type library, and early binding is not an option. This happens often in my Visual FoxPro interop code. You can use reflection to access the object members.

object excel = Activator.CreateInstance(Type.GetTypeFromProgID("Excel.Application", true));
excel.GetType().InvokeMember("Visible",BindingFlags.SetProperty, Type.DefaultBinder, excel, new object[] { true });
Console.WriteLine("Press Enter to close Excel.");
Console.ReadLine();
excel.GetType().InvokeMember("Quit", BindingFlags.InvokeMethod, null, excel, null);

The dynamic keyword makes for more natural code, but you can make reflection less cumbersome than above with your own wrapper methods.

For the adventurous, the Powershell team has created their own implementation of IDispatch. I don’t know how reusable this implementation is, but it may be worth a look.

If you’re using in-process (DLL) COM servers, be aware of 32-bit vs 64-bit issues. For web  applications, take a look a Rick Strahl’s post on STA components. I don’t know if these techniques are available in .NET Core.  In my experience, these issues do not apply with out-of-process (EXE) COM servers, but your mileage may vary.

Lastly, keep in mind that ASP.NET Core 2.0 continues to run on .NET Framework 4.x, in addition to .NET Core.  If you’re already restricted to Windows, there aren’t many reasons to prefer .NET Core over the full framework (yet), so it remains the best option for COM Interop.  That said, it’s good to know these possibilities exist with .NET Core.

Microsoft has been choosy about what they bring over to .NET Core.  Over time, they have been finding their way towards more parity with .NET Framework. It was recently announced that WinForms/WPF will be coming to .NET Core 3.0. They may find that a lot of existing code relies on late binding.  I would not be surprised if IDispatch makes a comeback.

COM Interop with .NET Core

Logo_DotNet

UPDATE: COM Interop with .NET Core 2.0

The short story is that COM interop does not function in .NET Core 1.0. .NET Core is Microsoft’s open-source, cross-platform implementation of the core libraries in the .NET Framework.  With its cross-platform focus, the exclusion of COM interop is intentional.

ASP.NET Core is Microsoft’s open-source web framework that sits on top of the base .NET framework.  If you require COM interop and want to take advantage of new features in ASP.NET Core 1.0, you are in luck. Scott Hanselman reminds us that ASP.NET Core runs not only on .NET Core, but also on the full .NET Framework 4.6.

Here is some typical code for instantiating a COM object in C#:

dynamic myObject = Activator.CreateInstance(Type.GetTypeFromProgID("My.COMObject", true));

If you try to compile this code with .NET Core 1.0, it simply won’t work because Type.GetTypeFromProgID() is not available in the API.

That describes the current situation, but what about the future?  A while back, there was actually talk about bringing pieces of COM to Mac and Linux.  I think those plans have been scrapped or would only have limited use.

.NET Standard is an effort to bring a minimum common API set to all .NET platforms, present (Full Framework 4.6, .NET Core, Xamarin, Mono) and future.  .NET Standard 2.0 will be implemented in .NET Core 1.1, and it brings in a lot of missing APIs from the full framework. (UPDATE: .NET Core 1.1 has been released since this was written. It includes many new APIs, but not full support for .NET Standard 2.0. That will show up in a future release of .NET Core.)   It should make porting existing code to .NET Core much easier.  One of the APIs slated for 2.0 (as of this writing) is Type.GetTypeFromProgID().  That means that COM interop will work on .NET Core 1.1, right? Wrong. Calling this method will throw a “Not Implemented” or “Platform Not Supported” error.  As I was told by a .NET Foundation member:

There is often incorrect assumption made that “included in .NET Standard” == “works in .NET Core”. There are going to be some APIs that will throw PlaformNotSupportedException on .NET Core, also this set be different between Windows and Unix.

That’s a bit counter-intuitive to me. First of all, it seems like .NET Core should be the “reference implementation” of .NET Standard.  Beyond that, the availability of APIs on unsupported platforms may lead a developer to believe an API is fully functional when it is not. To solve that issue, tooling is coming that will identify APIs that are not supported on certain platforms/runtimes (hopefully before a developer has gone through a porting effort). Also, keep in mind that a least-common-denominator approach has already been tried with Portable Class Libraries, and the .NET team is going for something better. The .NET Standard team is currently accepting feedback on GitHub, so feel free to post your thoughts and questions.

Looking beyond .NET Standard 2.0 and .NET Core 1.1, several COM/native interop pieces have already been segregated for a possible future extension. Also, the source code for the underlying functionality is readily available.  I think it is only a matter of time before COM interop will be available for use with .NET Core.

References:

Debugging in ParallelFox

ParallelFox 0.6 Beta was released today.  Earlier this week, I posted a couple of new training videos about Worker Events on VFPX.  When I started, I figured the training video would be 30-45 minutes to cover all the features in ParallelFox.  I’m now up to 4 videos clocking in at almost two hours.  Part of that is my slow presentation (sorry for that, I was doing most of it off the cuff), but if you’ve ever put together videos like this, then you know how time consuming they are.  I feel like it was important to actually demonstrate those concepts in action, but for the rest of the topics, that is not a requirement.  It will be quicker for you and me if I present the remaining topics in written form, and this is the first entry in that series.  I will link to all entries from the main ParallelFox page on VFPX.  Occasionally, a video may be called for, in which case I will create a short video and link to it from the associated blog entry.  These entries may someday make their way into a Help file.  Ok, let’s get started with the first topic…

Debugging
FoxPro does not allow you to debug into COM servers.  To spell that out, suppose you created a COM server in VFP, and then instantiated that object in VFP as well.  The code would look something like this:

Local loMyCOMObject as MyCOMServer.MyObject
loMyCOMObject = CreateObject(“MyCOMServer.MyObject”)

Set Step On

loMyCOMObject.DoSomething()

In the code above, Set Step On would open the debugger, but if you tried to step into the DoSomething() method, you would not be able to see the code inside the COM object.  VFP would execute the method then return to the calling program.  Calling Set Step On (or setting a breakpoint) inside the COM object doesn’t work either.  To work around this, you have to instantiate your objects as regular FoxPro objects and fully debug them before building them into a separate COM server.

Aside: Years ago, Robert Green and Calvin Hsia demonstrated debugging FoxPro COM objects from Visual Studio. This was around the time Microsoft introduced .NET, and even though they had already decided there would be no VFP.NET, VFP still needed a good integration “story” if it were to remain part of Visual Studio.  You know the rest of the story.  The Fox was taken “out of the box” before the release of VFP 7.0, so it no longer needed that story, and the feature was dropped.

This limitation could pose some problems for ParallelFox, because code is run in parallel by means of COM servers.  It is not convenient to simply instantiate an object in the main process, and the code you are running in parallel may not be inside a class anyway.  Even if it were convenient, the code would be not be running in parallel, only in the main process, so it’s not exactly an ideal situation.

Fortunately, ParallelFox makes it easy to debug your code by providing a debug mode.  To turn on debug mode, simply pass .T. as the third parameter to the StartWorkers() method:

Local Parallel as Parallel of ParallelFox.vcx
Parallel = NewObject(“Parallel”, “ParallelFox.vcx”)

Parallel.StartWorkers(“MyProc.prg”,,.t.)

This tells ParallelFox to start the workers in full instances of VFP, rather than as standard COM objects.  Simply SET STEP ON in your worker code (breakpoints may not transfer to worker instances), and the debugger will open in the worker instance.

To use debug mode, the main process needs to know where ParallelFox.vcx and WorkerMgr.vcx are, so make sure they are in your path.  If you’re going to use the Worker object, the worker processes need to know where ParallelFox.vcx is as well, so make sure your workers can find it before you instantiate the Worker object.

VARIANTs on Fox

As I mentioned previously, I have been experimenting with integrating our application with Maximizer CRM.  Working with their SDK, it doesn’t take long to figure out that the com interfaces were designed with VB and C++ programmers in mind.  Most of it works with VFP, but there are a few exceptions.  I ran into a problem with an interface that accepts a variant variable by reference.  I was surprised to find out that Fox can’t create a variant.

Shirley I can’t be serious?  Everyone knows that all variables in Fox are variants.  While that may be true, Fox can’t create a variable of type variant that can be passed to com components.  Fox always assigns a type to a variable.  Even a simple local myvariable statement creates a logical variable.  Attempting to pass the variable by reference to a com component may result in a “type mismatch” error.

This isn’t entirely Fox’s fault.  After all, should a COM component assume that the calling environment can create a variant?  Microsoft concedes this is a problem in C++ with MFC components and recommends using ATL instead (see link below).

So, what can you do if you run into this situation?  Ideally, the COM interface would be changed not to use variants, but I doubt Maximizer will change their interfaces, in place for years, just so I can use VFP.  VB does allow you to create variables of type variant, so I created a VB wrapper component.  VFP passes a string to the VB component, the VB component converts that to a variant and passes it to the Maximizer component.  It’s not an ideal solution, but it works.

Prb: ActiveX controls passing variant* back to VFP cause error