Nowadays, when incorporating database support into a .NET application , its common practice to use the Microsoft Enterprise Libraries. Since we are developing our application in Visual Studio 2008, we opted to use the most current version (4.1) of the Enterprise Libraries.

The application configuration file consists of three sections that define the target database and connection criteria as shown below:

The onfigSections ties the attribute name="dataConfiguration" to the Enterprise Libraries.

The <connectionStrings> section defines one or more connection strings that specify which database to connect to, logon credentials, machine source, etc.

Lastly, the dataConfiguration ties the Enterprise assemblies to a specific connection string specified by defaultDatabase="ConnectionStringName".

If the defaultDatabase name does not match a connection string, the following error will result:

12/21/2009 14:20:18 : [83c] [_ERROR_]Caught in: SomeWebServer::InitServer
Exception in: D:\ DBLayer\ DB.cs set_ConnectionString(177,4): The type initializer for 'DBLayer' threw an exception.

When Things Go Horribly Wrong

The client I am at decided that the app.config file was not to be deployed with the standard setup, but rather would be generated from XSLT scripts and deployed on a per machine basis. That seemed like a reasonable approach, but they didn’t test it until we were ready to deploy to production. I’m sure you can guess what happened next.

12/21/2009 14:20:18 : [83c] [_ERROR_]Caught in: SomeWebServer::InitServer
Exception in: D:\ DBLayer\ DB.cs set_ConnectionString(177,4): The type initializer for 'DBLayer' threw an exception.

My first reaction was to check the connection strings and the defaultDatabase name. They were a spot on match.

It gets worse.

Typically when using the Enterprise Libraries developers include and deploy the pre-built assemblies that come with the package. Unfortunately that means incorporating the individual projects into our solution.

The customer did not like this solution as it changes the binary footprint of the application even though we assured them it was only temporary.

XML Gotchas

It's obvious that there were differences between my app.config file and the resulting XSLT generated file. It's been my experience in the past that these problems are often caused by non-printable characters in the text files. So the obvious thing to do was to look at the app.config and the XSLT file in a binary editor.

When I opened up the app.config file in binary mode, I was surprised to see 3 bytes that preceded the first printable characters…

What are those? And more to the point, why don’t they show up as ‘.’ or some other funny characters?

Those turn out to be what is known as a Byte Order Mark (BOM).

The byte order mark (BOM) is a Unicode character used to signal the endianness (byte order) of a text file or stream. Its code point is U+FEFF. BOM use is optional, and, if used, should appear at the start of the text stream. Beyond its specific use as a byte-order indicator, the BOM character may also indicate which of the several Unicode representations the text is encoded in.

In prior versions of the Visual Studio IDE going back to 2002, the app.config file is produced as straight XML. In Visual Studio 2008, the editor now adds the BOM to the beginning of the file. The 4.1 parser for the Enterprise Libraries expect the BOM to be the first thing to appear in the app.config file. The XSLT scripts were not coded to insert this byte sequence.

That should have been the end of the story. Of course, there was one more catch…  When developers edit the app.config file, they typically use the IDE’s XML editor. When tags don’t match, the IDE will highlight the offending tags with a red squiggle line as shown below:

When the XSLT scripts generated the configuration files, we only looked at them in Notepad. Without the aid of the red squiggle lines, we couldn’t see the XML errors that crept in. One I used the IDE to look at the files, the mistake was obvious. In this case, the Transport section is missing the ‘/’ in the terminating <value> tag.

Lesson learned.

We just ported the entire application codebase for one of our clients from Visual Studio 2005 to Visual Studio 2008. While working on the codebase we also decided to deploy the code on Windows 7.

Many of our managed applications are actually a composite of C# assemblies, managed C++ assemblies and InProc COM components. We’ve spent many man hours addressing the precarious marriage between the managed and unmanaged world.

While unit testing our work, we ran into a peculiar problem. When we tried to run one of our unmanaged applications, the following message box was displayed:

 A closer look in the event viewer revealed the following:

That’s odd… until now we’ve never had to worry about managed applications requiring manifest files for side by side issues.

Managed C++ Assemblies

After pondering what we saw, it occurred to me that the incorporation of managed C++ assemblies was the most likely cause of the incompatibility. In the past looking at the About box in the version of the Visual Studio IDE used to build the target binaries was sufficient to determine the runtime version of the C/C++ assemblies. I clicked on the About box and made a note of the version of the runtime DLLs for this instance of the IDE.

 

The next step then is to add a manifest file to the target executable and set the version to the number in the About box. The resulting file looks something like this:

I compiled and ran the application… no dice.

It occurred to me that maybe we need to ensure that the managed C++ assemblies are binding to the correct resource versions for the runtime DLLs.

Managed C++ Manifest Support

The best way to ensure that the managed C++ assemblies bind to the correct runtime versions is to specify those version directives as linker directives. To do this, we create a new header file whose sole purpose is to emit the proper linker directives in a hands off fashion as follows:

 

I included this header in stdafx.h and proceeded to rebuild the DLL. The output I got was puzzling, to say the least…

Wait a minute… the IDE is set to 9.0.30729.1, yet the version the compiler sees is 9.0.21022.8. What is going on?

After digging around on the internet, I find that adding the following directive to the stdafx.h file should force the version to be the most current.

#define _BIND_TO_CURRENT_VCLIBS_VERSION 1  

I add this and recompiled. Now the output is producing even stranger results:

What?

Alright, if that’s the version that’s current then we’ll go with that. I recompile the application and run only to be faced with:

 

Clearly something is amiss…

New Windows 7 Debugging Tools to the Rescue

Windows 7 comes with kernel support for enhanced deep dive tracing and logging of any event you can possibly imagine. This means developers can now trap metrics at a level far deeper than ever before possible, and with OS support that does not require code change.

Since we are facing a Side by Side issue, I chose to use sxstrace. Setting up the trace tool is rather straight forward. To start the trace, I ran the following command in a separate DOS box:

I then ran the application directly using the Windows explorer tool. One I got past the message box telling me I couldn’t run the application, I went back to the DOS box and hit <ENTER> to stop the trace.

The next step is to parse the .etl file to produce human readable text. I ran the following command in the DOS box:

 

The resulting file told the tale:

 

Error in the XML configuration file? The IDE clearly shows that the app.config file parses fine and is not in error. A look at line 1 shows the standard XML header.

What is going on?

 

A Sad Tale of XML Standards

The answer was staring me right in the face. Let me explain.

In prior versions of the Visual Studio IDE going back to 2002, the app.config file is produced as straight XML. The encoding specified is the code page for the target language supported. Since that time the standards committee has adopted the specification of using UTF-8 or UTF-16. On further examination, there are non-printable bytes that precede the first text character know as Byte Order Mark (BOM). This tells the XML parser what type of character set to expect before it begins parsing that text.

 The byte order mark (BOM) is a Unicode character used to signal the endianness (byte order) of a text file or stream. Its code point is U+FEFF. BOM use is optional, and, if used, should appear at the start of the text stream. Beyond its specific use as a byte-order indicator, the BOM character may also indicate which of the several Unicode representations the text is encoded in.

Apparently, there are several different parse engines that parse the app.config file. The parser that reads the app.config for key-value pairs, etc understands the old VS2005 format. The manifest parser, however, chokes on the old format, and stopped on the keywords encoding="Windows-1252". The final piece of the puzzle fell into place when we changed that to the new standard:

encoding="utf-8"

But that’s only one part of the solution. To add the correct BOM we actually had to regenerate the app.config file in the VS2008 IDE. Once this last step was completed, the application ran fine.

 

In the beginning there was DOS. And lo DOS was crippled by the 640K limitation and the segmented architecture of the 8086/80286 real mode. To address this problem Microsoft said “let there be int 21h, function 40h. Let there be overlay support”.

But Microsoft saw that this was difficult for programmers to grasp (meaning Lotus had already incorporated this into their flagship spreadsheet product) and said “we need a way to give programmers a way to write overlay code without mucking with interrupt tables.”  They proceeded to incorporate overlay support into Interface Manager, which in 1985 was launched as Windows 1.0. And they said, “let us name this overlay support Dynamic Link Libraries so that no one will confuse them with UNIX loadable libraries.”

It was evening, it was morning, the first version of Windows.

But trouble soon began as these DLL files shared the extension of “.exe” with their process counterparts. Worse, these DLLs could not take advantage of the “new” memory addressing capabilities of EMS and XMS. They proceeded to release Windows 2.x and renamed these DLLs with the extension “.dll”. Windows 2.1 recognized EMS and XMS memory, and would wipe clean any attempt by Lotus 123 to utilize that memory space. And Microsoft saw it was good.

It was evening, it was morning, the Windows 2.x version of Windows.

Microsoft continued to utilize DLL technology to expand Windows bloat. But the developers were plagued by the lack of version control as they could not distinguish one version of a DLL from another. And the users named this plague “DLL Hell”.

But Microsoft cared not as they had solved the Intel segmented architecture problem, and implemented “Protect Mode” in Windows. They could now double and triple up the amount of memory Windows could consume, and they saw it was good!

It was evening, it was morning, the Windows 3.x/386 version of Windows.

As Microsoft pondered this “DLL Hell” problem their application team said “Lo! We have this new technology called Object Linking and Embedding (OLE), and we need to specify where Windows might find the correct DLL to load. Lets add to the registry bloat and specify the location of these DLLs.” And Microsoft saw it was good.

It was evening, it was morning, the Windows 95 version of Windows.

But developers still bemoaned the huge learning curve to write OLE support, and clamored for a solution to DLL hell which basically allowed the last OLE DLL registered to win, surprising application vendors with overwhelming calls to technical support. So Microsoft introduced the “new OLE” dubbed Component Object Model (COM) and its versioning scheme. Developers could specify a specific version to load, or a version independent request to load the latest DLL. Since Windows wasn’t slow enough, they added DCOM to allow calls to flood the network and slow by a factor of 10. And Microsoft saw it was good.

It was evening, it was morning, the Windows 98 SE version of Windows.

But Dave Cutler was brooding that developers were largely ignoring his Windows “New Technology” (Windows NT) OS in favor of “those toy OS’s”. Worse, corporate America was slow to adopt NT 3.5, 3.51 and 4.0. So Dave and his boys set out to increase the bloat and memory footprint by orders of magnitude, and to give NT the same look and feel as Windows 95/98, allowing employees to waste time and CPU power playing .wav files and MineSweeper. So DLL hell increased, and developers were forced to use undocumented methods for checking versions of DLLs and COM components.

It was evening, it was morning, the Windows 2000 version of Windows.

Now Microsoft had been sleeping at the switch, and allowed the internet to pass them by. So they declared “behold let us take back the internet and force everyone to switch from Netscape to Internet Explorer. Further, let us solve DLL hell once and for all by allowing websites to send COM DLLs across the wire without regard for security. Let’s also rename these DLLs from COM to ActiveX, and called this new paradigm Distributed interNet Architecture (DNA).” And Microsoft saw it was good.

It was evening, it was morning, the Windows XP version of Windows.

The Microsoft said “Let’s leave our developer base in the dust and introduce a new programming paradigm. Let’s call that new paradigm .NET, and let’s solve DLL hell once and for all by introducing assemblies. Since the registry is already over bloated, let’s create a new bloated database call the Global Assembly Cache (GAC). This will truly solve DLL hell, and allow different versions of DLLs to live in peace and harmony. Let's further make the lives of our developers miserable by introducing new security restrictions which should address the press reports of security holes in our beloved OS.” And Microsoft saw it was good.

It was evening, it was morning, the Windows XP SP2 version of Windows.

But the developers continued to wail about DLL hell. Raw DLLs were blindly overwritten. COM support defaulted to the “version independent” calls for loading DLLs. Assemblies were strewn all about the hard drive or competed for precious space in the GAC. DCOM “local servers” became process shells such as ConHost, SvcHost and RunDLL loading yet more DLLs. Yet security was breached on a daily basis. The most offensive parasites were the myriad versions of slipstreamed DLLs for the C/C++ Runtime (CRT), Microsoft Foundation Class (MFC), Active Template Library (ATL), and other quasi-OS DLLs. And Microsoft saw it was not good…

So Microsoft declared “Behold, we now give you Side by Side (SxS) DLL support, which will definitely absolutely solve DLL hell once and for all. Further, we will introduce and enforce user access control in our newest and spiffiest version of Windows which we will call… Vista. You will cease to use Windows XP, which also supports SxS DLL technology, nudge nudge wink wink.” And Microsoft saw it was good, and pulled OEM support for XP.

It was evening, it was morning, the Vista version of Windows.

But there was a hue and cry from the developer community as they now faced infinite combinations of DLL hell… legacy DLLs, legacy and “new” COM (on which the .NET foundation is built), assemblies, side by side DLL versions, downloaded ActiveX controls, and the inability to control the most simple of tasks without the dreaded UAC message box rearing its ugly head. And Microsoft saw it was not good…

So they went back to the drawing board one more time. This time they had the solution to DLL hell once and for all. This time they could not fail. They decided that declarative programming was the new old thing. Developers could write code in XML which would give developers triggers, setters, properties, events, and commands. They dubbed this new savior eXtensible Application Markup Language (XAML). Any of that “dirty C# code” could be tucked away in code behind files. DLLs would be managed for developers. The end of DLL hell.. and while they were at it, quietly fix all that was wrong with Vista and release it as… Windows 7.

It was evening… a very long evening. Windows 7 is here.

Please wake me up when the morning comes… maybe Microsoft will finally have a solution to DLL hell.

<sigh>

I was standing in line at the registration desk for the 2005 PDC in Los Angeles behind two developers deeply engrossed in conversation about the future of Microsoft technology. One of the developers was adamant that COM was dead and that all technology moving forward would be in .NET. I chuckled to myself realizing that these two developers had no real understanding of the role COM played (and continues to play) in .NET. In fact, COM is so deeply embedded in the .NET core and the Windows O/S that you can’t really talk about .NET without talking about COM.

Fast forward to the PDC 2009 where the future is full of buzzwords like Azure, cloud computing, WPS, DirectX 3D, XAML, Windows 7 UI and so forth. In the world painted by these two programmers one would believe that any last vestiges of COM are a long distant memory.

Nothing could be further from the truth!

In fact, not only is COM still a major technology on which the future of .NET is laid upon, but the .NET interfaces are little more than COM InterOp wrappers. Moreover, C++ is still considered an important method for writing and deploying the new functionality.

What does this mean to developers?

We cannot architect and design new systems without taking into account the impact of unmanaged code. Issues such as object lifetime, concurrency, remoting, marshaling and other plumbing and glue affect how systems are designed and even optimized. What should have been a straightforward task is now complicated by having to support the managed and unmanaged world.

It gets worse…

The other subliminal message that was imparted to developers was that there would be no new support for managed C++ development. The impact of this is huge, but may not be fully grasped until you realize that the only avenue for porting and supporting C++ code is COM! More than one speaker at the PDC said that no effort was put into supporting managed C++ in this current release (VS 2010 and .NET 4.0).

Developers can of course opt out of C++ development by implementing new COM object s in C# as CCW objects. That approach, however, does not address consuming COM objects (or their .NET wrapped counterparts) exposed by the operating system. All new DirectX (on which WPF sits on top of), Windows 7 UI enhancement, motion and input libraries are deployed as COM objects.

What should developers do?

1.       Review the basics of the tao of COM. Understanding how reference counted objects behave in a managed environment is key to developing robust systems.

2.       Review the fundamentals of COM InterOp and how the managed world interacts with the unmanaged world.

3.       Review P/Invoke mechanisms and how objects are marshaled between the managed and unmanaged world.

4.       When developing system avoid brittle dependencies and assumptions how objects are created, maintained and destroyed.