Update: This article is not limited to deploying applications to Vista, but also when changing files that are only writeable by administrators (e.g. files in the Program Files directory).

If you have not made specific changes to your installation to support UAC in Vista, this is for you. I hope this will help others to not spend many days of confusion, desperation and sleepless nights like I have.

Lately I've been getting reports from some of my coworkers running Windows Vista that the app.config file in our application was not updated after installation. I've also experienced this myself. We've solved them without knowing exactly how (which I hate). Typically "I uninstalled the application and installed to a different location, and it worked!" Well, this is a workaround that are not particulary popular to any of our customers, so I dived into the problem to find the exact reason.

To save you some reading I'll just say that the reason and solution is to provide your application with a manifest file to handle UAC in Vista as described in this article: Create and Embed an Application Manifest (UAC) I haven't actually tried this out yet, but I trust the article to be correct :-)

Virtualization
But that's not the interesting part. The interesting part is what happens (or what can happen) if you don't have an application manifest for this purpose? If your superstitious you might think that something weird is going on with your computer. If you're like me you KNOW that something weird is going on with your computer, and you need to find it and fix it. The weird thing is Vista virtualization.

"File virtualization addresses the situation where an application relies on the ability to store a file, such as a configuration file, in a system location typically writeable only by administrators"

"Virtualization is implemented to improve application compatibility problems for applications running as a standard user on Windows Vista."

As I write this I have exactly this problem on my computer. I've installed our application (Contiki ECM) to this location: C:\Program Files\CMA Contiki AS\Contiki ECM\. This is where I've always install our app. So today I installed and checked that the app.config was updated with all my changes done during installation. And it hadn't. Actually it was updated, but not with my changes. It looked like an old file that I had some time ago. How could this happen?

Textpad .vs. Notepad
To make this even more confusing, I discovered this reading the file in TextPad, but when opening the file in Notepad it was fine. All my changes was there! So I thought that this was a TextPad problem. I started our application, but it turns out that when .Net tries to read the config file, it reads the same old stuff as TextPad did, resulting in old config to be loaded into our app. Why does Notepad get the right file? I don't know. Please tell me if you do.

The solution
So after Googling for a while I found this article on MSDN, and things started to make sense. But first let me show you what it looks like on my computer. The file causing my headaches are Contiki.Windows.Application.exe.config.

 
Do you see when it's modified? 08:56 this morning, but the file that is being opened is from 13th of April! How do I know? Well, if you have a look at the virtualized files which you'll find in C:\Users\Username\AppData\Local\VirtualStore\[Your path], it might help out. So I looked at C:\Users\1jontor\AppData\Local\VirtualStore\Program Files\CMA Contiki AS\Contiki ECM and found this:

 
Modified back in April. I opened this file and it was exactly what I saw in TextPad. But this file is the same both in TextPad and Notepad! :-)

Explorer gives you a shortcut to the Virtual Store folder if any files are virtualized, and will show this button in Explorer:

The end
A couple of notes from the MSDN article:

"When you enumerate resources in folders and in the registry, Windows Vista will merge global file/folder and registry keys into a single list. In this merged view, the global (protected) resource is listed along with the virtualized resource."

"The virtual copy will always be present to the application first.... even if [some file] is updated"

Even if the file is updated!? Who though of that? Was that really such a good idea? I can see what they where thinking, but it would have saved me a lot of work getting an error message saying "Access denied"...

To end this off I picked another good quote from the article:

"Microsoft intends to remove virtualization from future versions of the Windows operating system as more applications are migrated to Windows Vista."

I appreciate that... :-)

Today I was struggling to figure out why I kept getting this error message when downloading a ClickOnce app:

The file 'C:\Documents and Settings\[user]\Local Settings\Temp\2\Deployment\RCAX3K4W.T8E\GRY27V4Q.GBE\SomeDll.dll' already exists.

First, let me just say that I don't use Visual Studio to create the ClickOnce deployment, but the Mage.exe utility. If you use Visual Studio this problem will be solved on Publish in Visual Studio.

I knew there was only one of this dll present on the server where ClickOnce downloads its files, but still it insisted that it was already downloaded. I checked my application manifest and there where only one entry for my dll. The dll causing this problem is a COM dll with generated manifest files (Reg-Free COM) as described here.

I started to inspect the Reg-Free COM manifest file generated by Visual Studio. It turns out that this manifest file use the same syntax as ClickOnce, or actually ClickOnce use the existing manifest schema that was introduced with Windows XP and Reg-Free COM. So in my ClickOnce manifest I had this:

...
<file name="My.dll" size="1121280">
  <hash>
    <dsig:Transforms>
      <dsig:Transform Algorithm="urn:schemas-microsoft-com:HashTransforms.Identity" />
    </dsig:Transforms>
    <dsig:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
    <dsig:DigestValue>UwpS1EpkD7aJk3uWdOtX+3BHiOA=</dsig:DigestValue>
  </hash>
</file>
...

...and in my Reg-Free COM manifest I had this:

...
<file name="My.dll" asmv2:size="1121280">
    <hash xmlns="urn:schemas-microsoft-com:asm.v2">
      <dsig:Transforms>
        <dsig:Transform Algorithm="urn:schemas-microsoft-com:HashTransforms.Identity" />
      </dsig:Transforms>
      <dsig:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
      <dsig:DigestValue>UwpS1EpkD7aJk3uWdOtX+3BHiOA=</dsig:DigestValue>
    </hash>
    <typelib tlbid="{2d5e2d34-bed5-4b9f-9793-a31e26e6806e}" version="4.1" helpdir="" resourceid="0" flags="HASDISKIMAGE" />
    ...COM Registry Entries...
</file>
...

It's exactly the same as ClickOnce except from the <typelib> part which is used by Reg-Free COM. ClickOnce treated this as any other ClickOnce manifest, and correctly stated that the file is about to be downloaded twice. The surprising part for me though was that the Reg-Free COM manifest was read at all. Since the deployment manifest is only linked to the application manifest...

So what's the solution? Just delete the file entry in the ClickOnce manifest and re-sign it! Deleting can be done like this:

ApplicationManifest manifest = (ApplicationManifest) ManifestReader.ReadManifest("ApplicationManifest", @"My.exe.manifest", false);
FileReference fileRef = manifest.FileReferences.FindTargetPath("My.dll");
manifest.FileReferences.Remove(fileRef);
ManifestWriter.WriteManifest(manifest);

Curious about how Visual Studio solves this? It merges all the manifest files into the application manifest and use the imported manifest entries instead (or something like that). At least that happened when I created a sample program to test this.

Server deployment of ClickOnce? Isn't ClickOnce client deployment? Yes, it is. However, you will have to deploy the ClickOnce files to a server, right? I feel this is a poorly documented step and I thought I share some of my experience around this.

So what's the problem? The main issue is the deployment manifest file. If you've worked with ClickOnce you know about this file, if not here's a (very) short description. The deployment manifest includes information about where the ClickOnce application is located (e.g. http://myServer/myWD/myApp.application) as showed in the following snippet:

...
<deployment install="true" mapFileExtensions="true" trustURLParameters="true">
    <subscription>
        <update>
            <beforeApplicationStartup />
        </update>
    </subscription>
    <deploymentProvider codebase="http://myServer/myWD/myApp.application" />
</deployment>
...

This file needs to be signed with a certificate to be valid, which means that any changes to this file makes it invalid and you'll have to resign it. This is a good thing for security, but it causes some problems regarding deployment. The url specified in this file must be changed when deployed at a server. Unless you want to get all information needed from your customer and create the deployment package before sending it to them or force them to set up a server named myServer ;-). I don't think that solution is very likely to be used in production systems. So what you need to do, is make the installation (e.g. msi) change the manifest file and resign it during installation.

Microsoft has provided us with a tool to do exactly this. However, you're not allowed to redistribute it. Another issue is that you need to have a certificate for signing and you don't want to have your corporate certificate laying around in your msi. The most common solution around this problem is to prompt the user during installation for a certificate (like suggested here). But then your app is signed by someone else's certificate, which is probably not what you want.

I will now suggest a different solution to this problem. In the System.Build.Tasks.dll you'll find much of the functionality needed to do the same thing as Microsoft's Mage.exe is doing. The nice thing about this dll is that it's a part of the default .Net Framework installation. Here's a short description of the classes needed:

DeployManifest
An object model for the manifest file. Gives you direct access to specific parts of the manifest and makes it really easy to make changes.

ManifestReader
Have a static method (ReadManifest) which let you read and return the manifest you are going to work with.

ManifestWriter
Use the static method WriteManifest to write your changes back to the manifest file.

X509Certificate2
Your certificate to sign the manifest file with.

SecurityUtiities
Has the static method called SignFile which let you sign your manifest.

By using the classes described above it should be quite easy to achieve the same as with the Microsoft utility. Here's an example:

using System;
using System.IO;
using System.Reflection;
using System.Security.Cryptography.X509Certificates;
using Microsoft.Build.Tasks.Deployment.ManifestUtilities;

...
string manifestPath = @"C:\Temp\myApp.application";
string deploymentUrl = "http://myServer/myWD/myApp.application";

DeployManifest manifest = (DeployManifest)ManifestReader.ReadManifest("DeployManifest", manifestPath, false);
manifest.DeploymentUrl = deploymentUrl;
ManifestWriter.WriteManifest(manifest);
X509Certificate2 certificate = new X509Certificate2(GetCertificateFile(), "password");
SecurityUtilities.SignFile(certificate, null, manifestPath);

There is one important step we need to solve. How do we protect the certificate we need to sign the manifest file. For this I suggest you incorporate this into your tool (either a command line tool or a .Net library) by using Build Action = Embedded Resource. This will make your certificate inaccessible for most people. If you want to secure it even more, you can crypt it in however way you want. I leave this up to you ;-) In my code above I've done this in the GetcertificateFile() method:

X509Certificate2 certificate = new X509Certificate2(GetCertificateFile(), "password");

There is of course one other issue which you'll have to consider. What if someone uses your custom tool to sign their ClickOnce app? So you need to protect this as well with a password or some other mechanism. If you have a good suggestion to this, please let me know.

Update: I've found a temporary solution. After inspecting which files was in our package I found a 3rd party component that we don't use anymore but still had a reference to. This was references by our business layer, which all our web services use, hence this dll was in every bin folder on every web service. Since the size of the file was quite big it was enough to get the installation down to a reasonable size and avoiding the described error message.

For a while now our msi file that contains the application server for our product has not been working. Not an msi that our customers have, but one we're about to ship in two weeks. I've had my hands full for a long time, but today I finally got around to check what was causing the problem. Here's a screen shot of the error I get:

After a quick Google search I found that this is related to a bug on Windows Server 2003 and Windows XP. Cause:

This problem occurs if the Windows Installer process has insufficient contiguous virtual memory to verify that the .msi package or the .msp package is correctly signed. 

Fantastic! I will not go and ask all our customers to install the hotfix on their servers. That's just to stupid. About now you're probably thinking something like "How big is their app-server really?" or "What product can get to big for an msi file?". I can tell you... The size of the msi is 576MB. No images, no database, just pure .Net dll's! And actually we're not alone. Visual Studio Service Pack 1 had the exact same problem! Go figure. And even better, there is no hotfix for this on Windows XP.

Have we been coding like mad men for a decade and producing tons of code? I wish (or maybe not) :) The answer is simpler than you might think. Our application server contains web services only. At the beginning of our project we took an architectural decision to have our web services separated in virtual directories (or projects in VS). This to be able to update parts of our application without affecting the whole system. This was part of a master plan around the smart client principle and would let us deploy small modules into our application which dynamically loads the new behavior. As it turns out we have almost never done this in production. The drawback of this is of course that you get many of the same dll's scattered around in many virtual directories. And when you have many web services (by web services I mean asmx files and not web methods, 43 to be exact), and each one is dependent on your business and data access layer (which is quite huge) and more, megabytes is piling up.

So what's the solution? Well, since we don't really use this as intended we can just merge our web services into one and we will be home free. Except you don't do that two weeks before deployment with acceptance testing coming up. So I have to think of something else. The only solution I can see now is to find some way of making the msi smaller, which at the current time I don't have the faintest idea of how to accomplish. I can use a setup launcher (exe) which will probably solve it, but that will not make our customers very happy. Hopefully I'll figure something out during the weekend, or maybe you have a solution?