Thursday, June 13, 2013

NuGet Install Is Broken With F#

There’s a very nasty bug when you try and use NuGet to add a package reference to an F# project. It manifests itself when either the assembly that is being installed also has a version in the GAC or a different version already exists in the output directory.

First let’s reproduce the problem when a version of the assembly already exists in the GAC.

Create a new solution with an F# project.

Choose an assembly that you want to install from NuGet that also exists in the GAC on your machine. For ironic purposes I’m going to choose NuGet.Core for this example.

It’s in my GAC:

D:\>gacutil -l | find "NuGet.Core"
NuGet.Core, Version=1.0.11220.104, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL
NuGet.Core, Version=1.6.30117.9648, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL

You can see that the highest version in the GAC is version 1.6.30117.9648

Now let’s install NuGet.Core version 2.5.0 from the official NuGet source:

PM> Install-Package NuGet.Core -Version 2.5.0
Installing 'Nuget.Core 2.5.0'.
Successfully installed 'Nuget.Core 2.5.0'.
Adding 'Nuget.Core 2.5.0' to Mike.NuGetExperiments.FsProject.
Successfully added 'Nuget.Core 2.5.0' to Mike.NuGetExperiments.FsProject.

It correctly creates a packages directory, downloads the NuGet.Core package and creates a packages.config file:

D:\Source\Mike.NuGetExperiments\src>tree /F
D:.
│ Mike.NuGetExperiments.sln

├───Mike.NuGetExperiments.FsProject
│ │ Mike.NuGetExperiments.FsProject.fsproj
│ │ packages.config
│ │ Spike.fs
│ │
│ ├───bin
│ │ └───Debug
│ │
│ └───obj
│ └───Debug

└───packages
│ repositories.config

└───Nuget.Core.2.5.0
│ Nuget.Core.2.5.0.nupkg
│ Nuget.Core.2.5.0.nuspec

└───lib
└───net40-Client
NuGet.Core.dll

But when when I look at my fsproj file I see that it has incorrectly referenced the NuGet.Core version (1.6.30117.9648) from the GAC and there is no hint path pointing to the downloaded package.

<Reference Include="NuGet.Core, Version=1.6.30117.9648, Culture=neutral, PublicKeyToken=31bf3856ad364e35">
<Private>True</Private>
</Reference>

Next let’s reproduce the problem when a version of an assembly already exists in the output directory.

This time I’m going to EasyNetQ as my example DLL. First I’m going to take a recent version of EasyNetQ.dll, 0.10.1.92 and drop it into to the projects output directory (bin\Debug).

Next use NuGet to install an earlier version of the assembly:

Install-Package EasyNetQ -Version 0.9.2.76
Attempting to resolve dependency 'RabbitMQ.Client (= 3.0.2.0)'.
Attempting to resolve dependency 'Newtonsoft.Json (≥ 4.5)'.
Installing 'RabbitMQ.Client 3.0.2'.
Successfully installed 'RabbitMQ.Client 3.0.2'.
Installing 'Newtonsoft.Json 4.5.11'.
Successfully installed 'Newtonsoft.Json 4.5.11'.
Installing 'EasyNetQ 0.9.2.76'.
Successfully installed 'EasyNetQ 0.9.2.76'.
Adding 'RabbitMQ.Client 3.0.2' to Mike.NuGetExperiments.FsProject.
Successfully added 'RabbitMQ.Client 3.0.2' to Mike.NuGetExperiments.FsProject.
Adding 'Newtonsoft.Json 4.5.11' to Mike.NuGetExperiments.FsProject.
Successfully added 'Newtonsoft.Json 4.5.11' to Mike.NuGetExperiments.FsProject.
Adding 'EasyNetQ 0.9.2.76' to Mike.NuGetExperiments.FsProject.
Successfully added 'EasyNetQ 0.9.2.76' to Mike.NuGetExperiments.FsProject.

NuGet reports that everything went according to plan and that EasyNetQ 0.9.2.76 has been successfully added to my project.

Once again the packages directory was successfully created and the correct version of EasyNetQ has been downloaded. The packages.config file also has the correct version of EasyNetQ. I won’t show you the output from ‘tree’ again, it’s much the same as before.

Again, when I look at my fsproj file the version of EasyNetQ is incorrect, it’s 0.10.1.92, and again there’s no hint path:

<Reference Include="EasyNetQ, Version=0.10.1.92, Culture=neutral, PublicKeyToken=null">
<Private>True</Private>
</Reference>

Yup, NuGet install is most definitely broken with F#.

This bug makes using NuGet and F# together an exercise in frustration. Our team has wasted days attempting to get to the bottom of this.

It seems that it’s a well know problem. Just take a look at this workitem, reported over a year ago:

http://nuget.codeplex.com/workitem/2149

After much cursing of NuGet, the problem actually appears to be with the F# project system rather than with NuGet itself:

“F# knows about this behavior and they will release the fix”

Hmm, it hasn’t been fixed yet.

We had a dig around the NuGet code. The interesting piece is this file snippet (from NuGet.VisualStudio.VsProjectSystem):

   1: public virtual void AddReference(string referencePath, Stream stream)
   2: {
   3:     string name = Path.GetFileNameWithoutExtension(referencePath);
   4:     try
   5:     {
   6:         // Get the full path to the reference
   7:         string fullPath = PathUtility.GetAbsolutePath(Root, referencePath);
   8:         string assemblyPath = fullPath;
   9:  
  10:         ...
  11:  
  12:         // Add a reference to the project
  13:         dynamic reference = Project.Object.References.Add(assemblyPath);
  14:  
  15:         ...
  16:  
  17:         TrySetCopyLocal(reference);
  18:  
  19:         // This happens if the assembly appears in any of the search
  20:         // paths that VS uses to locate assembly references. Most commonly, 
  21:         // it happens if this assembly is in the GAC or in the output path.
  22:         if (!reference.Path.Equals(fullPath, StringComparison.OrdinalIgnoreCase))
  23:         {
  24:             // Get the msbuild project for this project
  25:             MsBuildProject buildProject = Project.AsMSBuildProject();
  26:  
  27:             if (buildProject != null)
  28:             {
  29:                 // Get the assembly name of the reference we are trying to add
  30:                 AssemblyName assemblyName = AssemblyName.GetAssemblyName(fullPath);
  31:  
  32:                 // Try to find the item for the assembly name
  33:                 MsBuildProjectItem item = 
  34:                     (from assemblyReferenceNode in buildProject.GetAssemblyReferences()
  35:                     where AssemblyNamesMatch(assemblyName, assemblyReferenceNode.Item2)
  36:                     select assemblyReferenceNode.Item1).FirstOrDefault();
  37:  
  38:                 if (item != null)
  39:                 {
  40:                     // Add the <HintPath> metadata item as a relative path
  41:                     item.SetMetadataValue("HintPath", referencePath);
  42:  
  43:                     // Save the project after we've modified it.
  44:                     Project.Save();
  45:                 }
  46:             }
  47:         }
  48:     }
  49:     catch (Exception e)
  50:     {
  51:         ...
  52:     }
  53: }

On line 13 NuGet calls out to the F# project system and asks it to add a reference to the assembly at the given path. We assume that the F# project system then does the wrong thing by searching for the assembly name anywhere in the GAC or the output directory rather than referencing the explicit assembly NuGet is asking it to reference.

Interestingly, it looks as if the NuGet team have attempted to code a work-around for this bug from line 22 onwards. Could this be why C# projects don’t exhibit this behaviour? Unfortunately the work around doesn’t work in the F# case. We think it’s because F# doesn’t respect assembly versions and will happily replace any requested assembly with another one so long as it’s got the same simple name. At line 33, no assemblies are found in the fsproj file because the ‘AssemblyNamesMatch’ function does an exact match using all four elements of the full assembly name (simple name, version, culture, and key) and of course the assembly that the F# project system has found and added has a different version.

So, come on F# team, pull your finger out and fix the Visual Studio F# project system. In the meantime, in my next post I’ll talk about some of things our team, and especially the excellent Michael Newton (@mavnn) has been doing to try and work around these problems.

Update: Micheal Newton has written a guest post to explain some of things we are doing to work around these problems.

No comments: