Tuesday, March 27, 2007

How to add a command to a custom VS project type

In this post I'm going to show how to add your own commands to a custom project type's context menu in Visual Studio.

I've been having a lot of fun recently trying to create my own custom project type for VS. In my last post on this subject I described how you can create the most basic custom project type with no real functionality. When you create one of these with the VS SDK's MPF (Managed Package Framework), your project inherits from ProjectNode which seems to be designed to support a buildable project. This makes sense since VS is fundamentally a development tool and the vast majority of project types are going to be things that need to be built. This means that when you right click on your new project, it shows a context menu with commands like 'Set As Startup Project' and 'Debug' and there are hooks in the ProjectNode class to implement this behaviour. However the kind of project I want to create is more like a database project where there's no requirement to actually build anything and the project can't be run or debugged, unfortunately the only way to remove all the build and debug related functionality seems to be to create your own version of ProjectNode (and ProjectFactory too). I haven't explored this yet, so if you walk through this post to create your own context menu items as I describe you'll see those commands still displayed. However, they don't do anything if you select them. OK, so to start off you need to create your new project type following the steps in the last post. But we have to make one change. When the New Package Wizard runs, instead of leaving all the checkboxes blank on the 'Select VSPackage Options' page, you should check the 'Menu Command' box. When the wizard runs, in addtion to all the other project items, it creates a folder called 'CtcComponents' that includes the following files: CommandIds.h Guids.h Resource.h TestMenuPackage.ctc (or whatever you_called_your_project.ctc) If you right click on the ctc file and select properties you'll see that its build action is 'CtcFile'. This file has its own special compiler that builds a binary command table file that VS uses to build all the menus and toolbars you see in the IDE. The CtcFile compiler uses the C++ preprocessor, thus the C style header files, #defines and #includes. When the wizard builds the package project it creates a command for you that appears in the Tools menu. You can see how it's defined by looking in the ctc files. Basically, in the ctc file you define a command set with a unique guid and then a command group that belongs to that command set and a particular VS menu, then define some commands that belong to the command group. To have a command group that appears under your custom project type, you first need to define the command set guid in Guids.h (I just used the one that the wizard created)

// Command set guid for our commands (used with IOleCommandTarget)

// { bdc32849-d202-496c-96b4-837c0254e0e2 }

#define guidTestMenuPackageCmdSet { 0xBDC32849, 0xD202, 0x496C, { 0x96, 0xB4, 0x83, 0x7C, 0x2, 0x54, 0xE0, 0xE2 } }

#ifdef DEFINE_GUID

DEFINE_GUID(CLSID_TestMenuPackageCmdSet,

0xBDC32849, 0xD202, 0x496C, 0x96, 0xB4, 0x83, 0x7C, 0x2, 0x54, 0xE0, 0xE2 );

#endif

The C style syntax is a bit gnarly I agree, but you can get the Toos->Create Guid tool to generate it for you which is nice. Next you define the command group id and command id in the CommandIds.h file:

///////////////////////////////////////////////////////////////////////////////

// Menu Group IDs



#define MyProjectGroup    0x10A0



///////////////////////////////////////////////////////////////////////////////

// Command IDs



#define cmdidDoSomething   0x001

Next you link the command group to the command set and a VS menu in the ctc file under the NEWGROUPS_BEGIN section:

// my project group

guidTestMenuPackageCmdSet:MyProjectGroup, // group name

guidSHLMainMenu:IDM_VS_CTXT_PROJNODE,  // parent group

0x0001;          // priority

The parent group 'guidSHLMainMenu:IDM_VS_CTXT_PROJNODE' tells VS that this command group should appear under project nodes in the solution explorer. OK, so how do find out the name of that constant? It's a case of digging in a file called 'SharedCmdPlace.ctc' that's located at C:\Program Files\Visual Studio 2005 SDK\2007.02\VisualStudioIntegration\Common\Inc on my computer and having a best guess about which one seems to fit the bill. The next step is to link define the command in the ctc file under the BUTTONS_BEGIN section (all menu items are BUTTONs):

// Do Something Fun command

guidTestMenuPackageCmdSet:cmdidDoSomething,  // command

guidTestMenuPackageCmdSet:MyProjectGroup,  // parent group

0x0100,           // priority

guidTestMenuPackageCmdSet:bmpPic1,    // image

BUTTON,           // type

DEFAULTINVISIBLE  DYNAMICVISIBILITY,   // visibility

"Do Something Fun";        // caption

You can see we link the command id to our command group, give it a display priority (which controls where it appears in the command group), an icon (I've just used the one created by the wizard), define the visibility and the caption. The visibility is important. When we defined our command group we told VS that we wanted it to appear under a project node. This means that it will appear under every project node, not only ours. We need to have our command default to being invisible and then enable it only from our project node, that's why the visibility is set to 'DEFAULTINVISIBLE DYNAMICVISIBILITY'. OK, that's the ctc file done, now we can get our project node to enable the command and respond to it when it's clicked. Our project class (referring back to the previous post) inherits from ProjectNode which in turn inherits from HierarchyNode which implements IOleCommandTarget. If you read the documentation on VSPackages you'll see that IOleCommandTarget is the interface for any class that needs to intercept command invocations. IOleCommandTarget defines two methods, QueryStatus and Exec. These are handled by HierarchyNode and you can customise their behaviour by overriding QueryStatusOnNode and ExecCommandOnNode. QueryStatus gives the implementer of IOleCommandTarget a chance to modify the display of the menu item and is invoked whenever the project node becomes active. Because the default visibility of our command is 'DEFAULTINVISIBLE DYNAMICVISIBILITY', we need to enable it when our project is active by overriding QueryStatusOnNode in our project class:

protected override int QueryStatusOnNode(

    Guid guidCmdGroup, uint cmd, IntPtr pCmdText, ref QueryStatusResult result)

{

    if (guidCmdGroup == GuidList.guidTestMenuPackageCmdSet)

    {

        result = QueryStatusResult.SUPPORTED  QueryStatusResult.ENABLED;

        return VSConstants.S_OK;

    }

    return base.QueryStatusOnNode(guidCmdGroup, cmd, pCmdText, ref result);

}

We can find out if the command is one of ours by checking the guidCmdGroup argument against the guid for our command set (I've used the GuidList static class that's generated by the wizard for this). Any other commands we just delegate to the base class. Now if we run our project by hitting F5 and create a new instance of our custom project type we'll see our new command in the context menu of the project node. To respond when a user clicks on our new command we simply override ExecCommandOnNode:

protected override int ExecCommandOnNode(

    Guid guidCmdGroup, uint cmd, uint nCmdexecopt, IntPtr pvaIn, IntPtr pvaOut)

{

    if (guidCmdGroup == GuidList.guidTestMenuPackageCmdSet)

    {

        if (cmd == 13)

        {

            return ShowMessage();

        }

    }

    return base.ExecCommandOnNode(guidCmdGroup, cmd, nCmdexecopt, pvaIn, pvaOut);

}

Once again we look for any of our own commands by checking the guidCmdGroup argument and pick individual commands by checking the 'cmd' argument which holds the command id. This is kinda redundant here because we've only got the one command. ShowMessage is just a private function that pops up a message box in this case, but it's where you'd implement whatever functionality you wanted your command to execute. Again we delegate any other commands to the base class. And that's all there is to it. Now I know I'm going to start sounding like a broken record, but this VSPackage stuff is made far too hard by poor documentation. Needless to say, I spent ages wondering why I couldn't see my command as I debugged through QueryStatusOnNode, and that was after I spent just as long working out that I should be overriding QueryStatusOnNode. As for finding the IDM_VS_CTXT_PROJNODE constant, that was just luck. All the project base stuff really does need some kind of high level overview. Even the best samples are very hard to grep without it.

Sunday, March 25, 2007

Is LINQ the writing on the wall for C#?

C++ was the writing on the wall for C. C is a phenomenally successful low level systems programming language. If you want to write an operating system you use C. I remember first becoming aware of C as a teenage BASIC freak when it was discussed in the computing magazines I used to read. In those days the choice for budding programmers like me was between the ROM based BASIC interpreter, or writing in Assembly language. C was described as a way of getting the speed and power of assembler with the ease of BASIC and it pretty much took over the PC programming world. A lot of business applications were written in C because there was no higher level abstraction that could do the job. Visual Basic took years to get the necessary performance and functionality to write decent enterprise level applications. A lot of people would argue that it never got there. But now, no one would dream of writing a business application with C, we all use modern OO languages like Java or .NET. C++ is an OO language right? So why don’t we build modern business applications with C++? The reason is that C++ is a low level systems programming language with some OO stuff grafted on. You still have to do all your own memory management and worst of all you have to get your head around all that mind boggling pointer stuff that’s frankly beyond the average Mort. For a long time, though, a lot of business applications were built with C++. People were right to believe that OO programming gave them a fantastic tool to get on top of the complexity of large programs, but what they needed was the OO stuff without the C stuff and that’s where, first Java and then C# came into the picture. So why is LINQ the writing on the wall for C#? Well I’m being slightly provocative here, it’s just a thought experiment, but doesn’t this look like a repeat of the whole C + OO = C++ thing. LINQ is giving us functional programming in C#, which isn’t a functional programming language, it’s an imperative language. There’s no doubt that the current trend in programming is away from imperative and towards a more declarative style. SQL’s been doing this it for decades in it’s own tightly bounded domain and now we’re beginning to see general purpose declarative languages becoming more widely known. The problem with grafting some functional stuff on top of an imperative language is that the imperative stuff stops you from getting all the benefits of the functional stuff in the same way that the C in C++ stopped C++ from giving the programmer all the benefits if a language with first class types. Is C#+LINQ another intermediate Frankenstein's monster like C++? When we've all had plenty of practice with LINQ are we going to start thinking that we don't need all that imperative syntax? In ten years time will I be writing, “nobody would dream of writing a business application with an imperative language!”?

REST is the Betamax of Web Services

I while ago I blogged about REST. I was pretty excited about it at the time, it seemed to blow away a lot of the lava layers that have been accumulating around the SOAP protocol with a back to basics, or rather back to HTML approach that leverages a lot of the forgotten functionality of that protocol. OK, so should I be using it when my customers ask me to implement a Service Oriented Architecture? I think the answer has to be no. SOAP has gained far too much market share in the developer tools community. Especially in the Microsoft world, I’d be throwing away all the SOAP support built into the MS toolset and the .NET framework in particular that makes implementing web services with SOAP so easy. Also I’d be loosing any interoperability with other platforms, because with a RESTfull approach you’re basically designing your own protocol. It’s a shame, but sometimes you just have to go with the flow. Of course the world of IT (actually the world generally) is full Betamaxes. If the best tools always won, we’d be programming in Haskell, on BeOS running on a RISC machine.

Friday, March 23, 2007

Building a Visual Studio Custom Project Type

Recently I've been looking at Visual Studio Extensibility and specifically how to create my own custom project types. There are three fundamental ways of extending VS:

  • Macros. This is the easiest way to automate VS and can be as simple as recording a few key strokes or as complex as coding (in VB ugh!) entire extensibility applications against the VS automation model (known as EnvDTE).
  • AddIns. These allow deeper integration with VS and allow you to code in C# and ship your automation code as a compiled assembly.
  • Packages. This is the deepest level of integration and is how Microsoft creates features in VS, including all the languages.

Now there's a common misconception among .net developers (myself included, before I started looking into this) that Visual Studio is itself writen in .net, it's not, it's a evolution of the earlier C++ IDE and is fundamentally a COM framework with plugable COM components called 'Packages'. Apparently many of the VS packages are now implemented in .net, but they still have to deal with the VS COM framework via interop. The only way to do what I want and create a new project type is to use VS Packages. Unfortunately this means dealing with VS in the raw and coding against the gnarly old VS COM framework. Microsoft provide interop classes and a managed framework, the Managed Package Framework (MPF) to help with this, but it's not very well documented and hardly a walk in the park. OK, so after much fustration and effort I've managed to create the most basic non-functional custom project type. It doesn't do anything, but you can select it in the New Project dialogue box and create an instance of it that you can see in solution explorer. The steps below show how I did it. I'm not sure if it's the right way, but it works. First you'll need to download the Visual Studio SDK, currently at version 4. Once that's installed, you can use the new package wizard to create a basic package: 1. Open Visual Studio 2. Select File->New->Project, the new project dialog appears 3. Select Other Project Types->Extensibility->Visual Studio Integration Package 4. Give your package a name 5. Click OK, The Visual Studio Integration Package Wizard Appears, click Next 6. Select C# and Generate a new key file to sign the assembly, click Next 7. Enter some information about your package, this will appear in the VS about dialog, click Next 8. Leave all the following checkboxes blank, click Finish. The wizard does it's stuff and now you've got a working Visual Studio package (not project that comes later!), you can build it and run it by hitting F5. A new instance of VS appears and if you click on Help->About Microsoft Visual Studio, you can see that your package has been registered and appears under 'installed products' with the information you entered in the Wizard. The instance of VS that runs when you hit F5 uses what's known as the 'Experimental Hive' which is simply a copy of the VS registry settings that's created for developing and debugging extensions. You can reset it at any time by using the VsRegEx.exe tool. The part of the MPF that deals with creating new project types isn't provided as part of the MPF assemblies, but as source code, called 'ProjectBase', that you are supposed to include in your project. On my machine, the VS SDK installed it here:

C:\Program Files\Visual Studio 2005 SDK\2007.02\VisualStudioIntegration\Common\Source\CSharp\Project

To include it you have to edit your package's csproj file to include the following nodes:

<!-- This imports the files which makes up the project base classes -->
<PropertyGroup>
<ProjectBasePath>C:\Program Files\Visual Studio 2005 SDK\2007.02\VisualStudioIntegration\Common\Source\CSharp\Project</ProjectBasePath>
</PropertyGroup>
<Import Project="$(ProjectBasePath)\ProjectBase.Files" />

Once you reload the project you'll see that a folder called project base has been added. If you now try and build the project, you'll get a whole load of errors. To fix them, add the following references: EnvDTE Microsoft.VisualStudio.Designer.Interfaces and add the following lines to your AssemblyInfo.cs file:

using System;
...
[assembly: CLSCompliant(false)]

Now you should be able to build the project with the ProjectBase files included. In order for Visual Studio to create an instance of your project you have to create a 'ProjectFactory', you do this by extending the ProjectFactory class in ProjectBase (Microsoft.VisualStudio.Package). Here's my ProjectFactory. Note it must be given a fixed Guid to allow the class to be identified by COM:

using System;
using System.Runtime.InteropServices;
using Microsoft.VisualStudio.Package;
using Microsoft.VisualStudio.Shell;
using IOleServiceProvider = Microsoft.VisualStudio.OLE.Interop.IServiceProvider;

namespace Company.VsPackage1
{
    [Guid("AC959CD1-5A2C-4306-BE72-259804B01F08")]
    public class MyProjectFactory : ProjectFactory
    {
        public MyProjectFactory(Package package)
            : base(package)
        {
        }

        protected override ProjectNode CreateProject()
        {
            // TODO: create project here
            return null;
        }
    }
}

MyProjectFactory now has to be registered by the package. The MPF uses attributes on the package class to create the required registry entries so you need to add the following attribute to the package class created by the wizard. In my case it's called 'VsPackage1' in the file 'VsPkg.cs'. Here you should define the name of your project type and your project file's file extension:

[ProvideProjectFactory(typeof(MyProjectFactory),
"My Project",
"MyProject Files (*.myproj);*.myproj",
"myproj",
"myproj",
@"..\..\Templates\Projects")]

You also have to get the Package to register the project factory type when it initialises by calling RegisterProjectFactory and passing in a new instance of MyProjectFactory that takes a pointer to the package in its constructor:

protected override void Initialize()
{
    Trace.WriteLine (string.Format(CultureInfo.CurrentCulture, "Entering Initialize() of: {0}", this.ToString()));
    this.RegisterProjectFactory(new MyProjectFactory(this));
    base.Initialize();

}

Also at this stage we should change our Pakage to inherit from ProjectPackage (in Microsoft.VisualStudio.Package) rather than Package:

public sealed class VsPackage1 : ProjectPackage

If you hit F5 now to run your package you see that when you go to File->New->Project you'll see 'My Project' as one of the project types available. If you click on 'My Project' there are no projects to select. All the available project types are defined as templates, so the next step is to define a template file. In the ProvideProjectFactory attribute above the last parameter is the path to where VS should look to find your project templates, so create a new folder under your project called 'Templates' and under that another folder called 'Projects'. In that folder create a file called 'My Project.myproj'. Note that the extension should match the extension defined above in the ProvideProjectFactory attribute. Paste this minimal project template into the file:

<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <Name>"My Project"</Name>
    <SchemaVersion>2.0</SchemaVersion>
    <ProjectGuid>{AC959CD1-5A2C-4306-BE72-259804B01F08}</ProjectGuid>
  </PropertyGroup>
</Project>

Note that the ProjectGuid property should be the same as the MyProjectFactory class' Guid. Now when you hit F5 and then File->New->Project and navigate to 'My Project' you will see the 'My Project' template. If you select it and hit 'OK', you'll get a nasty error and VS will crash out. The next task is do define the project class and get the project factory to create it. Create a class called 'MyProject' and paste in the following code:

using System;
using System.Runtime.InteropServices;
using Microsoft.VisualStudio.Package;

namespace Company.VsPackage1
{
    [CLSCompliant(false)]
    [ComVisible(true)]
    [Guid("7B61EA22-333F-4b4e-A000-FB847CDD4A99")]
    public class MyProject : ProjectNode
    {
        public override Guid ProjectGuid
        {
            get { return typeof(MyProjectFactory).GUID; }
        }

        public override string ProjectType
        {
            get { return this.GetType().Name; }
        }
    }
}

And return a new instance of the project class from the project factory's CreateProject method. We also have to set the site of the project by providing the package's service provider (see my post on service providers for more on this):

protected override ProjectNode CreateProject()
{
    MyProject project = new MyProject();
    project.SetSite((IOleServiceProvider)
        (((IServiceProvider)this.Package).GetService(typeof(IOleServiceProvider)))
        );
    return project;
}

Now when you hit F5 and create a new 'My Project' project it works! You can see a new instance of your project in Solution Explorer under a new solution. You can right click on the solution select Add->New Project and add another instance of your project type to the solution. It took me ages (several days) to work out these few steps to creating my own custom project type. All the walkthroughs I could find only covered building simple single menu command, tool window or editor packages and none covered using the ProjectBase classes. In the end I had to work it out from the samples (mainly the NestedProject and IronPython samples) and a few hints and tips from blogs and the Visual Studio Extensibility forum. I'm expecting a similarly difficult and steep learning curve as I try to add functionality to my project. Getting to grips with buiding packages is also confused by having lots of samples and instructions based on the COM interfaces. I guess you've got to understand that architecture in order to effectively build VS solutions, but when I came to use the MPF I didn't really know how to translate what I'd learnt about the VS architecture into how to code against the MPF. I guess it's good that the VS extensibility team have taken an agile approach and are releasing stuff as they put it together without waiting for it to be fully documented. I'd much rather have the MPF without much documentation than have to code against the raw interop classes and the release cycle is pretty short so I imagine we'll see version 5 of the MPF pretty soon. Just to wrap up I have to mention Carlos Quintero's MZ-Tools site. It's a goldmine of information especially the resources page. Watching the Videos, Webcasts and Screeencasts he lists there is probably the easiest way to get up to speed with VS Packages. I've also recently read Inside Visual Studio .net by Johnson, Skibo and Young which, although it doesn't cover VS Packages is a goldmine of information on the DTE automation API and stuff to do with AddIns. The current version is also avaiable as a free PDF download when you register Visual Studio.

Friday, March 02, 2007

Writing a raw web service using an HttpHandler

The ASP.NET web service infrastructure provides a RPC style abstraction for web serivces. You write a web method that looks like any other C# function and simply add a [WebMethod] attribute. ASP.NET then reflects on the name of the method, your input parameters and return types and creates a web service end point for you. If you use the 'Add Web Reference' facility for the client, you never have to consider what the web service will look like in terms of WSDL or XML, it just looks like a remote procedure call. This is fine in many scenarios, but sometimes what you are interested in is the actual XML message itself and you don't need it deserialized as .net types. It's pretty straightforward to write a web service using an HttpHandler to grab the raw SOAP envelope. The IHttpHandler interface is the core endpoint interface for anything that handles a web request in .net. All you need to do is write a class that implements the interface and then register it in Web.config:
<httpHandlers>
<add verb="*" path="MyServiceEndpoint.asmx" type="MyService.MyHandler, MyHandler" />
</httpHandlers>
IHttpHandler provides a method 'PrcessRequest' that looks like this:
void ProcessRequest(HttpContext context);
You take the HttpContext.Request.InputStream and read the raw SOAP envelope from it. When you're ready to return the response, simply write your SOAP envelope response to the HttpContext.Response.OutputStream. A couple of other things to consider are providing a default information page and your web service's WSDL file. In ASP.NET if you execute an HTTP GET request against your web service, ASP.NET builds a nice web page naming your service and listing the operations. If you append WSDL to the querystring, it builds and returns the WSDL for the web service. You can replicate this behaviour by looking for a GET request (web service requests are typically POST) and then streaming either a descriptive web page or a WSDL document to the response. Here are functions that look for a GET request and stream either the WSDL file or the default page file into the response.
public void ProcessRequest(HttpContext context)
{
switch(context.Request.HttpMethod)
{
 case "GET":
  ProcessGetRequest(context);
  break;
 case "POST":
  ProcessPostRequest(context);
  break;
 default:
  throw new ApplicationException(
   string.Format("HTTP Method '{0}' is not supported.", context.Request.HttpMethod));
}
}

void ProcessGetRequest(HttpContext context)
{
// is this a request for the WSDL file?
string filePath;
if(context.Request.QueryString.ToString().ToUpper() == "WSDL")
{
 filePath = context.Server.MapPath(myWsdlPath);
}
else
{
 filePath = context.Server.MapPath(myDefaultPagePath);
}
RespondWithFile(context, filePath);
}

void RespondWithFile(HttpContext context, string filePath)
{
using(FileStream fileStream = new FileStream(filePath, FileMode.Open, FileAccess.Read))
{
 byte[] buffer = new byte[streamBufferSize];
 int bytes;
 while((bytes = fileStream.Read(buffer, 0, streamBufferSize)) > 0)
 {
  context.Response.OutputStream.Write(buffer, 0, bytes);
 }
 context.Response.OutputStream.Flush();
}
}
This technique can be usefull when you're writing stuff to route or map web services. Currently I'm working on a mapping service that takes a request and forwards it on to another web service with a different schema. Using the handler I can simply grab the SOAP envelope and run it through an XSLT to transform it to target schema and then run the response through another XSLT to transform it back to the originating web service's response schema.