bizmonade

Overview

Bizmonade is a new project being developed to enable unit testing of BizTalk orchestrations, in isolation of other components, without deploying the orchestrations to BizTalk and without doing any configuration (bindings).

It allows running orchestrations directly from unit tests written in C#, using NUnit, MSTest or any another .Net testing tool. The orchestrations therefore run in complete isolation from the whole BizTalk environment, reducing the time needed to do quick tests of new orchestrations or of modifications to existing ones. The orchestrations can be tested without waiting for deployment scripts to run, and without worrying about technical communication details (server adresses, authentication details, file system paths, message encoding, encryption, mappings...)

The goal of the project is to allow more agile BizTalk development process by giving a shorter feedback cycle between modifications to orchestrations and execution of unit tests. It allows a "Test Driven Development" model, where each small modification is tested immediately, instead of "batching" multiple modifications together to test them all later. This makes it easier to immediately see and fix problems introduced by a change, and is known to improve development productivity in plain C# projects (by allowing developers to stay focused on small incremental changes and getting immediate feedback on the impacts of these changes).

Examples

• Test sends a message to the orchestration;
• Orchestration sends another message (a "request" to an external system);
• Message is NOT sent to the external system, instead:
  • The test validates the message sent by the orchestration;
  • The test then builds a dummy "response" based on values from the request
• Test sends the "response" back to the orchestration.
• Test ensures the orchestration completed successfully.

The preview release includes a set of examples (in the Bizmonade.Examples.zip file).

Most examples use "boring" scenarios that are variations of the "Hello World" classic. They don't represent real-world business problems, but instead focus on testing one specific BizTalk functionality (the examples represent a large part of Bizmonade's unit tests).

The examples are split accross separate projects for Schemas, Maps, Orchestrations. However, for some examples, the schemas are directly in the Orchestrations projects. This was done this way to ensure that Bizmonade would correctly support both cases. (Depending on the project, there are legitimate reasons to either separate schemas from orchestration, or to include them in the same project)

Some of these examples are shown in this document, but other examples are available within the preview release as well.

Simple Order / PO / Invoice example

[Test]
public void TotalForOrder413ShouldBe14_99()
{
    // To test the SubmitOrder orchestration, the test must reference the C# class generated
    // from that orchestration. Generated C# classes are named the same name as the original
    // orchestration, with the "__Simulated" suffix.
    OrchestrationSimulator.Test<SubmitOrder__Simulated>()
        .When(MessageReceived.FromFile<PurchaseOrder>(
            Path.Combine(exampleInstancesPath, "PurchaseOrder_413.xml")
        ))
        // Ensure the orchestration publishes an updated "PurchaseOrder" with the correct price quote
        .ExpectMessageSent<PurchaseOrder>
        (
            msg =>
            {
                // NUnit asserts, if using another test tool, substitute with that tool's syntax
                Assert.AreEqual(14.99m, msg.DistinguishedFields["Total"]);
                Assert.AreEqual("priceQuoteSubmitted", msg.PromotedProperties[typeof(orderStatus)]);
            }
        )
        // Simulate that the client sent an "Approved Purchase Order" in response
        // to the price quote.
        .When(MessageReceived.FromFile<PurchaseOrder>(
            Path.Combine(exampleInstancesPath, "PurchaseOrder_413_Approved.xml")
        ))
        // Ensure the orchestration publishes the right Invoice in reaction to the approval
        .ExpectMessageSent<Invoice>
        (
            // Not using NUnit asserts this time, can be used with any test tool, 
            // but provides less deatailed errors in case of failure (for now)
            msg =>
                msg.GetDistinguishedField<string>("Number") == "1") &&
                msg.GetDistinguishedField<decimal>("TotalPrice") == 14.99m
        )
        // Ensure the orchestration completes successfully
        .ExpectCompleted()
    .ExecuteTest();
}

In this example:

• The test first sends a PurchaseOrder message which activates the orchestration;
• The orchestration, after applying some rules, decides that the price for the order should be 14.99, and submits a modified PurchaseOrder back to its caller (the OrchestrationSimulator in this case) for review;
• The test validates that the price is 14.99 as expected;
• The test then submits an Approved PurchaseOrder message, simulating that the client confirmed his order;
• The orchestration generates an Invoice for the order and submits it to its caller;
• The test validates that the total on the Invoice is also 14.99;
• The test validates that the orchestration was completed successfully, without any error or discarded messages.

Expect in any order example

In some cases, the test can't reliably anticipate the order in which messages will be sent by the orchestration. This frequently happens with orchestrations using parallel actions, or with tests involving multiple orchestration instances. In that case, Bizmonade allows definining a series of validation steps that can occur in any order.

[Test]
public void ShouldPublishTwoOutputMessages()
{
    OrchestrationSimulator.Test<Parallel__Simulated>()
        .When(MessageReceived.FromFile<ParallelInput>(
            Path.Combine(exampleInstancesPath, "ParallelInput_HelloWorld.xml")
        ))
        .ExpectInAnyOrder(
            ValidationSteps.Build()
                .ExpectMessageSent<ParallelInput>
                    (msg => Assert.That(msg.DistinguishedFields["Greeting.To"], Is.EqualTo("Universe")))
                .ExpectMessageSent<ParallelInput>
                    (msg => Assert.That(msg.DistinguishedFields["Greeting.To"], Is.EqualTo("Galaxy")))
                .ExpectCompleted()
        )
    .ExecuteTest();
}    
    

Request-Response Client Test

In this example, the orchestration "ReqRespClient" talks to a server. This server could be an external web service, a SQL stored procedure, another BizTalk orchestration,... However, the test doesn't need to know that and can be ran without the server: it is the test itself that simulates responses from the server. This allows testing the client in isolation from the server. Similarly, it's possible to test the server by itself without the client (not shown in this example, but can be found in Bizmonade samples).

[Test]
public void CanSubstituteServerWithTestCode()
{
    // in this test, the ReqRespClient orchestration is tested in isolation,
    // without the server. Instead of the server, the test itself responds
    // to the client's request, by building a response using the BuildDummyResponse method.
    OrchestrationSimulator.Test()
        .WithOrchestration<ReqRespClient__Simulated>()
        .When(MessageReceived.FromFile<Request>(
            Path.Combine(exampleInstancesPath, "ReqRespClient_Request.xml")
        ))
        .Expect(
            ValidationSteps.Build()
                .ExpectMessageSent<Request>(
                    msg => msg.GetDistinguishedField<string>("DataId") == "DataId_0"
                )
                // The RespondWith method allows Bizmonade to setup the correlation properties 
                // necessary for routing the response to the right orchestration instance.
                // It should be used instead of simply sending a message when working 
                // in a request-response pattern.                
                .RespondWith(requestMsg => BuildDummyResponse(requestMsg))
        )
        .ExpectMessageSent<Response>(msg =>
             {
                 Assert.That(msg.GetDistinguishedField<string>("DataId"), 
                             Is.EqualTo("DataId_0"));
                 Assert.That(msg.GetDistinguishedField<string>("Message"), 
                             Is.EqualTo("Response for DataId_0"));
                 Assert.That(msg.GetDistinguishedField<string>("To"), 
                             Is.EqualTo("ReqRespClient.Caller"));
             }
        )
        .ExpectCompleted<ReqRespClient__Simulated>()
        .ExecuteTest();
}

// Builds a dummy response, based on values from the request
private Message BuildDummyResponse(Message requestMsg)
{
    var responseMsgType = MessageType.Build().WithBodyPart<Response>();

    // TODO: Bizmonade should offer a better way to build messages at runtime
    var writer = new StringWriter();
    var xmlWriterSettings = new XmlWriterSettings()
                                {
                                    Indent = true, IndentChars = "\t"
                                };

    var xmlWriter = XmlTextWriter.Create(writer, xmlWriterSettings);
    var requestedDataId = requestMsg.GetDistinguishedField<string>("DataId");
    
    const string targetNamespace = 
        "http://Bizmonade.Tests.OrchestrationExamples.RequestResponse.Response";
    xmlWriter.WriteStartElement("resp", "Data", targetNamespace);
    xmlWriter.WriteElementString("DataId", requestedDataId);
    xmlWriter.WriteElementString("Message", "Response for " + requestedDataId);
    // will be concatenated with ".Caller" (giving "ReqRespClient.Caller") 
    // by the ReqRespClient orchestration
    xmlWriter.WriteElementString("To", "ReqRespClient");
    xmlWriter.WriteEndElement();
    xmlWriter.Flush();

    var response = responseMsgType.CreateInstance(writer.ToString());
    return response;
}    
    

Delivery notification NACK example

In this example, the test verifies that the orchestration behaves correctly in case of an error (on a port with Delivery Notification enabled). This makes it possible to test error conditions without "physically" reproducing the error (for example, by deleting a filesystem path or by putting down a web service).

The test simulates an error by producing a NACK (Negative Acknowledgement), which the orchestration will see as a DeliveryFailureException.

[Test]
public void WhenNACK_ShouldStillSendSecondOutputNotExpectingNACK_but_AlsoExceptionDetails()
{
    OrchestrationSimulator.Test()
        .WithOrchestration<SimpleDeliveryNotification__Simulated>()
        .When(MessageReceived.FromFile<DeliveryNotificationInput>(
              Path.Combine(exampleInstancesPath, "SimpleDeliveryNotificationInput_HelloWorld.xml")
        ))
        .Expect(
            ValidationSteps.Build()
                .ExpectMessageSent<DeliveryNotificationOutput>(
                    // expect a first message which is published regardless of ACK or NACK 
                    // (the orchestration will then wait for a NACK for this message)
                    msg => 
                        msg.GetDistinguishedField<string>("Greeting.Message") == 
                        "Output expecting delivery notification"
                )
                // The NACK method sends a negative acknowledgement to the orchestration. 
                // This is seen by the orchestration as a DeliveryFailureException. The NACK method
                // therefore takes as a parameter an expression to build that exception
                // based on the message.
                .NACK( 
                    msg => 
                        new Exception(
                            "Unable to deliver message with Greeting.Message=" + 
                            msg.GetDistinguishedField<string>("Greeting.Message")) 
                        )
               )
       )
       // [...remaining steps excluded...]
       .ExecuteTest();
}
    

Implementation details

Bizmonade does not depend on BizTalk's orchestration engine. Instead, it provides its own implementation of the XLang/s language (found in ODX files). It generates a C# representation from XLang/s code, and the generated C# code uses Bizmonade's "fake orchestration engine" implementation. This generation is done using an interpreter generated by SableCC. Another interesting tool that was considered instead of SableCC was Microsoft "Oslo"'s MGramar. However, development of Bizmonade was already started before the initial CTP release of MGrammar, so it was decided to keep the working implementation with SableCC for now. Migration to MGrammar could provide some interesting advantages and may be reconsidered later once, the project is mature enough.

Bizmonade's "fake orchestration engine" executes orchestrations in memory and therefore lacks some of BizTalk's features for reliability and scalability (for example, it does not persist orchestrations). This is not a weakness of the tool but rather a desired feature: the tool should allow testing without any dependency on the infrastructure, and adding any persistence would necessarily add an unwanted dependency.

Note that BizTalk also uses the same strategy of generating C# from ODX files. However, these are two completely different representations. Having a different representation allows working at an higher abstraction level (the XLang/s language), without worrying about BizTalk's internal implementation details (which makes the tool significantly easier to implement). The disadvantage of this strategy is that BizTalk's behavior needs to be reproduced as closely as possible in Bizmonade, which is a huge challenge. Therefore, tests developed using Bizmonade are not (yet) guaranteed to follow the same behavior in the real BizTalk (it only provides a "good enough" approximation for projects that were used while developing the tool). Hopefully, as the tool gets used in more real-world projects, it will be possible to have gathered enough examples to more closely follow BizTalk's behavior.

The tool still depends on some of BizTalk's assemblies. These assemblies are related to message context properties and pipelines execution (Bizmonade uses pipelines, called from the Winterdom PipelineTesting library, to promote/demote context properties)

Relationship to other existing BizTalk testing tools

Note that Bizmonade does not replace other BizTalk testing tools such as BizUnit, BizMock, or the Winterdom PipelineTesting library, but it can be used in addition to them.

• BizUnit is closer to integration tests than unit tests. It tests several BizTalk components integrated together, running within the BizTalk runtime:
  • Receive and send ports
  • Pipelines
  • Maps
  • Orchestrations
• Winterdom PipelineTesting enables unit testing of pipeline components. Bizmonade fulfills that same goal for orchestrations. (in fact Bizmonade uses PipelineTesting internally for property promotion/demotion)
• BizMock shares a similar design goal to Bizmonade: testing in abstraction of the underlying infrastructure (servers, file system locations,...). However, BizMock achieves that goal by providing a test BizTalk adapter, which means orchestration must still be deployed into BizTalk instead of being tested in isolation of the whole BizTalk environment. Like BizUnit, this tool is probably more appropriate for integration tests than unit tests.

Recommended BizTalk testing/development strategy

• Start with unit tests of individual components:
  • Schemas (including flat file schemas), using a tool such as the Winterdom PipelineTesting library:
    • Ensure a flat file is interpreted correctly by the flat file disassembler
    • Ensure a XML message is correctly converted to a XML file
    • Ensure a XML message is valid according to its schema
  • Pipeline components and pipelines: test using the Winterdom PipelineTesting library (within NUnit/MSTest/other .Net test tool)
  • .Net classes: Test using NUnit/MSTest/other .Net test tool
  • Maps:
    • Apply a map on a XML message and validate its output (either using XPath expressions or by comparing to an expected XML message). The map should be applied using the same classes as BizTalk, and not directly on the generated XSLT files, in order to ensure the map will correctly execute in the BizTalk environment. Bizmonade includes code for this (although it is primarily designed to be used internally for the "Transform" shape)
  • Orchestrations
    • Test using Bizmonade (within NUnit/MSTest/other .Net test tool)
• Deploy to BizTalk server
  • Configure bindings
  • Test "assembly" of components with integration tests:
    • Test by exchanging messages with BizTalk ports (FILE, MSMQ, HTTP,...). Suggested options:
      • BizUnit
      • BooUnit (provides simpler syntax and programming constructs over BizUnit)
      • BizMock

There will usually be some duplication between unit tests and integration tests. However, this does not mean we should forget about unit tests and directly do integration tests. Since unit tests can be run without deploying to BizTalk, they allow faster feedback after a modification (we can know much faster if a modification broke something, and avoid deploying to BizTalk until all our unit tests are successful). Also, when an integration test fails, it can be harder to pinpoint the source of the failure since multiple components are involved.

Unit tests are also not enough because they isolate the tested component from a major dependency: the BizTalk server runtime.

Since Bizmonade has its own equivalent of the BizTalk server runtime, it is not guaranteed to reproduce non-obvious behavior (or bugs/omissions) of that runtime. An ideal Bizmonade would have exactly the same behavior, but it is not complete enough yet. Therefore, it is possible to have a Bizmonade test that fails for something that would work once deployed in BizTalk. When this happens, Bizmonade should be modified accordingly (please report such issues, see Feedback)

How to use Bizmonade

Prerequisites

• .Net 3.5
• Visual Studio 2008
• BizTalk Server 2009 (required for some assemblies and for working in Visual Studio)
• NUnit 2.5.2 with nunit.framework.dll installed in the GAC (if using NUnit)

Installing Bizmonade to the GAC and installing the Visual Studio project template

• Extract files from BizmonadePreview.zip to a temporary location
• Deploy Bizmonade.Build.dll, Bizmonade.Simulator.dll and Winterdom.BizTalk.PipelineTesting.dll to the GAC (using drag and drop in Windows Explorer to C:\Windows\Assembly, or using gacutil.exe from the command line). These DLLs can also be installed to another location if you prefer not to use the GAC, but the project template assumes they are installed to the GAC (so projects generated from the template will have to be modified to reference the right path)
• Install the Visual Studio project template by double-clicking the "Bizmonade BizTalk 2009 NUnit Test Project.vsi" file.

Using the Visual Studio NUnit 2.5.2 project template

The simplest way to use Bizmonade is by creating a new project using the provided "Bizmonade BizTalk 2009 NUnit Test Project"

• In Visual Studio, open File - New - Project... - Test and select the "Bizmonade BizTalk 2009 NUnit Test Project" template; select the "Add to solution" option
• When the project is loaded for the first time on a user's machine, Visual Studio will show the following security warning:
  
  To have the test project work, select the option "Load project normally". This is necessary to have Bizmonade generate C# files from ODX files as part of the build process
  (Ideally, a future release should be fixed to avoid the security warning).
• The template assumes Bizmonade and NUnit 2.5.2 are deployed into the GAC, and that BizTalk is in c:\Program Files\Microsoft BizTalk Server 2009. If not, edit the project generated from the template to reference these DLLs from their correct location.
• On the new project, add references to BizTalk projects containing the orchestrations to be tested.
• Write tests by following the NUnit syntax, and using Bizmonade's OrchestrationSimulator.
• Run the tests with a test runner (such as NUnit GUI or ReSharper's runner)

BUG: it's sometimes necessary to compile the unit tests project twice before it runs the right version of an orchestration. This will be fixed in a later release.

Adding to another project type (using another unit testing tool)

For now, only a template using NUnit is provided. However, it's also possible to use Bizmonade with other testing tools, such as MSTest. To enable Bizmonade in an existing test project, open the project file (.csproj) in a text editor and add, after the
<Import Project="$(MSBuildBinPath)\Microsoft.CSharp.targets"/> line:

<UsingTask TaskName="Bizmonade.Build.CreateSimulatedOrchestrationsTask" AssemblyName="Bizmonade.Build, Version=1.0.0.0, Culture=neutral, PublicKeyToken=3743753380050984, processorArchitecture=MSIL" />  
<Target Name="BeforeBuild">
    <Bizmonade.Build.CreateSimulatedOrchestrationsTask ProjectFiles="@(ProjectReference)">
        <Output TaskParameter="Output" ItemName="Compile" />
    </Bizmonade.Build.CreateSimulatedOrchestrationsTask>
</Target>    
    

Also add the following references to the same .csproj file (in the <ItemGroup> containing references):

<Reference Include="Microsoft.BizTalk.DefaultPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
    <SpecificVersion>False</SpecificVersion>
    <HintPath>..\..\..\..\..\..\..\Program Files\Microsoft BizTalk Server 2006\Microsoft.BizTalk.DefaultPipelines.dll</HintPath>
</Reference>
<Reference Include="Microsoft.XLANGs.Pipeline, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
    <SpecificVersion>False</SpecificVersion>
    <HintPath>..\..\..\..\Program Files\Microsoft BizTalk Server 2009\Microsoft.XLANGs.Pipeline.dll</HintPath>
</Reference>
<Reference Include="Microsoft.BizTalk.Pipeline, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
    <SpecificVersion>False</SpecificVersion>
    <HintPath>..\..\..\..\..\..\..\Program Files\Microsoft BizTalk Server 2006\Microsoft.BizTalk.Pipeline.dll</HintPath>
</Reference>
<Reference Include="Microsoft.XLANGs.BaseTypes, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
    <SpecificVersion>False</SpecificVersion>
    <HintPath>..\..\..\..\..\..\..\Program Files\Microsoft BizTalk Server 2006\Microsoft.XLANGs.BaseTypes.dll</HintPath>
</Reference>
<Reference Include="Microsoft.BizTalk.GlobalPropertySchemas">
    <Name>Microsoft.BizTalk.GlobalPropertySchemas</Name>
</Reference>
<Reference Include="Bizmonade.Simulator, Version=1.0.0.0, Culture=neutral, PublicKeyToken=3743753380050984, processorArchitecture=MSIL">
    <SpecificVersion>False</SpecificVersion>
</Reference>        
    

Project Status

The current release is still considered a "preview" as it still lacks some important features, even though it is a working implementation that could be experimented with in some real-world projects. It probably won't allow testing all cases in a real-world project, but trying it in real projects would allow identifying the limitations that need to be resolved for a more complete version. Therefore, we would be interested in getting any comments on ways to improve the tool, or on ommited features/behavior that should be included (see Feedback).

Currently implemented features

• Receive messages;
• Send messages;
• Basic structures: if, loops, try/catch, scopes (without transactions/compensation for now);
• Call .Net classes from expression shapes;
• Transform (call BizTalk maps);
• Read/write Promoted Properties and Distinguished Fields (not all cases work yet);
• Filter Expressions;
• Correlation;
• xpath() function (read-only for now);
• Parallel actions;
• Listen;
• Delay
• Throw Exception;
• Sequential and parallel convoys;
• Discarded messages detection (zombies);
• Call/Start Orchestration: Message, Variable and Correlation parameters only for now;
• Delivery Notification (ACK/NACK);
• Request-Response
• Suspend and Resume (note that resume will not work correctly for Suspend shapes inside a Parallel branch, but the orchestration can still be Suspended - support for this special case of Resume will be added later)
• Terminate

Features planned for future releases

• Call Rules
• Transactions and Compensation
• XLang/s expression operators: succeeded, exists,...
• Property demotion (for now, the code in orchestrations must manually synchronize between promoted properties, distinguished fields and message contents to work properly in Bizmonade tests. This is obviously incorrect and needs to be fixed in a future release).
• Execute pipelines from orchestrations
• Multipart messages, .Net class messages and XLang messages (only single part XML messages are working for now, others are only partially implemented and not fully tested);
• Modify message contents using the xpath() function (it only allows reading from messages for now);
• Support all cases of Promoted Properties and Distinguished Fields manipulations.
  • They do not work at all on multipart messages;
  • The C# compiler is not always able to infer a property's type from the way it is being used. Therefore, the XLang/s to C# generation process should cast each property to the right type (as defined in the message's schemas). For now, these limitations can be worked around by manually doing the necessary conversion/cast when the compiler is not able to do it (for example by using System.Convert.ToXXXX or System.String.Format)
• Dynamic Send Ports
• Role Links
• Partner Ports and Self-Correlating Direct Binding (only Message Box Direct Binding is implemented)
• Promote relevant context properties. Note however that closely following ALL of BizTalk's context properties would be contrary to the tool's goal of infrastructure-ignorance: for example, it should not promote any property that exist only on messages coming from a FTP or FILE port. Therefore, it will only implement a minimal set of context properties: those that are required for some use case, and that don't depend on a specific protocol. (However, if really needed, a unit test can still promote protocol-specific properties)
• Implement missing types of parameters for Call/Start orchestration
• Substitute System.DateTime.Now with a "faked timer", to allow quickly running tests dependent on dates/times/delays (already partially implemented but relies on timing hacks to work)
• Better error messages when a test fails, to make it as easy as possible to locate the source of an error.

Features that will not be implemented

Some features would contradict the tool's goal of infrastructure-ignorance, and will therefore not be implemented in future releases. These still represent things that are important to test, but an integration testing tool (which tests the orchestrations integrated in the real BizTalk and with their real dependencies) would be more appropriate to test them.

• Orchestration Persistence
• Interaction with any protocol or BizTalk adapter
• Promotion of context properties coming from a specific BizTalk adapter. (Tests can still promote these properties, but they won't be promoted automatically by Bizmonade)
• Side by side versionning (test some messages with version 1.0 of an orchestration, and other messages with version 1.1 of the same orchestration, within the same test)
• BAM

Feedback

Since the product is still in development, it would be a good time to get feedback on ways the tool could be improved. Specifically, we are looking for answers to:

• Is there anything important that's missing (and not part of the features planned for a future release)?
• Suggestions on ways to improve the way tests are written?
• Does the tool have behavior that is different from the real BizTalk's behavior? (if possible, please provide sample orchestrations and unit tests, it would greatly help in implementing a fix).
• Suggestions on ways to automatically compare Bizmonade and the real BizTalk's behavior? (Currently, BizTalk's behavior is checked with manual tests, but it would be interesting to have a way to reuse Bizmonade automated tests in a real BizTalk context and validate that both contexts return the same results)

For now, feedback can be sent using the comments section of the original Bizmonade announcement blog post. A temporary email address has also been setup: bizmonade at gmail.com.

A more structured discussion board or issue tracking system will be setup later.