Matevž Gačnik's Weblog - Transactions

Demos from the NT Conference 2008

Matevz Gacnik — Thu, 15 May 2008 15:24:19 GMT

As promised, here are the sources from my NTK 2008 sessions [1].

Talk: Document Style Service Interfaces

Read the following blog entry, I tried to describe the concept in detail. Also, this blog post discusses issues when using large document parameters with reliable transport (WS-RM) channels.

Demo: Document Style Service Interfaces [Download]

This demo defines a service interface with the document parameter model, ie. Guid CreatePerson(XmlDocument person). It shows three different approaches to creation of the passed document:

Raw XML creation
XML Serialization of the (attribute annotated) object graph
XML Serialization using the client object model

Also, versioned schemas for the Person document are shown, including the support for document validation and version independence.

Talk: Windows Server 2008 and Transactional NTFS

This blog entry describes the concept.

Demo 1: Logging using CLFS (Common Log File System) [Download]
Demo 2: NTFS Transactions using the File System, SQL, WCF [Download]
Demo 3: NTFS Transactions using the WCF, MTOM Transport [Download] [2]

[1] All sources are in VS 2008 solution file format.
[2] This transaction spans from the client, through the service boundary, to the server.

Managed TxF: Distributed Transactions and Transactional NTFS

Matevz Gacnik — Mon, 23 Jul 2007 20:54:13 GMT

Based on my previous post, I managed to get distributed transaction scenario working using WCF, MTOM and WS-AtomicTransactions.

This means that you have the option to transport arbitrary files, using transactional ACID semantics, from the client, over HTTP and MTOM.

The idea is to integrate a distributed transaction with TxF, or NTFS file system transaction. This only works on Windows Server 2008 (Longhorn Server) and Windows Vista.

Download: Sample code

If the client starts a transaction then all files within it should be stored on the server. If something fails or client does not commit, no harm is done. The beauty of this is that it's all seamlessly integrated into the current communication/OS stack.

This is shipping technology; you just have to dive a little deeper to use it.

Here's the scenario:

There are a couple of issues that need to be addressed before we move to the implementation:

You should use the managed wrapper included here
There is support for TransactedFile and TransactedDirectory built it. Next version of VistaBridge samples will include an updated version of this wrapper.
Limited distributed transactions support on a system drive
There is no way to get DTC a superior access coordinator role for TxF on the system drive (think c:\ system drive). This is a major downside in the current implementation of TxF, since I would prefer that system/boot files would be transaction-locked anyway. You have two options if you want to run the following sample:
- Define secondary resource manager for your directory
  This allows system drive resource manager to still protect system files, but creates a secondary resource manager for the specified directory. Do this:
  - fsutil resource create c:\txf
  - fsutil resource start c:\txf
    You can query your new secondary resource manager by fsutil resource info c:\txf.
- Use another partition
  Any partition outside the system partition is ok. You cannot use network shares, but USB keys will work. Plug it in and change the paths as defined at the end of this post.

OK, here we go.

Here's the service contract:

[ServiceContract(SessionMode = SessionMode.Allowed)]
interface ITransportFiles
{
   [OperationContract]
   [TransactionFlow(TransactionFlowOption.Allowed)]
   byte[] GetFile(string name);

   [OperationContract]
   [TransactionFlow(TransactionFlowOption.Allowed)]
   void PutFile(byte[] data, string name);
}

We allow the sessionful binding (it's not required, though) and allow transactions to flow from the client side. Again, transactions are not mandatory, since client may opt-out of using them and just transport files without a transaction.

The provided transport mechanism uses MTOM, since the contract's parameter model is appropriate for it and because it's much more effective transferring binary data.

So here's the service config:

<system.serviceModel>
<bindings>
    <wsHttpBinding>
      <binding name="MTOMBinding"
          transactionFlow="true"
          messageEncoding="Mtom"
          maxReceivedMessageSize="10485760">
        <readerQuotas maxArrayLength="10485760"/>
      </binding>
    </wsHttpBinding>
</bindings>
<services>
    <service name="WCFService.TransportService">
      <host>
        <baseAddresses>
          <add baseAddress="http://localhost:555/transportservice">
        </baseAddresses>
      </host>
      <endpoint address=""
          binding="wsHttpBinding"
          bindingConfiguration="MTOMBinding"
          contract="WCFService.ITransportFiles"/>
    </service>
</services>
</system.serviceModel>

Here, MTOMBinding is being used to specify MTOM wire encoding. Also, quotas and maxReceivedMessageSize attribute is being adjusted to 10 MB, since we are probably transferring larger binary files.

Service implementation is straight forward:

[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
class TransportService : ITransportFiles
{
    [OperationBehavior(TransactionScopeRequired = true)]
    public byte[] GetFile(string name)
    {
        Console.WriteLine("GetFile: {0}", name);
        Console.WriteLine("Distributed Tx ID: {0}",
          Transaction.Current.TransactionInformation.DistributedIdentifier);
        return ReadFully(TransactedFile.Open(@"C:\TxF\Service\" + name,
          FileMode.Open, FileAccess.Read, FileShare.Read), 0);
    }

    [OperationBehavior(TransactionScopeRequired = true)]
    public void PutFile(byte[] data, string filename)
    {
        Console.WriteLine("PutFile: {0}", filename);
        Console.WriteLine("Distributed Tx ID: {0}",
          Transaction.Current.TransactionInformation.DistributedIdentifier);

        using (BinaryWriter bw = new BinaryWriter(
            TransactedFile.Open(@"C:\TxF\Service\" + filename,
              FileMode.Create, FileAccess.Write, FileShare.Write)))
        {
            bw.Write(data, 0, data.Length);

            // clean up
            bw.Flush();
        }
    }
}

Client does four things:

Sends three files (client - server) - no transaction
Gets three files (server - client) - no transaction
Sends three files (client - server) - distributed transaction, all or nothing
Gets three files (server - client) - distributed transaction, all or nothing

Before you run:

Decide on the secondary resource manager option (system drive, enable it using fsutil.exe) or use another partition (USB key)
Change the paths to your scenario. The sample uses C:\TxF, C:\TxF\Service and C:\TxF\Client and a secondary resource manager. Create these directories before running the sample.

Download: Sample code

This sample is provided without any warranty. It's a sample, so don't use it in production environments.

Managed TxF: Support in Windows Vista and Windows Server 2008

Matevz Gacnik — Fri, 20 Jul 2007 14:59:16 GMT

If you happen to be on a Windows Vista or Windows Server 2008 box, there is some goodness going your way.

There is a basic managed TxF (Transactional NTFS) wrapper available (unveiled by Jason Olson).

What this thing gives you is this:

try
{
   using (TransactionScope tsFiles = new TransactionScope
      TransactionScopeOption.RequiresNew))
   {
      WriteFile("TxFile1.txt");
      throw new FileNotFoundException();
      WriteFile("TxFile2.txt");
      tsFiles.Complete();
   }
}
catch (Exception ex)
{
   Console.WriteLine(ex.Message);
}

WriteFile method that does, well, file writing, is here:

using (TransactionScope tsFile = new TransactionScope
(TransactionScopeOption.Required))
{
Console.WriteLine("Creating transacted file '{0}'.", filename);

   using (StreamWriter tFile = new StreamWriter(TransactedFile.Open(filename,
          FileMode.Create,
          FileAccess.Write,
          FileShare.None)))
   {
      tFile.Write(String.Format("Random data. My filename is '{0}'.",
          filename));
   }

tsFile.Complete();
Console.WriteLine("File '{0}' written.", filename);
}

So we have a nested TransactionScope with a curious type - TransactedFile. Mind you, there is support for TransactedDirectory built in.

What's happening underneath is awesome. The wrapper talks to unmanaged implementation of TxF, which is built in on every Vista / Longhorn Server box.

What you get is transactional file system support with System.Transactions. And it's going to go far beyond that.

I wrote some sample code, go get it. Oh, BTW, remove the exception line to see the real benefit.

Download: Sample code

This sample is provided without any warranty. It's a sample, so don't use it in production environments.

Durable Messaging Continues

Matevz Gacnik — Wed, 30 May 2007 20:46:14 GMT

Nick is continuing my discussion on durable messaging support in modern WS-* based stacks, especially WCF.

I agree that having a simple channel configuration support to direct messages into permanent information source (like SQL) would be beneficial.

A simple idea that begs for an alternative implementation of a WCF extensibility point, has some questions:

What happens when messages are (or should be) exchanged in terms of a transaction context?
How can we demand transaction support from the underlying datastore, if we don't want to put limitations onto anything residing beind the service boundary?
What about security contexts? How can we acknowledge a secure, durable sessionful channel after two weeks of service ~~hibernation~~ down time? Should security keys still be valid just because service was down and not responding all this time?
Is durability limited to client/service recycling or non-memory message storage? What about both?

Is [DurableService]enough?

.NET 3.0 Middleware Technologies Day: Third Incarnation

Matevz Gacnik — Thu, 15 Feb 2007 05:09:56 GMT

Third incarnation of the .NET 3.0 Middleware Technologies day went through yesterday.

Here are the deliverables:

PPT: Download [1]
WCF Samples: Download
WF Samples: Download

If you did not get/notice the printed articles, here's what should put you to sleep over the weekend:

Also, the accompanying book is available here.

[1] Only available in Slovene.

Article: Transactional Semantics in Loosely Coupled Distributed Systems

Matevz Gacnik — Tue, 06 Jun 2006 08:59:30 GMT

Last article finished my XML series. This one focuses on transactional semantics in a service oriented universe.

Language: Slovenian

Naslov:

Transakcijska semantika v šibko sklopljenih, porazdeljenih sistemih

WCF - List of Changes from Beta 1 to Beta 2 RC0

Matevz Gacnik — Sun, 11 Dec 2005 19:12:52 GMT

The following is a list of Windows Communication Foundation (read Indigo) changes from September CTP (beta 1) to November CTP (beta 2 RC0).

Most of them are name changes. Nevertheless, you will need to update at least configuration, since element/attribute naming there has changed quite a lot.

Class name changes:

Fault<T> FaultException<T>
UnknownFault UnknownFaultException
ServiceSite InstanceContext

Parameters of [OperationBehaviour] attribute:

AutoCompleteTransaction TransactionAutoComplete
AutoEnlistTransaction TransactionScopeRequired

Other changes:

ServiceHost<T> ServiceHost(typeof(T)>

Binding names:

wsProfileBinding wsHttpBinding
netProfileTcpBinding netTcpBinding
netProfileDualTcpBinding netTcpBinding
netProfileNamedPipeBinding netNamedPipeBinding
basicProfileBinding basicHttpBinding

Configuration element/attribute name changes:

/services/service[@serviceType] /services/service[@type]
/endpoints/endpoint[@contractType] /endpoints/endpoint[@contact]
/services/service/endpoint[@bindingSectionName] /services/service/endpoint[@binding]
/behaviors/behavior[@configurationName] /behaviors/behavior[@name]
/bindings/*/binding[@configurationName] /bindings/*/binding[@name]

Attribute changes:

[BindingRequirements(TransactionFlowRequirements=RequirementsMode.Require)]
[TransactionFlow(TransactionFlowOption.Required)]
[BindingRequirements(QueuedDeliveryRequirements=RequirementsMode.Require)
[BindingRequirements(QueuedDeliveryRequirements=BindingRequirementsMode.Required)
[InstanceMode=InstanceMode.Singleton]
[InstanceContextMode=InstanceContextMode.Single]
[InstanceMode=InstanceMode.PrivateSession]
[InstanceContextMode=InstanceContextMode.PerSession]
[InstanceMode=InstanceMode.PerCall]
[InstanceContextMode=InstanceContextMode.PerCall]
[InstanceMode=InstanceMode.SharedSession]
[InstanceContextMode=InstanceContextMode.Shared]

This list may not be complete.

I do expect that QueuedDeliveryRequirements property of [BindingRequirements] attribute will be standalone when Indigo ships. There's no point in having a separate attribute for expected transaction semantics and leave out the queued delivery. It's the same concept being pushed down to administrators (binding requirements are fixed requirements which are supposed to be met by the admin - they are the developers' law of expected processing model).

Maybe this attribute should be named [QueuedDelivery], with appropriate property for demanding it (like QueuedDeliveryOption) and a property called OrderedSessionOption, which would handle ordered session maintenance.

Indigo - Multiple endpoints

Matevz Gacnik — Fri, 12 Aug 2005 11:02:47 GMT

There is a common functionality request in modern distributed applications that clients and services (client also being other services) could communicate using different communication protocols and/or different transport semantics inside a single protocol.

For this sample we're going to limit ourselves to HTTP messaging, but this does not limit the scope of the article. Other transport protocols can be added without obstructing other transport mechanisms.

To allow this scenario Indigo (Windows Communication Foundation, WCF) allows multiple endpoints per service to be setup. Service side bindings can, for example, be the following:

There are six different bindings setup:

Secure
SecureReliable
SecureReliableTransacted
SecureMTOM
SecureReliableMTOM
SecureReliableTransactedMTOM

The Secure binding uses WS-Security/WS-SecureConversation pair to secure all outgoing and incoming messages using encryption and digital signing. We’re not going to drill down into it in this post. I’m going to write another one where different certificate mechanisms will be discussed.

The SecureReliable binding also uses WS-ReliableMessaging with message ordering and guaranteed delivery turned on.

The SecureReliableTransacted endpoint also uses WS-AtomicTransaction to implement transaction semantics between the client and the service.
There are another three bindings (*MTOM) with the same transport semantics as well as MTOM encoding turned on. Since services can support transfer of large binary files, we have also set the maxMessageSize attribute in every binding to 128 MB.

Endpoint configuration for this service is:

It can be seen that almost every endpoint (two are bound to the same binding) is bound to a different binding.
The service is therefore exposing its functionality using the following endpoints:

The first endpoint in this example is considered a default one and is mapped to the same binding as the *securereliableMTOM endpoint.

The problem with this is that current WCF’s WSDLExporter maps different endpoints into WSDL ports, but WSDL ports have a requirement to be uniquely named. The default port name construction is done by using the binding type name (i.e. wsProfileBinding, basicProfileBinding), which is concatenated with the contract name.

Therefore, when one has defined multiple bindings from the same binding type a name collision occurs (six WSDL ports with the same QName (namespace and local name)). This can be avoided using a bindingNamespace attribute as well as bindingName attribute. In this example we set the bindingNamespace to http://webservices.gama-system.com/bindings for all bindings and made the name unique by using the bindingName attribute.

There are currently two weird design decisions in multiple binding support of WCF. First bindingNamespace and bindingName attributes are bound to endpoints and not bindings. It would probably be more appropriate to put binding’s namespace and name into the <binding> element. Second, binding name is not propagated into WSDL properly when the same binding is used in two or more endpoints (in this example http://localhost:666/TransferService.svc/ and http://localhost:666/TransferService.svc/securereliableMTOM endpoints are bound to Default binding. If you look closely into the generated WSDL, you will see that the matching WSDL ports are not named after bindingName attribute.

These are non-breaking issues and will probably be addressed for beta 2 of WCF.

In the next Indigo series post, we’ll look closely into the security characteristics of certificate based authentication.

OTS 2004

Matevz Gacnik — Sun, 20 Jun 2004 21:19:39 GMT

As last year, I spoke on OTS 2004 too.

This time, we discussed transactional semantics in weakly bound, distributed systems.

Hope to see you on 2005's annual reincarnation of a great conference.