Cross Domain MSDTC

I spent a lot of time troubleshooting an error from a BizTalk send port:

 

System.Runtime.InteropServices.COMException: The MSDTC transaction manager was unable to push the transaction to the destination transaction manager due to communication problems. Possible causes are: a firewall is present and it doesn't have an exception for the MSDTC process, the two machines cannot find each other by their NetBIOS names, or the support for network transactions is not enabled for one of the two transaction managers.

 

I thought I would summarize my findings.

 

What is MSDTC? Microsoft Distributed Transaction Coordinator. In a nutshell, MSDTC coordinates transactions between the BizTalk Server and SQL Server. For instance, if you have a polling statement in SQL that updates the database, you would not want it to commit if the message fails to get added to the BizTalk MessageBox database. Using MSDTC, the update call will rollback if the messagebox call fails. Without MSDTC, there is a risk of losing messages in this scenario.

 

MSDTC uses NetBIOS name resolution. This was one of the first hurdles I had to overcome as the names were not resolving to the IP address of the servers. To troubleshoot this, an entry was made in the hosts file to resolve the sql server NetBIOS name. No matter what you read, this is not the preferred method of dealing with name resolution. If the IP of the server changes, every host file would have to be modified causing an administration nightmare Because MSDTC uses NetBIOS resolution, not a fully qualified domain name, a way needed to be found to resolve the 'short' name with DNS. There is a setting in Advanced TCP/IP Settings in which you can append a suffix to the short name for resolution. Adding [domain] to this suffix list resolved the name resolution (http://technet.microsoft.com/en-us/library/cc959339.aspx). When this was resolved, we ran into firewall issues.

 

MSDTC uses port 135 to initiate the connection. After that, Remote Procedure Call dynamically allocates a port between 1024 and 65535. Obviously, from a security standpoint, opening that range of ports between servers is not suggested. There is a way to limit the port range using registry settings. At some point, this was done on the [biztalk server] box but was not on the BizTalk server. Changing the registry settings requires a reboot of the BizTalk server. After that was done on dev and the firewall was open to the new range of ports, everything worked correctly. We currently are allocating 200 ports for MSDTC connections between biztalk and [sql server] in development (http://support.microsoft.com/kb/250367).

 

The ability to turn off MSDTC is available in the binding settings of the send port. If the UseAmbientTransactions is set to ‘false,’ your send port will make its sql call without using transaction coordination. This is not a solution, be careful turning this off and know that distributed transaction are not required. This would be safe if your sql call included no updates, just gets.

 

There are also some setting in the Component Services->Computers->My Computer->Distributed Transaction Coordinator->Local DTC that need to be configured. In Properties->Security Network DTC Access needs to be checked as well as allowing all of the inbound and outbound connections. Also, since this is cross domain, No Authentication Required should be checked. Those settings need to match on both servers.

Byte Order Mark (BOM) in BizTalk message

I was sending out some eRx files to a folder to use to validate with the NIST validation tool. When we tried to import the message to the tool, it failed with an invalid xml error and was showing a 'special' character at the beginning of the file. Turns out, the send port was appending a Byte Order Mark (BOM). The following blog post describes the issue in better detail. A big question I need to dig into more, is why a pass-through pipeline was appending a BOM? Solution was to change the pipeline to xml transmit and set PreserveBOM to false.


http://mindovermessaging.com/2013/08/06/removing-the-bom-from-outgoing-biztalk-files/

Impressions of Day One of BizTalk Integration Summit

Day one of the BizTalk Integration Summit is over. Here is what I came away with.

Microsoft is commited to the BizTalk platform. They have scheduled major releases every other year, along with minor releases every other year. There will be a BizTalk Server 2013 R2 release in early 2014 with a major release in 2015. A lot of the minor 2014 release will focus on improving healthcare integration with support for HL7 2.6, HL7 Accelerator improvements, 64-bit MLLP, dynamic MLLP port. Also, R2 will bring JSON REST support, proxy support for SFTP, service bus adapters and map engine improvements.

Much of the day was spent discussing the future of Azure BizTalk Services and business project management. BizTalk Services went live and there are a number of Microsoft partners deploying solutions. Business project management is on the horizon and they are gathering information on what should be included. It probably will include a notation for modeling, unified tools, out of the box business activities templates, a web based Business Rules Engine, web-based configuration. It will be build on .Net workflow engine. It is clear that Microsoft is focused on cloud computing.

C++/CLI library in BizTalk component

For a BizTalk project I needed to decrypt a password stored in a database that was encrypted using a legacy unmanaged C++ Blowfish static library. In attempting to write a C# blowfish decrypter, it was discovered that the Blowfish library wasn't really a 'true' implementation of Blowfish. Since redoing the encryption was not an option due to many factors, I decided to write a managed C++/CLI wrapper class of the static libraries. Using this as a guide (http://tom-shelton.net/?p=95), I went about creating the wrapper class. Most of the challenges of getting the wrapper class to work revolved around matching data structures between the managed and unmanaged code. Special care should be taken to clean up any unmanaged objects.

A pipeline component was created that made the database lookup, decrypted the password using the wrapper class and inserted the password into the outgoing message. To use the wrapper dll, it needed to be strong named. I'm sure there is probably a way to do that in a post-build command, but I did it manually using

 sn -Rca //pathtodll keycontainer  

The dll needs to be gaced into the 32 bit Global Assembly Cache. Any BizTalk host that uses the pipeline component should be checked 32-bit only.

Biztalk retry pattern, deliver notification

I have a BizTalk orchestration that uses a retry pattern. After a code review, it was noticed that the failing ports should have delivery notification set to transmitted. Otherwise, there is a chance that on a restart, the send message could be resumed before the orchestartion. This would cause two messages. Here are some links describing the retry pattern and delivery notification:

http://realbiztalk.blogspot.com/2004/12/useful-retry-pattern.html 

http://kentweare.blogspot.com/2007/11/biztalk-delivery-notification.html 

Multi-Part Map instance

Testing a debugging a multi-part map in BizTalk 2010 can be a pain. There is no way to easily create an instance of an aggregate schema. This post shows a way to create an aggregate instance by inverting the map:

http://allcomputers.us/windows_server/biztalk-2010-recipes---document-mapping---testing-maps.aspx

Orchestration design disappeared

I opened a BizTalk 2010 orchestration file in the xml editor in Visual Studio 2010. When I tried to get back to the design view, it insisted on opening in xml. I found this blog posting that gave me the solution:

http://blog.tallan.com/2012/06/05/biztalk-orchestration-fails-to-open-in-design-view/


P.S.: I think this is my second solution from blog.tallan.com. I might have to bookmark them.