Building BizTalk Server
From InstallationWiki
| Official Page |
| Project Documentation |
| Download |
|
Contents |
[edit] BizTalk architecture
So how does BizTalk Server actually work? BizTalk Server at its core is an event-processing engine, based on a conventional publish-subscribe pattern. Wikipedia defines the publish-subscribe pattern as:
An asynchronous messaging paradigm where senders (publishers) of messages are not programmed to send their messages to specific receivers (subscribers). Rather, published messages are characterized into classes, without knowledge of what (if any) subscribers there may be. Subscribers express interest in one or more classes, and only receive messages that are of interest, without knowledge of what (if any) publishers there are.
|
Critical point This pattern enforces a natural loose coupling and provides more scalability than an engine that requires a tight connection between receivers and senders. In the first release of BizTalk Server, the product DID have tightly coupled messaging components, but thankfully the engine was completely redesigned for BizTalk Server 2004. |
Once a message is received by a BizTalk adapter, it runs through any necessary pre-processing (such as decoding) in BizTalk pipelines, before being subjected to data transformation via BizTalk maps, and finally being published to a central database called the MessageBox. Then, parties which have a corresponding subscription for that message can consume it as they see fit. While introducing a bit of unavoidable latency, the MessageBox database makes up for that by providing us with durability, reliability, and scalability. For instance, if one of our subscriber systems is offline for maintenance, outbound messages are not lost, but rather the MessageBox makes sure to queue messages until the subscriber is ready to receive them. Worried about a large flood of inbound messages that steal processing threads away from other BizTalk activities no problem! The MessageBox makes sure that each and every message finds its way to its targeted subscriber, even if it must wait until the flood of inbound messages subside.
There are really two ways to look at the way BizTalk is structured. The first is the traditional EAI view, which sees BizTalk receiving messages, and routing them to the next system for consumption. The flow is very linear and BizTalk is seen as a broker between two applications.
However, the other way to consider BizTalk, and the focus of this tutorial, is as a service bus, with numerous input/output channels that process messages in a very dynamic way. That is, instead of visualizing the data flow as a straight path through BizTalk to a destination system, consider BizTalk exposing services as on-ramps to a variety of destinations. Messages published to BizTalk Server may fan out to dozens of subscribers, who have no interest in what the publishing application actually was. Instead of thinking about BizTalk as a simple connector of systems, think of BizTalk as a message bus which coordinates a symphony of events between endpoints.
This concept, first introduced to me by the incomparable Charles Young (http://geekswithblogs.net/cyoung/), is an exciting way to exploit BizTalk's engine in this modern world of service-orientation. In the diagram below, I've shown how the central BizTalk bus has receiver services hanging off of it, and has a multitude of distinct subscriber services that are activated by relevant messages reaching the bus.
|
If the on-ramp concept is a bit abstract to understand, consider a simple analogy. In designing the transportation for a city, it would be foolish of me to create distinct roads between each and every destination. The design and maintenance of such a project would be lunacy. I would be smart to design a shared highway with on and off ramps, which enable people to use a common route to get between the numerous locations around town. As new destinations in the city emerge, the entire highway (or road system) doesn't need to undergo changes, but rather, only a new entrance/exit point needs to be appended to the existing shared infrastructure. |
What exactly is a message anyway? A message is data processed through BizTalk Server's messaging engine, whether that data is transported as an XML document, a delimited flat file, or a Microsoft Word document. The message content may contain a command (for example InsertCustomer), a document (for example Invoice), or an event (for example VendorAdded). A message has a set of properties associated with it. First and foremost, a message may have a type associated with it which uniquely defines it within the messaging bus. The type is typically comprised of the XML namespace and the root node name (for example ([http://CompanyA.Purchasing#PurchaseOrder]). The message type is much like the class object in an object-oriented programming language; it uniquely identifies entities by their properties. The other critical attribute of a message in BizTalk Server is the property bag called the message context. The message context is a set of name/value properties that stays attached to the message as long as it remains within BizTalk Server. These context values include metadata about the transport used to publish the message, and attributes of the message itself. Properties in the message context that are visible to the BizTalk engine, and therefore available for routing decisions, are called promoted properties.
How does a message actually get into BizTalk Server? A receive location is configured for the actual endpoint that receives messages. The receive location uses a particular adapter, which knows how to absorb the inbound message. For instance, a receive location may be configured to use the FILE adapter which polls a particular directory for XML messages. The receive location stores the file path to monitor, while the adapter provides transport connectivity. Upon receipt of a message, the adapter stamps a set of values into the message context. For the FILE adapter, values such as ReceivedFileName are added to that message's context property bag. Note that BizTalk has both application adapters such as SQL Server, Oracle, and SAP as well as transport-level adapters such as HTTP, MSMQ, and FILE. The key point is that the adapter configuration user experience is virtually identical regardless of the type of adapter chosen.
Receive locations have a particular receive pipeline associated with them. A pipeline is a sequential set of operations that are performed on the inbound message in preparation for being parsed and processed by BizTalk. For instance, I would need a pipeline in order to decrypt, unzip, or validate the XML structure of my inbound message. One of the most critical roles of the pipeline is to identify the type of the inbound message and put the type into the message context as a promoted property. As discussed earlier, a message type is the unique characterization of a message. Think of a receive pipeline as doing all the pre-processing steps necessary for putting the message in its most usable format.
A rece'ive port' contains one or more receive locations. Receive ports have XSLT maps associated with them that are applied to messages prior to publishing them to the MessageBox database. What value does a receive port offer me? It acts as a grouping of receive locations where capabilities such as mapping and data tracking can be applied to any of the receive locations associated with it. It may also act as a container that allows me to publish a single entity to BizTalk Server regardless of how it came in, or what it looked like upon receipt. Let's say that my receive port contains three receive locations, which all receive slightly different "invoice" messages from three different external vendors. At the receive port level, I have three maps that take each unrelated message and maps it to a single, common format, before publishing it to BizTalk.
|
Critical point By default, all messages pass through BizTalk Server as a stream of bytes, not as an XML message loaded into the server's memory. Therefore, when the message is published to the MessageBox, BizTalk Server has yet to look inside the message unless:
Note that custom pipeline components may also peek into the message content. If the message has promoted properties associated with it, then the disassembler pipeline component will extract the relevant data nodes from the message and insert them into the message context. |
Now that we have a message cleaned up (by the pipeline) and in a final structure (via an XSLT map), it's published to the BizTalk Server MessageBox where message routing can begin. For our purposes, there are two subscribers that we care about. The first type of subscriber is a send port. A send port is conceptually the inverse of the receive location and is responsible for transporting messages out of the BizTalk bus.
It has not only an adapter reference, adapter configuration settings, and a pipeline (much like the receive location), but it also has the ability to apply XSLT maps to outbound messages. If a send port subscribes to a message, it first applies any XSLT maps to the message, then processes it through a send pipeline, and finally uses the adapter to transmit the message out of BizTalk.
The other subscriber for a published message is a BizTalk orchestration. An orchestration is an executable business process, which uses messages to complete operations in a workflow. We'll spend plenty of time working with orchestration subscribers throughout this tutorial.
[edit] Setting up new BizTalk projects
What do you need to set up a brand new BizTalk project? First, you will want to have a development environment with Windows Server 2008, IIS 7.0, SQL Server 2008, Visual Studio 2008, and BizTalk Server 2009, installed in that order.
Consider using a standard structure for all of your BizTalk Server solutions. This makes it easier to package and share source code, while also defining a consistent place to store solution artifacts in each project. To build the structure below, I put together a VBScript file, which is available on my blog at: http://seroter.wordpress.com/2007/03/29/script-for-automatically-creating-biztalk-solution-structure/.
Note that BizTalk Server 2009 solutions can (and should) be centrally persisted in standard source control applications such as Subversion or Microsoft Team Foundation Server.
You can tell if you have successfully installed BizTalk Server in your development environment if you are able to see BizTalk Projects in the Visual Studio.NET New Projects menu option.
When a new BizTalk Project is added to a Visual Studio.NET solution, you should immediately right-click the project and select the Properties option. In BizTalk Server 2009, we can now set properties in the familiar C# project properties pane, instead of the BizTalk-only properties window. The BizTalk project type has been redesigned so that BizTalk projects are now simply specialized C# project types.
The first value that you need to set is under the Signing section. You can either point to an existing strong name key, or now in BizTalk Server 2009, generate a new key on the fly. BizTalk Server projects are deployed to the Global Assembly Cache (GAC) and must be strong named prior to doing so. After setting the necessary key value, navigate to the BizTalk-specific Deployment section, and set the Application Name to something meaningful such as BizTalkSOA.
Once you have a project created, the strong name key set, and application name defined, you're ready to start adding development artifacts to your project.
[edit] Configuring BizTalk messaging
Understanding how to design and arrange BizTalk messaging settings is an absolutely critical part of designing any BizTalk solution, let alone a service-oriented one.
Earlier in this crash-course on BizTalk Server, we discussed the BizTalk messaging architecture and its foundation in a publish and subscribe routing model. One of the most important parts of a messaging configuration is enabling the receipt of new messages. Without the ability to absorb messages, there's not much else to talk about. In BizTalk Server, messages are brought onboard through the combination of receive ports and receive locations.
Receive ports can be configured from within the BizTalk Server Administration Console. New receive ports support both "one-way" or "two-way" message exchange patterns. On the lefthand side of a receive port configuration, there are a series of vertically arranged tabs that display different sets of properties. Choosing the Receive Locations tab enables us to create the actual receive location which defines the URI that BizTalk will monitor for inbound messages. In the Transport section of a receive location's primary configuration pane, we can choose from the list of available BizTalk adapters. Once an adapter is chosen from the list, the Configure button next to the selected transport type becomes active. For a receive location exploiting the FILE adapter, "configuration" requires entering a valid file path into the Receive folder property.
The next step in configuring BizTalk messaging is to create a subscriber for the data that is published by this receiving interface. BizTalk send ports are an example of a subscriber in a messaging solution. Much like receive locations, send ports allow you to choose a BizTalk adapter and configure the transmission URI for the message. However, simply configuring a URI does not complete a send port configuration, as we must pinpoint what type of message this subscriber is interested in. On the left side of a send port configuration window, there is a vertical set of tabs. The Filters tab is where we can set up specific interest criteria for this send port. For example, we could define a subscription that listens for all messages of a particular type that reach the MessageBox.
|
A send port can be in three distinct states. By default, a send port is |
The BizTalk Server messaging engine is the heart and soul of a BizTalk solution. Here we saw how to create new input interfaces, and define subscribers for the published data.
[edit] Working with BizTalk orchestration
BizTalk Server includes a workflow platform, which allows us to graphically create executable, long-running, stateful processes. These workflows, called orchestrations, are designed in Visual Studio.NET and executed on the BizTalk Server. The Orchestration Designer in Visual Studio.NET includes a rich palette of shapes we can use to build robust workflows consisting of control flow, message manipulation, service consumption, and much more. The Orchestration Runtime is responsible for executing the orchestrations and managing their state data.
Orchestration is a purely optional part of a BizTalk solution. You can design a complete application that consists solely of message routing ports. In fact, many of the service-oriented patterns that we visit throughout this tutorial will not require an orchestration. That said, there are a number of scenarios where injecting orchestrations into the solution makes sense. For instance, instead of subscribing directly to the "new employee" message, perhaps a payroll system will need additional data (such as bank information for a direct deposit) not currently available in the original employee message. We could decide to create a workflow, which first inserts the available information into the payroll system, and then sends a message to the new employee asking for additional data points. The workflow would then wait for and process the employee's response and conclude by updating the record in the payroll system with the new information. BizTalk orchestrations are a good fit for automating manual processes, or choreographing a series of disconnected services or processes to form a single workflow.
Orchestration "shapes" such as Decide, Transform, Send, Receive, and Loop are used to build our orchestration diagrams like the one below. This particular diagram below shows a message leaving the orchestration, and then another message returning later on in the flow. How does that message know which running orchestration instance to come back to? What if we have a thousand of these individual processes in flight at a single point in time? BizTalk Server has the concept of correlation which means that you can identify a unique set of attributes for a given message which will help it find its way to the appropriate running orchestration instance. A correlation attribute might be as simple as a unique invoice identifier, or a composite key made up of a person's name, order date, and zip code.
Orchestration is a powerful tool in your development arsenal and we will make frequent use of it throughout this tutorial.
[edit] Source
The source of this content is Chapter 1: Building BizTalk Server of SOA Patterns with BizTalk Server 2009 by Richard Seroter (Packt Publishing, 2009).
