115
CHAPTER 4
Pipelining and Components
Pipelines are probably the least properly utilized tools in the BizTalk toolbox. Pipelines are
designed to do one thing well:
Examine and potentially modify messages or convoys of messages as they are received
and sent to/from the Messagebox
The most important words in the preceding statement are “Examine” and “modify.” As
stated previously, messages in BizTalk are immutable once they have entered the Messagebox.
The only proper way to affect a message is to change it in a pipeline either on the send side or
on the receive side. Before starting a new custom pipeline, it is important to understand that
a pipeline by itself does nothing. The real work is accomplished by the pipeline components
that are attached to the pipeline. If you are building custom pipelines, 99% of your time will be
spent coding the custom pipeline components, not building the actual pipeline.
These are some typical out-of-the-box uses of pipelines:
• Breakingupinbound documents into separate individual documents
• Verifyingorsigningdocumentswithdigitalsignatures
• Processingencodeddocuments(MIME/SMIME)
• Processingflattext files into XML and back
• Augmentingmessage data with information from external systems such as a database
or mainframe
These scenarios cover about 75% of the typical uses of a BizTalk solution that would need
to use a custom pipeline. There are, however, a multitude of other uses that we will dive into
later. Pipelines are attached to either a send port or receive location and executed when a mes-
sage is either received into or sent from the Messagebox. When the port processes a message,
the pipeline processing is activated when the message is passed to the start of the pipeline.
CHAPTER 4 PIPELINING AND COMPONENTS
116
Note If you were to monitor the typical newsgroups relating to BizTalk, you would find that a common
question often asked is, “How do I call a pipeline from an orchestration or from custom code?” Although it is
possible to do this, try to understand why you need to do this before tackling it. Pipelines are meant to oper-
ate on a message or a convoy of messages. A convoy is simply a set of related messages that are logically
grouped together by a promoted property called the
InterchangeID. In BizTalk 2004, calling a pipeline from
an orchestration was not a supported scenario. In BizTalk 2006 this has been greatly improved. Unless the
application needs to aggregate messages inside the orchestration, there is no real reason to do this. Mes-
sage aggregation is the primary reason why this feature was improved in BizTalk 2006.
ManynewBizTalkarchitectsoftenask,“Sowhynotjustadd all my custom code into the
pipeline?” The answer is threefold:
• Pipeline processing does not occur on multiple machines: Pipelines execute on only
one machine at a time since a message instance is picked up by a host instance. For
example, a receive pipeline will execute on the BizTalk machine where the message
is processed, not where it is received; if the pipeline has a significant amount of CPU
work to accomplish, then this machine can become unavailable while the pipeline is
executed. Note that multiple instances of a pipeline can execute simultaneously, but
each instance executes on one machine. Likewise, if the messages being processed
are quite large, the memory usage on that machine will grow accordingly if the entire
document needs to be loaded into memory. This is the primary reason why most
BizTalkarchitectureshavereceive/sendprocessingserversisolatedfromorchestration
processing servers.
• Pipelines are stateful: A pipeline executes in a stateful manner in that once the pipe-
line starts executing, it doesn’t stop until it is completed. Also, if that pipeline instance
terminates, that message becomes suspended and does not automatically restart on
another machine. This is unlike an orchestration, which can dehydrate itself when
the instance is waiting for a task to complete or rehydrate itself on another host if the
original machine becomes available. If there are a significant number of messages to
process, this can affect how the application will perform.
• Pipeline exception handling is problematic: If a pipeline throws an unhandled excep-
tion,thepipelineprocessingstops.Thisisamajorissueforsynchronousapplications.
Synchronousapplicationsthatrequireanotificationbesenttothecallingapplication
will have issues if the pipeline throws exceptions and no response message is sent
back. This issue will be explored later. It is possible in BizTalk 2009 to subscribe to the
message failure, but this does not help in a synchronous application scenario.

Get Pro BizTalk 2009 now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.