Hybrid Reference Implementation Using BizTalk Server, Windows Azure, Service Bus & SQL Azure

Integrating an on-premises process with processes running in Windows Azure opens up a wide range of opportunities that enable customers to extend their on-premises solutions into the Cloud environment.

Based off real-world customer projects led by the Windows Azure Customer Advisory Team (CAT), this reference implementation comprises of a production quality, fully documented hybrid solution that demonstrates how customers can extend their existing on-premises BizTalk Server infrastructure into the Cloud. The solution is centered on the common requirements for processing large volumes of transactions originated from the on-premises system and off-loaded into the Windows Azure to take advantage of elasticity and on-demand compute power of the Cloud platform. The reference implementation addresses the above requirements and provides an end-to-end technical solution architected and built for scale-out.

The main technologies and capabilities covered by the reference implementation include: Windows Azure platform services (compute, storage), Windows Azure Service Bus, SQL Azure and BizTalk Server 2010.

The reference implementation is founded on reusable building blocks and durable patterns widely recognized as “best practices” in Windows Azure solution development.

This project is implemented as a hybrid solution in which BizTalk Server represents a fundamental dependency. Logically, the BizTalk Server customers can install and use the reference implementation with a minimum of modifications. However, the solutions also carry many of the reusable patterns and building blocks which the developer audience could explore in isolation from the larger end-to-end reference implementation.

The accompanying source code is available for download from the MSDN Code Gallery. Please note that the source code is governed by the Microsoft Public License (Ms-PL) as explained in the corresponding legal notices.

Solution Components

BizTalk Server

The BizTalk Server 2010 is a core element in the hybrid solution architecture. The transactions that need to be processed on the Cloud are originated from a BizTalk application running on the premises. The BizTalk application is also used for hosting the service endpoints which make mission-critical capabilities such as data transformation, BAM and BRE available to the cloud-based applications.

Business Activity Monitoring (BAM)

The BAM is used to store activities that are generated and tracked by the BizTalk Sever application and cloud-based services. All activities are collected into an on-premises BAM database and visible through the BAM portal.

Business Rules Engine (BRE)

The BRE is used by both the on-premises BizTalk application and cloud-based services to power up the decision making and define the operational aspects of the data processing on the Cloud. In addition, the BRE is utilized for authorizing and managing complex application configuration used by the hybrid solution. Lastly, the BRE policies provide the extensibility mechanism upon which custom activities can occur at runtime depending on type or content of messages flowing from and to the BizTalk application.

XML Transformation

The BizTalk Mapper tool is used to author complex XML transformation rules and package them into maps. The BizTalk maps are then executed on the Cloud to transform XML documents between canonical and custom schemas.

Windows Azure Service Bus

The Service Bus provides the endpoint in the Cloud through which all traffic between the on-premises and cloud-based services (both directions) are securely relayed. The Service Bus is also leveraged to provide the inter-role communication between worker roles running on the Windows Azure platform.

Windows Azure Worker Role

The worker role provides the processing of messages, pulling each message from the queue, and shredding the collection of records into individual records that are stored in the SQL Azure database.

Windows Azure Storage

The storage queues are used to hold references to messages that are in line to be processed. Because the size of each item in the queue is limited to 8K, the actual messages are stored temporarily in a blob storage.

SQL Azure

The SQL Azure database provides durable persistence of the business transactions on the Cloud. The large transaction sets are stored in a SQL Azure database from which they are shredded (debatched) into smaller XML documents and are dispatched for processing.

Hybrid Solution Architecture

The following diagram depicts the architecture of the hybrid reference implementation:

Best Practices Applied

The following best practices were applied in the technical design and implementation of the hybrid solution:

• Reliable WCF communication using WCF clients & WCF hosts capable of recovering from transient communication errors.
• Windows Azure storage abstraction which provides a generics-based enhancement for Windows Azure Queue Storage API capable of storing very large messages (to overcome the queues’ 8KB limit).
• Transient Fault Handling Framework which delivers reliable SQL Azure data access layer capable of detecting and recovering from transient conditions such as SQL Azure throttling.
• Centralized configuration store capable of describing very rich configurations and storing them securely on-premises.
• Inter-role communication using event relay messaging provided by Windows Azure Service Bus.
• Hybrid instrumentation framework for high-performance real-time logging/tracing of cloud-based and on-premises processes into an ETW event collector.
• WCF custom endpoint behavior for securing and simplifying the integration of BizTalk Server and Windows Azure Service Bus.
• Windows Azure queue listener capable of auto-scaling up & down and reducing overall storage transaction costs.

For more information please refer to the resources linked to each of the individual best practices above.

Bookmark It

Hide Sites

(7 votes, average: 5.00 out of 5)