Windows Azure Service Bus & Windows Azure Connect: Compared & Contrasted

Late last year I published a blog that compared and contrasted cloud and hybrid cloud implementations and one of the big follow-up questions was this: can we have more elaboration on Windows Azure Connect and Windows Azure Service Bus (Relay Service)? This is also a common question in various MSDN Forums and Discussion Groups.

In this blog we take a scenario-driven approach to compare and—more importantly—provide guidance around when to use these two seemingly similar communication paradigms. Of course experts in the domain are aware that comparing these two technologies is akin to comparing apples and oranges. But the general perception is that these two technologies accomplish the same goal: supporting communication between cloud and on-premise resources.

Before we dive into the details, allow me to state that the Connect Services is a Layer 2 network peer integration technology while the Service Bus enables cross network, cross trust-domain endpoint federation for any combination of cloud and on-premise services. To summarize:

• Windows Azure Service Bus Relay Service is recommended wherever you want location transparency for message-based communication between application endpoints. You might use it, for example, to enable a WCF Service hosted in an Azure role to communicate with an on-premise WCF service. A typical scenario is elaborated in a recent publication, here.
  
• Windows Azure Connect is similar to an on-demand virtual private network (VPN) between on-premise computers and Azure Role Instances. You commonly use Connect in scenarios where you can have a common trust domain (as you might for a VPN), and need support for location transparency beyond message based communication. A typical example for Connect Services is accessing an on-premise file share from an Azure role instance.
  

An easy way to think of this is that the Service Bus works at the application level, providing connectivity to two web services applications – it is a pipe connecting two applications; in contrast, Azure Connect works at the machine level, providing full network connectivity between two machines.

Windows Azure Service Bus is an Internet-aware messaging service. It operates on the application layer and connects application endpoints and the brokered messaging artifacts used to mediate messaging flows between them. In order to provide location transparency for the clients and services that interact with the bus, Service Bus offers protocols and client-side binaries that support relayed communication.

Windows Azure Connect Service is a virtualized networking service for Windows Azure-based hybrid applications. Connect Service operates at the network layer and connects machines and VM nodes together into a single trust domain with shared access (as controlled by Active Directory) to all endpoints and services running on those machines/nodes.

Windows Azure Service Bus

A relay service enables an on-premises client to connect to another service via an intermediary (Figure 1). Both client and service connect to the relay service through an outbound port, and each creates a bidirectional socket for communication tied to a particular rendezvous address. The client can then communicate with the on-premises service by sending messages to the relay service targeting the rendezvous address. The relay service will then ‘relay’ messages to the on-premises service through the bidirectional socket already in place. The client does not need a direct connection to the on-premises service nor does it need to know where it resides. The on-premises service doesn’t need any inbound ports open on the firewall. This is how most instant messaging applications work today. Typical examples of such a solution (based off a relay service) are Instant Messaging, P2P Sharing, and On-line Games.

Figure 1: Typical Relay Service Pattern (Source – A Developer’s Guide to the Service Bus)

The Windows Azure Service Bus (“Service Bus” hereafter) provides a communications relay service in the cloud that operates at the application layer. It includes features for large-scale event distribution, naming and service endpoint publishing. It simplifies the task of connecting WCF, REST and other service endpoints. It overcomes the complexities introduced by firewalls, network address translation boundaries and dynamic IP-addresses. Depending on the endpoint types, communication can be secured using transport security, message security or both.

The Service Bus’s relay capability allows for global endpoint federation across network and trust boundaries. In the June 2011 CTP release (Windows Azure AppFabric SDK V2.0 CTP – June Update), a new set of cloud-based, message-oriented-middleware technologies were added to Service Bus that provide:

• Reliable message queuing.
  
• Durable publish/subscribe messaging.
  

Both occur over a simple and broadly interoperable REST-style HTTPS protocol. Other benefits include:

• Long-polling support.
  
• Throughput-optimized, connection-oriented, duplex TCP protocol.
  

Windows Azure Connect

Windows Azure Connect (“Azure Connect” hereafter) is part of the Windows Azure Virtual Network.

Azure Connect enables customers to easily build and deploy a new class of hybrid and distributed applications that span both cloud and on-premises environments. In particular, Azure Connect allows cloud applications running on Microsoft data centers to access existing resources located on a corporate network, such as:

• Databases.
  
• File systems.
  
• LOB applications.
  

In terms of functionality, Azure Connect provides a network-level bridge between roles and services running in the cloud and on-premises environments. Use Azure Connect to

• Facilitate the migration or integration of existing on-premises applications with the cloud.
  
• Help customers to leverage their existing IT investments and assets.
  

Azure Connect is less focused on inter-service communication than the Service Bus. It is akin to setting up a VPN without the typical hassles and complexities of configuring one. It operates at the network layer to provide IPv6 connectivity and DNS resolution for local computers and Azure role instances bi-directionally. Azure Connect transparently secures all connectivity, end-to-end with IPSec.

Figure 3: Example Configuration in Windows Azure Connect Scenario

Comparing the two technologies

In this section we will compare these two technologies via various pivots (e.g., communication scenarios, pricing models, etc.) and will conclude with observations to give you a better understanding of these two technologies.

Communication Pattern Scenarios

Let’s dive into the communication patterns that motivate the selection of these technologies. We begin by describing the high-level steps taken to enable communication for Service Bus and Windows Azure Connect.

The table below highlights the four communication patterns you will encounter with either the Service Bus or Azure Connect. For each, we describe the key implementation steps taken, in addition to the typical implementation workflow previously detailed.

Table 1: Comparing the Communication Patterns

Note1: Windows Firewall rule configuration may be required for accessing applications (e.g., PING, File Shares) that are typically blocked by Windows Firewall (default) rules.

Let’s walk thru some of these scenarios to highlight their application. These scenarios fall into the following three types: hybrid, pure cloud and pure on-premises.

On-Premises Scenario

Both the client and server are located on-premises (first row in the table).

It’s important to note that just because the servers are located on-premises does not mean that they are necessarily within the same premise. You may be communicating between local servers of two different organizations. For the Service Bus, the respective use scenario would be communication between services in disparate organizations, or between fixed or roaming on-premise clients and fixed or roaming on-premise servers.

While not a predominant scenario, Azure Connect provides connectivity between on-premises source and on-premises destination via a simplified VPN. Such a use not only benefits interactions between partner organizations, it’s also quite compelling for roaming users who need to access on-premises servers from their laptops. For example, a roaming user connects to a remote desktop or file share.

The Azure Management Portal (https://windows.azure.com/default.aspx) allows defining and configuring groups to connect existing Azure roles with a set of local computers where the Connect agent has been previously installed. In particular, the Azure Connect network enables connectivity between computers in the same group, so a consumer application running on a computer A can invoke a service hosted by a server B when both machines are configured in the same group, even if they physically live in different corporate networks.

Hybrid (cloud/on-premises) Scenario ?

For the Service Bus, an example scenario is this: a WCF service hosted is on-premise on Windows Server. The service provides access to data which is sensitive and therefore can only be stored on-premises, though some of it can be safely exposed to external clients. An Azure web role, for example, would call the service to get the data so that it can be consumed by users such as partner organizations.

It should be noted that with Azure Connect, the use scenarios are not limited to service communication. For example, it may be that your Azure role instances are domain-joined to an Active Directory server that runs on-premises, or that they use Windows integrated security to access an on-premises instance of SQL Server. Azure Connect can also facilitate simpler scenarios, such as allowing Azure role instances to access a network share volume (that only exists on-premises), or that ping on-premises resources periodically to ensure they are still reachable.?

To summarize:

1. The Service Bus can wrap and expose an existing corporate resource (e.g. a database, a LOB application, etc.) through a WCF (façade) service. The service can be accessed by cloud or on-premises client applications that are authenticated and authorized using the Access Control Service.
   
2. Azure Connect allows an existing web, worker or VM role to directly access an existing resource located on a corporate data center, but it is not allowed to publicly expose this resource to external partners.
   

Cloud Scenario

In this scenario, communication occurs between Azure hosted clients and Azure hosted servers (the last row in the table).

For the Service Bus, an example of this is a pub/sub solution whereby one role instance acts as a publisher of data and multicasts data using the NetEventRelayBinding. Registered subscribers run on separate role instances and possibly in different applications. This support for pub/sub is unique to the Service Bus, and would require additional effort across Windows Azure Connect.

Windows Azure Connect is primarily focused on the hybrid scenario, so it should be no be surprise to learn that inter-role instance communication is not a supported scenario. If you need to communicate between Azure role instances, you need to use an alternative method, such as communicating across input or internal endpoints. This is as explained here. ?

Scenario Notes

• Local endpoints may increase performance by making direct connections instead of first going through a cloud-hosted relay. Service Bus provides support for upgrading relay connections to direct connections dynamically. Under adverse conditions, direct connections can failover to a Relay mode. Today, Azure Connect cannot make direct connections; it only goes directly to the relay.
  
• In scenarios where you need to interrogate a service registry to list all running services, only the Service Bus provides out of the box support at the service level, and is accessible from an ATOM feed (Routers in the Service Bus). Windows Azure Connect only enables you to enumerate Azure roles and local endpoints, not any of the services within them, and this data is currently only available from the Windows Azure Portal.
  

Security

The Service Bus and Azure Connect also differ in how they authenticate and authorize servers and clients. The Service Bus authorizes services and clients using the Windows Azure Access Control Service (ACS). A service endpoint registered with the Service Bus must be authenticated by the ACS. Clients can optionally be required to authenticate via ACS as well. Azure Connect, on the other hand, controls access exclusively by configured Network Policy (e.g. Endpoint Groups) – the mechanism is the Connect Agent, on the client, authenticating itself on behalf of the machine where it’s running.

Pricing

For cloud resources, it is important to consider costs and your usage patterns.

Costs for the Service Bus are affected by the maximum number of daily concurrent connections, the number of ACS transactions made per month and data transfer costs. By purchasing “connection packs” you effectively pre-commit to using up to a certain number of connections monthly and are billed at a lower rate per-connection. At present, the Service Bus is priced to scale for a small (1-5) to medium (500) number of connections; if you need connections beyond that (say on an Internet scale with thousands of concurrent connections) it can be cost prohibitive.

Windows Azure Connect costs are currently (June 2011) TBD. While it is in CTP, it is free, but the actual pricing model will strongly affect how economical some of the aforementioned scenarios are. While the details are still being worked out, it is fair to guess that pricing for Windows Azure Connect will involve at the least data transfer costs. Depending on what these latter cost, it is likely that you would use Connect for a smaller number of connections (definitely not on an Internet scale). In this sense, Service Bus and Windows Azure Connect may well be comparable in terms of pricing in the small to medium scale.

Note: the preceding is per announced Windows Azure offers. For current and up-to-date information on pricing please check here.

Developer Requirements

A developer building applications that use the Service Bus will typically do so while developing WCF SOAP Web Services, WCF REST Web Services, or (non-WCF) REST-based services. These services will then be registered as endpoints on the Service Bus and available for use in both on-premises and cloud environments. Service Bus also enables developers to integrate with non-Microsoft client platforms through the use of message queues and REST-based APIs. Typically the developer will access the Windows Azure Services Portal using their Live ID with an active Windows Azure Platform subscription.

A developer building applications that use Windows Azure Connect may not need any specific expertise in building WCF or REST services, as not all use scenarios of Connect actually require building services. The developer will, need access to the Windows Azure Portal, which requires a Live ID and an active Windows Azure Platform subscription. While Azure Connect is in CTP, that subscription will need to be approved to be enabled for Connect.

System Requirements

Below are the system requirements for typical scenarios.

Service Bus

To use the Service Bus, services and clients can build on one of the following systems:

• Windows XP SP3+
  
• Windows Vista
  
• Server 2008/R2
  
• Windows 7
  

In addition, you must install

• .NET 3.5 SP or .NET 4.
  
• To build a WCF service or client, use the Windows Azure SDK,
  
• REST scenarios can be supported by any platform offering REST-enabled HTTP programming.
  

In terms of connectivity, clients and services require HTTP(S) or TCP connectivity outbound to the Internet via a range of ports.

Windows Azure Connect

Azure Connect requires that all participants run Windows (Windows Vista SP1, Windows Server 2008/R2 or Windows 7), as they must be capable of installing and running the Windows Azure Connect Agent.

In terms of connectivity, Azure Connect only requires that participants be able to make outgoing HTTP(S) connections on port 443.?

Implementation approach

Connecting Windows Azure Service Bus Services and Clients

The general process for implementing Service Bus clients and services is as follows:

1. Create a WCF service.
   
2. Choose a host.
   
   1. Self-host in a .NET application or Windows Service.
      
   2. Host a service in Windows Server, Windows Azure Web role, worker role or VM role.
      
3. Configure a WCF service to register with the Service Bus
   
   1. Choose a communication approach (see Table 2 below).
      
   2. Expose metadata as appropriate.
      
   3. For Azure hosted services, ensure the Service Bus assembly is packaged together with the cloud application, and that the configuration is applied programmatically; or use the RelayConfigurationInstaller and ensure that your role is configured to run with Full Trust. See here for details.
      
4. Start the host.
   
5. Create the client.
   
   1. Use the desired communication approach.
      
   2. For Azure hosted clients, ensure the Service Bus Assembly is packaged together with the cloud application and that configuration is applied programmatically; or use the RelayConfigurationInstaller and ensure that your role is configured to run with Full Trust. See here for details.
      

Table 2: Service Bus – Communication Patterns

Connecting Services and Clients using Windows Azure Connect

The general process for implementing connectivity between clients and the services using Windows Azure Connect is as follows:

Figure 4: Windows Azure Connect Interface

1. Configure Windows Azure Connect on all participants.
   
   1. Web & worker roles: Configure the service model to activate Windows Azure Connect prior to deployment. For more information, see How to Activate Windows Azure Roles for Windows Azure Connect.
      
   2. VM Role: Install the WA Connect Agent in the base image and configure the service model to activate Windows Azure Connect prior to deployment.
      
   3. Local computers: Install WA Connect Agent.
      
2. If using applications (such as Ping) that require application firewall configuration, configure firewall rules on all participants.
   
   1. Local servers.
      
   2. Azure Web, Worker and VM Roles: script a StartupTask that uses the netsh advfirewall firewall command to add or configure firewall rules when an instance first loads.
      
3. Configure Network Policy.
   
   1. Define Endpoint Groups which specify which roles can communicate with which local servers, and can be nested so that groups of computers can be related easily (e.g. Local servers, Azure roles and nested endpoint groups).
      

Observations

In conclusion, here are some useful observations that can help you in the selection process.

1. If Interoperability with non-Microsoft platforms is important to your scenario, you should use the Service Bus because its message queuing functionality enables connectivity from any REST-enabled HTTP clients.
   
2. Azure inter-role communication is not supported with Windows Azure Connect, so if this is a requirement you may want to use Service Bus as explained here. (For more information, see Overview of Enabling Communication for Role Instances).
   
3. For scenarios where your required protocols might change, and you don’t know what they are in advance, Windows Azure Connect is the best choice because enabling new protocols boils down to configuring application firewall rules to enable communication.
   
4. Roles are limited in the number of internal and input endpoints they can define, so if you are likely to exceed these limits (currently no more than five endpoints), you should consider Windows Azure Connect which will remove your need to define specific endpoints.
   
5. There is no need to define input or internal endpoints for Azure hosted services communicating with either the Service Bus or Windows Azure Connect, as both rely on making outgoing connections via their own transport channels and ports.
   
6. If you require a simplified approach to enabling access to network services like Active Directory supporting integrated security when communicating to SQL Server instances hosted on-premises, or accessing network file shares, you should use Windows Azure Connect. While it is possible to make the Service Bus support any custom protocol exchanging binary data, this is significantly more complex than simply enabling Windows Azure Connect.
   
7. Windows Azure Connect requires that on-premise machines install the WA Connect Agent, which means that that they must be running Windows. There is currently no support for looping in non-Windows based machines or network devices.
   

Additional Resources

1. Windows Azure
   
2. Windows Azure Platform MSDN Premium Pricing
   
3. Service Bus Port Settings
   
4. Windows Azure Connect Port Settings
   
5. How to configure a Windows Azure Hosted Service Bus Service or Client
   
6. Tutorial: Setting up Windows Azure Connect
   
7. Clemens Vasters Blog – Cloud Development and Alien Abductions
   

Acknowledgements

Significant contributions of Abhishek Lal, Clemens Vasters, Jason Chen, Keith Bauer, Paolo Salvatori, Sidney Higa, Todd Holmquist-Sutherland and Valery Mizonov are acknowledged.