UDP Multicast
– Sam Mitchell, Senior Cloud Solutions Architect at CohesiveFT
While not on the top of the list for sexy new lingo, UDP multicast is a vital part of data center networking that is being left out of the cloud conversation. Multicast helps to create and connect High Performance Computing (HPC) grids, for example. Essentially, multicast is the bullhorn that lets you send information out to hundreds of thousands of receivers worldwide.
This technological slight is having a huge effect on how companies use (or choose not to use) public cloud.
What is multicast?
Multicast is a technique for one-to-many communication over the internet protocol (IP) infrastructure, or the vast network that makes the internet, our private networks, and the cloud all possible. Communicating via multicast does not require prior knowledge of who or how many receivers there are. If you’ve streamed video conferences, been re-routed in a call center, or taken place in distance learning, you probably used the most common transport layer, User Datagram Protocol (UDP) multicast.
UDP multicast is relatively simple, transaction-oriented, and works well in unidirectional communications for things like voice and video traffic. Multicast uses the network infrastructure efficiently, by only sending the packets of data once, even if there are a large number of receivers. In fact, UDP multicast is widely deployed in enterprises, commercial stock exchanges, and multimedia content delivery networks.
In networking, multicast delivers a packet or message to a group simultaneously in a single transmission from the source. The down side of sending a packet to every single host in the network makes it rather “chatty.” UDP multicast is more efficient and scales well for large networks.
So why is multicast disabled in public clouds?
Public cloud networks are usually on a shared, virtual local area network (LAN or vLAN). The shared network and shared infrastructure is called a multi-tenant environment, similar to a multi-tenant apartment building. Allowing a “chatty” protocol inside a cloud network could have a serious impact on the performance of the cloud as a whole.
For this reason, public cloud providers would rather disable UDP multicast without an option to re-enable. Public cloud customers also are not allowed to access the lower network layers, such as Layer 2. But what about all those uses for multicast applications and use cases in the cloud?
What can enterprises do? Take back control.
To re-allow multicast traffic, enterprises can build their own network on top of their data center, private cloud, hosted virtual data centers, and public cloud networks. It may sound challenging, but an overlay network simply creates a private, sealed network on top of an existing network. These ‘overlay networks’ give complete control of the network in any cloud, including the ability to use UDP multicast.
Think of it this way: you can rent a server from AWS by the hour. AWS assigns a network interface card (NIC), and an IP address. You can pick whether your IP address is assigned to you from your private vLAN address pool or from the great un-washed shared vLANs address pool. Your IP and NIC connect your server to the AWS cloud network. Inside the AWS network, you have to play by their rules. AWS firewalls block your UDP multicast traffic from broadcasting to all others hosts in the vLAN.
But, by using an overlay network on top of the AWS cloud network, you get to make your own network rules. You can configure a second vLAN (using tun0), connected to your other cloud deployments in the same network. In this respect, your overlay network manager becomes the overlay network’s cloud switch. Your overlay interface is free from any cloud provider’s firewall conditions, because it is inside your control and inside your sealed overlay network. UDP multicast is back in business.
Adding more servers to your overlay network means that your UDP multicast packets can now flow from your first server, through the manager, and then on any other servers inside your overlay network. The same logic applies if you want to use IPsec or BGP network protocols inside or connected into your overlay network.
Real customer examples.
We have gathered a wide and varied collection of customer use cases where the customer needed to use multicast applications in the cloud. For me, the most interesting use case is our customer’s HPC grid bursting. Here’s a quick preview:
Rather than spending time and capital on managing infrastructure, our customer, a large mutual fund based in the US, hoped to expand their high performance computing (HPC) grid into the public cloud. Their goal was to quickly expand the existing HPC workloads securely while reducing infrastructure costs, increasing flexibility, using existing resources, and complying with industry security standards.
Traditional HPC environments are expensive to own, manage and operate. Often organizations require extra compute resource for irregular one-off jobs, which contain sensitive data such as trading transactions and intellectual property. Public cloud infrastructure offers lower capital costs and on-demand flexibility, but can increase enterprise risk profiles and cause networking headaches.
In the public cloud, our customer created an overlay network with added end-to-end and data-in-motion encryption. The overlay network, and the re-allowed UDP multicast networking features, allowed the new HPC workloads to act like the existing HPC grid network. By controlling the cloud network and their data center network, the large mutual fund was able to pass internal and external security tests.
The mutual fund company can now securely burst into public cloud IaaS as if it is a natural extension to their HPC grid. Public cloud infrastructure saves expensive physical servers from sitting idle and lets them “flex up” processing power up as needed, and back down when no longer in demand.
