Multicast on OCI - Connecting multiple regions in a Mesh

In our last multicast blog post Multicast on OCI - High Availability, wXcked Eye and multicast traffic testing we have introduced a very important feature, the HA feature. We analyzed the configuration part together with the multicast traffic testing path. Continuing our multicast journey, we will introduce another great feature we can use, the Mesh feature.

If we recall from Multicast on OCI - overview and configuration, Mesh is a feature that can be used to connect multiple cloudSwXtches in the same cloud region, different regions or between different cloud providers (as soon as the IP connectivity is established between the control plane IP addresses of cloudSwXtches). The final scope is to be able to send/receive multicast traffic to/from other cloudSwXtches in the same mesh.

On the other hand, the Mesh feature can be used to group two or more cloudSwXtches together for acting just as a single cloudSwXtch to gain network performance.

We will expand our networking topology previously used to include a new OCI region and to create a mesh of cloudSwXtches using the two OCI regions. The new networking topology is listed below:

The right side is the newly added region, connected with our first region using the Remote Peering Connection. In this way the will assure that the IP connectivity between the cloudSwXtches control plane IP addresses is achieved. Our new diagram includes three cloudSwXtches so our mesh will be formed by the three depicted cloudSwXtches.

Any member forming the mesh is called swxtch-node or for short node and as we discussed above, it can be part of a Mesh if it is IP reachable. We have a restriction, a node that is part of an HA path cannot be included in a mesh and vice-versa, the Mesh and HA are mutually exclusive.

A node can be part of a single mesh network.

In the past we discussed about the xNIC installation and how a VM with xNIC installed is associated with a particular cloudSwXtch. In our current architecture we have the following associations: 

xnic-vm1 and xnic-vm2 -> cloudSwXtch-1

xnic-vm3 and xnic-vm4 -> cloudSwXtch-2

xnic-vm5 and xnic-vm6 -> cloudSwXtch-3

Suppose that our multicast producer is xnic-vm1 and the consumer for the same group is xnic-vm6. In which way we can make the multicast traffic to make its way from cloudSwXtch-1 to cloudSwXtch-3 via the RPC and reach the consumer? Your guess is right, by configuring the Mesh feature.

Before going into configuration details and traffic testing, we should say that mesh membership doesn't mean all the multicast traffic will be sent to every other node in the mesh. Thus, the multicast traffic for a multicast group from a producer will be sent only and only to the nodes with active consumers that joined the multicast group.

Let's have an example, suppose that xnic-vm1 is a producer for multicast group of 239.1.1.1:3490 and xnic-vm6 is a consumer for 239.1.1.1:3490. Taking into account the phrase above, the multicast traffic path from xnic-vm1 to xnic-vm6 will be: xnic-vm1 -> cloudSwXtch-1 -> cloudSwXtch-3 -> xnic-vm6. As we can observe, cloudSwXtch-2 does not have any consumers joined for 239.1.1.1:3490 and will not receive any multicast traffic for this group.

Mesh configuration

For Mesh configuration we will use wXcked Eye, please refer to the previos blog post where you can find the details regarding wXcked Eye.

In order to complete the configuration you can connect to any of the cloudSwXtches using wXcked Eye and from there using the Configuration -> Settings -> Mesh -> Create we can have:

When you click Create, the node where you are connected will be automatically included in the mesh, so you only need to specify the remaining nodes. Once you have added all the nodes in the mesh, all the nodes should be marked in green and reachable, as below:

Every other node in the mesh will show the very same information if you connect with wXcked Eye. We can say now that our mesh network is ready and we can proceed with the next step, multicast traffic testing over our mesh network.

Multicast traffic testing over the Mesh Network

xnic-vm1 will be a producer for the multicast group at 239.1.1.1:3490 and xnic-vm6 will be the consumer for the same group:

Our multicast traffic is working as expected and xnic-vm6 is receiving the multicast traffic since it is a member of 239.1.1.1:3490.

One interesting point is the Mesh RX counter started to increase and this is telling us that cloudSwXtch-3 has started to receive the multicast traffic over our mesh network.

Let's verify is cloudSwXtch-2 receives any multicast traffic:

cloudSwXtch-2 does not receive any multicast at this point since there are no consumers that joined the 239.1.1.1:3490.

Let's make xnic-vm4 a consumer for 239.1.1.1:3490 and verify if cloudSwXtch-2 is starting to receive the multicast traffic for 239.1.1.1:3490:

\

What a surprise, xnic-vm4 has started to receive the multicast traffic for 239.1.1.1:3490 and cloudSwXtch-2 Mesh RX is confirming that this multicast traffic is received over our mesh network. Wonderful, isn't it?