Avoiding Multicast Hell

24.01.2014 Gilbert Liu
Paris-traffic-jam

Multicast messaging is a commonly overlooked cause for saturated control networks. To a control systems engineer, there exist a plethora of reasons that a program or field device may present with unexpected behaviour. How is it possible that multicast messaging would be the culprit of a seemingly unrelated problem?

What is multicast?

In a computer network environment, multicast messaging is an IP message delivery mechanism that allows sending a single message to a group of recipients. The one message that is delivered is copied and distributed to devices belonging to the group only at the network routing hardware level. This mechanism differs from other more commonly known message delivery mechanisms such as unicast and broadcast. Unicast allows a device to send a single message to another device, and broadcast allows a device to send a message to every device in the network.

Advantages of multicast

In a typical control network environment, multiple PLCs may require access to a single remote IO drop or master PLC. In this scenario, multicast improves the efficiency and performance of the network by decreasing the number of times the IO drop needs to resend the same message. If there are 10 PLCs that require the information, unicast means the source device must send the message to each PLC individually – meaning 10 identical packets. Multicast allows the message to only be sent once, while network routing hardware duplicates the message where needed - decreasing network congestion.

Why is multicast important to consider in a control network?

In high criticality networks used for control and functional safety, thousands of tags from a remote IO drop may need to be updated many times per second. This generates an enormous amount of traffic that must be managed carefully. If careful measures and design decisions aren’t considered, multicast can quickly turn into broadcast messages and subsequently saturate any hardware on the network that wasn’t designed for multicast.

How does multicast distribute messages?

Multicast first requires devices to join multicast groups. Messages are then sent to devices based on their group memberships. IGMP (Internet Group Management Protocol) is  a communication protocol used to establish group memberships and an integral part of multicast. Devices join multicast groups according to the following procedure:

  1. The IGMP querier generates a message to query all devices to determine if they wish to join a multicast group.
  2. If a device wishes to join, a unicast response is sent back from the device to the querier. This querier must be a router or a switch with IGMP querying support.

On a large network, the querier and the device may be separated by multiple switches. If these switches in between have IGMP snooping support, they will have the ability to store a map of which links need IP multicast streams.

Importance of choosing the right hardware

Set up properly, multicasting can be used to good effect in the control systems world. Correct hardware selection and network design will prevent the occurrence of a saturated network. Here are some important measures and considerations to ensure multicast is properly implemented.

Use managed switches with adequate IGMP support. Using an unmanaged switch that doesn’t recognise multicast messages will instead lead to broadcast like behaviour. Managed switches have both hardware and IP multicasting support, ensuring that multicast traffic isn’t inadvertently duplicated throughout the entire network. When choosing a managed switch, ensure that it has support for IGMP snooping. This will allow the switch to inspect the contents of the IGMP packets.

Choose switches with port mirroring. Port mirroring allows a port to be mirrored for monitoring purposes. This can be helpful in inspecting and diagnosing multicast packets and their behavior using Wireshark, discussed in previous blog posts.

Importance of good network design

Have IGMP queriers in strategic locations. An IGMP querier is responsible for initiating multicast group subscriptions. More importantly, queriers must be located at strategic locations to ensure that all multicast traffic paths are captured by the IGMP subscription mechanism, to prevent unwelcome broadcast traffic resulting.

Correct network architecture. Incorrect network topology is a common cause for multicast issues. In a typical control environment, the remote IO drops are the producers of multicast messages while PLCs are consumers of multicast messages. Correct topology ensures that consumers of multicast messages span off switches that are aware of both the consumer and producer. This normally requires these switches to be in the path from unicast IGMP response to the IGMP querier.

Verify the TTL (Time to live) of the multicast packets. Multicast packets from control systems protocols such as Ethernet/IP generally have a TTL of 1 hop. Since every router decrements the TTL value, the multicast Ethernet/IP packet will expire from one route. Correct understanding of the packet behavior is critical to good design.

Leveraging the power of multicast

Multicast can be leveraged in everyday control environments. When handled correctly, multicast serves as an efficient messaging mechanism. By understanding multicast behaviour, early design decisions can reduce the risk of a saturated network and the unpredictable behavior that could result.

____

More by Gilbert Liu

IP Addresses: Lost and Found (Part 1 of 2)

IP Addresses: Lost and Found (Part 2 of 2)