Low-Latency Networks for Control System Applications

In control system applications the efficient transfer of data from sensors to the processing system and from the processing system to the actuators and motors is often essential for optimal system functionality and performance. Low-latency networks enable the efficient transfer of data between system components, which makes them the backbones of many control systems.

At the most basic level, a control system consists of a Plant, i.e. the process or device we want to control, and a Controller, which provides inputs to the Plant that direct its behaviour. In most control systems, outputs from the Plant are fed back to the Controller, which continuously updates the inputs to the Plant in order to obtain the desired outputs/behaviour. This is called a closed-loop control system. For example, consider a stepper motor. A user specifies a motor position to the Controller, which drives the stepper motor inputs. An encoder on the stepper motor provides feedback to the Controller so it can confirm the motor position and adjust the manner in which it is driving the stepper motor accordingly.

When establishing the requirements for low-latency networks, a number of control theory parameters need to be determined. These parameters are highly dependent on the primary elements in the control system. In this article, we will discuss the following parameters:

• Frequency Response
• System Update Time
• Latency
• Bandwidth Requirements
• Network Types and Topology
• Error Handling

Frequency Response

The first parameter to determine in a control system is the frequency response of the Plant. How fast will the outputs of the Plant change in response to changes in the inputs? Frequency response is highly dependent on the properties of the Plant and on the underlying physical processes being controlled. For example, the temperature of an electronic device may change relatively slowly with changes in the device processing load and the applications being executed. This enables thermal management tasks such as temperature measurement, fan speed adjustment and other cooling methods to be effective even when initiated at a relatively low speed. A design engineer can therefore focus on minimizing component costs when designing the thermal management solution for this type of device. Alternately, a scientific measurement device may include a highly sensitive optical receiver that responds very quickly to changes in laser light input. In that scenario, the designer must employ high-speed circuits that will quickly detect, analyze, and respond to these rapidly changing inputs.

System Update Time

A control parameter closely related to Plant frequency response is the system update time. In most analog systems the Controller continuously adjusts the analog inputs to the Plant in response to the analog outputs. In most digital systems however, Plant outputs are first gathered and analyzed by the Controller, then the next set of Plant inputs are calculated and sent to the Plant. The duration between each iteration of this cyclical process is called the system update time, or system period. In order to effectively control the Plant outputs, the system period (or frequency) must be faster than the frequency response of the Plant. In the thermal management scenario it is acceptable for the system update time of the thermal management control system to be slow in response to the slowly changing device temperature. The system update time of the laser optics systems on the other hand needs to be significantly faster because of the fast response time of the optical sensor.

Nuvation Engineering designed a system controller for electrostatic braking devices on an industrial conveyance system that transports packages at an order fulfilment facility. The devices required a system update time greater than 50 Hz in order to enable the conveyance system to operate at a high speed without causing packages to fall off the line at high-risk points such as inclines and sharp turns. The alternative – which is the norm for these types of conveyance systems – would have been to operate the entire conveyance system at a slower speed to ensure safe package handling at all points on the line. Nuvation Engineering delivered an electrostatic braking controller that exceeded the required 50 Hz update time requirement, which enabled our client to implement their innovative package conveyance solution.

Latency

A third parameter to consider is latency. This parameter can be defined as the delay between when the Plant output is observed and when the Controller updates the Plant inputs. There can be several delays in a control system that contribute to overall system latency, including sensor delays, network transfer delays, and processing delays. Typically, control systems work to minimize latency. The answer to the question “how much latency is too much?” is highly dependent on the system context.

Bandwidth Requirements

Bandwidth requirements for networks can be calculated by identifying the number and type of actuators/motors and sensors in the system, data formats, and sampling/update rates. System-level control and health monitoring messaging also need to be included in the network bandwidth calculations. Both average and peak bandwidth requirements need to be considered to ensure the networks have sufficient bandwidth capacity and are not introducing additional latency into the control system.

Network Types and Topology

Networks can be broadly sorted into three main categories in terms of their latencies:

• Standard computing networks
• Industrial automation networks
• Custom networks

At the high end of the spectrum are the standard computing networks such as Ethernet and USB. These network types are supported by most microprocessors via mature and robust communication stack software. They are well suited for applications with long system periods and systems that are tolerant to variable latency.

Industrial automation network protocols include EtherCAT, Profibus, and CAN. The hardware and software elements of these networks take into account the need for bandwidth efficiency, lower latency, and communication robustness. These networks are often used in automation control applications.

For systems with very low latency requirements and fast sampling/update rates, a custom network may be required. These networks often use an existing physical layer and utilize customized protocols that are optimized for the types of data transferred over the network. These networks are often implemented in programmable logic (CPLDs or FPGAs) in order to minimize latency by off-loading network protocol handling from the CPU.

Nuvation Engineering implemented a low-latency EtherCAT network for an industrial control system that transported chip wafers along the conveyance system of a semiconductor fabrication facility. The network needed to support up to 10,000 nodes throughout the large facility. The system utilized real-time distributed and synchronous motor control, machine vision, and inventory tracking /control. The EtherCat networking protocol was selected because of EtherCat’s reliable performance in low-latency networking applications. To meet the high level of data processing overhead from the numerous nodes and diversity of control systems, Nuvation’s engineers implemented custom control algorithms in FPGA and DSP processors throughout the control system.

In addition to network types, network topology is also very important. Daisy-chain networks reduce network wiring requirements as well as the number of network interfaces at the Controller. However daisy-chain networks can also introduce latency and increase the bandwidth requirements of the links closest to the Controller. Alternatively, a star network topology minimizes latency and bandwidth on each link, but may require a network hub at the Controller and a significant number of network interfaces. Hybrid networks are a potential solution that combines these two topologies to meet both link bandwidth and latency requirements.

Nuvation Engineering designed a hybrid network topology and protocol for a manufacturing automation application. The system collected position sensor data for a controller that provided current settings to linear motors that moved pallets through the manufacturing process. A traditional daisy-chain network topology was unable to support the data bandwidth and fast system period requirements, so Nuvation designed a hybrid topology. This custom topology consisted of several parallel daisy-chain networks that utilized a Nuvation-designed FPGA-based custom network protocol to meet the bandwidth and system period requirements.

Error Handling

Other areas of consideration include network robustness, health monitoring and error handling. In the event of system warnings, errors, faults, or dropped network messages, methods to identify these events and respond to them must be defined. For example, if the Plant outputs change slowly, it may be sufficient to use previously received data and wait for the next system period to obtain new data. For fast changing Plant outputs, error and fault handling may need to be implemented in programmable logic devices or in hardware to respond quickly to ensure safety and protect the hardware from damage.

Summary

Low-latency networks are the backbone of control systems because they enable the efficient transfer of data between system elements. The product engineering design team must factor in a wide range of control theory considerations in order to ensure that all points in the design where latency issues can arise have been identified and solved for. These solutions tend to involve a combination of design choices, from hardware component selection to the choice of which networking protocol to implement. For complex high reliability systems with fast response time requirements, custom control algorithms and networking protocols may need to be developed to meet the unique properties of the target control application.

Nuvation Engineering has over 20 years of experience designing control systems for a wide variety of applications. Contact us for help with your control system project.