Data Corruption - Part 1 (Signal Filtering)

Data Corruption in Control Systems

In the rush to implement advanced analytics, machine learning, artificial intelligence, and other smart systems, organizations often overlook a fundamental truth: these systems are only as good as the data feeding them and the field outputs they are controlling. Even the most sophisticated control, safety, and supervisory systems will be severely limited (and/or will reach incorrect conclusions) if they are provided bad data or if the field devices aren't behaving as assumed. 

The reality is that very few plants are even properly implementing data for use by standard PLC/DCS control systems - let alone ready to do amazing things with even higher levels of intelligence.

This series explores the most common—yet frequently overlooked—problems affecting industrial control and safety system performance and associated data & analytics. 

In this series, we will cover:

Part 1: Signal Filtering - Eliminating noise without sacrificing responsiveness   

Part 2: Digital Resolution & Degradation - Preventing data quality loss as data traverses different systems 

Part 3: Input Accuracy & Precision - Ensuring your measurements reflect reality

Part 4: Output Device Behaviors & Impacts Accounting for the behaviors and characteristics of real-world output devices such as positioners, actuators, control elements / valves, etc.., and the impact these devices can have on processes.  

One of the most common problems in the realm of 'bad data' is improper signal filtering. Until we remove noise from the signals, it's tough to ensure end to end accuracy, and the other common mistakes and problems - so we will start with Signal Filtering for Part 1. 

Part 1: Signal Filtering

The Hidden Cost of Noisy Signals

Consider a 120 inch tall tank with a 4-20mA level transmitter whose current level is at 60 inches. Due to turbulence from incoming flow and resultant wave action in the tank, the surface liquid level at the sensor fluctuates between 57 to 63 inches, even though the actual level centerline is at 60 inches. This is a classic example of 'process' noise.

If this +/- 3 inch fluctuation is not properly filtered out, the following problems arise (among others):

• Aliasing Errors - Due to the variation in signal input, the analog input (AI) card snapshot of the data may fall anywhere in the +/- 3 inch area of the noise. See graphic below: 

• • Analog input cards (AI cards) usually have moderately slow sample rates and network update timing as compared to typical noise fluctuations.  

• • Unless the AI card can sample much faster than the frequency of the noise, the samples end up being just random snapshots of the signal at various error points and the AI card is unable to properly filter out the noise.

• • Aliasing errors are difficult to overcome unless the fluctuations are slow and/or the sampling and updating rate is fast. 

• • To prevent this aliasing error issue, you must remove the noise before the 'sampling' point - in other words, remove the noise at the source. 

• • Note - It is incorrect to think that the data can be effectively filtered past the first sample point (AI card) - because the aliasing errors are embedded at that point. More on this later in the article...

• Problems for PID loops & Controls - Any incoming signal noise on the PV is likely to be passed through PID loops and control functions, which can result in at least one of the following problems:  

• • Excessive wear on output elements due to constantly trying to correct for 'noise' signals that it assumes are actual PV's. 

• • Or sadly, the controllers are sometimes detuned as a misguided attempt to 'slow down' the outputs, greatly degrading the performance of the system. Solve the root cause of the problem (incoming noise) - don't just cover it up or treat the symptoms. 

• • Note - Noise on a PID loop which includes Derivative action becomes a nightmare (impossible really) because the controller sees the noise and interprets it as PV changes, or as Error (SP-PV) changes, and it tries to adjust for it, resulting in erratic controls.

• Hides other errors - If noise is transmitted from the source, any additional noise picked up enroute (such as electromagnetic induction of noise onto the signal transmission line) will be hidden inside the source noise. It becomes very difficult to identify or remove these additional noise components further down in the data flow path. 

So now, we agree that we need to filter signals properly to remove the noise. How do we do it?

Filtering Options (From Best to Worst): 

When it comes to mitigating signal noise, three common approaches are used. We will start with the 'best practice' option and move to the 'least effective' option: 

Option A: Filter at the Source (Transmitter) - Best Practice

By filtering directly at the transmitter (or source of the noise), you can eliminate nearly all of the process noise before it is initially sampled at the AI card. This maximizes the signal to noise ratio (SNR) and provides for maximum data quality and integrity. The biggest advantage of this approach is that it helps minimize Aliasing Errors. This option typically results in:

• • High quality data
  • Prevention of cascading or compounding errors
  • Optimized network bandwidth and processing resources 

By filtering noise at the source, the AI card sees a smooth stable signal that is truly representative of the process variable (PV).

Many PLC and DCS AI cards are setup to only report new data if the value has changed. A noisy signal will appear to be constantly changing and will dramatically increase AI card updates and network traffic. Along with reducing network traffic, this can reduce the data storage and memory levels on systems setup to report only on change of data values (most modern systems)

BUT - some engineers and techs prefer NOT to perform filtering at the field device, here's why: 

If handling signal filtering at the field transmitters, it can be harder to track and control any filtering (damping value) adjustments! And frankly, many techs do not adequately understand damping settings well enough to be making those adjustments. So engineers sometimes opt to handle it all in the controller or AI cards. 

A better solution is to simply train personnel to understand the concepts involved and establish policies and systems for how damping is managed. 

There is validity in wanting to control these field adjustments - but it comes at a big cost in terms of data quality and controllability of the plant. 

If one does choose to filter somewhere other than the actual source (field transmitter, etc.), filtering at the AI card is the next best option.

Option B: Filter at the Analog Input Card - A Compromise, But at a Cost!

Most modern control systems have functionality built into the AI cards to handle signal processing, including signal filtering and even scaling. Older and simpler systems that lacked processing power in the IO cards didn't have this capability and like many 'new improvements in the I&C field' new functions are often overlooked or neglected (or simply unrealized). 

When filtering at the AI card:

• • You'll still get some aliasing errors if your sampling rate is too slow as compared to the predominant noise frequency.  

• • • In the example of the tank level with wave noise, it is possible that the noise is slow enough that AI card filtering could actually work to some degree - But on something like a DP flow signal (with higher frequency noise), aliasing errors become very likely if filtered at the AI card.  

• • Example: Assume a typical AI card has a sample & update rate of 100msec (10 hertz) - For AI card filtering to work, the noise frequency would have to be slower than 5 hertz to avoid gross aliasing errors, and ideally slower than 1hertz or even 0.1 hertz to maintain good data quality. Read up on Nyquist Criterion and Aliasing Errors in Signal Processing for more info. 

Option C: 'Smoothing' within Controller Logic - An Undesirable Practice 

When filtering (or "smoothing") the data in the control logic, you get all the problems of filtering in the AI card, and more. 

By the time a signal reaches your controller logic the aliasing error has entrained and hidden other errors along the way and has likely compounded the errors due to various digitization issues along the way (to be discussed in part 2 of this article). In short, there are lots of errors embedded in the data - and you are unlikely to be able to truly isolate or remove them. 

This option would only work in situations where drastically slowed data, or as I call it, 'smoothed' data will suffice. I specifically use the word smoothed instead of filtering because this is not real filtering due to all the other effects compound the errors and because it is not actually filtering any errors out - it is simply slowing the data down so much that it appears 'smooth'. 

Any analysis (human or otherwise) MUST recognize that this data has entrained errors, and is likely way slower than real world process changes.

Attempting to 'smooth the data' in the control logic would also require the following: 

• • Consistent (precision clocked) timing for input data updates. 
  • The filtering logic must be performed in a timed task or routine.
  • The logic execution must be synchronized with the incoming data rate.
  • Any 'downstream' use of the 'smoothed' data (such as in PID / control blocks) must be coordinated and synchronized with the timing of the smoothed data. 

This approach results in much slower responding data than the other methods, due to the 'smoothing' approach. This is can be a big problem. For example: 

A savvy engineer (or a brilliant artificial intelligence system) is analyzing a system looking for patterns of how the tank level control interacts with the 3 main feeding flows and the outflow demand.

If the data has simply been 'smoothed' with all the aliasing and digitization errors still embedded in the data, the analysis could 'imagine' correlations between the tank and other systems because of 'ghost patterns' and/or erroneous data, and reach incorrect conclusions. 

This is the classic GIGO (garbage in = garbage out) setup. 

Filtering Approaches - In summary:

It is best to filter the signal at the source, or as close to it as possible. There is a reason that maximizing SNR by filtering noise at the source is considered best practice in electronics, electrical systems, and all other fields. But it is often overlooked in the I&C field. 

Just as 'smoothing' data in the control logic causes problems, so will attempting to smooth signals further downstream in the data flow path such as servers, data collectors, historians, and various intelligent processing systems. If the signal noise was not properly filtered prior to digitization, processing, and communications stages, it only gets more convoluted and difficult to utilize at each stage. More on this in the next part of this article. 

So if you are convinced you need to properly filter the incoming signals - let's discuss the concepts needed to understand how to accomplish proper filtering and explain the basics of the filtering procedure. 

Understanding Damping and Time Constants

To properly filter incoming signals, you need to understand two key concepts: The first is what is signal (or transmitter) damping to understand the general shape and reaction of damping; and the second is the closely associated topic of time constants, and how they work to visualize how the time constant settings will impact the signal. 

What is Transmitter Damping?

Transmitter damping mimics a first-order system. The value changes at a slower rate as the difference between the initial point and final point equalizes. 

Imagine placing a cold temperature probe into hot water:

• At first, the temperature reading climbs rapidly
• The rate of change gradually decreases
• Eventually, the reading stabilizes at the actual water temperature

Another example might be charging or discharging a capacitor, or pressurizing a small tank from 0 to 100 psi through a small needle valve, or bleeding it down to atmosphere. You can change the speed by the size of the opening (or the resistance inline for a capacitor), but the shape of the response will follow the curve above...

This characteristic of a quicker change during a large mismatch and a slower change when there is minimal mismatch makes for excellent filtering - and is exactly how most filters are implemented in the I&C realm. To fully understand damping, you need to fully grasp time constants. 

Time Constants: The Key to Proper Damping

A time constant represents how quickly your system responds. Specifically, it's the time needed for a system to complete approximately 2/3 (actually 63.2%) of a change.

After one time constant, your system has moved about 2/3 of the way to its final value. After two time constants, it's moved 2/3 of the remaining distance (reaching about 86% total). This progression continues until the system effectively reaches its final state, and the same trend can apply moving downwards. 

How Filtering is Accomplished

"Mow the Grass, Not the Hill" 

Finding the right damping value is about balance and usually requires an empirical (trial and error) approach. I use this analogy: noise on a signal is like grass on a hill with the hill being the actual process variable.

Your goal is to mow the grass (eliminate noise) without excavating the hill - so that you minimize the noise - but don't slow or impact the response of the actual process variable.

See below for a comparison of an example of the tradeoffs between too little damping and not enough damping:

In the example, bump tests are performed by taking the pressure controls of a tank to manual control and adjusting output from say 50 to 55% output and the PV is recorded:

The top trend shows a typical noisy signal.

The second trend shows that signal filtered enough to remove most of the noise, but not enough to impact the overall response rate. 

The third trend shows a scenario where the filtering is excessive and has begun to 'slow' the response of the signal to the real PV vs just filtering out noise. This would be a bad situation...

The simplified process to establish filtering is pretty straight forward. 

1. Connect a Scope-meter or other way of recording a real-time trend of the process reaction. 
   1. In some cases, you may have to increase sampling / update rates to clearly see the noise. 
2. Perform bump tests (up and down, across range of normal operation). 
3. Record and analyze responses
4. Add small amounts of damping incrementally until you have removed all of the noise and/or are approaching the point where the process response is slowed. 

Too little damping leaves noise that causes the problems previously mentioned. 

Too much damping may prevent your system from responding to real changes in a timely manner and/or misleading (slow) data which can cause control problems and/or safety issues. 

*Notes

1) There are some generalities assumed in this example and it is greatly reduced for brevity - but the idea is that you filter as much of the noise out as you can without altering the overall response of the system.

2) The closer the noise frequency is to the process reaction frequency, the more difficult it is to find a balance - and sometimes you can realistically only remove 'most' (or some) of the noise. 

3) Tip - If there are less than about 10-20 noise cycles (peaks) during the bump test slope you are probably going to have to compromise significantly on how much noise you can filter out, because the frequencies are close enough together that by filtering all of the noise you'd also be filtering (slowing down) the response of the actual value.

A Cautionary Tale: When Damping Becomes Dangerous

Be super cautious about adjusting damping (wherever you adjust it) and ensure that personnel who might make such changes are fully trained and competent to do so. 

A single untrained technician may adjust the damping on many devices to produce a 'smoother' output (it happens all the time!)... In some cases, that tech may work on the process control system, and the process safety system for the same protective function, meaning they have single-handedly compromised each major layer of protection for a hazard situation... 

I have observed numerous scenarios where technicians unwittingly inserted excessive amounts of damping in order to 'smooth out' a signal for various (misguided but well intended) reasons. 

In some of these scenarios, the techs actually disabled protective functions - including one that was on an important SIF. 

The damping in that case was slow enough that it would not have provided the necessary protection when needed...

Luckily it was discovered during the next test cycle by a savvy technician who noticed and questioned a 30-second damping value in the transmitter and took appropriate actions to resolve and correct the issue.

This illustrates a critical point: damping settings must account for worst-case scenarios, not just normal operation.

Moving Forward

Signal filtering is just the first step in ensuring data quality. In our next article, we'll explore some of the digital data degradation, and quantization errors that develop as signals pass through your systems, and how this subtle issue can significantly impact your operations.

See Next: Part 2 - Digital Degradation