"Noisy" tpsDOT

[robs]

I'm not entirely sure the TPS_PERSIST is needed either. Where I think it may be helpful is in seeing a trend in noisier sensors. I don't think my car is particularly noisy, but I see fluctuations around 15 in my TPSadc values. To cope with this I need my TPS_EPS set to at least 10 (1%). If you only compare consecutive samples of the sensor, and throw away movements of less than 1%, you will only be able to detect movements faster than 100%/s which seems fairly fast -- I'd expect there are people who would like to detect some movements slower than this. My first thought was to sample at wider intervals (the first patch), but the two parameter solution is better. People with a good clean TPS can specify a denominator (i.e. TPS_PERSIST) of 0.01 seconds and get quicker recognition of slower trends. Larger denominators mean slower trends will take longer to spot (but at least they can be spotted). Fast trends will be detected equally quickly by either approach.

One complication of my algorithm is that the TPS_PERSIST parameter also determines how long tpsDOT lives after throttle movement has stopped, which can obviously change the behaviour of enrichment.

I took the car for a tuning run last night and Murphy's law kicked in. AFR was reading wrong, sitting just about 12. It ran well until it warmed up and EGO adjustment started. At first I thought it was my change and that the EGO sensor was never being read. So I took a really close walk through the code to make sure it was OK. Running it with the Stim sees it updating fine. So I guess it's my LC-1 playing up.

Have fun,

Rob.

[robs]

I haven't had another chance for a tuning session, but have had a good logging drive. Here are a few notes.

In light of what you said, Ken, I reduced the PERSIST value. I agree that the old value was a bit silly, able to measure very slow throttle movements to no good purpose.

In light of something I said, I decided to run my TPS lag value at 100% as this moving anchor approach makes the lag averaging redundant. This deals with the noise a different way, and it will be more responsive without the lag value.

I rebuilt with a TPS_EPS of 29, and TPS_PERSIST of 480.

How did I decide on these values? The algorithm will catch fast throttle movements fine, more or less regardless of the values you choose. What needs to be nailed down is the slow movements. Given that I use mapDOT for moderate accel enrichment, with tpsDOT aimed at faster bursts, I figured that 50%/s was about as slow as I needed to worry about. So the slope needs to be 50%/1s. Given that the code is called 100 times per second, this is 0.5%/.01s. Now, because I have quite a bit of noise in my tpsADC logfile readings, I can't reliably detect movements as small as 0.5%. Something like 2.5% is needed. A bit more for luck, and we get 2.9%/.06s (.06/.000128 ~ 480). Ok, it's not science, but it makes some sense doesn't it?

This highlights the advantage of having a denominator parameter. It allows you to tune out the noise, at the small cost of taking a bit longer to tell the speed of slower throttle movements, since it has to find them in amongst the noise. But it can find them. Following Matt Yates's idea on my car would miss opening rates slower than 300%/s, six times faster than my value of TPS_PERSIST allows.

On the noise, I see that the tpsADC values typically jitter by something near 20 for closed throttle, but only around 5 at full throttle. Can anyone explain this? Is it that the electrical noise has less influence at higher voltages, or is it that the rarely visited full throttle areas have less mechanical wear?

Anyhow, I had thought that these changes would result in a smoother TPS graph. Wrong. With the short TPS_PERSIST, it settles after .04 seconds which, with no lag based smoothing, means it actually jitters around more than ever. But the tpsDOT is very clean. The only bumps in that graph are real acceleration events. For example:

tpsDOT response

tpsvtpsdot.jpg (15.01 KiB) Viewed 999 times

I'm pretty happy with the result and am interested to hear how it goes for anyone else who tries it.

Oh, and my WB problem was down to my not having hit Calibrate AFR. Sheesh.

Have fun,

Rob.

[Black99rt]

Good work!

I'm not setup to compile code, but I can't wait to try this when/if it gets encorporated into a beta/release.

[ewflyer]

I'm not setup to compile code, but I can't wait to try this when/if it gets encorporated into a beta/release.

Exactly! I too am the owner of a noisy TPS signal. I'm eagerly looking forward to seeing the "robs patch" become part of the code.

Thanks for addressing this problem.

[robs]

I spent some time retuning accel with this change a few weeks ago. I still aim for MAP-based enrichment for moderate acceleration, with TPS focussed on quick burst stuff. The results have been interesting. Here are a few points:

• running no lag (i.e. 100% lag factor) on TPS
• reduced MAP lag factor to 40% to smooth it out
• TPS wanted more enrichment than it used to for faster rates
• Needed a bit more enrichment than I used to have for slower MAP rates
• TPS graph with no lag is an ugly mess, but tpsDOT is clean

Car's performing well, not weak on the quick blip anymore, but most impressive is the improvement to fuel economy. I'm just about up to my second fill since the change and it looks like I'm using around 15% less fuel -- and this has been just about all city driving (not to mention accel testing). Not sure why the improvement should be so large. I think it may be due to the change to MAP lag, the smoother readings making the car behave a bit more consistently. It used to have occasional unpredictable lean moments during gentle acceleration -- my reaction being to give a bit more throttle. It is now easier to drive smoothly. Still hard to see how that accounts for so much fuel -- not that I'm complaining.

Think I might try a similar mod for mapDOT and see how that goes. Have to think about it though, because if the smoothed values have improved the car's behaviour, the scratchy un-lagged values won't do it much good. Perhaps use the raw values just for mapDOT calculation, but keep the lagged ones for ordinary fuelling calcs. Thoughts anyone?

Have fun,

Rob.

[muythaibxr]

I've done the minimum change code for mapDOT and tpsDOT in ms3, y8s reports that they work pretty well, and he's been able to dramatically increase his lag factors and get better response. I'm not convinced the extra time factors are needed for that reason. I'll probably give these changes a shot in my own car sometime in the near future.

Ken

[Black99rt]

Awesome, definitely still looking forward to this.

I have a MEAN mapDOT signal from 2-3500rpm under moderate to heavy loads. I have to deal with false triggers to get reasanable tip-in performance. They only result in 101% enrichment though.

[robs]

I've done the minimum change code for mapDOT and tpsDOT in ms3, y8s reports that they work pretty well, and he's been able to dramatically increase his lag factors and get better response. I'm not convinced the extra time factors are needed for that reason. I'll probably give these changes a shot in my own car sometime in the near future.

Ken

I agree, it's not needed. But I'm suggesting something a little more radical than simply adding a time value along with the threshold parameter you and y8s have been experimenting with. If you add this time value you can eliminate the TPS lag value completely. The benefit being that you can spot a TPS change rate sooner than you could before. The drawbacks I can think of seem less significant. What drawbacks do you see?

Have fun,

Rob.

[JasonC SBB]

Been a while since I posted anything code/design related, the last being the epic on "noisy" RPM values. Like in that thread, I am running the risk of being told that I'm looking at this stuff too closely and, once again, I'm going to be saying that it's not me but the MS code that's looking too closely.

I'm sure I'm not the only one here to look at the TP values at idle, fluctuating around between 0 and 1. Like this:

TP.jpg

and compare it with the corresponding tpsDOT values which seem very volatile for such small movements. Like this:

The best filtering for a given signal depends on the characteristics of the signal and the noise.

One type of filtering that works very well for the type of noise in the first post is a moving hysteretic window.
The window has a certain "width" and it gets "pushed" up or down by the data. To push up, it has to exceed the upper part of the window. To push down, it has to be lower than the lower part of the window. The output is the location of the window. This type of filtering works very well for coolant and air temps too.

For a window width of 2 it will go like this. Input data is a slow rise with noise on it

in out
10 10
11 11
10 11
12 12
10 12
9 11
12 12
13 13
11 13
14 14
15 15
13 15

and so on

[24c]

...One type of filtering that works very well for the type of noise in the first post is a moving hysteretic window...

Interesting. I remember reading they use a moving window approach on scanning microscopes to reduce image distortion.

[gslender]

Hey Rob,

Are you still just running with the patch you posted a while back or has your code evolved since then?

G

[JasonC SBB]

The moving hysteretic window is probably good for all inputs - RPM (idle), MAP, etc.
Such a filter would help prevent say, the CLT signal from causing the fan to dither on and off.

It is especially helpful for quantization noise in A/D converters - i.e. last bit changing back and forth as in the first post.
This would require a window that has a width of 1.

Aside from this window, an analog lopass filter is very helpful.

The intake manifold has pulsey pressure and it is also good to reduce the pulsation that the MAP sensor sees.
I have a small restrictor feeding the hose that goes to my MAP sensor. In addition, I have a 2-pole Bessel analog filter after it. Bessel filters' claim to fame is lack of ringing in their response to a step input. My Bessel filter also has a feature (a diode) to speed up its response to a big fast rise or fall in MAP.

I tested the rise time when jabbing the throttle. Filters always introduce some signal lag, and lopass filters slow down rise times. I made sure the lag wasn't enough to affect engine operation.

This link shows the results:
http://www.miataturbo.net/showthread.php?t=48665

[robs]

Firstly:

Are you still just running with the patch you posted a while back or has your code evolved since then?

Yep -- though I don't think the patch is using my current values:

Code: Select all

#define TPS_EPS         29              /* 2.9% TPS variation */
#define TPS_PERSIST     480             /* 0.06 second memory */


(in ms2_extra_main.c)

These values suit the noisiness of my TPS and are meant to be tuned. In any "official" release, they would be described somewhere in the INI file and settable in Tuner Studio.

One type of filtering that works very well for the type of noise in the first post is a moving hysteretic window.

This is interesting, but incomplete, I think. The "window" must decay away after some time. As you described it, the window would end up 0:100. I think you'd need to track the window over some number of samples. The approach Ken and I have been looking at also has a window, based on a centre and radius, once the TPS value moves beyond that radius it sets a new centre. This is less flexible than the filter you describe, because it won't adapt to different noise levels, but looking at my graphs, the noise levels seem pretty consistent, so the fixed radius is probably good enough. Where Ken and I are disagreeing (I believe) is whether there is any real benefit in having this window persist beyond the next sample.

Filtering is a tricky business. For things like MAP and RPM, where the most recent sample is used directly to decide how much fuel to inject, advance to apply, etc., having a stable input is important in order to get a stable tune. But TPS is different. For the most part its value is ignored. What is important is its rate of change. When you see that the throttle is opening rapidly, you know that in a moment the MAP is going to increase and, for once, you can be ahead of the game and get extra fuel in in time for the extra air that's coming. To actually be ahead of the game though, you need to spot this trend quickly. These windowed algorithms are well suited to this: the moment you get a sample outside the window you can get a pretty good estimate of how quickly things are changing.

The current MS2/Extra code misses out on two counts:

1. It doesn't use a windowed filter, instead averaging the most recent two samples by the lag value.
2. It computes the rate of change after this lag filter has been applied.

Ken is on top of item 1, and I think we can expect to see tpsDOT, mapDOT and maybe rpmDOT with some sort of "jitter radius" parameter in due course.

Ken may also be (planning to) sort out item 2 and compute the DOT values before the lag filter has been applied. Mathematically, this would be good. Computing DOT values after lag averaging blends the last two samples and means the rate of change calculation will be "blurred". But computing it before averaging is also tricky. Some people may have quiet TPS values, but RPM and MAP are pretty much intrinsically noisy. The width of "window" you have to specify for these values would be so wide that you will not be able to spot some significant trends unless you look over several samples -- which is why I have been pushing my PERSIST notion.

To sum up my thoughts on getting rid of spurious DOT spikes:

1. Add RADIUS and PERSIST values for each DOT variable defining the window width and duration
2. Compute the DOT value on the raw sample, without any lag smoothing
3. Continue to compute lag values for RPM, MAP and TPS (in the latter case, just to keep the graph pretty)

This would allow running quite low lag values and getting smooth adjustments of metering for steady running, while spotting accel events quicker than before. Are there any gotchas here?

Have fun,

Rob.

[JasonC SBB]

One type of filtering that works very well for the type of noise in the first post is a moving hysteretic window.

This is interesting, but incomplete, I think. The "window" must decay away after some time. As you described it, the window would end up 0:100.

You may have misunderstood. The width of the window stays constant...

[JasonC SBB]

I will take back some of what I wrote.

The hysteresis window filtering method is most appropriate for the output of the feedback loop, for filtering the signal going into an actuator, such as the idle control motor/solenoid. The hysteresis window filter output will reduce the motion of an actuator. This is good for cases where moving an actuator around a lot will cause rapid wear.

The best filter for a noisy input such as the TPS will be something like a "Bessel" lopass filter. Bessel filters have no overshoot to step inputs. The filter obviously should be slow enough to get rid of the noise but not so slow that the relevant information in the signal is dulled (such as for triggering throttle enrichment in a timely manner). If the noise is so bad that this is not possible, then something else needs to be done (such as using shielded wire, proper grounding, etc).

The way a Bessel filter can be implemented in code is by doing a "convolution" with the appropriate Bessel filter's impulse response.
A simple running average will be inferior to a Bessel filter.

The AEM EMS has a user-settable filter settingfor the TPS input in the throttle enrichment setup screen, FWIW.

I'll repeat, I think a hysteresis window on the RPM signal (for the purpose of idle control), is a bad idea. GregG explained why.

[muythaibxr]

Been a while since I posted anything code/design related, the last being the epic on "noisy" RPM values. Like in that thread, I am running the risk of being told that I'm looking at this stuff too closely and, once again, I'm going to be saying that it's not me but the MS code that's looking too closely.

I'm sure I'm not the only one here to look at the TP values at idle, fluctuating around between 0 and 1. Like this:

TP.jpg

and compare it with the corresponding tpsDOT values which seem very volatile for such small movements. Like this:

The best filtering for a given signal depends on the characteristics of the signal and the noise.

One type of filtering that works very well for the type of noise in the first post is a moving hysteretic window.
The window has a certain "width" and it gets "pushed" up or down by the data. To push up, it has to exceed the upper part of the window. To push down, it has to be lower than the lower part of the window. The output is the location of the window. This type of filtering works very well for coolant and air temps too.

For a window width of 2 it will go like this. Input data is a slow rise with noise on it

in out
10 10
11 11
10 11
12 12
10 12
9 11
12 12
13 13
11 13
14 14
15 15
13 15

and so on

We do something similar to this in ms3 to get a more useful rpmdot output.

Ken

[JasonC SBB]

Here is a sample convolution.
A lopass filter is like a running average filter but with special weighting.
For example, you might have a running average using just the past 3 values. Your weighting is 1 1 1 - or when scaled, 0.3 0.3 0.3

A lopass filter using just the past 3 values would have some special weighting of the past 3 values, with the most recent value having the most weighting.
One such weighting might be 1 0.5 0.25

So at any given time the output filter will be the sum of
1 * most recent
0.5 * last most recent
0.25 * the one before that

Then the sum is divided by 1.75 (the sum of the weighting)
Compared to a simple running average, this filter will have less "smear" of the signal, and the present value will have the most influence on the output, and the previous 2 samples have diminishing influence.

Cheers.

[jsmcortina]

The existing lag filters in the code are a low-pass filter implementation.

James

[JasonC SBB]

Is it a running average?

[robs]

You may have misunderstood. The width of the window stays constant...

Yes, you're right, I misunderstood. I still don't quite see how it works though. I tried Googling "hysteretic window filter" but that didn't come up with anything terribly promising. If you added two more columns to your example -- i.e. tabulate: in, out, wmin, wmax -- I might get a better grasp of it.

On your most recent question, the existing lag filters are a weighted average of the last two samples and, if I haven't misunderstood again, your convolution is a weighted average of the last three. Both seem reasonable enough for smoothing direct readings.

Neither is good though, when you're trying to derive the slope from the result. For this, you need to know:

1. when the throttle started to move, and
2. how far it has moved since then.

Blending samples muddies the waters on both, particularly #1, which is why I think the way to go is to derive the xxxDOT values from window-filtered raw values and use weighted averages for the corresponding cooked xxx values.

Have fun,

Rob.