"Noisy" tpsDOT

This is a forum for discussing the development and testing of alpha MS2/Extra code. Documentation
(Runs on MS2 and Microsquirt)

Moderators: jsmcortina, muythaibxr

Black99rt
Experienced MS/Extra'er
Posts: 364
Joined: Tue Mar 14, 2006 10:38 am

Re: "Noisy" tpsDOT

Post by Black99rt »

muythaibxr wrote:Matt Yates (y8s) had an idea that I'm trying on ms3 that could easily be backported to ms2:

If the TPS value doesn't change more than <user settable> %, set TPSdot to 0. I *think* this will allow things to remain just as responsive while not false triggering. Essentially I've implemented half your changes.
This will help for detecting changes off 0% TPS well. It won't help with false triggers at cruise.

While we are giving standard AE love- Can we re-visit decel fuel enleanment? I believe it is only MAPdot based, regardless of what blend you run for AE. Also, it is active even at WOT. Any or all of the following things would make it more useful

TPS based option
A maximum TPS for it to be active
A maximum MAP (or load)
Less erratic TPSdot / MAPdot values everywhere (an modification that only works at 0% tps will not help)
Seperate threshold

I can't use this feature with how noisy my map line is. I run small false triggers on map AE out of nessecity, but they only result in say 101% enrichment. This doesn't make much difference, but pulling 10% on a negative false trigger sure does! My truck bucks when I let off the throttle. I know I should use EAE ultimately, but I want to science out each section before I decide on what to ultimately use long term.
1999 Dakota 5.9L R/T-
MSIIe 3.1.2 Batch Fuel/MSD single coil dizzy/Stepper idle
LC-1
JasonC SBB
Helpful MS/Extra'er
Posts: 120
Joined: Wed Oct 26, 2011 7:36 pm

Re: "Noisy" tpsDOT

Post by JasonC SBB »

The moving window hysteresis filter is good for filtering an output signal that goes to an actuator, such as the idle solenoid.
It's not good for filtering an input signal such as RPM.

See below example.

The red trace is the "jittery" output signal. The blue traces are the top and bottom of the window. The signal can push the "top" of the window up, or push the "bottom" of the window down. The filtered output is the center of the window. (X's)

Notice that the red signal jitters up or down with every sample. Imagine this is your idle solenoid. It's constantly moving.
In contrast, the window has periods where it is flat - it doesn't change for a couple of cycles. So the idle solenoid doesn't jitter as much. It will move in fits and starts, but it stays put a good percentage of the time. In this example, >50% of the time.
JasonC SBB
Helpful MS/Extra'er
Posts: 120
Joined: Wed Oct 26, 2011 7:36 pm

Re: "Noisy" tpsDOT

Post by JasonC SBB »

robs wrote:On your most recent question, the existing lag filters are a weighted average of the last two samples
Are the weights 1 and 1? (This would make it a simple 2-sample moving average) Or more like 1 and 0.5?

The former would be inferior to my suggested weighted average of 3 samples with weights = 1, 0.5, 0.25.
The latter is a ... Bessel filter of one flavor.
and, if I haven't misunderstood again, your convolution is a weighted average of the last three.
BTW weighted averages *are* a convolution operation. A simple running average is convolution, but with even weighting. In signals theory, what you are doing is "convolving" your signal with the *impulse response* of your filter. So a weighted average say of 3 samples, (meaning weights are 1 1 1), is a convolution with specific type of *comb filter*, whose impulse response is 1 1 1.

A Bessel filter's impulse response, resembles my suggested weighting of 1, 0.5, 0.25.

Comb filters are inferior to Bessel filters for the purpose of TPSdot.
Let's look at the response of a Bessel to a step input like this
0 1 1 1

The step response of a Bessel filter goes like this.
0 0.7 0.9 1

Note that the instant you step on the throttle, its output instantly hits 0.7. TPSdot will detect it.

If it's not a real step, but a spike of noise like this,
0 .2 0 0

A Bessel's response will be more like
0 .12 .06 .03 0

A 3-sample running average filter (comb) response to the same impulse (spike) will be
0 .06 .06 .06 0

The Bessel will initially react more strongly to the noise spike, but it doesn't hang around for as long.

The other problem with a comb filter is that it has peaks in its response to multiple frequencies way out in its passband.
What this means is that if the noise has a certain periodicity to it, it gets through the filter, even if the frequency is significantly higher than where you expect the cutoff frequency to be. Bessels are superior this way.
JasonC SBB
Helpful MS/Extra'er
Posts: 120
Joined: Wed Oct 26, 2011 7:36 pm

Re: "Noisy" tpsDOT

Post by JasonC SBB »

Note:

When an input signal has a ramp to it, e.g.

0 1 2 3 4 5

The Bessel filter output will have the same slope to it. So the rate of change information gets through.
What happens is that there will be a lag before its output begins ramping:

0 .3 .8 1 2 3 4
JasonC SBB
Helpful MS/Extra'er
Posts: 120
Joined: Wed Oct 26, 2011 7:36 pm

Re: "Noisy" tpsDOT

Post by JasonC SBB »

There are many many types of filters, e.g. Chebyshev, Equiripple, Elliptic, Butterworth, etc.
What is "best" depends on the signal and noise characteristics, and how the information is processed from the signal.
For most ECU signals, I think the Bessel is pretty good.
robs
Master MS/Extra'er
Posts: 564
Joined: Sun Jan 17, 2010 4:26 pm
Location: Sydney, Australia

Re: "Noisy" tpsDOT

Post by robs »

Black99rt wrote:
muythaibxr wrote:Matt Yates (y8s) had an idea that I'm trying on ms3 that could easily be backported to ms2:

If the TPS value doesn't change more than <user settable> %, set TPSdot to 0. I *think* this will allow things to remain just as responsive while not false triggering. Essentially I've implemented half your changes.
This will help for detecting changes off 0% TPS well. It won't help with false triggers at cruise.
I think you may have misread Ken's posting. It's set tpsDOT to 0, not TPS. It should work at cruise. I'll leave your other points for the experts.

And Jason, I think we are at cross-purposes. My basic point is that the DOT values, the slopes, are best derived from unsmoothed data. So the sort of pseudocode I recommend would be:

Code: Select all

rawMAP = read_atd(MAP_UNIT)
MAP = smooth(rawMAP, ...)
mapDOT = calc_slope(rawMAP, ...)
as opposed to using the cooked values

Code: Select all

rawMAP = read_atd(MAP_UNIT)
MAP = smooth(rawMAP, ...)
mapDOT = calc_slope(MAP, ...)
Which is more the way the code currently is.

But, a confession. I was wrong again: I lied when I said that the lag value was a weighted average at all. It's actually an infinite convolution of sorts. The tuner gets to specify a lag percentage. The output of the lag calculation is the value that percentage of the way between the last output value and the most recent raw value. So a lag value of 0 locks the input forever where it started. A lag of 100 simply gives the most recent sample. A lag value of (say) .75 gives you T(n) = T(n-1) + .75[T(n)-T(n-1)]. Since T(n-1) depends similarly on T(n-2), there is actually a little feedback from everything that has ever happened, except that truncation of fractions soon wipes that out.

Have fun,

Rob.
Black99rt
Experienced MS/Extra'er
Posts: 364
Joined: Tue Mar 14, 2006 10:38 am

Re: "Noisy" tpsDOT

Post by Black99rt »

robs wrote: I think you may have misread Ken's posting. It's set tpsDOT to 0, not TPS. It should work at cruise. I'll leave your other points for the experts.
You are right, my mistake. I am still interested in what the decs have to say about the rest of my suggestions.

I'd also like to add that the notion of w/e MAPdot calculation scheme we used, if the lag was independant is desirable. This means that the lag factor can be reserved for smoothing MAP proper, for lookup and also MAP multiply. This will reduce pulse width jitter for sections of the operating map that have intrinsically noisy MAP signal.

I can be against the converter for seconds at a time at essentially one load point, and the pulse wdith jitter from map multiply follows in the AFR.
1999 Dakota 5.9L R/T-
MSIIe 3.1.2 Batch Fuel/MSD single coil dizzy/Stepper idle
LC-1
JasonC SBB
Helpful MS/Extra'er
Posts: 120
Joined: Wed Oct 26, 2011 7:36 pm

Re: "Noisy" tpsDOT

Post by JasonC SBB »

robs wrote:
Black99rt wrote:
muythaibxr wrote:Matt Yates (y8s) had an idea that I'm trying on ms3 that could easily be backported to ms2:

If the TPS value doesn't change more than <user settable> %, set TPSdot to 0. I *think* this will allow things to remain just as responsive while not false triggering. Essentially I've implemented half your changes.
This will help for detecting changes off 0% TPS well. It won't help with false triggers at cruise.
I think you may have misread Ken's posting. It's set tpsDOT to 0, not TPS. It should work at cruise. I'll leave your other points for the experts.

And Jason, I think we are at cross-purposes. My basic point is that the DOT values, the slopes, are best derived from unsmoothed data. So the sort of pseudocode I recommend would be:

Code: Select all

rawMAP = read_atd(MAP_UNIT)
MAP = smooth(rawMAP, ...)
mapDOT = calc_slope(rawMAP, ...)
as opposed to using the cooked values

Code: Select all

rawMAP = read_atd(MAP_UNIT)
MAP = smooth(rawMAP, ...)
mapDOT = calc_slope(MAP, ...)
Which is more the way the code currently is.

But, a confession. I was wrong again: I lied when I said that the lag value was a weighted average at all.... A lag value of (say) .75 gives you T(n) = T(n-1) + .75[T(n)-T(n-1)]. Since T(n-1) depends similarly on T(n-2), there is actually a little feedback from everything that has ever happened, except that truncation of fractions soon wipes that out.
Ah OK, this is an implementation called an IIR (infinite impulse reponse) filter. This particular on is a 1st order lopass filter with a variable time constant (lag). If you do it twice on the data (with the same lag value), what you are implementing is actually a 2nd order Bessel filter. These work very well.
And Jason, I think we are at cross-purposes. My basic point is that the DOT values, the slopes, are best derived from unsmoothed data.
Ahh. Given that, there are 2 approaches.

1) 2 separate filtered MAP values. 1st one for the lookup tables (I presume that's what the filtered MAP value in your pseudocode is for) and a 2nd for the MAPdot.

2) The 2nd approach is to filter the MAP "properly" so that it is useful for both the MAPdot and the other usage.

The problem with using unfiltered MAP values as you know is that noise will cause false triggering. If you say, "well, we can put a threshold on the MAPdot before triggering accel enrichment, and put a duration threshold before you trigger accel enrichment", well, that *is* a form of filtering. What I'm saying is, the proper overall filtering and triggering strategy *should* include *some* IIR or Bessel filtering.

As I alluded to in an earlier post, the proper filtering depends on the signal and noise characteristics, and the characteristics of the information contained in the signal. And, one of the rules of system design is: always band-limit your signal to that which is needed. Any unnecessary bandwidth will just allow noise to enter. Just as an extreme example. Let's pretend that the sampling rate of MAP is 1 MHz. Obviously a 500 kHz signal riding on it is noise and should be ignored.

With this in mind, here are a few questions to help us design the filtering scheme:

1) What is the sampling rate of TPS and MAP?
2) Can we assume that the largest acceptable fueling error is 10%?
2a) What is the shortest amount of time (fastest) that MAP can actually change by 10% due to a throttle input, or due to turbo spoolup?
3) What is the fastest that TPS can change?


Also, let's take a step back. Remember that the whole purpose of acceleration enrichment is this:

-- if MAP is rising rapidly the MAP may be higher than what the last injection point was calculated for. Adding fuel as a function of MAPdot is an attempt to correct for this. If deltaT is the time between the time injW was last calculated and the time time the intake valve closes, then the shortfall in fuel is proportional to MAPdot*deltaT.

-- the filtering on the MAP signal which *needs to be slower* than the TPS signal (due to intake manifold pressure pulses, see my thread)
http://www.miataturbo.net/showthread.php?t=48665
will cause an additional deltaT delay. (This goes back to what I said about signal and noise characteristics, determining the filtering you need) This is not a problem for when the MAP has been rising for some time already. The problem this creates is that when MAP actually *begins* to rise, the MAPdot will NOT show up until after some other delay.

-- the onset delay in MAPdot can be addressed by TPSdot.

-- what I'm saying is that you need a *combo* of MAPdot and TPSdot. MAPdot is used for most of the accel anrichment, while TPSdot is used to fill in just the initial part of tip-in where MAPdot is delayed. Importantly, MAPdot addresses the lag in the MAP signal as it rises as a turbo spools up *after* the throttle is already closed.
JasonC SBB
Helpful MS/Extra'er
Posts: 120
Joined: Wed Oct 26, 2011 7:36 pm

Re: "Noisy" tpsDOT

Post by JasonC SBB »

When designing a TPSdot strategy, there needs to be a accel fuel multiplier vs throttle position and also vs RPM.

The reason is that the TPS to MAP response varies with both TPS and RPM.
Going from 0-10% TPS has a much larger effect on MAP than going from 90-100% TPS.
This effect is more pronounced at 1000 RPM than at 6000 RPM.

This is true for n/a, turbo, and s/c engines. BTW DBW throttle systems attempt to flatten this curve to make parking lot driving less jerky.
robs
Master MS/Extra'er
Posts: 564
Joined: Sun Jan 17, 2010 4:26 pm
Location: Sydney, Australia

Re: "Noisy" tpsDOT

Post by robs »

Jason,

I haven't made my terms clear, I suppose, and I guess we may have somewhat different backgrounds. When I have referred to "smoothing", I have been meaning any kind of blending of values. This includes MS2/Extra's lag filter and the Bessel filters you have been referring to. I prefer to use the term "filter" to refer so something that discards values by some criterion -- strains out the junk, like a fuel filter.

By nature, the DOT values are looking for when some value sticks its head above (or below) the general run of values. Obviously, smoothing the values will make these sudden changes less sudden, reducing sensitivity. But we do want the non-DOT values to be less jittery, so your favoured approach #2 would require a filter that has the effect of getting rid of jitter. Funnily enough, that was the very effect of running a high value for PERSIST. If no sample broke through the threshold, all samples were recorded as the same and my TPS values were smooth as could be (not that it matters much for TPS). Perhaps I'll revisit this when I experiment with MAP.

It is enjoyable to theorise about how an ECU can best run the show, but given that the problem is trying to extrapolate the future from the past (i.e. TPS, MAP, RPM, etc. are all from a moment ago, and you want to decide when to start/stop injection and spark in a moment's time), you can only get so far. Not saying that MS2/Extra can't do better, but I think at some point the experiment has to move from brain to engine. Empirically, for me, the change described earlier in this article has given tangible benefits. I'm now making the corresponding changes to MAP/mapDOT, and will report back here when I've given it a couple of weeks' try out.

Have fun,

Rob.
y8s
Master MS/Extra'er
Posts: 478
Joined: Wed Jan 07, 2009 3:21 pm

Re: "Noisy" tpsDOT

Post by y8s »

I actually just posted a new thread in the ms3 support area about the noise--I hadn't seen your thread here. But this is exactly the problem I'm running into and why I asked Ken about the minimum change per time settings.

Unfortunately, the noise is great enough that for TPS I have to run 1.5% per 10 milliseconds for the setting and it effectively eliminates TPS enrichment even when I want it. Do the math and TPSdot has to be greater than 150 to enrich at all.

So when I set my tps based enrichment lower limit to 50%/s for slow takeoffs, I get nothing.

So while it DOES help for eliminating transients at idle and cruise, I can't dial in my tps based enrichment where I want.

thread:
http://www.msextra.com/forums/viewtopic ... 31&t=42830

I'm going to reread this (not my) thread again and think about other strategies that might improve the situation.
JasonC SBB
Helpful MS/Extra'er
Posts: 120
Joined: Wed Oct 26, 2011 7:36 pm

Re: "Noisy" tpsDOT

Post by JasonC SBB »

robs wrote:Time to cut to the chase. The interval that the 3.1.1 code uses is 10ms -- 100 times per second. I have put in a small change in ms2_extra_main.c to only sample every 40ms -- 25 times per second. The car still seems to run fine (on a few revs in neutral test) and the noise in tpsDOT has fallen quite a bit, with typical jitter of around 20 being reduced to around 4, but the peak values still looking much the same as before.

I don't think the chief benefit of this is in reducing noise on fixed throttle, but in slightly more accurately estimating the real rate of change of throttle position thus improving throttle based enrichment.
I strongly feel that you are better off keeping your sampling at 10 ms *but* with the proper filtering done, as opposed to reducing sampling rate to 40 ms.
Reducing sampling rate to 40 ms can be a problem because the TPS signal rise time can be on the order of 80 ms or faster. You can miss the first half of a rapid tip-in, with a resulting hesitation, especially off-idle at low RPM.

Question:
Does TPSdot simply look at Delta_TPS (unscaled for sampling rate) or does it do Delta / sampling_rate?
JasonC SBB
Helpful MS/Extra'er
Posts: 120
Joined: Wed Oct 26, 2011 7:36 pm

Re: "Noisy" tpsDOT

Post by JasonC SBB »

robs wrote: The red curve samples the sine curve, truncated to integer, every unit. The blue curve samples the integer truncated sine curve only every fourth unit. tpsDOT is the slope of the curve -- which slope better estimates the slope of the real sine curve? IOW, sampling less frequently is a win. Oversampling leads to successive over and under estimates of the actual slope. This will be worst at areas of slow change.
Here is a counterpoint. The red trace is the fast sampling TPSdot. The blue is slow sampling. Notice how the blue trace has a lag.
The sampling rate can be so slow that it misses a big part of the initial rise.
Last edited by JasonC SBB on Mon Nov 07, 2011 9:04 am, edited 1 time in total.
JasonC SBB
Helpful MS/Extra'er
Posts: 120
Joined: Wed Oct 26, 2011 7:36 pm

Re: "Noisy" tpsDOT

Post by JasonC SBB »

Here is a sample of a noisy TPS and TPSdot, and a 2-pole Bessel filtered noisy TPS and its "dot":

Notice how the jaggies in the TPSdot are greatly attenuated.
robs
Master MS/Extra'er
Posts: 564
Joined: Sun Jan 17, 2010 4:26 pm
Location: Sydney, Australia

Re: "Noisy" tpsDOT

Post by robs »

JasonC SBB wrote: Here is a counterpoint. The red trace is the fast sampling TPSdot. The blue is slow sampling. Notice how the blue trace has a lag.
The sampling rate can be so slow that it misses a big part of the initial rise.
Indeed. But a faster sampling rate isn't always better and what you want is the right sampling rate. I was slightly worried by a comment in the code saying to sample more frequently because MS2 was now sampling slower than MS1. I would prefer to see a physical argument for a sampling rate. Fastest throttle movement might be from closed to wide open in 0.1s. Least significant throttle movement is 2%. Sample at twice this rate -- every 0.01s. Something like that.

In any case, this is from my earliest thoughts in this thread. Please read the other postings and I hope you'll see that I have moved on -- though the car was actually better with my experiments with the slower sampled approach, it's much better with my later approach.

Have fun,

Rob.
robs
Master MS/Extra'er
Posts: 564
Joined: Sun Jan 17, 2010 4:26 pm
Location: Sydney, Australia

Re: "Noisy" tpsDOT

Post by robs »

y8s wrote:I actually just posted a new thread in the ms3 support area about the noise--I hadn't seen your thread here. But this is exactly the problem I'm running into and why I asked Ken about the minimum change per time settings.

Unfortunately, the noise is great enough that for TPS I have to run 1.5% per 10 milliseconds for the setting and it effectively eliminates TPS enrichment even when I want it. Do the math and TPSdot has to be greater than 150 to enrich at all.

So when I set my tps based enrichment lower limit to 50%/s for slow takeoffs, I get nothing.

So while it DOES help for eliminating transients at idle and cruise, I can't dial in my tps based enrichment where I want.

thread:
http://www.msextra.com/forums/viewtopic ... 31&t=42830

I'm going to reread this (not my) thread again and think about other strategies that might improve the situation.
Please do. I'll be interested to hear what you think. It was because I saw this as a problem I would face that I added the TPS_PERSIST argument to "remember" when the TPS value entered a given range so that slower rates could be determined when it broke out of that range. I'm running 2.9% TPS for my noise radius (yours might be worse too if you were running 100% lag value), and any movement greater than this within 0.06 seconds (six samples) will calculate a tpsDOT value. It's quite sensitive to slow movements, but has abolished the noise from the tpsDOT graph.

I am currently trying something similar with MAP. It's a bit more complicated for several reasons. Perhaps my biggest reservation with it is the low resolution. While the AD converter is in the [0:1023] range, that's to 250kPa. Slowpokes running NA never see much over 360 from the AD converter, so each unit from the AD converter is about .25kPa. I'm sure that's enough, but there's not a lot to spare for averaging. So I think my code will do a decent job with mapDOT, but it may not open the door to all that much smoother sampling for MAP.

Have fun,

Rob.
JasonC SBB
Helpful MS/Extra'er
Posts: 120
Joined: Wed Oct 26, 2011 7:36 pm

Re: "Noisy" tpsDOT

Post by JasonC SBB »

Here's an analogy for the sampling rate problem.

A friend embarked on a weight loss program. Knowing that day to day variations are water weight, he decided to record his weight weekly.
I pointed out that the data still had the same water weight problem.
I told him to weigh himself everyday, and then do a week-long running average.

Much better results.
JasonC SBB
Helpful MS/Extra'er
Posts: 120
Joined: Wed Oct 26, 2011 7:36 pm

Re: "Noisy" tpsDOT

Post by JasonC SBB »

swnp wrote:The solution to my particular problem was two-fold. First, I configured my tps signal line in twisted-pair. That helped plenty. But to completely remove the noise from the board, I took the ground that the IGBTs were using completely off the board. Then I got what you see above.
This is a really important point.

The sensor grounds should return to the ADC input ground. If you follow the path of the current in this ground return, at no point must there be a portion of the path that is shared with high currents. The correct term is "shared impedance path" aka "common impedance coupling". The common, lousy name is "ground loop".

It is a BIG MISTAKE to design a system with high current ground returns, and low-voltage, low-current returns, and then at some point, have them share a conductor.

The 2 ground systems (high current ground, and "analog ground") should only be connected at one point, possibly inside the uC if it has the ADC's built in and has an internal connection between the analog front end and the ADC output ground. If they are connected at 2 different points, now you the 2 currents will share a common impedance. The high current will induce a voltage drop in said common impedance, which will contaminate the low-voltage signal.

I do switching power supply design for a living, and I regularly lay out boards with IC's that have a power ground pin for switching 10's of amps in 10's of nanoseconds, and an analog ground pin that is sensitive to signals in the 10's of mV which can respond to signals in the 10's of nanoseconds. You must pay careful attention to PCB layout. I also do EMI troubleshooting. 80% of customer issues that I help with, are PCB layout related. Even in my field, the majority of engineers have a poor understanding of noise coupling phenomena.
jsmcortina
Site Admin
Posts: 39585
Joined: Mon May 03, 2004 1:34 am
Location: Birmingham, UK
Contact:

Re: "Noisy" tpsDOT

Post by jsmcortina »

JasonC SBB wrote:The sensor grounds should return to the ADC input ground.
It seems very difficult to make people follow the wiring diagrams in the manual that show the sensors grounded back AT the Megasquirt.

James
I can repair or upgrade Megasquirts in UK. http://www.jamesmurrayengineering.co.uk

My Success story: http://www.msextra.com/forums/viewtopic ... 04&t=34277
MSEXTRA documentation at: http://www.msextra.com/doc/index.html
New users, please read the "Forum Help Page".
JasonC SBB
Helpful MS/Extra'er
Posts: 120
Joined: Wed Oct 26, 2011 7:36 pm

Re: "Noisy" tpsDOT

Post by JasonC SBB »

robs wrote:There are two controlling parameters: TPS_EPS and TPS_PERSIST. TPS_EPS is a threshold value -- throttle movements are ignored until they have added up to at least this value. TPS_PERSIST is a time value -- if there has been no significant throttle movement for this length of time, the throttle is not moving.
The effect of TPS_PERSIST is the same is you use some kind of lopass filter with a "triggering thershold*, without the disadvantage of the slow sampling delay. This is because for the lopass output to rise up, the rise in the TPS signal needs to persist for some duration.

What I mean by "triggering threshold", is, the TPS_filtered_dot must exceed some value before it does anything.
Last edited by JasonC SBB on Mon Nov 07, 2011 6:19 pm, edited 2 times in total.
Post Reply