"Noisy" tpsDOT

[robs]

Been a while since I posted anything code/design related, the last being the epic on "noisy" RPM values. Like in that thread, I am running the risk of being told that I'm looking at this stuff too closely and, once again, I'm going to be saying that it's not me but the MS code that's looking too closely.

I'm sure I'm not the only one here to look at the TP values at idle, fluctuating around between 0 and 1. Like this:

TP noise

TP.jpg (8.7 KiB) Viewed 3081 times

and compare it with the corresponding tpsDOT values which seem very volatile for such small movements. Like this:

tpsDOT noise

tpsDOT.jpg (12.12 KiB) Viewed 3081 times

The name tpsDOT relates to calculus. The mathematical ideal of this is the *instantaneous* rate of change, approached by looking at average change over ever smaller time intervals.

But just because it's mathematically ideal doesn't mean that it's ideal for our purposes. For deciding how much extra fuel to squirt in (or whatever) the instantaneous change is irrelevant. What is really relevant is the average change over some meaningful interval.

I'm afraid I don't have the training to say what that meaningful interval would be. One cylinder stroke? The time it takes the gas, or a sound wave, to travel along the intake tract? I don't know, and this might make a talking point for those with some engineering training. What I feel reasonably confident in saying is that MS is looking at too short an interval.

Here's another graph, two different estimates of part of a sine curve:

Oversampling example

jaggies.jpg (12.2 KiB) Viewed 3081 times

The red curve samples the sine curve, truncated to integer, every unit. The blue curve samples the integer truncated sine curve only every fourth unit. tpsDOT is the slope of the curve -- which slope better estimates the slope of the real sine curve? IOW, sampling less frequently is a win. Oversampling leads to successive over and under estimates of the actual slope. This will be worst at areas of slow change.

Time to cut to the chase. The interval that the 3.1.1 code uses is 10ms -- 100 times per second. I have put in a small change in ms2_extra_main.c to only sample every 40ms -- 25 times per second. The car still seems to run fine (on a few revs in neutral test) and the noise in tpsDOT has fallen quite a bit, with typical jitter of around 20 being reduced to around 4, but the peak values still looking much the same as before.

I don't think the chief benefit of this is in reducing noise on fixed throttle, but in slightly more accurately estimating the real rate of change of throttle position thus improving throttle based enrichment.

Here's a patch file which applies my changes. Most of the changes are to indent, or adding comments, the meat of it is to wrap appropriate lines in "if (tpssample_time > 320) { }".

Grrr. Maximum 3 attachments. I'll post the patch in a followup.

Have fun,

Rob.

[robs]

FWIW, here's the patch file.

tpssamp.zip

Reduced TPS sampling patch

(1.14 KiB) Downloaded 97 times

Have fun,

Rob.

[tpsretard2]

would love to see what the developers say about this, i have a HELL of a time with this an my car.

[Black99rt]

The next issue I was going to tackle on my installation is why I can't get it to jitter less than ~60%/s. I even installed a brand new TPS and that only very slightly improved it. I was going to try preloading it some, in case it was a mechanical vibration. Very curious to see how this pans out.

[Reverant]

Very interesting, this is one of my pet peeves with the MS2.

Dimitris

[swnp]

I searched for the pictures that I had to prove this, but I had TERRIBLE amounts of noise in my TPS. My tpsDOT is completely flat now.

tpsDOT and mapDOT

graph.png (21.71 KiB) Viewed 3016 times

The solution to my particular problem was two-fold. First, I configured my tps signal line in twisted-pair. That helped plenty. But to completely remove the noise from the board, I took the ground that the IGBTs were using completely off the board. Then I got what you see above.

[roarin_mouse]

I changed the TPS input resistor from 1k to 10k. much better now!

[robs]

I searched for the pictures that I had to prove this, but I had TERRIBLE amounts of noise in my TPS. My tpsDOT is completely flat now
.
[snippped graph]

The solution to my particular problem was two-fold. First, I configured my tps signal line in twisted-pair. That helped plenty. But to completely remove the noise from the board, I took the ground that the IGBTs were using completely off the board. Then I got what you see above.

That's very interesting, and answers one thing that I've been puzzled about. I couldn't see why the TPS had to be noisy. The map sensor, fair enough, it's essentially a microphone in a very noisy auditorium. The TPS is just a simple potentiometer.

However, your success in eliminating TPS noise doesn't completely negate my reasoning above. I'll try to explain why.

First, just to go over the point of TPS vs MAP. Both are useful. The MAP tells you state of the manifold a moment ago, when the sample was taken. The TPS is ahead of the game and gives you a clue where you might expect the MAP to be in a moment's time; it doesn't have to wait for the air to actually flow. It fairly naturally falls out that the MAP is most useful for steady state, with TPS used to prepare for changes that are coming.

Being the steady state monitor, the MAP sensor wiggles can safely be averaged, in code, or using a vacuum tank of some sort. Or it's fine to sample at a point in the cycle so you're (with a bit of luck) comparing apples with apples.

With the TPS samples, on the other hand, the individual values are not what matters. It's the difference between values which tells the ECU how quickly the throttle is being opened or closed, allowing it to adjust pulses for what's about to happen.

As a sweeping statement, all we want to know from the TPS is how quickly it's changing, i.e. the slope of its graph. Even with no noise, the estimation of this slope will suffer from the quantization effects shown in the sine curve graph above.

However, it has only just dawned on me that that graph was unfair to MS2, since it was quantized to the nearest percent. The AD circuits quantize to [0:1023], so the resolution is much better. If everyone had as clean a TP signal as yours, the fast sampling would almost certainly be a win. In my defence, the TP signal in my car would only be accurate to about 1%, so the graph reasonably shows my situation and, I gather, many others'.

Having driven my car across Sydney yesterday, including dozens of traffic light starts, I'm confident that the patched code has made it noticeably less twitchy for creeping along in traffic, and nicely responsive for accelerating away. The only thing that doesn't seem as good as before is that it is less responsive to quick blips (i.e. for downshifts). I don't think this is a direct effect of the slower sampling rate, but rather that the less spiky tpsDOT values mean that I need to richen it up a little for faster movements. I expect I will also be able to drop the tpsThresh value quite a bit from its current 75 which may further improve the response to gentle movements.

It'll all come out when I get a chance to do a bit of post-patch accel tuning. Right now though, it seems promising.

Have fun,

Rob.

[robs]

A few more thoughts on this stuff, then I'll give you peace for a while again.

Despite the way that I might come across, I think it is important not to pretend this is some sort of rock solid science. MS bases its core fueling decisions on the ideal gas law, but I don't think there is anything so solid for accel enrichment. As I said earlier, what you want is to use the throttle change as advance warning of an upcoming change in the airflow. But exactly how big a change and how much advance warning it gives will vary according to the engine design, engine speed and load, and a squillion other things. And that's why it comes down to tweaking magic numbers, whether you use old-style AE or EAE.

Just as an aside, one thing that I wondered about on my morning walk today was: are all percentages of TPS equivalent? By geometry, roughly the first 60° (i.e. 2/3 of a right angle) of butterfly spindle rotation is required to open up half the cross sectional area of the throttle body. The next 30° doubles the amount of air that can get in. I haven't taken the time to check how my TPS voltage varies with rotation, but if it is just proportional to rotation it means that the first 5% TPS increment isn't the same as the last 5%.

Well, let's set that rambling aside. What I want to post is a few simulated accel events with "perfect" and "noisy" TPS sensors and compare the current approach with my patched approach.

Firstly, here is the noise-free version.

Noise free sim

quiet.jpg (34.44 KiB) Viewed 2942 times

On the left is the fast-polling standard algorithm, on the right is the 1/4 sampled "patched" version. Each event is 0.1 seconds long, they look similar, but the first is only opening the throttle a little, the last going all the way from closed to wide open. Bear in mind, what is important is the slope of each line segment. At each each "o" in the graph, the slope of the line before it is going to be applied after it. It's quite clear that the patched version is inferior. It underestimates the slope at the start of the event and then overestimates at the end. Mind you, if you look closely at the third row you can see the alternating under and over estimating of the slope in the blue graph.

All in all though, for noise-free TPS, the existing code is better.

Now let's try it with a random normal wiggle added (std deviation 0.5%) which makes the graph look reasonably like my own TPS.

Noisy sim

noisy.jpg (35.69 KiB) Viewed 2942 times

This time the winner is not so clear cut. For slow events, the patched version is much better. It still has the slow ramp in and out, but the slope in the middle of the event is much closer to the money. It's a bit of a toss-up which is better in the middle row and the traditional algorithm gets the nod in the third case.


Here is the R code for this (R is a freely available statistical analysis language).

Code: Select all

par(mfrow=c(3,2))

showit<-function(x, y, c='black') {
   plot(x, floor(y)/10, type='b', col=c, ylab='TP', xlab='t (0.01s)', xlim=c(0,31));
}

pairplot<-function(x, y, r) {
   showit(x, y, 'blue')
   showit(x[r], y[r], 'red');
}

set.seed(1);
w<-5      # amplitude of "noise"
x<-0:30
qr<-(0:7)*4
b<-15

# Slow opening rate -- 100% in 1.0 seconds
y<-(c(rep(b,10),b+(0:10)*(10/11), rep(b+10,10)))*10 + w*rnorm(31)
pairplot(x, y, qr)

# Typical opening rate -- 100% in 0.5 seconds
y<-c(rep(b,10),b+(0:10)*(20/11), rep(b+20,10))*10 + w*rnorm(31)
pairplot(x, y, qr)

# Fast opening rate -- 100% in 0.1 seconds
y<-c(rep(0,10),(0:10)*(100/11)+rnorm(11), rep(100,10))*10 + w*rnorm(31)
pairplot(x, y, qr)


If you get rid of the set.seed(1), you'll get different plots from the above. Dropping the noise value from w<-5 to w<-0 changes the standard deviation of the Gaussian noise, getting you the noise-free versions.

I seeded it with 1 just to show that I didn't go out of my way to generate convincing graphs. You soon see much worse cases if you generate a few at random. The red column generally looks the same where the blue column can be quite a deal noisier than shown here.

CONCLUSION

It's a toss up. If your TPS is noisy, the patched version is probably better. If not, stick with the original. For now, I'll continue with the patched version -- bear in mind that I drive like a granny.

I agree that getting rid of the noise is a good thing. But an algorithm that is looking for significant movements shouldn't be as sensitive to relatively small amounts of noise as the current one.

The code currently waits a particular amount of time and checks the throttle position. It then comes up with a tpsDOT value. How would it be if this was turned on its head: the code could wait until a particular (configurable) amount of throttle position change has occurred, then determine tpsDOT from the amount of time that has passed? Seems to me this would be a lot more tolerant of noise, while still allowing for a rapid response to changes.

Have fun,

Rob.

[piledriver]

Nice.

As to the throttle airflow/effectiveness, as you noted, it's highly nonlinear and tpsdot it would be best on a curve of some sort.
(if they aren't already... seems the only way it could really work)

Of course that's assuming tpsdot was the only input.

[Gokart]

robs,
can you help me of how to apply the patch?
It just refused to compile.

thanks.

[robs]

robs,
can you help me of how to apply the patch?
It just refused to compile.

thanks.

Firstly, I should have made clear that I'm working with 3.1.1. It will probably apply to 3.1.2, but I haven't tried. I'll also assume that you are able to build the standard release. I'll talk in terms of Linux, but this should work in Cygwin under Windows.

1. unzip the patch file. Let's say it ends up in /tmp/tpssamp.pch
2. get into the source directory e.g. cd /var/tmp/ms/ms2extra_3.1.1_release/ms2extra
3. apply the patch: patch -p1 < /tmp/tpssamp.pch
4. build the .s19 file: make

In case the above is too difficult, and because the only file involved is ms2_extra_main.c, and the change is pretty trivial, I'll describe how to do it manually.

1. go to the source directory and edit (using whatever text file editor you like) ms2_extra_main.c
2. get to line 1499. It will have unsigned int tmp_tpssample_time; on it.
3. insert if (tpssample_time > 320) { before this declaration
   

   Code: Select all

   if (tpssample_time > 320){
  unsigned int tmp_tpssample_time;

4. get to line 1549 or so (depending on how many newlines you inserted). You want a line saying ego_get_sample();
5. insert a } before it.
   

   Code: Select all

   }
ego_get_sample();


Save the file and build as usual. My s19 file ended up 302004 bytes long -- the original was 301990 bytes.

Please post here if you try it in your car. I'll definitely be interested. In the meantime, I'm looking at trying out the idea I mentioned at the end of my last posting. Don't think it'll be too hard, but I'll take my time with it.

Have fun,

Rob.

[Gokart]

Thanks robs, I'll try out and see how it comes.

[ol boy]

Long time since you posted Rob. I was wounder what you were up to..

I've used a lot of averaging factor under the general lags table to smooth out my noisy TPS and MAP signals. In my case I'm opening 4 throttle plates in the 42mm range so slight TPS changes yield large MAP changes. I can run down the freeway at 80mph turning 2300rpm with only 7% TPS. A 2% TPS change will move % fuel load from 50% to nearly 75% at a steady state cruise. Without changing the code, can one just increase C9 to a 33uF or R9 to 10K to smooth out this unwanted noise? I solved unwanted noise for my baro sensor this way.

Later, Ryan

[robs]

For those who want to try the patch, probably best to hold off.

I made a silly mistake in there. You're actually going to be sampling your EGO sensor every time through the main loop. The flag flagbyte0_50ms needs to be cleared regardless of tpssample_time.

I don't plan to issue a fix to the patch. As I said, I'm looking at the other approach which I think I should have working in a day or two and I'm pretty sure it'll be better. I realised I'd made this mistake while planning out the changes.

That said, there's nothing much to worry about. I'll keep running the patch in my daily driver -- it's running fine, mistake and all.

And hi Ryan. I've been around, but haven't posted anything thought provoking for quite a while. One of the postings was on my electronic illiteracy, so all your C9s and R9s go clear over my head. I guess it's a low pass filter to dampen down the noise. swnp sounds like he really nailed the problem though.

Have fun,

Rob.

[robs]

OK, time to report some results.

The new "moving anchor" tpsDOT algorithm I mentioned earlier is now running in my car and is looking good. I don't think there are any silly mistakes this time, but would appreciate review.

Here is a somewhat human-friendly description of the algorithm.

There are two controlling parameters: TPS_EPS and TPS_PERSIST. TPS_EPS is a threshold value -- throttle movements are ignored until they have added up to at least this value. TPS_PERSIST is a time value -- if there has been no significant throttle movement for this length of time, the throttle is not moving.

Code: Select all

each 0.01 second:
  read current TPS value
  compare it with the TPS anchor point
  if the difference is more than TPS_EPS
      compute tpsDOT based on the change in TP / time since last anchor
      set anchor point at the current TPS and time
   else if the time since last anchor is more than TPS_PERSIST
      tpsDOT = 0
      set anchor point at the current TPS and time


Perhaps easier to explain from a small detail in a log file from my run in the car.

Small TPS wiggle

small.jpg (28.07 KiB) Viewed 2820 times

This is a 5 second snippet, there is about 0.1 seconds between each 'o' (i.e. there have been ten evaluations done by the MS between each dot). Even without full detail, you can see the stair step effect. This is because the TPS value now anchors itself to points of significant movement, and re-anchors periodically when there has been no movement. tpsDOT is computed at the first point in each new rung, looking back to the first point on the previous rung. Hopefully, you can now look at the two-stage acceleration and see that the slope from dots #6-9 is about the same as from dots #9-11 and this is reflected in the tpsDOT graph below. Then there's no significant change from 11-14, but 14-16 is an even steeper slope. Etc.

Notice also that this is only an accel event with TPS between 3-9%, and that the tpsDOT value is in the range [-16:10], yet is noise free. With my car at least, you would have been hard pressed to get anything as sensible as this for such a small, slow movement. And this algorithm is executing 100 times per second. If the throttle is moving rapidly, it won't miss anything, which could have been a problem with my previous "improvement".

Here is a patch file which implements the moving anchor TPS algorithm. The patch is public domain.

newtpsalgo.zip

Patch file for moving anchor TPS algorithm

(1.9 KiB) Downloaded 79 times

Now, the bad news...

While it is fully functional, the values are not easily configurable. Ideally, the values TPS_EPS and TPS_PERSIST should be able to be set in TunerStudio, but that is something that I don't want to embark on unless this code is to be adopted into the main tree. A period of trying out in the meantime will be no bad thing.

The values are set in #defines in the code and you need to recompile in order to change them. I have set the values to TPS_EPS 15 (i.e. 1.5% change) and TPS_PERSIST 1600 (0.2 seconds -- the units are in 0.000128s ticks, the rate of the MS RTC interrupt). This looks to be sensitive enough to me, but maybe you might want greater sensitivity to small throttle movements, or less or more delay on the resetting of the anchor point. As I said before, it's not an exact science.

Of course you can expect to have to tune this afterwards. It's only a few lines, but a pretty big change. I'm hopeful that the tuning will be more responsive than TPS accel was before.

On tuning, I think there is going to be merit in setting your TPS lag value to 100% (i.e. no lag). Likewise, I have brought my TPSthresh value down to 35 and am seeing no unwanted accel events. That said, I haven't got to tuning it in earnest yet.

In the bigger picture, there may be scope for adopting a similar noise-tolerant approach with mapDOT and RPMdot computation. As in TPS, these are both trying to extract slopes from sampled values, but those values are intrinsically noisier than TPS, so the benefits could be even greater.

I'll be interested to hear how this goes for anyone willing to brave it.

Have fun,

Rob.

[gslender]

Good work Rob - I think the TunerStudiochanges are really just INI settings that are supplied along with the firmware, so technically the "patch" or updated code could just be a Rob's tpsDOT release that includes new settings in TunerStudio.... food for thought I guess.

G

[kritip]

Maybe worth mentioning this in the beta 3.1.2 thread thats running. See if it can be incorporated into that.

My car is off the road at the moment, but I will certainly try it out once its running in a few weeks.

Kristian

[robs]

gslender, the INI changes are part of the story. We'd also need to find slots in the standard flash data areas. Last time I looked, there were a few spares, but it's better if Ken and James make the decisions on how to incorporate this (and whether they want to incorporate it at all).

Kristian, I'm not eager for it to get into the beta yet. Better if it's tried out by a few people with the stable version, with only this change to consider. Glad you're interested in giving it a go. I'm hopeful that this is a real improvement, but it's best to take things steady.

Have fun,

Rob.

[muythaibxr]

Matt Yates (y8s) had an idea that I'm trying on ms3 that could easily be backported to ms2:

If the TPS value doesn't change more than <user settable> %, set TPSdot to 0. I *think* this will allow things to remain just as responsive while not false triggering. Essentially I've implemented half your changes.

I'm not entirely sure the second half is needed aside from not having it might cause very slow throttle movements to get missed. I think in practice it won't be a real issue though.

I've also done the same thing for MAPdot as well. So we'll see how well (or not well) it works for y8s.

Ken