by LT401Vette » Sat Aug 04, 2012 5:01 am
There are 2 representations for setting values, Human Readable and the binary controller values.
The binary values in the controller are usually integer based and may represent something like the number of clock ticks in the MegaSquirt. As a human it is easier to read that as ms. So TunerStudio converts all values from the byte msValues to human userValues based on the scale and translate defined in the ini using this formula:
msValue = userValue / scale - translate
userValue = (msValue + translate) * scale
Then in the ini the number of decimal places is defined. So looking at a setting like 'PWM Time Threshold'
injPwmT = scalar, U08, 613, "ms", 0.12800, 0.128, 0.00, 32.64, 1 ; * ( 1 byte)
It is has a scale of 0.128, a translate of 0.128 and is defined to display 1 decimal place.
so a userValue of 0.7 would be converted to binary as such:
msValue = (0.6 / 0.128) - 0.128
msValue = 4.5595
But now that it is to be stored as a byte, the decimal places are lost, a byte can only hold integer values 0-255. So it will be rounded to a 5 and stored.
So then when it gets converted back to a user value:
userValue = (5+ 0.128) * 0.128
userValue = 0.656384
as 1 decimal place is defined, this will then be displayed back as 0.7.
With the defined scale and translate there is not enough resolution to hold a representation for every 0.1 value
You shouldn't need to close and reopen the dialog, it should be converted to the closest possible value as soon as you tab out of the text area.