Feature request: show SD log length in TS
Moderators: jsmcortina, muythaibxr
Feature request: show SD log length in TS
I know this impacts both TS and the firmware but I figure the latter is probably the harder part so I'm asking here...
Would it be possible when viewing the SD log file list within TunerStudio, to see not only the file size but the length of the actual recorded data (as either bytes or time)?
Since my setup always starts SD logging at key-on, I get a fair number of useless logs from when I power the car on to make tuning changes etc. The trick is that such logs are usually very short, so if the actual data size were shown in the list, it would be easy to identify the logs I really want to look at.
Would it be possible when viewing the SD log file list within TunerStudio, to see not only the file size but the length of the actual recorded data (as either bytes or time)?
Since my setup always starts SD logging at key-on, I get a fair number of useless logs from when I power the car on to make tuning changes etc. The trick is that such logs are usually very short, so if the actual data size were shown in the list, it would be easy to identify the logs I really want to look at.
Eric Law
1990 Audi 80 quattro with AAN turbo engine: happily running on MS3+MS3X
2012 Audi A4 quattro, desperately in need of tweaking
Be alert! America needs more lerts.
1990 Audi 80 quattro with AAN turbo engine: happily running on MS3+MS3X
2012 Audi A4 quattro, desperately in need of tweaking
Be alert! America needs more lerts.
-
- Site Admin
- Posts: 39587
- Joined: Mon May 03, 2004 1:34 am
- Location: Birmingham, UK
- Contact:
Re: Feature request: show SD log length in TS
Technically not possible at present. When you key-off the MS3 is off, so there is no way to re-write the directory entry with a new file size.
Jamed
Jamed
I can repair or upgrade Megasquirts in UK. http://www.jamesmurrayengineering.co.uk
My Success story: http://www.msextra.com/forums/viewtopic ... 04&t=34277
MSEXTRA documentation at: http://www.msextra.com/doc/index.html
New users, please read the "Forum Help Page".
My Success story: http://www.msextra.com/forums/viewtopic ... 04&t=34277
MSEXTRA documentation at: http://www.msextra.com/doc/index.html
New users, please read the "Forum Help Page".
Re: Feature request: show SD log length in TS
I wasn't thinking of resizing the files, but when they're enumerated isn't there a way to tell how much valid data each one contains?
When TS converts the files it somehow determines when logging stopped, I figured a similar technique could be used in the firmware.
When TS converts the files it somehow determines when logging stopped, I figured a similar technique could be used in the firmware.
Eric Law
1990 Audi 80 quattro with AAN turbo engine: happily running on MS3+MS3X
2012 Audi A4 quattro, desperately in need of tweaking
Be alert! America needs more lerts.
1990 Audi 80 quattro with AAN turbo engine: happily running on MS3+MS3X
2012 Audi A4 quattro, desperately in need of tweaking
Be alert! America needs more lerts.
-
- Site Admin
- Posts: 39587
- Joined: Mon May 03, 2004 1:34 am
- Location: Birmingham, UK
- Contact:
Re: Feature request: show SD log length in TS
Only by reading each file in full sequentially. That could take 10-15 minutes.elaw wrote:I wasn't thinking of resizing the files, but when they're enumerated isn't there a way to tell how much valid data each one contains?
James
I can repair or upgrade Megasquirts in UK. http://www.jamesmurrayengineering.co.uk
My Success story: http://www.msextra.com/forums/viewtopic ... 04&t=34277
MSEXTRA documentation at: http://www.msextra.com/doc/index.html
New users, please read the "Forum Help Page".
My Success story: http://www.msextra.com/forums/viewtopic ... 04&t=34277
MSEXTRA documentation at: http://www.msextra.com/doc/index.html
New users, please read the "Forum Help Page".
Re: Feature request: show SD log length in TS
Ah, now I get it. That amount of time is a bit more than would be optimal.
Eric Law
1990 Audi 80 quattro with AAN turbo engine: happily running on MS3+MS3X
2012 Audi A4 quattro, desperately in need of tweaking
Be alert! America needs more lerts.
1990 Audi 80 quattro with AAN turbo engine: happily running on MS3+MS3X
2012 Audi A4 quattro, desperately in need of tweaking
Be alert! America needs more lerts.
-
- Site Admin
- Posts: 39587
- Joined: Mon May 03, 2004 1:34 am
- Location: Birmingham, UK
- Contact:
Re: Feature request: show SD log length in TS
Unfortunately yes. I know exactly what you are wanting and why though!
James
James
I can repair or upgrade Megasquirts in UK. http://www.jamesmurrayengineering.co.uk
My Success story: http://www.msextra.com/forums/viewtopic ... 04&t=34277
MSEXTRA documentation at: http://www.msextra.com/doc/index.html
New users, please read the "Forum Help Page".
My Success story: http://www.msextra.com/forums/viewtopic ... 04&t=34277
MSEXTRA documentation at: http://www.msextra.com/doc/index.html
New users, please read the "Forum Help Page".
Re: Feature request: show SD log length in TS
It probably wouldn't be worth the trouble but if you had random access to the files there might be a solution to that problem: a binary search.
Say for example the log file is 8000 bytes long. Read byte 8000, if it has data you know the log file is 100% full. If not, read the 4000th byte (the midpoint in the file) and see if it's populated. If it is, check the 6000th byte (3/4 point), if not, check the 2000th byte (1/4 point), and so on.
If my mental math is correct, you could determine the "fullness" of the file to within 10% by reading 5 bytes, and 10% accuracy would probably be good enough for most people.
Say for example the log file is 8000 bytes long. Read byte 8000, if it has data you know the log file is 100% full. If not, read the 4000th byte (the midpoint in the file) and see if it's populated. If it is, check the 6000th byte (3/4 point), if not, check the 2000th byte (1/4 point), and so on.
If my mental math is correct, you could determine the "fullness" of the file to within 10% by reading 5 bytes, and 10% accuracy would probably be good enough for most people.
Eric Law
1990 Audi 80 quattro with AAN turbo engine: happily running on MS3+MS3X
2012 Audi A4 quattro, desperately in need of tweaking
Be alert! America needs more lerts.
1990 Audi 80 quattro with AAN turbo engine: happily running on MS3+MS3X
2012 Audi A4 quattro, desperately in need of tweaking
Be alert! America needs more lerts.
-
- Super MS/Extra'er
- Posts: 9128
- Joined: Sun May 02, 2004 6:51 am
- Location: Quebec, Canada
- Contact:
Re: Feature request: show SD log length in TS
Actually, 5 reads in a binary search would put you closer to 3% (1/32).
Jean
Jean
Re: Feature request: show SD log length in TS
Good point!
I realized afterward the approach I outlined above has one "flaw" that in some circumstances would be helpful and in others not.
For a true binary search, the first read would not be at the endpoint of the file, it would be at the midpoint and you'd go from there. But reading the last byte first would enable you to quickly identify full files without having to do any more reads on those files. So if you have an abundance of full log files it would speed things up, if most of your logs are partial it would slow things down.
I realized afterward the approach I outlined above has one "flaw" that in some circumstances would be helpful and in others not.
For a true binary search, the first read would not be at the endpoint of the file, it would be at the midpoint and you'd go from there. But reading the last byte first would enable you to quickly identify full files without having to do any more reads on those files. So if you have an abundance of full log files it would speed things up, if most of your logs are partial it would slow things down.
Eric Law
1990 Audi 80 quattro with AAN turbo engine: happily running on MS3+MS3X
2012 Audi A4 quattro, desperately in need of tweaking
Be alert! America needs more lerts.
1990 Audi 80 quattro with AAN turbo engine: happily running on MS3+MS3X
2012 Audi A4 quattro, desperately in need of tweaking
Be alert! America needs more lerts.
-
- Site Admin
- Posts: 39587
- Joined: Mon May 03, 2004 1:34 am
- Location: Birmingham, UK
- Contact:
Re: Feature request: show SD log length in TS
There's no per-byte random access on SDcards, you have to read in full sectors of 512 bytes.
James
James
I can repair or upgrade Megasquirts in UK. http://www.jamesmurrayengineering.co.uk
My Success story: http://www.msextra.com/forums/viewtopic ... 04&t=34277
MSEXTRA documentation at: http://www.msextra.com/doc/index.html
New users, please read the "Forum Help Page".
My Success story: http://www.msextra.com/forums/viewtopic ... 04&t=34277
MSEXTRA documentation at: http://www.msextra.com/doc/index.html
New users, please read the "Forum Help Page".
Re: Feature request: show SD log length in TS
I have no knowledge of the SD card system, how hard would it be to reserve the first 10 bytes of the log and update them with the size of the log with every frame written (or every 10 frames)? This is a really neat idea (showing the actual SD log size in TS I mean) and I would love to have it during drag racing.
The man behind MS Labs
2005 Audi A3 2.0L TFSI DSG AWD - Extreme MS3
2002 Mazda Miata 1.8 6sp - Enhanced MS3 1.4.0, sequential injection, sequential ignition, big turbo, lots of boost
2005 Audi A3 2.0L TFSI DSG AWD - Extreme MS3
2002 Mazda Miata 1.8 6sp - Enhanced MS3 1.4.0, sequential injection, sequential ignition, big turbo, lots of boost
-
- Experienced MS/Extra'er
- Posts: 372
- Joined: Sat Dec 31, 2011 6:10 am
- Location: Netherlands
- Contact:
Re: Feature request: show SD log length in TS
I'm probably overlooking a bunch of things, but isn't this is a simple calculation of:
LogDuration = TimeEnd(or LastWrittenDataTimestamp) -minus- TimeLogStart ?
LogDuration = TimeEnd(or LastWrittenDataTimestamp) -minus- TimeLogStart ?
Daily driver: 1973 Dodge Dart - 360ci engine - 11.3:1cr - MS3x - ignition only. 42RH/A500 OD+LU transmission / 3.23 gears
--------------
Website: https://www.bigblockmopar.nl/
YouTube Channel: https://www.youtube.com/c/BigBlockMopar
--------------
Website: https://www.bigblockmopar.nl/
YouTube Channel: https://www.youtube.com/c/BigBlockMopar
-
- Site Admin
- Posts: 39587
- Joined: Mon May 03, 2004 1:34 am
- Location: Birmingham, UK
- Contact:
Re: Feature request: show SD log length in TS
How do you determine "TimeEnd" ?BigBlockMopar wrote:I'm probably overlooking a bunch of things, but isn't this is a simple calculation of:
LogDuration = TimeEnd(or LastWrittenDataTimestamp) -minus- TimeLogStart ?
There is no "LastWrittenDataTimestamp".
That would halve the max log rate and greatly complicate the code.Reverant wrote:how hard would it be to reserve the first 10 bytes of the log and update them with the size of the log with every frame written (or every 10 frames)?
James
I can repair or upgrade Megasquirts in UK. http://www.jamesmurrayengineering.co.uk
My Success story: http://www.msextra.com/forums/viewtopic ... 04&t=34277
MSEXTRA documentation at: http://www.msextra.com/doc/index.html
New users, please read the "Forum Help Page".
My Success story: http://www.msextra.com/forums/viewtopic ... 04&t=34277
MSEXTRA documentation at: http://www.msextra.com/doc/index.html
New users, please read the "Forum Help Page".
Re: Feature request: show SD log length in TS
Adding a keep-alive circuit to keep the ECU powered for a couple of seconds (to write the log size in the first frame) is trivial. The same circuit would also allow us to permanently write the long term trim table to flash only on shutdown, instead of using a button. Thoughts?
The man behind MS Labs
2005 Audi A3 2.0L TFSI DSG AWD - Extreme MS3
2002 Mazda Miata 1.8 6sp - Enhanced MS3 1.4.0, sequential injection, sequential ignition, big turbo, lots of boost
2005 Audi A3 2.0L TFSI DSG AWD - Extreme MS3
2002 Mazda Miata 1.8 6sp - Enhanced MS3 1.4.0, sequential injection, sequential ignition, big turbo, lots of boost
-
- Site Admin
- Posts: 39587
- Joined: Mon May 03, 2004 1:34 am
- Location: Birmingham, UK
- Contact:
Re: Feature request: show SD log length in TS
Using the RTCC to date-stamp the files' creation time resolved the issue for me. I don't care about the reported length, just the time of the run.Reverant wrote:Adding a keep-alive circuit to keep the ECU powered for a couple of seconds (to write the log size in the first frame) is trivial. The same circuit would also allow us to permanently write the long term trim table to flash only on shutdown, instead of using a button. Thoughts?
James
I can repair or upgrade Megasquirts in UK. http://www.jamesmurrayengineering.co.uk
My Success story: http://www.msextra.com/forums/viewtopic ... 04&t=34277
MSEXTRA documentation at: http://www.msextra.com/doc/index.html
New users, please read the "Forum Help Page".
My Success story: http://www.msextra.com/forums/viewtopic ... 04&t=34277
MSEXTRA documentation at: http://www.msextra.com/doc/index.html
New users, please read the "Forum Help Page".
Re: Feature request: show SD log length in TS
its only 2 wires to connect a datalog buttton, and one or two more if you want the led indicator, then you can choose when to log
Re: Feature request: show SD log length in TS
Yeah but for me that's kind of counter to the purpose of SD logging.
When I expect something to happen or know I want to look at some parameter(s), I'll datalog on my phone or laptop. For me, the SD log is there for when something unexpected happens, so you can examine the log afterward to see what went wrong. And since by definition unexpected things are... unexpected, I'm not going to think to press the button beforehand.
I think I've sort of fixed the issue by setting logging to trigger on RPM > 25. That should catch everything I'm interested in (I'm hoping logging starts fast enough to catch cranking), but should not create a log when I just turn the key on without starting the engine.
When I expect something to happen or know I want to look at some parameter(s), I'll datalog on my phone or laptop. For me, the SD log is there for when something unexpected happens, so you can examine the log afterward to see what went wrong. And since by definition unexpected things are... unexpected, I'm not going to think to press the button beforehand.
I think I've sort of fixed the issue by setting logging to trigger on RPM > 25. That should catch everything I'm interested in (I'm hoping logging starts fast enough to catch cranking), but should not create a log when I just turn the key on without starting the engine.
Eric Law
1990 Audi 80 quattro with AAN turbo engine: happily running on MS3+MS3X
2012 Audi A4 quattro, desperately in need of tweaking
Be alert! America needs more lerts.
1990 Audi 80 quattro with AAN turbo engine: happily running on MS3+MS3X
2012 Audi A4 quattro, desperately in need of tweaking
Be alert! America needs more lerts.
-
- Experienced MS/Extra'er
- Posts: 372
- Joined: Sat Dec 31, 2011 6:10 am
- Location: Netherlands
- Contact:
Re: Feature request: show SD log length in TS
jsmcortina wrote:How do you determine "TimeEnd" ?BigBlockMopar wrote:I'm probably overlooking a bunch of things, but isn't this is a simple calculation of:
LogDuration = TimeEnd(or LastWrittenDataTimestamp) -minus- TimeLogStart ?
There is no "LastWrittenDataTimestamp".
James
Doesn't every logfile have a Last Modified date given on disk?
EDIT: Nevermind, it occured to me that the MS ECU would have to have a real time clock probably.
Daily driver: 1973 Dodge Dart - 360ci engine - 11.3:1cr - MS3x - ignition only. 42RH/A500 OD+LU transmission / 3.23 gears
--------------
Website: https://www.bigblockmopar.nl/
YouTube Channel: https://www.youtube.com/c/BigBlockMopar
--------------
Website: https://www.bigblockmopar.nl/
YouTube Channel: https://www.youtube.com/c/BigBlockMopar
-
- Site Admin
- Posts: 39587
- Joined: Mon May 03, 2004 1:34 am
- Location: Birmingham, UK
- Contact:
Re: Feature request: show SD log length in TS
No, there's no last modified date written. As discussed above when the ECU is off it can't write anything. Writing during the log would waste half the SPI bandwidth.
James
James
I can repair or upgrade Megasquirts in UK. http://www.jamesmurrayengineering.co.uk
My Success story: http://www.msextra.com/forums/viewtopic ... 04&t=34277
MSEXTRA documentation at: http://www.msextra.com/doc/index.html
New users, please read the "Forum Help Page".
My Success story: http://www.msextra.com/forums/viewtopic ... 04&t=34277
MSEXTRA documentation at: http://www.msextra.com/doc/index.html
New users, please read the "Forum Help Page".