Shadow Logger MS 1.02 Released

For discussion of other tuning and logging softwares

Moderators: jsmcortina, muythaibxr

gslender
Super MS/Extra'er
Posts: 1072
Joined: Fri Sep 16, 2011 5:29 am
Location: Brisbane, Australia
Contact:

Re: Shadow Logger MS 1.02 Released

Post by gslender »

Phil,

About to start using ShadowLogger, and was wondering if the possiblity to add a modified MSQ to the site is available so that when you next connect the Droid device to the ECU, it would offer an option to update using the tune uploaded to ShadowTuner.com ??

That would allow you to log one day, that night review and update the tune, then return to the vehicle next day to upload and log again. Good way to review VE and Spark tables and such.

G
Mazda MX5 + MS3 Pro
rx7_ben
Helpful MS/Extra'er
Posts: 67
Joined: Tue Jan 01, 2008 8:05 am
Location: Melbourne, Australia

Re: Shadow Logger MS 1.02 Released

Post by rx7_ben »

Thank you very much for your help Phil.

I can't recall what signature is displayed in shadowlogger, but I don't remember any of the data appearing to be truncated. I won't be with the car for a week or so, but I'll capture a new log then and see what happens.

Thanks again.
Keithg
Super MS/Extra'er
Posts: 2413
Joined: Sun Mar 06, 2005 9:15 am
Location: Chicago, IL, USA
Contact:

Re: Shadow Logger MS 1.02 Released

Post by Keithg »

also, any ETA on a version that uses the error checking capability of the new serial comms?

Keith
LT401Vette
Super MS/Extra'er
Posts: 12684
Joined: Sat Jul 16, 2005 8:07 am
Location: Moorseville, NC
Contact:

Re: Shadow Logger MS 1.02 Released

Post by LT401Vette »

You can update the msq on shadowtuner.com and time.
You can then do as you said, upload again and have the new settings take effect.

There was an issue with the site not clearing cached projects, that did prevent it from using the new settings, but the cached project should be getting cleared now when you upload a new msq, ini or inc file.
also, any ETA on a version that uses the error checking capability of the new serial comms?

Keith
That will be in the next release, but I am currently focusing on TunerStudio enhancements for the spring release, so they have sufficient beta test time. I'll be back to android next month.
Phil Tobin
EFI Analytics, helping to simplify EFI
Next Generation tuning software.
Supporting all MegaSquirt versions and firmwares.
http://www.TunerStudio.com
http://www.efiAnalytics.com/MegaLogViewer/
Support the firmware running your engine:
http://www.msextra.com/doc/donations.html
gslender
Super MS/Extra'er
Posts: 1072
Joined: Fri Sep 16, 2011 5:29 am
Location: Brisbane, Australia
Contact:

Re: Shadow Logger MS 1.02 Released

Post by gslender »

LT401Vette wrote:You can update the msq on shadowtuner.com and time.
You can then do as you said, upload again and have the new settings take effect.

There was an issue with the site not clearing cached projects, that did prevent it from using the new settings, but the cached project should be getting cleared now when you upload a new msq, ini or inc file.
Ok, I'm not sure if we're talking 'bout the same thing here.... I want to use my droid to capture a log, send to ShadowTuner.com, and then when back home, use MLV and TS to review said log and tune, make changes to tune, upload MSQ to ShadowTuner.com, and when next back in car, have my droid update the ECU with the new MSQ from ShadowTuner.com and begin a new log using that tune etc...

G
Mazda MX5 + MS3 Pro
LT401Vette
Super MS/Extra'er
Posts: 12684
Joined: Sat Jul 16, 2005 8:07 am
Location: Moorseville, NC
Contact:

Re: Shadow Logger MS 1.02 Released

Post by LT401Vette »

Ok, I'm not sure if we're talking 'bout the same thing here.... I want to use my droid to capture a log, send to ShadowTuner.com, and then when back home, use MLV and TS to review said log and tune, make changes to tune, upload MSQ to ShadowTuner.com, and when next back in car, have my droid update the ECU with the new MSQ from ShadowTuner.com and begin a new log using that tune etc...
Ahh, yes that is different..
That is planned for the tuning app code base. Shadow Logger doesn't understand an msq or ini file. Of coarse that would be possible if the server just told Shadow Logger what bytes to send as the server does understand ini files and msq's. But I think I would rather go straight to the Android tuning app that does understand the ini files and msq.
Phil Tobin
EFI Analytics, helping to simplify EFI
Next Generation tuning software.
Supporting all MegaSquirt versions and firmwares.
http://www.TunerStudio.com
http://www.efiAnalytics.com/MegaLogViewer/
Support the firmware running your engine:
http://www.msextra.com/doc/donations.html
gslender
Super MS/Extra'er
Posts: 1072
Joined: Fri Sep 16, 2011 5:29 am
Location: Brisbane, Australia
Contact:

Re: Shadow Logger MS 1.02 Released

Post by gslender »

LT401Vette wrote:
Ok, I'm not sure if we're talking 'bout the same thing here.... I want to use my droid to capture a log, send to ShadowTuner.com, and then when back home, use MLV and TS to review said log and tune, make changes to tune, upload MSQ to ShadowTuner.com, and when next back in car, have my droid update the ECU with the new MSQ from ShadowTuner.com and begin a new log using that tune etc...
Ahh, yes that is different..
That is planned for the tuning app code base. Shadow Logger doesn't understand an msq or ini file. Of coarse that would be possible if the server just told Shadow Logger what bytes to send as the server does understand ini files and msq's. But I think I would rather go straight to the Android tuning app that does understand the ini files and msq.
I wouldn't underestimate the usefulness of being able to simply upload and transfer the MSQ bytes from the server directly to the ECU via the droid app.

After logging, and then reviewing on a remote PC, it would be really handy to then be able to upload a modified MSQ (the bytes) to the ECU without having to have a full blown laptop/pc connected. I also doubt the droid app would be as convenient to adjust a VE table etc, nor navigate the complex and lengthy dialogs - so many times it might be more suitable to tune on the PC with a large screen to review logs, make tune adjustments and then upload. I could also see TS having an "upload to ShadowTuner.com" option to make the process easy.

G
Mazda MX5 + MS3 Pro
LT401Vette
Super MS/Extra'er
Posts: 12684
Joined: Sat Jul 16, 2005 8:07 am
Location: Moorseville, NC
Contact:

Re: Shadow Logger MS 1.02 Released

Post by LT401Vette »

I don't think I'm under estimating the usefulness, but wouldn't want to have the phone sending bytes blindly without a 2 way communication established for proper validations. And this is in the plans... So I think the update of an msq, should follow this. Also, I'm putting in TS the ability to update the msq on the server and download logs from inside just to keep it in sync, but that would also get 1 step closer, saving the need to upload a new msq after you tune.
The longer term idea behind the name ShadowTuner.com is that it can act like a pipeline between TunerStudio and your phone, so you can connect your tuning app (TunerStudio) to ShadowTuner and see any phones that are online with a controller and you have permissions to, then connect to it. Based on your permissions you would be able to just data log and view runtime data, or full out remote tune.
To do that it will need that full 2 way communication, then having it apply a new msq from the server would not be that big a deal to implement.
Phil Tobin
EFI Analytics, helping to simplify EFI
Next Generation tuning software.
Supporting all MegaSquirt versions and firmwares.
http://www.TunerStudio.com
http://www.efiAnalytics.com/MegaLogViewer/
Support the firmware running your engine:
http://www.msextra.com/doc/donations.html
Keithg
Super MS/Extra'er
Posts: 2413
Joined: Sun Mar 06, 2005 9:15 am
Location: Chicago, IL, USA
Contact:

Re: Shadow Logger MS 1.02 Released

Post by Keithg »

The longer term idea behind the name ShadowTuner.com is that it can act like a pipeline between TunerStudio and your phone, so you can connect your tuning app (TunerStudio) to ShadowTuner and see any phones that are online with a controller and you have permissions to, then connect to it. Based on your permissions you would be able to just data log and view runtime data, or full out remote tune.
I guess I don't fully get this. I think the idea proposed by gslender makes pretty good sense: Log on ST, upload, remotely edit the msq 0on the laptop/desktop, upload a new one and allow the phone to upload it to the MS.

If I have the laptop in the car, I see little reason for ST to be connected to the MS, as the laptop will be doing that. What I do see is less and less of need for the laptop. Tablets are powerful enough and have large enough screens that these could be used as a platform for a full tuning app. As for the what app runs on a phone, It would be nice if it could also be used in a pinch to edit a value or 2 in the tune. Not a VE tune, but changing a value in the CL idle, slight adjustment to a PID parameter, etc.

$0.02
Keith
gslender
Super MS/Extra'er
Posts: 1072
Joined: Fri Sep 16, 2011 5:29 am
Location: Brisbane, Australia
Contact:

Re: Shadow Logger MS 1.02 Released

Post by gslender »

LT401Vette wrote:I don't think I'm under estimating the usefulness, but wouldn't want to have the phone sending bytes blindly without a 2 way communication established for proper validations.
What validations? It could be as simple as a smooth upload, or just report the error that it can't... simple. To get something out early (with tremendous value) would be to simply allow the phone to update the MSQ with the ST version based on date (ie in the phone, show the MSQ's by date uploaded and the user can decide to either replace the config, or not). Perhaps even take a copy of the current config, to reapply the last ECU config if the ST copy failed or was a bad tune etc.

I think having 2 way communication is nice, but not critical to going forward as I'm unlikely to want to have TS open on my desk PC, connecting in real-time to the phone, which is connected to the car, which is on, which is able to two-way update the tune. Sure, it might happen, but unlikely for 90% of users who aren't racing with a pit crew etc...

G
Mazda MX5 + MS3 Pro
LT401Vette
Super MS/Extra'er
Posts: 12684
Joined: Sat Jul 16, 2005 8:07 am
Location: Moorseville, NC
Contact:

Re: Shadow Logger MS 1.02 Released

Post by LT401Vette »

You are thinking on the golden path. Only 5% of TS's code goes toward handing when things go right, most is around making sure it is right or handling the side conditions.
I'm talking 2 way communications from the server to the Android...
With no 2 way communication just sending a raw dump, there is no validation that the bytes are right, on an error the phone doesn't know how to handle it, it doesn't even know if it is talking to the right firmware. I am not going to support that, I would be buried :)
Phil Tobin
EFI Analytics, helping to simplify EFI
Next Generation tuning software.
Supporting all MegaSquirt versions and firmwares.
http://www.TunerStudio.com
http://www.efiAnalytics.com/MegaLogViewer/
Support the firmware running your engine:
http://www.msextra.com/doc/donations.html
gslender
Super MS/Extra'er
Posts: 1072
Joined: Fri Sep 16, 2011 5:29 am
Location: Brisbane, Australia
Contact:

Re: Shadow Logger MS 1.02 Released

Post by gslender »

LT401Vette wrote:You are thinking on the golden path. Only 5% of TS's code goes toward handing when things go right, most is around making sure it is right or handling the side conditions.
I'm talking 2 way communications from the server to the Android...
With no 2 way communication just sending a raw dump, there is no validation that the bytes are right, on an error the phone doesn't know how to handle it, it doesn't even know if it is talking to the right firmware. I am not going to support that, I would be buried :)
Fair point. I'm aware I was making it very simple - thinking was that you'd place the responsiblity of getting right ECU and right MSQ config to be the users problem (with the app having no ability to verify anything at all).

I was hoping the simple TS -> MSQ -> ST -> Bytes -> Android -> ECU would be a straight forward copy/transfer process. If anything goes wrong, the last firmware obtained by the Android (before the new tune bytes are sent to the ecu) would be used to restore a known working condition. You could even make TS upload raw bytes to ST to ensure the right ini is loaded to avoid another step that could lead to errors in the ST/TS process.

Sounds like we are talking about the same goal - was hoping to encourage you to fast track a simple solution of just getting the raw tune to the ECU after a logging session back at home. Getting the Android app to fully understand the tune is certainly the final goal, just a longer way of getting to what I was hoping to achieve in a shorter (but yes riskier) way.

Goes without saying that I can't wait - must save up for the big-screen Android Tablet ;-)

G
Mazda MX5 + MS3 Pro
scudderfish
Helpful MS/Extra'er
Posts: 123
Joined: Sat Nov 28, 2009 5:10 am
Location: Hertfordshire

Re: Shadow Logger MS 1.02 Released

Post by scudderfish »

I thought about doing something like that with MSLogger, but the more I thought about it, the more it scared me. The BT connection WILL drop mid write (maybe not today, but it will at somepoint), and then how do you recover? You'd need to keep a known good 'spare' and hope that you can write that back to the ECU over an unreliable link.

Regards,
Dave
MSDroid More than just a dashboard.

Image

Do you like the application? Make a donation to Cancer Research UK on our behalf http://www.justgiving.com/MSDroid/
Keithg
Super MS/Extra'er
Posts: 2413
Joined: Sun Mar 06, 2005 9:15 am
Location: Chicago, IL, USA
Contact:

Re: Shadow Logger MS 1.02 Released

Post by Keithg »

The signature of the MS is broadcast at startup, so the phone/tablet could check and not send an MSQ that does not have a matching signature.

As for dropped packets, I thought that was why the 'new serial' was written, to allow CRC checks. As it is, I get bad data in my logs on the phone as well b/c it does not yet CRC check the packets. Once this is added, why could it not send an MSQ over the BT link? How about reducing the amount of data by sending over only the differences between the currently loaded MSQ and the new one? Is that even possible?

KeithG
LT401Vette
Super MS/Extra'er
Posts: 12684
Joined: Sat Jul 16, 2005 8:07 am
Location: Moorseville, NC
Contact:

Re: Shadow Logger MS 1.02 Released

Post by LT401Vette »

The signature of the MS is broadcast at startup, so the phone/tablet could check and not send an MSQ that does not have a matching signature.
If it is loading an msq, it would check the signature, but in this scenario where the phone doesn't understand the ini or msq, we are talking about raw bytes that were sent from the server. So no signature to validate against. This is where I am referring to needing 2 way communication. Seeing that the phone doesn't understand an msq, and it is the one with connectivity to the MS, it would need to read the signature from the MS and pass it back to the server, then the server can validate the signature and send the bytes to the phone to send to the MS.
In this model the phone app can remain stupid and just act as a proxy server between the shadowtuner.com and your MS. It will just blindly take bytes from the server and send them, then send the result bytes back to the server.

While that model is cool and has a place, it adds complexity and dependencies. The benefit is that ShadowTuner.com has TS on it and already knows how to deal with bytes so has the potential to short cut.

But ultimately I believe the right solution is the Android understanding the ini and msq in a standalone fashion. I don't plan to ever make Shadow Logger do that... It is planned to just be a logger and dash, and perhaps take on the proxy server role.

It is a different code base that I have been working on for Android tuning that is leveraging/porting TS components and shadow logger comms, then all the logic takes place on the phone it self.

As for dropped packets, I thought that was why the 'new serial' was written, to allow CRC checks. As it is, I get bad data in my logs on the phone as well b/c it does not yet CRC check the packets. Once this is added, why could it not send an MSQ over the BT link? How about reducing the amount of data by sending over only the differences between the currently loaded MSQ and the new one? Is that even possible?
It can and will, but like I said above, Shadow Logger is pretty much read only. So even with the new protocol it is just validating the CRC after a read, it is bad, drop it and post a bad read. I'm not expecting it to ever handle writes with error handling. The other code will.
Phil Tobin
EFI Analytics, helping to simplify EFI
Next Generation tuning software.
Supporting all MegaSquirt versions and firmwares.
http://www.TunerStudio.com
http://www.efiAnalytics.com/MegaLogViewer/
Support the firmware running your engine:
http://www.msextra.com/doc/donations.html
gslender
Super MS/Extra'er
Posts: 1072
Joined: Fri Sep 16, 2011 5:29 am
Location: Brisbane, Australia
Contact:

Re: Shadow Logger MS 1.02 Released

Post by gslender »

Phil, just started using Shadow Logger MS 1.05 and the coolant F/C display only does F even though you change it to C - so the value and label is not updating based on preferences.

Also, I'm having a lot of "BT connect, but can't connect to controller" errors, but then it can after a big pause. Once connected to the controller it works fine and the results are cool - I like having the phone display the data and log at the same time. 8) 8)

I'm using one of your serial BT devices connected to a DIYPNP MS2

G
Mazda MX5 + MS3 Pro
gslender
Super MS/Extra'er
Posts: 1072
Joined: Fri Sep 16, 2011 5:29 am
Location: Brisbane, Australia
Contact:

Re: Shadow Logger MS 1.02 Released

Post by gslender »

Another issue - I'm trying to update my default project ini file and it doesn't. I'm using this one attached (which is my mod but it works in TS fine). :? :?
megasquirt-ii.ms2extra.ini
G
Mazda MX5 + MS3 Pro
gslender
Super MS/Extra'er
Posts: 1072
Joined: Fri Sep 16, 2011 5:29 am
Location: Brisbane, Australia
Contact:

Re: Shadow Logger MS 1.02 Released

Post by gslender »

gslender wrote:Another issue - I'm trying to update my default project ini file and it doesn't. I'm using this one attached (which is my mod but it works in TS fine). :? :?
megasquirt-ii.ms2extra.ini
G
Fixed it. I had to upload the MSQ first. Obviously the website needs a little work to make it more user friendly, but the outcomes are awesome.

Great work Phil! :yeah!:

G
Mazda MX5 + MS3 Pro
LT401Vette
Super MS/Extra'er
Posts: 12684
Joined: Sat Jul 16, 2005 8:07 am
Location: Moorseville, NC
Contact:

Re: Shadow Logger MS 1.02 Released

Post by LT401Vette »

BT connect, but can't connect to controller
I believe I know what that is. I've just been trying to stay focused on what I'm working on... When I start jumping between apps too much, I get less done. :)

I'll have to get another release of Shadow Logger out to take care of that and the crc check soon.
I had to upload the MSQ first. Obviously the website needs a little work to make it more user friendly
The web site has been functionally sound, but man it needs a face lift :), I mean that more with the navigation even than the look..
Phil Tobin
EFI Analytics, helping to simplify EFI
Next Generation tuning software.
Supporting all MegaSquirt versions and firmwares.
http://www.TunerStudio.com
http://www.efiAnalytics.com/MegaLogViewer/
Support the firmware running your engine:
http://www.msextra.com/doc/donations.html
Morten VJ
Helpful MS/Extra'er
Posts: 131
Joined: Fri May 07, 2004 12:21 pm
Location: denmark

Re: Shadow Logger MS 1.02 Released

Post by Morten VJ »

After some problems getting the software on the phone, and understanding the server, setup.
I'm only smiles, it just works :D
Volvo 245 turbo
B 23 et engine
MSnS-E hires 10g
Post Reply