MSQ's and why they'll never be supported

[jsmcortina]

Request for poll - Megatune compatability - loading and saving of MSQs in native form.

James

[djandruczyk]

Request for poll - Megatune compatability - loading and saving of MSQs in native form.

James

unfortunately that is NOT possible, and NEVER will be. This has been discusssed before.

Even users are finding out that they can't take MSQ's for MS-II versions 2.3 and load them against a 2.687 firmware without it hosing things within the SAME version of megatune.

MSQ support will NOT be done in megatunix, the MSQ format is proprietary (it's kind of looks like XML, and kind of like an INI file but complies with NEITHER spec, thus making it a BITCH to parse), and is NOT independant as it marries itself to megatune's proprietary .ini like (BUT NOT to SPEC) gui config files. This marrying of those two together, is what LOCKS MSQ's to megatune and keeps it from being cross-application capable. Any other SW that would want to interface COMPLETELY with it, would be forced into using megatune's model of config files, which is VERY RESTRICTIVE, and cumbersome, and WILL NOT be done by me whatsoever.. There are some apps that DO "semi interface" with MSQ's. (table editors, loggers, etc). this can be done as it requires a MUCH simpler parser, than something that would require full data interchange. Those app writers have it easy...

I will NOT perpetuate a (in what I belive is ) a very flawed file format (i.e. a format dependant on a SEPARATE application SPECIFIC set of files), as that'll end up HURTING end users in the long run. (people are already experiencing it now by taking MSQ's from old versions of MS-II and finding it bork's their settings all to hell when trying to use it against the 2.687 MS-II firmware.)

[muythaibxr]

We know how you feel about the format. That doesn't mean that having that feature is unnecessary.

Personally, as long as I can add things to the GUI easily (according to you, I can, although I've not had a chance to try), just some kind of "translator" would be good enough for me... so a standalone, commandline program to read in an ini and msq, and output in whatever format you think is best.

Also, I'm not sure I like what megatunix currently does either as far as file format goes... as I don't think it's very cross software either... If you don't like the formats that the ini and msq are in, then wouldn't creating an ideal cross-software format be better than just saying "no, I won't support it." Then at least someone could add support to megatune for that format.

Ken

[djandruczyk]

We know how you feel about the format. That doesn't mean that having that feature is unnecessary.
Personally, as long as I can add things to the GUI easily (according to you, I can, although I've not had a chance to try), just some kind of "translator" would be good enough for me... so a standalone, commandline program to read in an ini and msq, and output in whatever format you think is best.

I understand. the problem is that to be able to load a megatune MSQ, you would HAVE to have the .INI's that are ASSOCIATED with it in order to load it. (megatune requires this, the diff, is that the ini's come with it). The problem comes along when someone say's here's an MSQ load it. if you do NOT have the exact .ini it was created with you are NOT guaranteed that it'll load correctly, as the names in the msq needs to be dereferenced in the .ini to determine a memory location and attributes of the variable (bitfield, scaler, array ,etc) . (this is a problem with megatune too, but since the devs have been fairly careful to not change the names of things in the .ini's largely stays intact) A translator would be a good idea, but I don't have the time myself to invest into it. Perhaps someone else can step up.

Also, I'm not sure I like what megatunix currently does either as far as file format goes... as I don't think it's very cross software either... If you don't like the formats that the ini and msq are in, then wouldn't creating an ideal cross-software format be better than just saying "no, I won't support it." Then at least someone could add support to megatune for that format.

Ken

MegaTunix was NOT designed to provide a cross platform data exchange. It was designed to be a cross platform APPLICATION, that would run on just about any major OS (Linux, OS-X, windows, etc..) I'm the first to admit that's it's model isn't readily adaptable to other SW's as well. I haven't come up with a way that I think would be best. It's a very difficult thing to do when there's so many constant changes going on in the megasquirt world. I'll bet it's a problem not easily solved at all.

I think that only the best we can hope for is a better table exchange system (a replacement for the VEX files) for transferring the tables (fuiel, AFR, ignition, boost, etc..) as that is one of the bits that affects almost al lMS related SW as many of the tools out there are table editors, scaler,s loggers and could benefit from a CLEAN, common, readily standards compliant file format (XML comes to mind) to promote that sorts of data exchange..

The rest of the data is much harder, as there is no easy set clean way to represent it between competing softwares. MegaTunix takes the less flexibily but infinitely simpler method of a simple text comma separated list of data for each page of data which works great as long as none of hte data has MOVED internally. if things have moved, then things can break badly. (megatunix uses a signature match to prevent loading of incompatible data.).

MegaTune uses a text indexed based system that is highly dependant on the corresponding .ini files for that firmware. you're not guaranteed to be able to load an MSQ against different FW releases esp if the .ini's were modified. i.e. if any of the var names spellchecked or changed to make the names more intuitive, as the name changes in one of the other, the file breaks and your load state for those variables is undefined.

as i see it there's no clean cut perfect solution to that problem. so I feel it's best that if the users wants to use one or the other, they may have to type in a few more things by hand (which takes a lot less time thanyou think when the gui is laid out well) I think the best solution is a outside translator app that can go either way. Hell it could even be web based. (upload your MTx backup, and get an MSQ back or vice versus. the website would have to maintain copies of the .ini's and MTX interrogation profiles for all firmware combinations, but that's not much space. If I happen to win the lottery and no longer need to work for a living, I'll get right on that....

[muythaibxr]

I understand. the problem is that to be able to load a megatune MSQ, you would HAVE to have the .INI's that are ASSOCIATED with it in order to load it. (megatune requires this, the diff, is that the ini's come with it). The problem comes along when someone say's here's an MSQ load it. if you do NOT have the exact .ini it was created with you are NOT guaranteed that it'll load correctly, as the names in the msq needs to be dereferenced in the .ini to determine a memory location and attributes of the variable (bitfield, scaler, array ,etc) . (this is a problem with megatune too, but since the devs have been fairly careful to not change the names of things in the .ini's largely stays intact) A translator would be a good idea, but I don't have the time myself to invest into it. Perhaps someone else can step up.

I know you'd have to have the INI, which isn't a big deal... it's not like they're difficult to find, and the msq file actually says which data format it needs, so you'd know which ini to grab.

I personally don't have time to do it either though it would go a long way towards me using MTx (I'd really like to stop using windows for my development for example, and move over to FreeBSD).

MegaTunix was NOT designed to provide a cross platform data exchange. It was designed to be a cross platform APPLICATION, that would run on just about any major OS (Linux, OS-X, windows, etc..) I'm the first to admit that's it's model isn't readily adaptable to other SW's as well. I haven't come up with a way that I think would be best. It's a very difficult thing to do when there's so many constant changes going on in the megasquirt world. I'll bet it's a problem not easily solved at all.

I don't think anyone is looking for "best" here... Most would settle for "works"

The rest of the data is much harder, as there is no easy set clean way to represent it between competing softwares. MegaTunix takes the less flexibily but infinitely simpler method of a simple text comma separated list of data for each page of data which works great as long as none of hte data has MOVED internally. if things have moved, then things can break badly. (megatunix uses a signature match to prevent loading of incompatible data.).

MegaTune uses a text indexed based system that is highly dependant on the corresponding .ini files for that firmware. you're not guaranteed to be able to load an MSQ against different FW releases esp if the .ini's were modified. i.e. if any of the var names spellchecked or changed to make the names more intuitive, as the name changes in one of the other, the file breaks and your load state for those variables is undefined.

We try to keep any name changes to a minimum, This is part of what I was saying though, the model that Megatune uses is common. You need to seperate the schema from the data, and I can't think of a better model to do it. The file formats involved could be changed to more standards-based formats, but the idea of keeping the data and schema separate is a good one... Also, the chances of using megatune to load an old msq on a new firmware and having it work are greater than with any other method so far. There is no way to ensure, 100%, that an old msq will load properly on a new firmware. At least this way, as long as we don't change the names of variables, the user has a good chance of having everything work. Using this method, without changing the names, we could rearrange pretty much ALL of the data, and the user would still be able to load it again.

as i see it there's no clean cut perfect solution to that problem. so I feel it's best that if the users wants to use one or the other, they may have to type in a few more things by hand (which takes a lot less time thanyou think when the gui is laid out well) I think the best solution is a outside translator app that can go either way. Hell it could even be web based. (upload your MTx backup, and get an MSQ back or vice versus. the website would have to maintain copies of the .ini's and MTX interrogation profiles for all firmware combinations, but that's not much space. If I happen to win the lottery and no longer need to work for a living, I'll get right on that....

As I see it, if there's no clean-cut perfect solution, the "next best" solution wins... In this case, the "next best" is what we have, possibly with more standards compliant files, and maybe without defining the GUI in the "schema" file (although having it there is very convenient).

I'm not suggesting that you spend all your time doing this, If anyone knows what it's like to be short on time, it's me; but I do see this as the key to getting a lot more people on MTx. I don't know if you care about that or not.

Ken