Concatenating Log Files?
Moderators: jsmcortina, muythaibxr
-
- Super MS/Extra'er
- Posts: 875
- Joined: Wed Dec 05, 2007 1:40 pm
- Location: San Diego, CA
- Contact:
Concatenating Log Files?
What does it take to string a bunch of .xls log files together from megatune? I want to autotune my maps, and I have many drives, with several different wastegate configurations, and this covers a wide range of steady-state boost values. I only tried just putting them in, and I think I ran into issues with the marks and the resets not being numbered correctly?
So, I would like to put them all together, sort out the lines that say reset/mark, and stuff them in MegaLogViewer and VE analyze. Is this reasonable? How do you do it?
So, I would like to put them all together, sort out the lines that say reset/mark, and stuff them in MegaLogViewer and VE analyze. Is this reasonable? How do you do it?
2000 VVT Miata turbo, MS3Pro
Contact me if interested in a MS-II 2nd gen NB Miata PnP board.
Contact me if interested in a MS-II 2nd gen NB Miata PnP board.
-
- Site Admin
- Posts: 8230
- Joined: Thu Oct 14, 2004 12:48 pm
-
- Super MS/Extra'er
- Posts: 875
- Joined: Wed Dec 05, 2007 1:40 pm
- Location: San Diego, CA
- Contact:
But, is that the right way to do it? When I first pasted them together, I got errors, and I would like to understand them.
Ideally, a script that renumbers the resets and the marks and retains them would be best, but I don't know why it doesn't work.
But a series of 10 minute logs won't work as well as a single hour long log, for autotuning or manual sorting...
Ideally, a script that renumbers the resets and the marks and retains them would be best, but I don't know why it doesn't work.
But a series of 10 minute logs won't work as well as a single hour long log, for autotuning or manual sorting...
2000 VVT Miata turbo, MS3Pro
Contact me if interested in a MS-II 2nd gen NB Miata PnP board.
Contact me if interested in a MS-II 2nd gen NB Miata PnP board.
-
- Site Admin
- Posts: 8230
- Joined: Thu Oct 14, 2004 12:48 pm
-
- Super MS/Extra'er
- Posts: 875
- Joined: Wed Dec 05, 2007 1:40 pm
- Location: San Diego, CA
- Contact:
Ok, but there's no more "header" information that that? Just as long as "RESET" and "MARK" were in order, you're ok?
And purely for autotuning, you could just take them all out, correct?
That's good enough for most things anyway. Now I just need to con a programmer into doing it.
And purely for autotuning, you could just take them all out, correct?
That's good enough for most things anyway. Now I just need to con a programmer into doing it.
2000 VVT Miata turbo, MS3Pro
Contact me if interested in a MS-II 2nd gen NB Miata PnP board.
Contact me if interested in a MS-II 2nd gen NB Miata PnP board.
-
- Super MS/Extra'er
- Posts: 875
- Joined: Wed Dec 05, 2007 1:40 pm
- Location: San Diego, CA
- Contact:
I guess I am out of uploads, and don't know how to get rid of the old ones... So here's the source code for a script that a guy on MiataTurbo.net wrote. Took him like ten minutes and it was greatly appreciated. I'm sure it'll help someone here, too. It was the best autotune I've gotten in a long time. I had to double and redouble the memory allotment for megalog viewer, but nearly every point got touched and it's smooth too.
Code: Select all
using System;
using System.Collections.Generic;
using System.Text;
using System.IO;
namespace MegaLogCombine
{
class Program
{
static void Main( string[] args )
{
double timeStampOffset = 0;
double currentTimeStamp = 0;
int iLog = 0;
int fieldCount = 0;
foreach ( string arg in args )
{
string[] lines = File.ReadAllLines( arg, Encoding.ASCII );
for ( int iLine = 0; iLine < lines.Length; ++iLine )
{
string line = lines[ iLine ];
// strip out the first two lines of all logs besides the first
if ( iLine < 2 )
{
if ( iLog == 0 )
Console.WriteLine( line );
continue;
}
// Don't want mark lines
if ( line.Contains( "\"MARK" ) )
continue;
string[] fields = line.Split('\t');
// don't do empty lines
if ( fields == null || fields.Length < 1 )
continue;
if ( fieldCount == 0 )
fieldCount = fields.Length;
else if ( fieldCount != fields.Length )
throw new System.FormatException( String.Format( "Found {0} fields, expected {1} fields, on line: {2}", fieldCount, fields.Length, line ) );
double timeStamp;
if ( !double.TryParse( fields[0], out timeStamp ) )
throw new System.FormatException( String.Format( "Failed to parse timestamp from line {0}", line ) );
currentTimeStamp = timeStamp + timeStampOffset;
fields[ 0 ] = currentTimeStamp.ToString();
StringBuilder builder = new StringBuilder();
for ( int i = 0; i < fields.Length; ++i )
{
if ( i > 0 )
builder.Append( '\t' );
builder.Append( fields[i] );
}
Console.WriteLine( builder.ToString() );
}
timeStampOffset = currentTimeStamp;
++iLog;
}
}
}
}
2000 VVT Miata turbo, MS3Pro
Contact me if interested in a MS-II 2nd gen NB Miata PnP board.
Contact me if interested in a MS-II 2nd gen NB Miata PnP board.
-
- Super MS/Extra'er
- Posts: 1331
- Joined: Mon May 21, 2007 8:36 am
-
- Super MS/Extra'er
- Posts: 875
- Joined: Wed Dec 05, 2007 1:40 pm
- Location: San Diego, CA
- Contact:
Well, either you compile it, or I upload the .exe - but last time I tried to put something on here I spent so long trying to make space I gave up. Maybe someone here can compile it for you?
Or if someone knows of an attachment finder so I can delete my old attachments I'll do it.
Once installed, just do this:
MLC.exe file1.xls file2.xls file3.xls > combined.xls
that's about it. It would be nice if it did a directory, easier to set up. But it doesn't and I haven't bothered to tweak it. I just renamed all my files to 1, 2, 3, 4, 5 so it was easier to type. If you don't put in the ">" it will output to the screen, which can take like 20 minutes and then you have no file. If you do it right, it takes a second or two.
Or if someone knows of an attachment finder so I can delete my old attachments I'll do it.
Once installed, just do this:
MLC.exe file1.xls file2.xls file3.xls > combined.xls
that's about it. It would be nice if it did a directory, easier to set up. But it doesn't and I haven't bothered to tweak it. I just renamed all my files to 1, 2, 3, 4, 5 so it was easier to type. If you don't put in the ">" it will output to the screen, which can take like 20 minutes and then you have no file. If you do it right, it takes a second or two.
2000 VVT Miata turbo, MS3Pro
Contact me if interested in a MS-II 2nd gen NB Miata PnP board.
Contact me if interested in a MS-II 2nd gen NB Miata PnP board.
-
- Super MS/Extra'er
- Posts: 1331
- Joined: Mon May 21, 2007 8:36 am
can you email it to me? I have tons of space & bandwidth, I can post it up on my website for people to download. chris AT 325ix DOT comAbeFM wrote:Well, either you compile it, or I upload the .exe - but last time I tried to put something on here I spent so long trying to make space I gave up. Maybe someone here can compile it for you?
Or if someone knows of an attachment finder so I can delete my old attachments I'll do it.
Once installed, just do this:
MLC.exe file1.xls file2.xls file3.xls > combined.xls
that's about it. It would be nice if it did a directory, easier to set up. But it doesn't and I haven't bothered to tweak it. I just renamed all my files to 1, 2, 3, 4, 5 so it was easier to type. If you don't put in the ">" it will output to the screen, which can take like 20 minutes and then you have no file. If you do it right, it takes a second or two.
-
- Super MS/Extra'er
- Posts: 1331
- Joined: Mon May 21, 2007 8:36 am
-
- Super MS/Extra'er
- Posts: 1331
- Joined: Mon May 21, 2007 8:36 am