Analyzing Dyno Data

[jsiedlicki]

I'm trying to look at data from my recent dyno sessions and want to display multiple runs at the same time to easily pick out the gains. I also want to be able to display a dyno graph and the TS log from the same run. To that end I have a couple of questions.

1. Can you get MLVHD to use RPM on the X axis of log graphs instead of time? This would help to align multiple runs since time is not available in the dyno data and would not be accurate to align multiple runs even if it were.
2. Is there a way to compare two logs with different fields? I.E. a dyno log and a TS log side by side and use RPM in to align them.

Any help or other ideas on how to relate dyno logs and TS logs in MLVHD would be helpful.

Thx

[LT401Vette]

1. Can you get MLVHD to use RPM on the X axis of log graphs instead of time? This would help to align multiple runs since time is not available in the dyno data and would not be accurate to align multiple runs even if it were.

Only fields that continuously go up can be used on the X axis. Otherwise it turns into a scatter plot, so you use the scatter plots.

1. Is there a way to compare two logs with different fields? I.E. a dyno log and a TS log side by side and use RPM in to align them.

No, it does not let you load different logs simultaneously.

[jsiedlicki]

Can you get MLVHD to use RPM on the X axis of log graphs instead of time? This would help to align multiple runs since time is not available in the dyno data and would not be accurate to align multiple runs even if it were.

Only fields that continuously go up can be used on the X axis. Otherwise it turns into a scatter plot, so you use the scatter plots.

For the dyno pull data logs I can trim the beginning and ends of the file so the RPM only climbs but how do I get MLVHD to use that as the X axis on the log traces (not the scatter plots). It seems to only use record number. Also If I do that with an MSL file it always defaults to time on the X axis. Where do I go to change that?

[retro1850]

Is the dyno data in MSL or CSV or excel format? If so, if you can post a copy on one dyno log and post one MSL or MLG. I want to see the data field labels and format.
I'll see what I can build to compare the two in megalogviewer using scatterplots. I think this is a data issue to make it fit the software. Anyway, what do you think? Is it worth a try?

[jsiedlicki]

Is the dyno data in MSL or CSV or excel format? If so, if you can post a copy on one dyno log and post one MSL or MLG. I want to see the data field labels and format.
I'll see what I can build to compare the two in megalogviewer using scatterplots. I think this is a data issue to make it fit the software. Anyway, what do you think? Is it worth a try?

The dyno data is in CSV so it's easy to work with. I can load them in MLVHD now and use scatter plots but I want to be able to use the Log Viewer tab to compare multiple runs. The key is to be able to align them by RPM so the comparison between each run's power output is accurate. Every run starts at a different RPM (human error while using the gas pedal) and the spacing between RPM points is not consistent between each file, you need something that can interpolate and smooth the data. MLVHD does those things well so I I'm trying to leverage MLVHD's features to view these data sets.

It does not look like it can be done natively in MLVHD. I can write a program or spreadsheet to do it, but if MLVHD could do it easily, it would have been a no brainer.

Here are two dyno runs in CSV as an example.

[retro1850]

Not sure if this will suit your needs, but here is what was done.

The data set was opening in Excel.
One column was added on the extreme right hand side of the data and a column named CASE created.
This column was populated with an integer.
These steps can be accomplished using an Excel macro.

Spreadsheet modified with CASE column.png (430.42 KiB) Viewed 2572 times

Once the data sets had been created with the CASE column added, I opened a series of these files in MLVHD.

Loading multiple files.png (143.1 KiB) Viewed 2572 times

In the scatter plot screen, the X and Y axis hold the data of interest and the Z axis contains the CASE index.
The Z-axis shows a different color for each data set so they can be visually recognized.

Multiple Data Files with CASE.png (221.21 KiB) Viewed 2572 times

If there are more than two data sets, the use of filters can remove one or more individual CASEs to allow better visibility of the remaining data sets.

Case Data Filters.png (69.37 KiB) Viewed 2572 times

Using Data Filter.png (259.62 KiB) Viewed 2572 times

That's all folks!

[jsiedlicki]

@retro1850 - Thanks for the input. I already had basically the same thing done with scatter plots. As I mentioned in earlier posts, my real interest is to get the data displayed in the Log Viewer tab so I can overlay at least two plots. That way I can add smoothing, extra fields, multiple graphs etc...

If I can get that to work, ultimately I want to be able to merge the dyno data with the MSL data so I can see what the motor is doing at each step in the dyno plot.

[ShirtUser]

Here you go. Try the attached CSV. Make sure you have auto detect for field delimiter (mis-labeled "file delimiter" in MLV), or commas.

I added a 127 to the end of the fields from the 127 file, and 134 for the 134 file. So you can easily tell which field is from which file. I don't know why MLV is adding all of the extra fields with just #'s, maybe something to do with extra spaces in the field names, but you can ignore those and select the fields that you want to compare. For example, you can select "CTorq Clb ft 127" and "CTorq Clb ft 134" and compare those torque curves.

If you select the two RPMs (EngSpd RPM 134/127), you'll see that they never exactly each other. One of them will match the time, and the other will be an average of that file's values adjacent to the RPM's. I think that it's close enough for your purposes, but some additional logic could be incorporate to make them always match.

One of the files forces the Time indicator at the bottom of the graph to also match RPM. If you don't care about the time indicator (you could just add the Time field to the graph), then use the "No-Zero-Start" file.

12-134 - Power Comparison.png (20.06 KiB) Viewed 2480 times

Notice that I manually set the Min/Max values for the fields to 0/400 so that they would be directly comparable.


The way that I achieved this:

-Remove the trailing rows where RPMs were decreasing on each of the two original spreadsheets.
-Combine the two header rows into one row for each of the two spreadsheets.
-Add either 127 or 134 to the end of each column name to differentiate them.
-Rename the second RPM column (the files had two duplicate RPM columns) to "Time"
-Add .127 or .134 to the end of every Time value so that there are no duplicates between the two spreadsheets and you know which spreadsheet the value came from.
-Combine the two files by adding the 134 columns after the 127 columns, and starting the 134 rows below the 127 rows (but under the correct 134 columns).
-Move the 134 "Time" column to be under the 127 column, then moving the entire Time column to be the first column.
-Sort the entire combined spreadsheet by Time.
-Fill in the empty cells between rows by averaging them.
-For empty cells at the top and bottom, just copy the adjacent row into them.
-Add a row at the top with all 0's so that it forces MLV to display the Time as is in the CSV so that Time=RPM. Otherwise, it takes the first row and sets that as 0 and all other Times are a differential, not actual.
-Export to CSV.


It sounds like a lot of steps, but it's actually pretty quick and easy to execute. I wrote formulas for the column names, time field modifications, and empty cell averaging. You could create a spreadsheet with formulas to do everything, but I figured a one off was good enough.

[ShirtUser]

If you're comfortable with Excel, you could just create a graph with both data sets on it and not have to go through all of the extra steps to get it into MLV.

[ShirtUser]

Since you want to integrate your MSL data, the easiest way would be to copy the RPM field in the MSL to a new Time field (so you'll need to convert it to a CSV and remove the existing Time field), and then trim the rows that are above and below the dyno run so you have no sequences with decreasing Time/RPM.

At that point, you'll be able to combine the modified MSL with your dyno log the same way that I combined your two dyno logs (adding columns and rows so there is no overlap), sort by Time (which is really RPM) and then fill in the blank cells with averages. Since you'll just be combining one dyno run with one MSL, there is no need to add anything to the Time field to differentiate them. Maybe you'll get lucky and some of the Time fields will be identical, reducing the need for averaging for empty cells.

Then, you can load it in MLV and see your dyno graph along side the log data from your MSL.

[jsiedlicki]

Wow! Thanks for the hard work. This is what I was looking for. It takes a bit more data manipulation than I wanted but it gets me there. I'm trying to save expensive dyno time. The next time I'm there I need to analyze runs from 0-50 degree intake cam advances and a Hi/Low cam lift profile all at the same time, 20+ runs back to back. I don't want to spend a lot of time manipulating and reviewing data and graphs as I get charged by the hour whether or not the dyno is running. I plan to write a program to align all the runs and spit out the best cam angle and lift combination for the entire rpm range. Then I can quickly reset the VVT intake table and the Hi/Low cam profile logic for the optimum run and move on to other tuning tasks.

...
One of the files forces the Time indicator at the bottom of the graph to also match RPM. If you don't care about the time indicator (you could just add the Time field to the graph), then use the "No-Zero-Start" file.

These two files are identical. I assume you meant to remove the first line of zero entries in the "No-Zero-Start" file.

[ShirtUser]

Given your parameters, I would write a script to do what I did manually. I would set it up to take any files dropped into a specific folder, combine them, move the original files to a new sub-folder in an archive folder, and place the final result into another folder and give the file a name that includes the date/time/seconds as well as the sub-folder's name. So, your on dyno process would be...drop the dyno files into a folder, run the script, open the result. The file name would help you keep track of the order of progression, as well as identify the original files if need be.

If you drop a MS log file in the conversion folder, the script should be able to handle adding it as well. You can tell it not to add suffixes to the MS log file's columns based on the MS log file name or on finding a specific column name. This gives you the ability to review the results side by side as needed. And to create different combinations of runs on the fly.

It would be easy to add a routine to log the power output (torque, presumably) results into a table with RPM range rows and power output columns. Dump each set of results to flat file that identifies the combination in the name and the body so the results aren't lost between script executions. Then a final program to take all of the results from the flat files and select the highest output from each RPM range and build a list with the combinations that gave the highest output. Formatted so it's easy to cut and paste into TunerStudio. Then manually adjust the results for smoothness.

With a bunch of tunes preconfigured and ready to load, and the analysis scripts, you could run through all of the dyno runs back to back, run the scripts in a few seconds, and have the output ready to load into TS. Manually smooth it, do a couple of confirmation dyno runs, and move on. Theoretically.

I'm sure you've already worked through the above. It's an interesting challenge to automate the analysis to make best use of the time. With the caveat that manually review and dyno confirmation are needed. I would be interested in hearing what you come up with and how it works out.

What language will you be writing the program in?

[jsiedlicki]

...
I'm sure you've already worked through the above. It's an interesting challenge to automate the analysis to make best use of the time. With the caveat that manually review and dyno confirmation are needed. I would be interested in hearing what you come up with and how it works out.

What language will you be writing the program in?

I have been prototyping it out in Excel and MLVHD but with the amount of manipulation and files involved I have begun writing it in the Ruby scripting language. I'm parsing both the dyno logs and MSL logs and storing them in a back-end database for easy access and manipulation later in the process. Once I get the kinks worked out I will move on to interpolating the optimum values and generating output files that TS can import.

It's actually a fun project to work on while waiting for the next dyno session.

[ShirtUser]

For simple text ETLs like this, I like basic scripting for portability purposes, but if you don't mind running MySQL or PostgreSQL or something like them on your laptop so you can use it at the dyno, your approach will be very robust and flexible. And easy to build on for future needs.

I agree, challenges like this are a lot of fun

Good luck and let us know how it works out!