ElectricMotorcycleForum.com

  • April 20, 2024, 09:47:08 PM
  • Welcome, Guest
Please login or register.

Login with username, password and session length
Advanced search  

News:

Electric Motorcycle Forum is live!

Pages: 1 2 [3] 4 5 ... 7

Author Topic: Log file parser utility  (Read 10938 times)

Kocho

  • Hero Member
  • *****
  • Posts: 552
    • View Profile
Re: Log file parser utility
« Reply #30 on: May 05, 2016, 07:17:46 PM »

It does not seem to work this way. Just downloaded the "bike" logs 3 times in a row and each time I get the same 269KB or so file with the same contents each time. Each time the same 3 months of data are covered - 10/2015 - 12/2015. It is hard to make heads or tails from the decoded logs too - looks like at some point some external chargers were connected to the bike, that at some point the battery hi-temp protection kicked-in, that it has been ridden hard to a point where the overheat protection had kicked-in, etc. And "PL536 Read Attempts Failed" occurs multiple times, suggesting some issue with the TI BMS unit (or what it is measuring).

I got an appointment to get the bike over to the dealer, probably next week, just curious to learn more myself and possibly give them hints on what to look at/for.

Oh, and the log parser seems to get confused by the heading info (model and firmware revision section at the top of the log) for the BMS logs but is fine for the Bike logs.

The parser gives me some errors in part-way through parsing the BMS logs too, so something is possibly not right with the parser too. But I am looking at the .bin files directly for the above-mentioned dates.

Oh, sorry, just keep exporting and emailing. I have a suspicion that the bike continues from the last time the logs were exported up to 128k of data. Worth trying and do please let us know if you get more recent entries next time.
Logged
'15 Zero SR

Kocho

  • Hero Member
  • *****
  • Posts: 552
    • View Profile
Re: Log file parser utility
« Reply #31 on: May 05, 2016, 07:19:40 PM »

Oh, and whether I choose to download the "Bike log" only or the "Bike and BMS" logs from the Zero app, in both cases I get the same size file, even though it takes 10 times longer to download from the bike to the phone the version with the BMS...
« Last Edit: May 06, 2016, 03:38:30 AM by Kocho »
Logged
'15 Zero SR

MostlyBonkers

  • Hero Member
  • *****
  • Posts: 1323
    • View Profile
Re: Log file parser utility
« Reply #32 on: May 06, 2016, 03:36:22 AM »

It's not perfect. Phreak mentioned an issue with the parser that sometimes results in just receiving the header in the output and no log entries. I wonder, how big are the output files? It makes me think that the utility is exiting prematurely for some reason. The data might be exported, but not all of it is being converted.

Try sending the logs to Zero and ask your dealer to send you the output. Zero should send the results to the dealer. They might say no, but it's worth asking. Don't ask zero for the output directly because they'll definitely say no.  Make sure you email yourself at the same time and keep the bin files. Just in case you do get the output, post the bin files and Zero's output here. At some stage in the future, either Phreak or someone else with coding skills might be able to make use of them to spot and fix some bugs.  Time is the big issue. I'm amazed that Phreak got so far with his utility.

Otherwise, just don't bother and leave it to the dealer. :-)
Logged

ges

  • Newbie
  • *
  • Posts: 4
    • View Profile
Re: Log file parser utility
« Reply #33 on: May 14, 2016, 02:13:39 AM »

I've tried the parser, but unfortunately I only get the header with Serial number, VIN, Firmware rev., Board rev. and Model. The parser is failing with the following information:

4172 entries found
Traceback (most recent call last):
  File "zero_log_parser.py", line 257, in <module>
    parse_log(log_file, output_file)
  File "zero_log_parser.py", line 229, in parse_log
    (length, entry) = parse_entry(log, read_pos)
  File "zero_log_parser.py", line 78, in parse_entry
    raise ValueError('Invalid entry header byte')
ValueError: Invalid entry header byte
Logged

MostlyBonkers

  • Hero Member
  • *****
  • Posts: 1323
    • View Profile
Re: Log file parser utility
« Reply #34 on: May 15, 2016, 02:00:39 AM »

I've seen that a couple of times ges. Just try extracting the logs again, possibly after some more activity.
Logged

ges

  • Newbie
  • *
  • Posts: 4
    • View Profile
Re: Log file parser utility
« Reply #35 on: June 04, 2016, 12:12:41 AM »

Tried the parsing again today but still struggle with "ValueError: Invalid entry header byte" from the BMS file.
Logged

PhreaK

  • Full Member
  • ***
  • Posts: 143
    • View Profile
Log file parser utility
« Reply #36 on: July 12, 2016, 10:25:35 PM »

Hi guys, haven't had a chance to post here / have a look at this for a while. It was summer so most spare time was spent out on a bike. That being said it's now back to crappy, cold, wet winter in my part of the world... so updates, yay!

Thanks to an awesome pull request from rDacted (who cracked an encoding used on the event entries) there's an updated parser that provides a bunch more useful info. It now also handles situations where you're log file wraps around the maximum log length and starts overwriting itself.

This is successfully decoding most of the sample logs I have on hand. I do have two that are breaking it though. Not sure if this is corruption of the logs themselves or something still missing in the log file format. If you happen try your decoding your log file and it does't work with the current version it'd be awesome if you can drop me a copy of the *.bin and when I have a few I'll see what I can do to unbreak it.

Oh also, I know a few people have been trying this on the BMS logs too. Currently this is untested and will likely (/almost certainly) not work.
« Last Edit: July 13, 2016, 04:35:46 AM by PhreaK »
Logged
2014 Zero SR

giacomo

  • Jr. Member
  • **
  • Posts: 93
    • View Profile
Re: Log file parser utility
« Reply #37 on: July 13, 2016, 04:56:10 AM »

Hello,  I  tried the latest code on my old logs but I get the following error =  type error: type str doesn't support the buffer API. I am using python 3.3. It looks like the data.find() search for string instead of bytes...
Any idea? thanks

Sent from my A0001 using Tapatalk

Logged

PhreaK

  • Full Member
  • ***
  • Posts: 143
    • View Profile
Re: Log file parser utility
« Reply #38 on: July 13, 2016, 11:33:23 AM »

Try that now. I was doing something stupid that python 2 let me get away with when it shouldn't have.
Logged
2014 Zero SR

BrianTRice@gmail.com

  • Unofficial Zero Manual Editor
  • Hero Member
  • *****
  • Posts: 4014
  • Nerdy Adventurer
    • View Profile
    • Personal site
Re: Log file parser utility
« Reply #39 on: July 13, 2016, 11:35:40 AM »

Thanks for keeping at this!
Logged
Current: 2020 DSR, 2012 Suzuki V-Strom
Former: 2016 DSR, 2013 DS

MostlyBonkers

  • Hero Member
  • *****
  • Posts: 1323
    • View Profile
Re: Log file parser utility
« Reply #40 on: July 13, 2016, 11:44:48 AM »

Thanks from me too. I've found it very useful on a number of occasions.  It puts power in the hands of the customer and helps keep everyone honest. It has also taught me a few things about my bike. :-)
Logged

giacomo

  • Jr. Member
  • **
  • Posts: 93
    • View Profile
Re: Log file parser utility
« Reply #41 on: July 19, 2016, 09:14:54 PM »

Try that now. I was doing something stupid that python 2 let me get away with when it shouldn't have.
it works like a champ now, thanks

Sent from my A0001 using Tapatalk

Logged

Swiebo

  • Newbie
  • *
  • Posts: 8
    • View Profile
Re: Log file parser utility
« Reply #42 on: August 25, 2016, 11:01:59 PM »

Tried to decode the log file of my DS, but got this error:

Code: [Select]
3371 entries found
Traceback (most recent call last):
  File "zero_log_parser.py", line 569, in <module>
    parse_log(log_file, output_file)
  File "zero_log_parser.py", line 531, in parse_log
    f.write('{0:18} {1}\n'.format(k, v))
  File "/usr/lib/python2.7/codecs.py", line 706, in write
    return self.writer.write(data)
  File "/usr/lib/python2.7/codecs.py", line 369, in write
    data, consumed = self.encode(object, self.errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xbd in position 19: ordinal not in range(128)

Logged

PhreaK

  • Full Member
  • ***
  • Posts: 143
    • View Profile
Re: Log file parser utility
« Reply #43 on: August 30, 2016, 05:44:10 PM »

Hey Swiebo, can you drop me that log file in a PM?
Logged
2014 Zero SR

kashography

  • Jr. Member
  • **
  • Posts: 96
    • View Profile
    • e-motorrad.ch
Re: Log file parser utility
« Reply #44 on: October 08, 2016, 03:39:44 AM »

Thanks a lot for this tool! With my first export it worked perfectly. for the second i get also this:
Code: [Select]
8483 entries found
Traceback (most recent call last):
  File "zero_log_parser.py", line 569, in <module>
    parse_log(log_file, output_file)
  File "zero_log_parser.py", line 541, in parse_log
    (length, entry) = parse_entry(event_log, read_pos)
  File "zero_log_parser.py", line 101, in parse_entry
    raise ValueError('Invalid entry header byte')
ValueError: Invalid entry header byte

i will try with the next export another time. If you need the file to debug: https://www.dropbox.com/s/0wvjtc1pnp0izxj/kw2.bin?dl=0
Logged
  • 200km with one charge (was too easy)
  • 300km in a day
  • 400km in a day
  • 500km in a day (550+km)
  • 600km in a day
    driven in [5] out of 6 nearby countries
Pages: 1 2 [3] 4 5 ... 7