E46 Can bus project.

[e30polak]

Great work on this Thaniel! How much more of a monster is the E46 M3 cluster to work with in comparison to the non-M cluster?

[e30polak]

Double tap

[SeanCollins]

PIDs. Think I may have talked about them before. I was about to write a message about how they a bad idea or won't work......But I got to thinking more. For the E46 side yes they don't work but the LSx side (providing the ECU is capable) it might work.

The drawback with PID's is that each value wanted has to be asked for. Each and every time. And the ECU response is specific to the request. So this is all additional traffic on the bus. And the messages are low priority. So if the bus gets busy there will be lag or delays. When just listening to the modules anounce their information causes no additional traffic.

But on the LSx engine in a BMW what else is it talking to?....nothing. So maxing out the bus bandwidth is not an issue. Just talking out loud let's do a short pro con list.

Using PID's

Con:
Must request the information from the ECU instead of just Listen.
Could cause lag and the guages may not respond quick enough (this worries me).
More Bus traffic (probably a non issue)
More processing for the Arduino (probably not an issue)
This seems somehow like the less elegant way to interface.
Can not send Can Bus specific information to imitate a missing modules

PRO:
PID's have standards for OBDII readers. Once set up the device could work on any ECU that uses the same protcol.
The programming to read PID's already exists.

Ok. So simple sometimes is better. The Can bus sheild I have had a free software for the Arduino that actually Sends PID's and grabs responses. I've already tested this with my Reno laguna and it works fine. To make it work for the LS1 would be no big deal.

Problem with this is I need 2 can bus sheilds (or the equivalent can chips) to do any testing. Which I don't have. I cold look at the programming side but someone would need to test it.

Commenst or thoughts anyone?

Thaniel

Well, your stuck with pids, when GM dropped class 2 and went to c.a.n. networks "I am so glad they did" they moved to pids, now a interesting thing about GM ecu's, they regularly broadcast for the cluster a ton of messages, things like RPM, oil pressure, coolant tempature etc, when GM went all c.a.n. somewhere around 2007, they went all in.

polling pids is very easy, and your not likely to max out bus bandwidth, iirc its in the mb/s range.

[siado]

Thaniel, I can't thank you enough for sending me your code.

I have my shield set up and I'm trying to connect to my car. Nothing useful for you guys unfortunately as I traded my E46 for an E60. :/

I have an SD error however. I'm using an Arduino Mega, could that have anything to do with it? I'm going to comment out the SD code and see if that helps.

[Pzary3233]

You were mentioning in your other thread about some parts you may need.... what are you looking for?

[Thaniel]

Great work on this Thaniel! How much more of a monster is the E46 M3 cluster to work with in comparison to the non-M cluster?

I'd expect the rpm and temp and check engine light to function the same. So no changes required for those to work. The mpg is replaced with an oil guage (temp or pressure?). To get that guage to work would just require finding the right can code and some adjustments to the program.

Well, your stuck with pids, when GM dropped class 2 and went to c.a.n. networks "I am so glad they did" they moved to pids, now a interesting thing about GM ecu's, they regularly broadcast for the cluster a ton of messages, things like RPM, oil pressure, coolant tempature etc, when GM went all c.a.n. somewhere around 2007, they went all in.

polling pids is very easy, and your not likely to max out bus bandwidth, iirc its in the mb/s range.

I have a difficult time believing the abs computer gets the throttle position or other data from the ECU using PIDs. It is not how the can bus system is designed. But as you say polling the pids is easy and could make development easy. I'll try to look at finding a way to get another set of can chips. But my Swedish is not good so shopping for odd items here can be difficult

Edit: just bought the 2 chips to make a second can interface. Shipping from thailand so will be a few weeks. I picture driving around in my laguna with an e46 cluster being driven by the can bus :-).

I have an SD error however. I'm using an Arduino Mega, could that have anything to do with it? I'm going to comment out the SD code and see if that helps.

The mega could have the control pins on differnt pins than the UNO. It's no big deal. Just have to change it in the program. But first check that the sd card is formatted to the FAT file system. Then use some of the example arduino sd programs to make sure it works.

You were mentioning in your other thread about some parts you may need.... what are you looking for?

Thanks for your interest. I put some info in the other thread now.

To give you guys an idea of what I do to get stuff. Swedens prices are very high on things (Sweden has a very socalist goverment. They seem quite happy with it but it does make prices high. Especially anything to do with cars) so not finding deals on anything. But the UK has some decent deals on things and I read the language much better but they tend to not want to ship out of country or charge way too much (because they don't want to). I bought my instrument cluster from the UK and paid the high shipping. I had to go to the UK again and so I bought some parts off eBay that were a good deal and had them shipped to our uk office. I'm headed back today with them in my suitcase

Thaniel

[SeanCollins]

I have a difficult time believing the abs computer gets the throttle position or other data from the ECU using PIDs. It is not how the can bus system is designed. But as you say polling the pids is easy and could make development easy. I'll try to look at finding a way to get another set of can chips. But my Swedish is not good so shopping for odd items here can be difficult

Edit: just bought the 2 chips to make a second can interface. Shipping from thailand so will be a few weeks. I picture driving around in my laguna with an e46 cluster being driven by the can bus :-).




Thaniel

Why so hard to belive ? they are sending a ton of ecu and bcm pcm data all over the car on the bus. Belive it, GM is doing 99% of module communication on can. If you can make can work for you, the obd2.5 compliant ecu's will make life simple, since they standardized a ton of the protocol. I will continue my mining efforts however.

[Thaniel]

Hello Everyone. Wanted to update earlier but couldn't get on the board. We'll see how much I remember still :-)

As mentioned I ordered the needed chips (MCP2551 and MCP2515) to create another interface with a Can Bus and ardruino. Plan is to construct one that can capture can data from a can bus on a LSx or whatever and pass to the current hardware to send the messages to the E46. So going to build a circuit like this one http://fazjaxton.net/products/fazcan...v2.1-schem.pdf (only the top portion). While waiting for the chips to come in I did some web surfing to see if others had done to interface with the Arduino and send PID's instead of just monitor.

Found that on ecomodder.com forum they have used the arduino to capture the injector pulses and speed pulses to make some MPG guages (sound familiar) and then later created using a ELM327 or ELM327 clone an OBDII interface to capture the data using PID's. In many ways similar to what we are capturing or want to capture. So I've downloaded their arduino programs and are looking through them. I find it ironic though to be looking at the ecomodder forum to get tips to help us put in HUGE engines into our cars Let's not mention it to them. Here is a link to the OBDII version summary http://en.wikipedia.org/wiki/OBDuino

In looking at how they use the hardware and capturing the messags my brain finally caught on to what now seem very obvious facts. Perhaps you guys have this figured out already. The hardware for polling the ECU with PID's or for streaming is esentially the same. PID's which are used by OBDII scanners. Many of them are based on the ELM327 chip or a clone of the ELM327. In reading I was noticing the protocol for the GM cars and thought. That looks the same as the GM Class 2 serial specifications I want to interface with and was wondering how to do it. And then remembered how I'd use my OBDII interface with my LS1 on my LS1 E36 to read codes and Live data. It finally hit me. Duh it's the same wire and same protocol. Only difference is what commands are sent on it. Seems so obvious now.

I already have my car wired with an OBDII port in the glove box. To capture the streaming data from the LS1 computer I just have to plug in my OBDII scanner and instead of using the OBDII scanner software I can use a terminal program and give the OBDII scanner a command to display all the traffic it sees. Simple. But I don't have my LS1 handy to try it out. So I hook the OBDII scanner up to my instrument cluster with my can bus shield still attached. Use a terminal program (hyperterm) Set the baudrate etc. Send a command to the code reader to set the protocol and to show headers and then montior all (ATMA). Volia. Like majic a stream of can bus data comes on the PC screen. And I check and it is the same data I'm sending to the cluster from my can sheild and the few responses from the instrument cluster. Sweet

OK. "listening" to the bus is quite easy and can be done with a cheap OBDII scanner that uses ELM327 commands. Most OBDII scanners are set up to handle multiple protocols so it can listen to the canbus stream or the Class 2 serial data or ISO stream etc. Quite handy. Those of you with OBDII scanners should give it a try.

Settings for the hyper term to talk to a ELM327 or ELM327 clone
Baud rate 38400 (or 9600)
8 data bits
no parity bits
1 stop bit
May need to mess with carrage returns and echo settings to get the data to look right on the screen.

Send command ATZ and the code reader should reset and respond with the version of the reader. Then type AT SP2 to set the protocol to SAE j1850 VPW (LS1 class 2 serial). or ATSP6 (ISO 15765 can 11 bit 500kbps, e46 and newer LSx with can bus). Then type ATH1 to turn on headers to see the message ID's. Then type ATMA to monitor all. If the response to any of the commands is ? then try again.

So great fun. But we don't want to spit data out on a PC's screen we want it in the Arduino so it can drive the guages. We can either hack the OBDII scanner to wire it to the Arduino or build a circuit with the correct chips to interface with our device (like what my original plan is at the top). Either way will work and we can make a "digital to digital" version of the instrument cluster driver. Just have to wade through a bunch of code to make it happen.

Why so hard to belive ? they are sending a ton of ecu and bcm pcm data all over the car on the bus. Belive it, GM is doing 99% of module communication on can. If you can make can work for you, the obd2.5 compliant ecu's will make life simple, since they standardized a ton of the protocol. I will continue my mining efforts however.

It's not that I don't believe they use Can bus. I do believe they send the messages over the bus. My point is that PID's are for diagnostics. The way I understand it, during normal comunication between modules on the can bus the messages are broadcast and not asked for (PID's not used during normal use). Any module that want's the information just listens for the broadcast and 2 or more modules can recieve the same data at the same time. With PID's one message asked for, by a specific module, and one message sent, to a specific module. Great for diagnostics and may be adequate for our purposes. The same information should be available by just listening to the broadcast.

[SeanCollins]

It's not that I don't believe they use Can bus. I do believe they send the messages over the bus. My point is that PID's are for diagnostics. The way I understand it, during normal comunication between modules on the can bus the messages are broadcast and not asked for (PID's not used during normal use). Any module that want's the information just listens for the broadcast and 2 or more modules can recieve the same data at the same time. With PID's one message asked for, by a specific module, and one message sent, to a specific module. Great for diagnostics and may be adequate for our purposes. The same information should be available by just listening to the broadcast.

a pid

proportional
integral
derivative

is really not a proper name for the data packets. They are simply,. data packets, and they broadcast with a specific ID, I can likely get the message id stuff relatively easily, but beyond that, its gonna get hard. Pids are fine, request away, they will live update, worst part is intializing communication at power on, after that it is relatively straight forward.

I had a pdf with the obd2 c.a.n. generic data list and id's as mandated by the epa, but I can't find it right now.

[Thaniel]

To give you guys an idea of what I do to get stuff. Swedens prices are very high on things (Sweden has a very socalist goverment. They seem quite happy with it but it does make prices high. Especially anything to do with cars) so not finding deals on anything. But the UK has some decent deals on things and I read the language much better but they tend to not want to ship out of country or charge way too much (because they don't want to). I bought my instrument cluster from the UK and paid the high shipping. I had to go to the UK again and so I bought some parts off eBay that were a good deal and had them shipped to our uk office. I'm headed back today with them in my suitcase

Forgot to mention this. The pedals and shifter went over with Londons airport security about as good as carrying a lb of brown sugar did. (emptied everything out, put through the X-ray separately etc). But I made it home with them.

My son also found a site for electrical components in Sweden that had an English language option. Was going great until I went to check out and the whole thing switched to Swedish. With my sons Swedish help we finished the transaction though. Hopefully a few more needed components will arrive before long.

Thaniel

[SeanCollins]

Thaniel, I got a sneak peak at confidential can bus data stuff, umm, yeah, just go with pids, none of it is standardized across any of the manufacturers. Use the obd2 diagnostic data monitoring functions, preferably I'd stick with generic data and request at high speed.

[Thaniel]

Hello. Did a little more playing with the ELM 327 clone. The one I have actually uses a PIC18F2XK80 chip. It has a MCP2551 chip for interfacing with can bus and other hardware for interfacing with the ISO and VPW etc protocols. Unfortunately USB has become more popular that the standard serial port. The ardruino doesn't comunicate easily to other micro controllers through the USB (I can't find anyone doing anyway). So I opened up the box to connect into the PIC18F2XK80 chip Rx and TX lines between the chip and the USB controller.

IMG_5548.jpg

Some really small solders needed to be made for the Rx and TX line (the white lead with the tiny read and green leads out of them). I Feel lucky that I got it right. Even on the first try. The blue and red wires in the top of the picture are to supply 5V for the board and a common ground.
IMG_0865.jpg

Back in it's original box (silver box in the pic) and connected to the arduino. Using the leonardo at the moment as the Uno is connected to the can bus shield and converting input signals to can so the ELM327 clone has some thing to read. What you can't see is on the laptop there is a stream of can bus messages coming through the arduino Leonardo which came through the ELM327. The ELM327 is not directly connected to the PC. The messages include the ones sent by the can shield and ones from the cluster itself. You can see the OBD rx light lit and USB (serial port now) TX light lit indicating it's streaming messages in and out.
IMG_0867.jpg

The PIC18F2XK80 uses mostly the same commands as an older version ELM327. The documentation for the ELM327 is pretty good. http://elmelectronics.com/DSheets/ELM327DS.pdf The information around page 28 I found particularly interesting. Page 30 states how to get an RPM using PID's. And page 61 has some good information about how fast one can interact with the ECU using PID's. A topic I have much debated here.

The most important statement for 2002 and earlier LS1's is this statement on page 30. "Prior to APR2002 release of the J1979 standard, sending J1850 requests more frequently than every 100 msec was forbidden." So that means only 10 quires (PID's) per second could be sent. And using PID's the device needs to wait for a reply as well So let's say 5 messages per second transmitted to leave time to recieve the response. And we need temp, RPM, fuel consumed and MIL status. Each has to be sent by seperate request using PID's. IF we update each one that means it will be updated roughly once per second. Even at double that speed, Twice per second, it is WAY too slow for RPM and MPG instrumentation. This has been my experience with my OBDII code reader with my LS1. Using PID's on the class2 bus is not a good solution. But we should still be able to monitor the class 2 bus using the ELM327 hardware (just listen not send) and get the data at a faster rate.

For Can Bus the PID's can be sent faster. But how much faster can it or will it go? The E46 instrument cluster does not respond to PID's. Well it does it says "no data". I can test and see with my rental car (I already have a PID sending program for the can bus shield). But it's cold and snowy outside.

Thaniel

[Thaniel]

A short update.

After looking through the OBDuino code there is a lot going on in that code that is unnesessary for our purposes. I spent a few hours working to strip it down to the basics. In the end I pushed it to the side. The code in this library http://sourceforge.net/projects/obdcon/?source=navbar is more easily converted.

I made some alterations to code to use Software serial inputs instead of the Pin 0 and 1 ports. This enables the USB interface to still be utilized which makes debugging and code modification much easier IMO.

Now have a working framework of a program that initializes the ELM327 and starts comunication then can send and recieves PID's. After initialization it could also just listen to the data stream and look for specific commands. The nice feature about this setup is the basic code framework and hardware can be used for a variety of bus protocols (VPW, Can, Iso etc.) with little to no code modification.

Still have to flush out some details and test on a car. Then I'll combine with the previous output program and do a can to can demonstration. More to come. :-)

Thaniel

[Thaniel]

Had to travel again for work this week and also caught a bad cold. So didn't do much this week. But did take some time to see if I could get the ELM device to read the K-bus. The standard baud rate for OBDII messages isn't the same as the E46 K-bus but the baud rate on the ELM iso connection can be configured. After setting the baud rate and turning on headers (to see the full message) I was able to capture a data stream. Sadly the formatting for the OBDII messages and the K-bus messages are not the same. So the ELM device sees the entire message as just headers to a message with no data packets. Or in other words. It takes one message and breaks it into 3 or 4 lines of headers (2-3 bytes each). instead of a complete message of say 7 bytes.

This wouldn't be so bad as the message could be put back together in software except it thinks one of the bytes, per line it displays, is the CRC and doesn't display it. I've found no way to get it to just pass the entire data without trying to decode it. So looks like reading the K-bus data with the ELM doesn't work. Not a big deal as a K-bus interface can be as simple as a few transistors and resistors. but would have been nice to have a do it all device.

Back to looking at can bus.

Thaniel

[02PRUV]

Wow this is a lot of work. Credit to you. Wouldn't it just be easier to leave the dme in the car and use it as your can bus interface though? Would save lots of work.

[Thaniel]

Wow this is a lot of work. Credit to you. .

Thanks. It's aways good to get a compliment.

Wouldn't it just be easier to leave the dme in the car and use it as your can bus interface though? Would save lots of work.

The easiest thing to do would be to leave the BMW motor in Leaving the ECU in is an option that's been done and the pro's anc cons discussed in other threads. This project is for those that want to take the ECU out and not have to deal with it anymore.

There are also benefits to having this information and skill available for other things as well. Easy data logging. Modules that can automatically turn off the MIL or codes, Mask or emulate certain perameters from a diagnostic tool etc. Lots of fun could be had.

[02PRUV]

Thanks. It's aways good to get a compliment.


The easiest thing to do would be to leave the BMW motor in Leaving the ECU in is an option that's been done and the pro's anc cons discussed in other threads. This project is for those that want to take the ECU out and not have to deal with it anymore.

There are also benefits to having this information and skill available for other things as well. Easy data logging. Modules that can automatically turn off the MIL or codes, Mask or emulate certain perameters from a diagnostic tool etc. Lots of fun could be had.

Ah ok far enough. I haven't seen it discussed in other threads.

[Thaniel]

More Can bus fun. Spent way to long on this one but that's how it goes. Trying to reuse others programs can be difficult. So I ended up basically rewriting the ELM327 inteface program. But I finally got it together for a Demo to show if it can be done.

Here a pic and video of it hooked up my rental car (laguna renault) hooked to the E46 instrument cluster. This car has can bus at the OBDII socket so I connected the ELM327 to the socket. Prompted by the Arduino the ELM327 sends the requests to the ECU using OBDII style PID's and then collects the message. The Arduino collects the response from the ELM327. Then converts the response to BMW can bus messages and sends out via the Arduino can bus shield (MCP2515 and MCP2551 chips). The temp sending is a bit slow during this video due to the extra steps required to gather the info from the 327 and the IKE gets bored waiting for temp messages. So it turns off the needle (makes it go full hot). Could be corrected by reducing the delay time in the program but with all the wires dangling I didn't feel like messing with it.

(The service engine is light due to it's time to take it back to Hertz for service on Friday)
Photo:
IKE in Laguna.jpg
Video


In this car anyway the RPM gathering seems to be quick enough. Infact I think the E46 RPM moves quicker than the Laguna. So it would be a workable solution. Can't test the MPG as it needs a speed input to the cluster which is not can bus. The program would just need a little polish but would be better done on a car with the cluster (and new motor) installed. If anyone want's to pursue this further I can offer assistance.

Related to this; my son's been working on getting the new MCP2515 and MCP2551 chips to comunicate on the bus like the premade sheild (same chips in the can bus shield). Another solution for can to can would be to use the MCP chips instead of the ELM. But the ELM does work and was about the same cost as the extra chips.

It for now.

Thaniel

[bimerok]

Amazing work!

It does look like e46 RPMs are faster to respond then Laguna's.

Also, it is funny to see the interior of the car and passenger seat occupant all covered up in wiring and modules

[V8Z4]

When will you be back in the states for good (so you can start your swap)?

[rao]

More Can bus fun. Spent way to long on this one but that's how it goes. Trying to reuse others programs can be difficult. So I ended up basically rewriting the ELM327 inteface program. But I finally got it together for a Demo to show if it can be done.

Here a pic and video of it hooked up my rental car (laguna renault) hooked to the E46 instrument cluster. This car has can bus at the OBDII socket so I connected the ELM327 to the socket. Prompted by the Arduino the ELM327 sends the requests to the ECU using OBDII style PID's and then collects the message. The Arduino collects the response from the ELM327. Then converts the response to BMW can bus messages and sends out via the Arduino can bus shield (MCP2515 and MCP2551 chips). The temp sending is a bit slow during this video due to the extra steps required to gather the info from the 327 and the IKE gets bored waiting for temp messages. So it turns off the needle (makes it go full hot). Could be corrected by reducing the delay time in the program but with all the wires dangling I didn't feel like messing with it.

(The service engine is light due to it's time to take it back to Hertz for service on Friday)
Photo:

In this car anyway the RPM gathering seems to be quick enough. Infact I think the E46 RPM moves quicker than the Laguna. So it would be a workable solution. Can't test the MPG as it needs a speed input to the cluster which is not can bus. The program would just need a little polish but would be better done on a car with the cluster (and new motor) installed. If anyone want's to pursue this further I can offer assistance.

Related to this; my son's been working on getting the new MCP2515 and MCP2551 chips to comunicate on the bus like the premade sheild (same chips in the can bus shield). Another solution for can to can would be to use the MCP chips instead of the ELM. But the ELM does work and was about the same cost as the extra chips.

It for now.

Thaniel

Continued great work!

[zr6sbc]

Nice work. I look forward to tagging along once my project is resurrected. Your passenger looks a bit like a hostage.

[Thaniel]

Thanks guys. Yah there are a few chords running around. I'm thinking I should mount the cluster and some key connections to a board so it's easier to move around. In theory with just one statement change in the program it would run of the LS1 VPW OBDII commnds.

When will you be back in the states for good (so you can start your swap)?

I'll return to USA end of June. When i'm back I'm betting the programming will take a back seat to other projects. So get your requests in early :-)

Nice work. I look forward to tagging along once my project is resurrected.

I'd be interested in any findings you would have trying the programs out on your car.

Your passenger looks a bit like a hostage.

Yah it does look pretty funny. The modules on his lap he brough with him and he has done quite a bit of the programming. Pretty good start for a 13yr old I think. Speaking of which. Garrett went through and removed the library calls to the Can bus and MCP2515 by including them into the program on tabs. Many programers will scream at this. However to use 2 sets of MCP2515 and MCP2551 chips we would have to modify the libraries. And modifying library files is a pain not only for us but for anyone else trying to use our programs. So now it's all combined and no one has to wonder if they have the right library file etc.

Attached is the program run on the Laguna can to E46 can test video.
Thaniel

[mktech]

Hi !
I've got question to thread author. Where from You get original frame names (like 0x153 = ASC1, 0x545 = DME4) ?

[Thaniel]

Hi !
I've got question to thread author. Where from You get original frame names (like 0x153 = ASC1, 0x545 = DME4) ?

Hello mktech.

Thank you for the question. Let me try to explain how the information was uncovered.

The first portion of the name 0x153 is the arbitration ID which is gathered from listening to the E46 can bus. The "0x" means the number is in hex format. The 153 is the can bus message ID. The "ASC1", "DME4" notation was found in the Error codes in the IKE. Using a PA soft type scanner the errors can be read from the IKE. Below is a screen shot of an error screen.

IKE_error.JPG

In the screen shot it states "No CAN message (DME1)" by sending different can bus messages then resetting and rereading the errors I could match which Arbitration ID matched which module. "DME1" in this example. (DME1 and DME2 are both from the same DME just it sends messages using two different ARBID's.)

If this does not answer your question feel free to ask more.


Update:
Spent some time working on making the bare MCP2515 and MCP2551 chips communicate with the instrument cluster. Wired it up and it should have worked but nothing was happening. Electronics hardware is not my strong point. So I start trying different connections and in the process bump a wire to where it shouldn't be and fried the 2551. They are cheap so I ordered another one. Well 10 more just incase I fry another one. Shipping was about as much as the chips and a 10 pack was only a few $ more.

In the mean time my Son whipped up a program so we could read the registers in the MCP2515. We were able to communicate with it just fine. But then noticed soon as we brought on another module (IKE or our Pulse generator) we would loose connection. After a ton of trouble shooting found that connecting the reset line to the Arduino solved the problem. Imagine that. Hooking up ALL the wires is required. If I hadn't fried that chip we'd be good to go with this configuration. It doesn't really give us anything new but has some advantages and disadvantages over the ELM.

Thaniel