Showing posts with label communication. Show all posts
Showing posts with label communication. Show all posts

Monday, July 21, 2014

Communication with embedded devices

It is always a hassle to communicate with embedded device. I do manage one time solution e.g. through USB to UART converter but it is always too much interleaved with application code to later switch to integrated USB peripheral.

Sometimes in the past I got requests to provide new or additional way of communication for device. Another use case I wrote this library for is when I found myself repeating writing same code. I had point to point serial communication implemented on some project. On another project I used serial bus approach (with CAN physical drivers - not important) and the communication was identical with addition of the byte header for destination address at the beginning of each packet. Repetition also happened on client and host side.

In short, Comm library offers a way to make communication entities modular thus easily replaceable. It also makes them stackable to make the whole through communication extending easy. First library implementation is in C [Google Code repository].

Communication is meant to be of  query - response from master (client) to slave (host). The use of library enforces a way of transmission and reception but the same design can be used on both sides of communication channel. This means that layer can have same implementation on client and host  side if both running on the same hardware platform.


From library point of view the communication entity is self sufficient communication layer implementing a set of predefined functions i.e. implementing an interface. As it turned out layers are of two types. In one stack there are more data formatting layers and single physical layer.

Physical layers are in charge of data storage and thus having communication buffer. Example of a layer on top of physical would be an encryption layer. Application interacts with Comm library functions when transferring data. Library then calls Comm layer implementation as needed. All functions have C++ style “this” pointer parameter. In the application accessible library functions this is a predefined Comm stack type holding a pointer to array of another predefined type instances each describing single Comm layer implementation. Comm layer instance is used as the "this" pointer in layer implementation functions and in library functions used for data transfer between layers. Comm layer instance has 5 pointers:
  • vague void pointer member to store reference to a custom type object for specific implementation
  • pointer to predefined type object holding function pointers of layer implementation
  • pointers to upper and lower layer if any
  • pointer to Comm stack to which layer belongs.

Instance custom pointer and pointer to functions are initialized in extra initialization function call which is custom for each specific layer implementation and additionally exposed to the application together with a custom instance type. This initialization function is not to be mistaken with layer implementation initialization function.
Pointers to surrounding layers and pointer to parent stack are initialized in library initialization call which should result in established connection after return.
Or everything again in objective language words: main communication stack object has methods for communication and holds references to layer objects each implementing the defined interface and custom init method which is only method called by application directly.
Below is a general sequence and internal states of the objects after each initialization step:

Initial state.

After first layer custom initialization call.

After last layer custom initialization call.

After Comm library initialization call.
Next to custom and implementation initialization method a Comm layer implements data transfer, event notification, and process methods.

Data transfer methods:

  • Send
    To send data in direction from application towards physical layer. Layer can optionally implement this method to transform data or add data to sent data. Additional data must be then in same fashion removed on receiving side in OnData call.

  • Store
    To save data in comm buffer. Called from OnData call back from upper layer. Usually only implemented in lowest, physical layer.

  • OnData
    To optionally unpack received data. Called from lower layer ReceiveProcess when new data received. Here is a chance to undo the changes to data made in Send and in PacketStart and in PacketEnd on transmitting side. If implemented it must forward remaining data with chained OnData call to upper layer or call Store if there is no upper layer like library does.

Event notifications:

  • PacketStart
    First call from application when sending a packet. Lower layers get this call first, upper last. Each layer can already put own header to transmitted packet with call to Send. This call gives layers a chance to clean up from previous application packet transfer. Send call(s) will follow.

  • PacketEnd
    Last call from application when sending a packet. Upper layers get this call first to append custom data to the packet with call to Send if needed. Physical layer can start actual transmission in this call.

  • OnNewPacket
    Call originates in physical layer ReceiveProcess method when start of new packet is detected. Here is a chance for layers to prepare for new packet. OnData call(s) will follow.

  • OnPacketEnd
    Called from physical layer ReceiveProcess  after complete packet was received.

Process methods:

  • TransmitProcess
    Called repeatedly from application after PacketEnd. Usually only implemented in physical layer. Returns Active status while data is being sent to medium and returns Idle status after all data is sent.

  • ReceiveProcess
    Called repeatedly from application to receive a packet. Usually only implemented in physical layer. Returns Idle status while there is no incoming data, optionally returns Active status while data is being received and returns NewPacket status after whole packet is received. Together with NewPacket status it returns the size of the packet and pointer to received data.

Flow of calls to send and receive the data with Comm library is as depicted below.

First call in client to send a packet to client; first call on host to send response.
PacketEnd behaves identically. Physical layer can start sending data in PacketEnd call.

Multiple calls to send data on client and host.

Multiple calls while transmitting data on client and host.

Repeatedly called on client while expecting a response from host; repeatedly called on host while waiting for new packet. OnPacketEnd behaves identically.

Repeatedly called on client while expecting a response from host; repeatedly called on host while waiting for new packet.
For test implementation I did a pair of embedded STM32F4 project and Windows client application communicating over USB. Windows side uses WinUSB library for communication and firmware reports a OS USB descriptor associating device with WinUSB driver so drivers or .inf files are not needed. Firmware code is in C and Windows client application uses C++ in Windows only code.

WinUSB communication with STM32 round 2 (Part 2: Client)


Previous client application is linked against WinUSB and SetupAPI libraries. This was a bit of a bother with WDK and project settings. They are loaded dynamically in new client test application. Two thin wrappers load the libraries and provide needed function calls with same prototypes as original library functions. SetupAPI is needed at low level for enumeration and when creating connection. WinUSB is further wrapped in an object representing USB device. And later wrapped in Comm interface explained in another post.
USB device object uses two USB pipes to transfer data which takes two USB endpoints in bulk mode at device side. One for sending data to device and other to receive it. Messaging control is done through USB control endpoint. Transfer sequence is optimized from previous implementation. Sending a complete application packet consists of sending a control message with the 32 bits length of the packet and then writing the packet data to bulk pipe. Receiving a packet is done with reading the return size through the control endpoint util nonzero length is returned and then reading the packet data from second endpoint. Application packet is meant to be a complete data set needed for single operation on device and can be larger than endpoint buffer.
Comm wrapper mentioned earlier is an implementation of the layer interface as specified by Comm library. Comm library core utilizes this interface to make transferring of data easier to application. WinUSB Comm layer is written to be the last layer in stack, also referred to as "physical" layer as is the closest to the physical medium on the master (client) side - Windows PC.
Application is written to test the connection and speed. Device is expected to return same data as sent by application. Speed is calculated from time it takes to send and receive back the packet.

Application uses Comm library methods to transfer the data. It is just that the Comm library object is initialized with comm layer using USB. It could instead be some other communication type, e.g. serial port or TCP/IP.

[Source files]

This is the second post in this series (of total 2 posts). See 1.

Tuesday, May 1, 2012

WinUSB communication with STM32 (Part 4: Software)


Time for some bulk communication. VS project that I added to sources shows steps which are needed for communication. Project uses the Setupapi for device enumeration and obtaining the handle to device. It further uses the winusb to communicate with device.

After obtaining the handle to device and initializing it there can be multiple communication cycles. The communication cycle has the following steps:

  1. reset the communication channel (control EP)
  2. send data (bulk EP)
  3. send end of data message (control EP)
  4. poll status until "READY" or "ERROR" (control EP)
  5. read data size of response (control EP)
  6. read response data (bulk EP).
The source is mostly copied from MSDN with some changes and additions. An improvement to the library would be an object type which would encapsulate all this functionality and expose nice interface to communicate with device. An alternative could be  libusb-win32 or libusbK or other USB libraries. I haven't checked these in detail. If interested check libwdi. There is some information on Windows 8 integrated support for devices like this (no driver needed for WCID devices!).

[Update - WinXP (SP2) and W7 support driver information descriptors; See my next project]

Test application output.

Check project wiki to get build instructions.

This is the forth post in this series (of total 4 posts). See 1, 2, 3.

[Update - I have a new WinUSB device project with STM32F407 without the need for Windows side driver info (.inf and .cat files) ]

Sunday, April 29, 2012

WinUSB communication with STM32 (Part 3: Driver)

WinUSB driver must be linked to device. This means creating a driver INF file specifying WinUSB as driver for device with specific USB VID and PID. To complicate a bit further this information needs to be signed. That is done by signing a catalog file (CAT) created from INF file. Windows Driver Kit (WDK) includes all the tools to do it.
Driver (code) signing was introduced to help prevent the installation of software that is not trusted. From what I understand about driver signing is that Windows trust some global certificate authorities (CA). They then issue a code signing certificates containing the publishers' information to software publishers. By this they are saying that the publisher was verified by them. They do it manually like call the person/company which issued a request for certificate and verify their information. For this they charge a yearly fee. The result is a class 3 certificate which publisher uses to sign his software. On installation/execution Windows pop up a dialog asking if you trust the publisher named as specified in signature of the executable which you wish to install/run. Publisher can also send his driver directly to Microsoft for testing and signing. It is called WHQL Testing. I'm not sure but I think that drivers tested and signed by MS are installed without any warnings. Read more about it on wiki. And I believe this isn't free either.
Free of charge alternative is an open source code signing certificate issued by same trusted CA (e.g. http://www.certum.eu). The requirement is that "Open source" keywords are stated in certificate information (e.g. the Name field of certificate is "<publisher name>. Open Source Developer")
Another free of charge alternative for debugging is so called self-signed certificate. This is a certificate created by developer. I created a script that:
  1. creates the certificates,
  2. copies the redistributables from WDK,
  3. creates usbID.h file containing the IDs for USB descriptor which must match the INF file,  
  4. creates the INF file,
  5. checks the INF file,
  6. creates the CAT file from INF file and
  7. signs the CAT file with created certificate
I did this rather than publish a driver ready to install because it depends on personal information. Well I could distribute my self-signed certificate but I don't own a USB vendor ID. And since you have to have your own in the INF file the signed driver is no use to you because you must change it.

USB Vendor ID is another story to this. If you are serious and plan to sell USB devices you should apply for USB VID. It will be assigned to you after one time purchase. To additionally use the USB logo on the device you must pay a yearly fee... I think.

The script actually creates two certificates. First is your CA certificate. Second is code signing certificate issued by your CA. I omitted the installation of created CA certificate to "Root trusted certificates" because it seems to make no difference. First I thought that there will be no warnings if I put the created CA certificate next to other trusted CAs. But the Windows still pop up even more annoying warning saying that publisher can not be verified. I also tried generating the certificates with OpenSSL with no success.


Anyway, driver installation completes successfully if you choose to install anyway. The device is then displayed in the Device Manager as shown below.


Automatic installation can be done with dpinst.exe toll (part of WDK redistributables). But I didn't deal with why dpinst warning pops up saying that the publisher of dpinst is unknown. Maybe because it was ran from script?


To see details of all operations see the script.

This is the third post in this series (of total 4 posts). See 1, 2, 4.

[Update - I have a new WinUSB device project with STM32F407 without the need for Windows side driver info (.inf and .cat files) ]

Tuesday, April 24, 2012

WinUSB communication with STM32 (Part 2: Firmware)

The sources are available at Google code project.

I decided to make a communication pipes implementation as library so that it can be used with other USB hardware. The library uses two bulk endpoints and control endpoint. One bulk endpoint is for receiving the data and the other for sending the data. The default control endpoint (EP0) is used for communication control.
User's code should repeatedly call single function in library which does nothing while no data is received and calls user's function when new packet arrives.

Larger communication buffers are better for faster communication. To improve it even more the USB implementation should use double buffered endpoints - not the case at the moment.

This is the second post in this series (of total 4 posts). See 1, 3, 4

[Update - I have a new WinUSB device project with STM32F407 without the need for Windows side driver info (.inf and .cat files) ]

Tuesday, April 17, 2012

WinUSB communication with STM32 (Part 1: Intro)

Aren't you fed up with putting a USB to serial converter in your embedded design while you have a USB capable device already in the design? Any other approach opens a can of worms though. Custom driver, certificate to sign it, USB vendor ID (2000$ at the moment), how to communicate with device on a layer above USB? But it is nice when OS pops up your name when your device is plugged in USB port. WinUSB is middle ground with tradeoffs one might consider.

I wanted to test the use of WinUSB with STM32F103 for some time now. When I finally got time I created a demo project. It shows basic request - response communication.
WinUSB kernel mode driver enables raw communication through USB pipes. In other words you can communicate with your USB device directly from application on the PC. So there is only code for application and code for firmware. Presuming you are not making standard USB class device like Mass Storage or HID. You might not even need custom PC side software in this case.
Furthermore the libraries to communicate are written for both ends to some extent. There SetupAPI and WinUSB on Windows side while ST has libraries for everything.
To get a taste check images below. I will post more about it in the future and probably publish the sources after I clean and organize them.

VS showing sent data.

winIDEA showing same received data on STM32.

This is the first post in this series (of total 4 posts). See  2, 3, 4.

[Update - I have a new WinUSB device project with STM32F407 without the need for Windows side driver info (.inf and .cat files) ]