WinUSB communication with STM32 round 2 (Part 1: Firmware)

Introduction

I did another WinUsb device. This time I've used STM32F4. Project in Google Code repository is for STM32F4DISCOVERY board featuring STM32F407VGT6.

I wanted to make it so that it wouldn't need Windows side driver information in a form of .inf file. I did it with Windows OS USB descriptors. They are described in subsection on MS "Dev Center - Hardware" Drivers page (links at the end of post). 

Result of device providing these descriptors is Windows loading one of MS provided drivers and associating the device with it as instructed in descriptor gotten from device when plugged in USB port.

"MS provided drivers" are not to be mistaken with "USB device class drivers included in Windows" which are drivers for external HDDs, webcams, HIDs...

WinUSB is one of MS provided drivers providing raw communication with device, e.g. WriteBuffer/ReadBuffer. This means that PC side application and firmware in device must follow some defined communication protocol. These are well defined for standard class USB devices like mouse and keyboard or external HDD but with WinUSB custom protocol can/must be defined. It can be quite simple like adding 4 byte header to transferred buffer holding the length of buffer.

[Update] Windows enumeration

While preparing PC side demo application to demonstrate communication I found out that categorization differs for single interface devices and composite multiple interface devices. Image above shows 2 devices in  device manager each plugged in own USB port and each with single USB interface associated with WinUSB driver. So does the image below with additional Properties window.

Single device with single WinUSB driver associated interface. 

But notice how composite device (having multiple USB interfaces) is shown on next image. Two WinUSB  associated interfaces were reported in descriptors by device:

Single device with two WinUSB driver associated interfaces.

Three devices are shown in Device Manager. One Composite device and two WinUSB devices - one for each interface. WinUSB devices have Composite device set as parent. Note how "Bus reported device description" differs for single interface device from dual interface device.

To test both configuration un/comment WINUSBCOMM_DUAL_INTERFACE_TEST macro in usbd_WinUSBComm.h file.

Windows OS USB descriptors

Text below refers to 'interfaces'. Term is used for a set of endpoints/pipes used in a group. Interfaces have own descriptors.

MS OS String Descriptor

When device is plugged in USB port for the first time on a Windows machine it also receives a standard USB string descriptor request requesting the string descriptor with index 0xEE among other requests during enumeration. For first few indices device responds with standard string descriptors which are Manufacturer, Product and Serial strings (also Language IDs for index 0). Device can ignore 0xEE string descriptor request or respond with data predefined by MS. If so Windows will then query for additional MS specific descriptors:

• Extended Compat ID OS Descriptor
• Extended Properties OS Descriptor
• others not relevant here

The content of MS OS String Descriptor must be:

Value	Field	Size in Bytes	Description
0x12,	bLength	1	Length of the descriptor
0x03,	bDescriptorType	1	Descriptor type - string
0x4D, 0x00, 0x53, 0x00, 0x46, 0x00, 0x54, 0x00, 0x31, 0x00, 0x30, 0x00, 0x30, 0x00,	qwSignature	14	“MSFT100” Signature field
MS_VendorCode,	bMS_VendorCode	1	Vendor-specific Vendor code
0x00	bPad	1	Pad field

MS_VendorCode is a value (defined as custom macro in project) which Windows will use to form requests for other descriptors. This gives device developer some leeway to add OS descriptors functionality without clashing with other custom USB requests which device might already support. For new and simple USB devices supporting only single WinUSB interface the vendor code is not that much relevant. Just make it non zero so that it won't cause problems on USB 3 hosts (see specification for more).

Note that Windows queries this descriptor only once. It can be a hassle during development. Information that OS descriptors have been queried for some device is stored in registry under

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\usbflags\VVVVPPPPRRRR
(VVVV - vendor ID; PPPP - product ID; RRRR - revision).

Delete VVVVPPPPRRRR key and also uninstall the device with utility like USDDeview to always get fresh device plug in behavior.

Extended Compat ID OS Descriptor

After device responds with proper 'MS OS String Descriptor' Windows sends a setup packet with request for 'Compat ID OS Descriptor'. Format of request is as follows:

Setup request field	Value
bmRequestType	b11000000 (device to host transfer direction; vendor type request; recipient is device)
bRequest	MS_VendorCode
wValue	0x0000
wIndex	0x0004 (request for an extended compat ID OS descriptor)
wLength	0x0010 or full length of descriptor

This a chance for device to inform Windows which driver should be loaded. Content below shows WinUSB driver association:

Value	Field	Size in Bytes	Description
0x28, 0x00, 0x00, 0x00,	dwLength	4	The length, in bytes, of the complete extended compat ID descriptor
0x00, 0x01,	bcdVersion	2	The descriptor’s version number, in binary coded decimal (BCD) format
0x04, 0x00,	wIndex	2	An index that identifies the particular OS feature descriptor
0x01,	bCount	1	The number of custom property sections
0, 0, 0, 0, 0, 0, 0,	RESERVED	7	Reserved
0,	bFirstInterfaceNumber	1	The interface or function number
0,	RESERVED	1	Reserved
0x57, 0x49, 0x4E, 0x55, 0x53, 0x42, 0x00, 0x00,	compatibleID	8	The function’s compatible ID ("WINUSB")
0, 0, 0, 0, 0, 0, 0, 0,	subCompatibleID	8	The function’s subcompatible ID (none)
0, 0, 0, 0, 0, 0,	RESERVED	6	Reserved

Descriptor is queried twice. First with only header length (0x10) in wLength field. Then, after full length is known to Windows (from dwLength field) with full length in wLength field.
Each interface in device can have own association. Multiple sections follow the header (in yellow) in case of more interfaces. Descriptor is queried on device level and all associations must be listed in single descriptor. bFirstInterfaceNumber denotes the index of associated interface in each section.

Extended Properties OS Descriptor

After associating the driver with device Windows queries for interface properties. These are meant to provide information which is usually provided in .inf files of drivers, e.g.:

[USB_Install.HW]

AddReg=Dev_AddReg

[Dev_AddReg]

HKR,,DeviceInterfaceGUIDs,0x10000,"{EA0BD5C3-50F3-4888-84B4-74E50E1649DB}"

Properties Descriptor request is also issued twice (or more for descriptors larger than 64kByte). First with wLength set to 0x0A to only retrieve the header and later with wLength set to full length gotten from dwLength.

Following is the layout of USB Setup request:

Setup request field	Value
bmRequestType	b11000001 (device to host transfer direction; vendor type request; recipient is interface)
bRequest	MS_VendorCode
wValue	0xppii (pp – PageNumber for descriptors larger than 64kB; ii – interface index)
wIndex	0x0005 (request is for an extended properties OS descriptor)
wLength	0x000A or full length of descriptor

Note that properties are queried per interface. So if your device has multiple interfaces the request is send at least twice for each interface (BTW the number of interfaces is known from standard USB configuration descriptor). Following descriptors defines two properties:

• DeviceInterfaceGUID
  This identifies interface with GUID which is used later in Windows application when querying the OS for devices with SetupDiGetClassDevs
• Label
  This should be a friendly name displayed for device in Device manager. But apparently it only works form Windows 8 on

Value	Field	Size in Bytes	Description
0xCC, 0x00, 0x00, 0x00,	dwLength	4	The length, in bytes, of the complete extended properties descriptor
0x00, 0x01,	bcdVersion	2	The descriptor’s version number, in binary coded decimal (BCD) format
0x05, 0x00,	wIndex	2	The index for extended properties OS descriptors
0x02, 0x00,	wCount	2	The number of custom property sections that follow the header section
0x84, 0x00, 0x00, 0x00,	dwSize	4	The size of this custom properties section
0x01, 0x00, 0x00, 0x00,	dwPropertyDataType	4	Property data format
0x28, 0x00,	wPropertyNameLength	2	Property name length (PNL)
'D',0, 'e',0, 'v',0, 'i',0, 'c',0, 'e',0, 'I',0, 'n',0, 't',0, 'e',0, 'r',0, 'f',0, 'a',0, 'c',0, 'e',0, 'G',0, 'U',0, 'I',0, 'D',0, 0, 0,	bPropertyName	PNL	The property name (DeviceInterfaceGUID)
0x4E, 0x00, 0x00, 0x00,	dwPropertyDataLength	4	Length of the buffer holding the property data (PDL)
'{',0, 'E',0, 'A',0, '0',0, 'B',0, 'D',0, '5',0, 'C',0, '3',0, '-',0, '5',0, '0',0, 'F',0, '3',0, '-',0, '4',0, '8',0, '8',0, '8',0, '-',0, '8',0, '4',0, 'B',0, '4',0, '-',0, '7',0, '4',0, 'E',0, '5',0, '0',0, 'E',0, '1',0, '6',0, '4',0, '9',0, 'D',0, 'B',0, '}',0, 0, 0,	bPropertyData	PDL	PDL Format-dependent Property data ({EA0BD5C3-50F3-4888-84B4-74E50E1649DB})
0x3E, 0x00, 0x00, 0x00,	dwSize	4	The size of this custom properties section
0x01, 0x00, 0x00, 0x00,	dwPropertyDataType	4	Property data format
0x0C, 0x00,	wPropertyNameLength	2	Property name length (PNL)
'L',0, 'a',0, 'b',0, 'e',0, 'l',0, 0, 0,	bPropertyName	PNL	The property name (Label)
0x24, 0x00, 0x00, 0x00,	dwPropertyDataLength	4	Length of the buffer holding the property data (PDL)
'W',0, 'i',0, 'n',0, 'U',0, 'S',0, 'B',0, 'C',0, 'o',0, 'm',0, 'm',0, ' ',0, 'd',0, 'e',0, 'v',0, 'i',0, 'c',0, 'e',0, 0, 0	bPropertyData	PDL	PDL Format-dependent Property data (WinUSBComm Device)

These properties are stored in Windows registry at:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\USB\VID_xxxx&PID_xxxx\sssssssss\Device Parameters
Use USDDeview or similar utility to uninstall device and delete this entry while developing.

HW Info

To supply the board from OTG USB micro AB connector you need to connect PA9 pin to 5V pin. This is possible because VBUS is (strangely) connected directly to PA9 VBUS sensing input.

[Update] Comm library

Communication is implemented by implementing Comm layer so that it can be used with Comm library explained separately.

This is first post in this series (of total 2 posts). See 2.

Reference

Links tree below is for my future reference. You probably want to see descriptor specification (in bold).

• Windows Driver Kit (WDK)
• Device and Driver Technologies
• Bus and Port Drivers
• Universal Serial Bus (USB)
• Building USB devices for Windows
• Microsoft OS Descriptors for USB Devices
• Microsoft OS 1.0 Descriptors Specification << this one
  (Windows XP and on)
• Microsoft OS 2.0 Descriptors Specification << and this
  (Windows 8.1 and on)
• Microsoft-provided USB drivers
• WinUSB (Winusb.sys)

Kinetis TWR-K70 main board mod for external debugger with trace

When plugging TWR main board to USB the integrated debugger enumerates and wants to install drivers. This can be annoying when you use external debugger and don't have drivers for on-board debugger installed at all. This debugger can be disabled with jumper connecting pins 2 and 4 on BDM connector J14. But with this change only the power switch for the rest of the board is permanently switched off. It can be forced to on state with applying high level on pin 4 of U2. Supply voltage is available on BDM connector.

Trace clock signal for parallel trace needs to be connected to external debug connector via R137 which is not populated by default.

WinUSB communication with STM32 (Part 4: Software)

Time for some bulk communication. VS project that I added to sources shows steps which are needed for communication. Project uses the Setupapi for device enumeration and obtaining the handle to device. It further uses the winusb to communicate with device.

After obtaining the handle to device and initializing it there can be multiple communication cycles. The communication cycle has the following steps:

1. reset the communication channel (control EP)
2. send data (bulk EP)
3. send end of data message (control EP)
4. poll status until "READY" or "ERROR" (control EP)
5. read data size of response (control EP)
6. read response data (bulk EP).

The source is mostly copied from MSDN with some changes and additions. An improvement to the library would be an object type which would encapsulate all this functionality and expose nice interface to communicate with device. An alternative could be  libusb-win32 or libusbK or other USB libraries. I haven't checked these in detail. If interested check libwdi. There is some information on Windows 8 integrated support for devices like this (no driver needed for WCID devices!).

[Update - WinXP (SP2) and W7 support driver information descriptors; See my next project]

Test application output.

Check project wiki to get build instructions.

This is the forth post in this series (of total 4 posts). See 1, 2, 3.

[Update - I have a new WinUSB device project with STM32F407 without the need for Windows side driver info (.inf and .cat files) ]

WinUSB communication with STM32 (Part 3: Driver)

WinUSB driver must be linked to device. This means creating a driver INF file specifying WinUSB as driver for device with specific USB VID and PID. To complicate a bit further this information needs to be signed. That is done by signing a catalog file (CAT) created from INF file. Windows Driver Kit (WDK) includes all the tools to do it.

Driver (code) signing was introduced to help prevent the installation of software that is not trusted. From what I understand about driver signing is that Windows trust some global certificate authorities (CA). They then issue a code signing certificates containing the publishers' information to software publishers. By this they are saying that the publisher was verified by them. They do it manually like call the person/company which issued a request for certificate and verify their information. For this they charge a yearly fee. The result is a class 3 certificate which publisher uses to sign his software. On installation/execution Windows pop up a dialog asking if you trust the publisher named as specified in signature of the executable which you wish to install/run. Publisher can also send his driver directly to Microsoft for testing and signing. It is called WHQL Testing. I'm not sure but I think that drivers tested and signed by MS are installed without any warnings. Read more about it on wiki. And I believe this isn't free either.
Free of charge alternative is an open source code signing certificate issued by same trusted CA (e.g. http://www.certum.eu). The requirement is that "Open source" keywords are stated in certificate information (e.g. the Name field of certificate is "<publisher name>. Open Source Developer")

Another free of charge alternative for debugging is so called self-signed certificate. This is a certificate created by developer. I created a script that:

1. creates the certificates,
2. copies the redistributables from WDK,
3. creates usbID.h file containing the IDs for USB descriptor which must match the INF file,  
4. creates the INF file,
5. checks the INF file,
6. creates the CAT file from INF file and
7. signs the CAT file with created certificate

I did this rather than publish a driver ready to install because it depends on personal information. Well I could distribute my self-signed certificate but I don't own a USB vendor ID. And since you have to have your own in the INF file the signed driver is no use to you because you must change it.

USB Vendor ID is another story to this. If you are serious and plan to sell USB devices you should apply for USB VID. It will be assigned to you after one time purchase. To additionally use the USB logo on the device you must pay a yearly fee... I think.

The script actually creates two certificates. First is your CA certificate. Second is code signing certificate issued by your CA. I omitted the installation of created CA certificate to "Root trusted certificates" because it seems to make no difference. First I thought that there will be no warnings if I put the created CA certificate next to other trusted CAs. But the Windows still pop up even more annoying warning saying that publisher can not be verified. I also tried generating the certificates with OpenSSL with no success.

Anyway, driver installation completes successfully if you choose to install anyway. The device is then displayed in the Device Manager as shown below.

Automatic installation can be done with dpinst.exe toll (part of WDK redistributables). But I didn't deal with why dpinst warning pops up saying that the publisher of dpinst is unknown. Maybe because it was ran from script?

To see details of all operations see the script.

This is the third post in this series (of total 4 posts). See 1, 2, 4.

[Update - I have a new WinUSB device project with STM32F407 without the need for Windows side driver info (.inf and .cat files) ]

WinUSB communication with STM32 (Part 2: Firmware)

The sources are available at Google code project.

I decided to make a communication pipes implementation as library so that it can be used with other USB hardware. The library uses two bulk endpoints and control endpoint. One bulk endpoint is for receiving the data and the other for sending the data. The default control endpoint (EP0) is used for communication control.

User's code should repeatedly call single function in library which does nothing while no data is received and calls user's function when new packet arrives.

Larger communication buffers are better for faster communication. To improve it even more the USB implementation should use double buffered endpoints - not the case at the moment.

This is the second post in this series (of total 4 posts). See 1, 3, 4

[Update - I have a new WinUSB device project with STM32F407 without the need for Windows side driver info (.inf and .cat files) ]

WinUSB communication with STM32 (Part 1: Intro)

Aren't you fed up with putting a USB to serial converter in your embedded design while you have a USB capable device already in the design? Any other approach opens a can of worms though. Custom driver, certificate to sign it, USB vendor ID (2000$ at the moment), how to communicate with device on a layer above USB? But it is nice when OS pops up your name when your device is plugged in USB port. WinUSB is middle ground with tradeoffs one might consider.

I wanted to test the use of WinUSB with STM32F103 for some time now. When I finally got time I created a demo project. It shows basic request - response communication.
WinUSB kernel mode driver enables raw communication through USB pipes. In other words you can communicate with your USB device directly from application on the PC. So there is only code for application and code for firmware. Presuming you are not making standard USB class device like Mass Storage or HID. You might not even need custom PC side software in this case.
Furthermore the libraries to communicate are written for both ends to some extent. There SetupAPI and WinUSB on Windows side while ST has libraries for everything.
To get a taste check images below. I will post more about it in the future and probably publish the sources after I clean and organize them.

VS showing sent data.

winIDEA showing same received data on STM32.

This is the first post in this series (of total 4 posts). See  2, 3, 4.

[Update - I have a new WinUSB device project with STM32F407 without the need for Windows side driver info (.inf and .cat files) ]