USB driver installer in NSIS

Creating a Windows Installer and Uninstaller for a custom USB driver using NSIS

The USB driver I created for the Xbox Big Button Controller has been sorely lacking a decent installer. I finally decided on writing one in the Nullsoft Scriptable Install System or NSIS for short.

The NSIS system is incredibly powerful and quite easy to learn. I quickly learned however that creating a fully automatic installer for a Windows driver is very tricky.

The full code for the installer can be found on GitHub

Show me the installer code

Installer

The installer was a quite straight-forward process. There is a lot of information already out there on how to install device drivers and do all sorts of fancy device install operations.

The features of the installer are as follows:

  1. Performs an uninstall of any older install of the same product

  2. Copies the correct binary files depending on the architecture being targetted (32 or 64 bit)

  3. Installs the driver into the Windows DriverStore

  4. Triggers a re-scan of the "unknown" plug and play devices to force the device to install itself correctly

The dpinst.exe (Driver Package Installer) utility was helpful in abstracting away the complexities of installing a driver and the installer is heavily centered around that step. The dpinst.xml file contains all the configuration options for this utility.

Uninstaller

Now this was a massive headache!

Simply speaking, performing a clean uninstall of a device driver on Windows is an incredibly tricky process. For some reason Microsoft has made it incredibly difficult to ensure that you can unload and delete any installed INF files you might have installed on your system programatically.

The features that I wanted for the uninstaller were:

  1. Uninstalls any USB devices from the Device Manager

  2. Completely removes the driver files from the Windows DriverStore

Neither of these steps were easy to solve given the information on the internet. The biggest problems were that

  1. The uninstall utilities such as PnPutil.exe only partially remove the device driver. The INF file is still left on your system in the form of the originally installed OEM driver INF file. This results in issues if you ever offer driver upgrades the user might actually not load the newest driver after upgrading if the old INF file is present.

  2. Uninstalling the USB device from the Device Manager was particularly difficult due to a problem with the devcon.exe (Device Console) program constantly erroring out for some reason when executed from NSIS.

  3. The return codes from devcon were not trackable from within the NSIS script.

My final solution was to push all the heavy lifting of uninstalling the device and INF file to an external bat file as the NSIS system was having difficulties calling and parsing the correct return codes from the devcon tool.

DriverStore Explorer [RAPR] is a really useful tool to debug the DriverStore and clean up any lingering INF files.

The biggest problem was to figure out that the reason why the devcon program was failing is that it requires the working directory to be set to its current directory before running. SetOutPath solved that issue

The uninstall batch script was a bit tricky because your driver installs under a generic OEMxx.inf name so you need to figure out this name before you uninstall the device to be able to successfully remove the INF file from the system.

Gotchas

  1. Finding the right device in devcon.exe was tricky as the device category must be correctly specified. In my case I install the driver under the built in "Sound, Video and game controllers" which has the name "Media" behind the scenes. By specifing =media in my remove query I was able to find it successfully.

  2. Working directory must be set before devcon.exe is called.

  3. devcon.exe dp_delete function uses the generic OEM name of your INF file. You must use dp_enum first to obtain the name that Windows assigned your of the INF file when installed.

Show me the entire installer code

This article is also available on Linked in:  https://www.linkedin.com/pulse/creating-fully-automatic-installer-uninstaller-driver-sigmundarson



Software Developer
For hire


Developer & Programmer with +15 years professional experience building software.


Seeking WFH, remoting or freelance opportunities.