Local Particle Build and Debug via Docker for Mac

As part of my journey to create my first product I found it important to have my local development environment set up in such a way that I could build firmware without relying on the Particle cloud and also be able to debug that firmware if necessary. What I found were resources scattered everywhere that covered pieces of the process or weren't very clear on certain steps. This post will show you step-by-step how to set up your local environment for Particle dev. This includes:

• Building the core Particle system firmware locally

• Building your application firmware locally

• Debugging both the system and application firmware with gdb

Install Docker and VirtualBox

Besides having the source code the only things you need to install on your Mac are Docker and VirtualBox (and Git if you want to follow along). If you don't care about debugging and only need to build firmware then you can skip installing VirtualBox. Here are links to the downloads.

• Docker for Mac

• VirtualBox for Mac

• VirtualBox Extensions for Mac (on the same page just below the download link for VirtualBox)

I got an error after installing VirtualBox on my Macbook saying the installation didn't finish successfully because of security policies. To get around this you have to go into Security Preferences and allow the install to run but it's already too late because it failed. Just uninstall it (delete it from the Application folder) and run the install again. This worked for me. Make sure you also install the VirtualBox extensions for Mac as they are necessary for debugging to work.

Get the Particle Source

Whether you plan to build the Particle system firmware or not you need to have the source pulled down for building your own application firmware locally. Clone the repo to your local machine.

git clone https://github.com/particle-iot/firmware.git
cd firmware
git checkout release/v0.6.3

The code above checks out the 0.6.3 branch which, as of this writing, is the current stable release version of the firmware. The great part about building locally is you can change this to whatever branch you want and even get the latest development changes that are still being worked on.

Configure Your Application Source

For things to work smoothly your application project should be organized to conform to the Particle extended structure. All that really means is that your main project folder should have a src directory under which your code resides. You can also have libraries as well which would all be located under a lib folder.

To help you get started I created a super simple sample repository that you can reference. It's essentially a copy of the Tinker firmware with some very minor changes to limit access to a couple of pins which are needed for debug. I will use that repository for the rest of this document but it is very easy to modify the steps to apply to your source code. To follow along with this writeup you can get the sample repo with the command below. Remember to clone it into it's own location and not under the Particle firmware folder.

git clone https://github.com/makercrew/particle_debug_sample.git

Before going further I have the following setup.

• Docker and VirtualBox (including extensions) installed on my Macbook

• The Particle firmware repo and my application repo cloned onto my machine in their own folders as shown.

Building the System Firmware

The first thing we will do is build the Particle system firmware. You don't have to build this before building your application firmware but I want to show you how in case you ever want to mess around with the Particle source. When building application firmware through the Particle Build or Dev IDE the system firmware has already been built for you and consists of multiple parts. The following build process will generate those parts as well as the default Tinker application which you can then program onto your device using dfu-util. Because we have Docker installed it's a single command from your terminal window. Be sure to populate the local path to where you cloned the Particle repo.

docker run -ti --rm --workdir="/particle_src/modules" -v [PATH TO ROOT OF PARTICLE REPO]:/particle_src makercrew/particle_build make clean all PLATFORM=photon

This command spins up a Docker container based on an image I built that has all of the dependencies required to build the firmware. This saves you a ton of time trying to figure out and install all of the dependencies locally. The very first time you run this command it will pull down the image from Docker Hub automatically. The build itself takes about 10-15 minutes and when finished the modular firmware binaries are available in the firmware repo at the following paths for a Particle Photon.

• firmware/build/target/system-part1/platform-6-m/system-part1.bin

• firmware/build/target/system-part2/platform-6-m/system-part2.bin

• firmware/build/target/user-part/platform-6-m/user-part.bin

If you build for another platform like the Electron the only difference will be the platform-6-m part of the path. It will change based on the target platform you choose.

Building your Application Firmware

Unless you are looking to tweak the system firmware you don't need to do a full build. You're probably more interested in just building your firmware locally. Again, the advantage to this is that you don't have any dependency on the Particle cloud for builds and aren't constantly uploading your source to their servers. This build is much faster and is also accomplished with a single Docker command. Remember to modify the command to include the right paths to where you cloned the repos.

docker run -ti --rm --workdir="/particle_src/main" -v [PATH to ROOT OF PARTICLE REPO]:/particle_src -v [PATH TO ROOT OF YOUR APP]:/app_src makercrew/particle_build make all PLATFORM=photon APPDIR=/app_src TARGET_FILE=my_app

This will build your application for the Particle device specified and either spit out compilation errors or produce a binary file called my_app.bin in a target folder under your application folder. To change the name of the output file simply modify the TARGET_FILE= portion of the command. Here is how I ran it with the sample repo.

And here is where the output ends up.

You can program your device directly with this file either via USB serial or DFU mode with the Particle CLI.

Program over Serial USB

1. Put your device in Listening Mode.

2. particle flash --serial target/my_app.bin

Program with DFU

1. Put your device in DFU Mode by holding down both buttons and applying power. Continue holding both buttons down until the RGB LED begins to flash yellow.

2. particle flash --usb target/my_app.bin

Building for Debug

To be able to hit breakpoints in our code we can't just connect to the firmware we previously programmed onto the device because it doesn't have any of the debug information in it. We have to rebuild with debug enabled. This is another single liner for Docker.

docker run -ti --rm --workdir="/particle_src/main" -v [PATH TO ROOT OF PARTICLE REPO]:/particle_src -v [PATH TO ROOT OF YOUR APP]:/app_src makercrew/particle_build make clean all PLATFORM=photon MODULAR=n DEBUG_BUILD=y USE_SWD=y APPDIR=/app_src TARGET_FILE=my_app

This time the my_app.bin file in the target folder of your application will be MUCH bigger (several hundred KB). It is a single monolithic build of the entire Particle system firmware as well as your application code with debugging information baked in.

To program it onto your device:

1. Put the device in DFU Mode by holding down both buttons and applying power. Continue holding both buttons down until the RGB LED begins to flash yellow.

2. particle flash --usb target/my_app.bin

Hardware For Debugging

To debug your application firmware you need a bridge device that connects the debugger process on your machine to the Particle device running your firmware. There are multiple options available but I chose the ST-Link v2 Mini. It is a SWD (serial wire debugger) device and only cost $13 through Amazon with Prime shipping. It is a knockoff version of the official ST-Link debugger but it worked perfectly for me. I imagine there are some out there that don't work as well or at all. This is a link to the specific one I purchased.

https://www.amazon.com/dp/B00PVJ8Q4C

There are reports of the debugger not working on the USB 3.0 ports on the Macbook. I used a USB 2.0 hub with no problems.

Connect the Hardware Debugger to Your Device

Connect the ST-Link device to your Particle Device as shown.

SWDIO ---> D7
SWCLK---> D6
GND------> GND

Notice that the debugger requires a connection to the onboard LED D7. This means that your firmware cannot use D6 or D7 while trying to debug. The debugger has the ability to power your device but I don't use that option. I power the Particle through whatever means my end project is already using (battery or USB). That means that during debug you will have the ST-Link powered via the USB port and the Particle getting power from some other means.

Docker Modifications For Debugging

As of this writing Docker for Mac does not support sharing a USB device with a container. This is easy to remedy and requires a few tweaks to how we run Docker. In fact, we basically need to run Docker the "old way" on our Mac so that we can pass the debugger device into a container for GDB to use. This is where VirtualBox comes in handy. The following steps will create a boot2docker VM named docker that we can run Docker commands against. From a terminal run the following.

docker-machine create --driver virtualbox docker
docker-machine start docker
eval $(docker-machine env docker)

That last eval command might seem strange but if you simply run docker-machine env docker you will see that it's just a bunch of export statements to set environment variables.

Running this command sets up your terminal window to pass all Docker commands to your VM for execution instead of to the native Mac Docker install.

Note: Running the eval command makes the changes locally to your terminal session. If you close your terminal window, open a new tab, or reboot your machine you must run that command again to reconfigure your Docker environment.

Give boot2docker USB Access

Now that our VM is running and ready to handle all of our Docker commands we need to let it know about our SWD debug device so we can pass it into a container. Doing so is very straightforward and explained in the following steps.

1. Open VirtualBox

2. Verify your boot2docker VM is in the Running state. If it's not just run docker-machine start docker from a command prompt.

3. Plug in your ST-Link V2 Mini device into a USB 2.0 hub attached to your Macbook. The LED will flash a couple of times and then be solid red.

4. In VirtualBox open the Settings for your docker VM

5. Click on the Ports tab and then the USB sub-tab

6. Check the box Enable USB Controller and select the USB 3.0 (xHCI) Controller option.

7. Next to the USB Device Filters area click on the icon to add a new USB filter.

8. A menu will appear showing all of the devices detected. Select STMicroelectronics STM32 STLink.

9. Click OK.

Now every time you plug in your USB debugger device it will be visible to your boot2docker VM.

Start Debugging

You're finally ready to start stepping through your code! Even though your code is built with debug symbols it will still run normally. That means when you power on your device it's not going to do anything special to wait for you to debug it. It's just going to do it's thing. So how do we actually step through the code?

The easiest way to get started debugging is to put your Particle device into DFU mode (flashing yellow). In DFU mode debugging is always enabled for you regardless of whether your application code was built in debug mode or not. What that means is that we can attach to it and get our debug session started. It's our doorway in.

With the device in DFU mode we have one last Docker command to get us debugging. Make sure your ST-Link device is attached to your Particle device and powered on. Run the following replacing the paths with those in your scenario.

docker run -ti --rm --device=/dev/bus/usb -v [PATH TO ROOT OF PARTICLE REPO]:/particle_src -v [PATH TO ROOT OF YOUR APP]:/app_src makercrew/particle_build arm-none-eabi-gdb -ex "target remote | openocd -f interface/stlink-v2.cfg -f target/stm32f2x.cfg -c 'gdb_port pipe; log_output openocd.log'" /app_src/target/my_app.elf

The final part of that command is what loads the right sources for debug. If you changed the TARGET_FILE output name above you'll need to adjust the name of the .elf file to match.

If everything is working your ST-Link should be flashing green and your terminal will be at a gdb prompt as seen below.

 

From the gdb prompt we need to reset our Particle device and break on the first instruction. This is done by running the command

(gdb) monitor reset halt

At this point you are in a normal gdb debug session. Set a breakpoint in the setup function with command

(gdb) break setup
(gdb) continue

Now you can hit c or type the word continue and the Particle will connect to the cloud and then break as it enters the setup function.

You are officially debugging your application firmware! If you are new to gdb here is a nice cheatsheet you can reference.

What About An IDE For Debugging?

Because this approach uses gdb it can be extended into things like the Eclipse IDE. The Particle website has some instruction on that but I wanted to keep it raw and simple so I didn't take my dev environment to that point. Also, I hate Java.

Wrapping It Up

Once your system is set up to build and debug it's really quite simple to do so via Docker. I created aliases in my shell resource file for the long docker-compose commands so I don't have to remember how to type them every time I need to use them.

If you have any feedback on how I could improve upon this setup please leave a comment. I want to make it as easy and as awesome as possible for other people to use.