Quirks of the Linux Serial Line – InApps Technology 2022

Something we haven’t discussed much is the data being generated by my latest project, Hedley the robotic skull. That’s right, deep down, Hedley is an “Internet of Things” device. He has sensors and those sensors produce data that can then be used for analysis, activity logging and as input to applications or other devices.

For example, I’d like to build some portable, networked spotlights that can travel with Hedley, when I take him to my next tech talk. He’ll sit on a little table at the front of the room, tracking me as I move around on stage. Wouldn’t it be cool to have a couple of spotlights that follow me, as well? Or, maybe a robotic video camera, to record my session. Data from Hedley could direct both of those gadgets to the right spots, during my presentation. Hey, I’m a one-man show and leverage the latest tech to get the job done.

Right now, the main data producer in Hedley is the JeVois smart machine vision camera. It provides both an augmented USB video feed and a hardware serial line output. The augmented video is great for monitoring what the sensor is interpreting, according to the particular on-board vision analysis application, running at the time. The hardware serial line sends position data to an Arduino, that then moves a servo attached to the skull, keeping the camera centered on its point of interest.

There’s also a software serial feed over the USB cable. This is a mirror image of what flows over the hardware serial line and you can use it to feed other applications and devices. Today, we’ll concentrate on sorting out the USB serial line setup so we can actually get the data to where it will be useful.

Prelude to MQTT and Python Scripts

A while back I explored using the MQTT pub-sub model to ferry data across machines, devices and servers. To me, MQTT represents a great communication protocol for the skull because it’s a one-to-many and many-to-one model. In other words, the skull can pipe its USB serial data, maybe through a Python program to an MQTT server and multiple clients, which can then grab and use the data as they see fit.

Using MQTT also abstracts the origin of the data. It won’t matter whether the spotlights get pointing data from the skull or from a little battery-powered ESP8266 clicker I hold in my hand. Either or both will work with MQTT. Not only that, since we can send data in either direction in a closed-loop feedback, error-correction is also possible. Just publish the data back to the MQTT server from the spotlights, for instance, and have the skull do something with that data.

Python has good support of MQTT, through libraries, so writing programs to automate how we use and where we send the data, will surely be in Hedley’s future. Best of all, since the skull has a Raspberry Pi 3, we can simply install an MQTT server right there. And, Python is readily available too.

We’ll tackle MQTT on the skull in an upcoming story. It’s all academic until we get the serial data process working reliably.

Quirky Linux Serial Lines and the JeVois Camera Commands

Getting serial data flowing to/from the JeVois sensor into and through the Raspberry Pi is tricky. For one thing, you first need to tell the sensor what neural-network application to run. The easiest way is to run the guvcview command and select an appropriate resolution. Naturally, a monitor, keyboard and the Raspberry Pi need to be hooked up to the JeVois device, for this to work.

Once the camera is running and you are monitoring the video feed, you’d think you could just type a simple command, in on of the Pi’s terminals and have the data stream out, since everything in Linux, including devices, are thought of as files. But, no!

For example, sometimes I’ll type ‘cat /dev/ttyACM0‘ and get absolutely nothing. The command line just sits there silently mocking me.

The way around this is to use the stty command to configure the serial port. stty has a bunch of options, although for our purposes we just need a few. In the skull’s Raspberry Pi terminal I just typed the following.

sudo is used to avoid any conflicts with /dev file permissions. The -raw option pipes data back and forth, without any mods or filtering. The -F specifies the device, in this case /dev/ttyACM0. Finally, 115200 sets the serial line speed.

Once the stty command executes, it should be smooth sailing for running commands like cat.

Not so fast. You have to explicitly set the JeVois sensor to send data OUT OF the USB serial port. This is easily done by sending data INTO the serial port from the Raspberry Pi.

Since the JeVois sensor is actually a little Linux machine running on the ARM processor in the device, it has a rudimentary command line that’s used for configuration. There are a handful of commands you can send to change the camera resolution (which neural-network program it will run), get info about the device and so on. Just send it a command over the USB serial line.

The form of the actual command is ‘<command> <parameter> <value>’.

Here it is all rolled up, sending the command to the JeVois, then having the resultant camera data output coming back out through the cat command.

Again sudo takes care of permissions. We then run the echo command, under the bash shell and send the “setpar serout All” text string INTO the /dev/ttyACM0 serial port. That’s the one hooked up to the JeVois camera. Lastly, a second command, cat is used to display what comes back FROM the serial port. In this case, it’s positional data from the JeVois camera.

It’s important to use the ‘All’ option with the setpar command because that turns on data flow for both the hardware serial line and the USB serial output.

Here’s a screenshot of some data.

JeVois sensor data as seen on the Raspberry Pi 3 terminal display

One especially quirky behavior of the JeVois sensor hooked up to the Raspberry Pi USB serial port is that the video feed will occasionally freeze, when data isn’t being sent.

It’s easy to fix though. Just execute another cat command. The video will start again, as will the serial data appearing on the terminal screen.

What’s Next

It’s been a quick trip through the Linux USB serial line configuration and usage maze. Keep in mind that while you can redirect or pipe data (using the Linux redirect ‘>‘ and pipe ‘|‘ symbols, respectively) on the command line, to other applications and devices, Python programs that run in the background are a more practical way to accomplish our automation tasks.

You have to crawl before you walk, so knocking out the serial line part of your project is always an important developmental exercise. If you can get the data to the terminal display, you can surely get it into a program or out on the network.

Be sure to look for Hedley’s MQTT installation and Python script development in the near future.

Feature image by Christopher Burns on Unsplash.

InApps Technology is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Prelude.