Integrating Machines Using Android and Python - Part 2
For the next iteration of our machine, a different machine was selected because the first one wasn’t quite up to the task. Besides functioning poorly (for our purposes, at least), the Bluetooth connection wasn’t reliable enough.
Asking around, one of the entrepreneurs’ CTO heard of a (supposedly) better machine that had been “hacked” by one of the distributor’s clients. A few days passed, and we got to play with a new (secondhand) machine, already half disassembled by one of the mechanical engineers already modeling its parts.
First thoughts, again
Since the machine’s electronics were mostly out, I first tried figuring out what’s what. I found a microcontroller board at the back of the machine, and found that it was driven by a microcontroller I sort of know - STM32F103. I tried figuring out what each IO was doing, and doing so I found the TX/RX lines going to the touch screen at the front of the machine. I then followed SparkFun’s tutorial on using a logic analyzer with Sigrok, to capture the UART comm.
At this point my goal was to record the communication sent back and forth between the uC and the touch screen, and to figure out how to trigger certain functions.
It turned out to be a good learning experience, but not a worthwhile one. The communication protocol used in this particular machine was too complicated for me to decipher, as it was using what seemed to be changing in length with every operation. Without some prior knowledge about the communication scheme, it seemed like a lot of work was still ahead of me if I were to continue down this road.
As the communication seemed too complex to replicate, I took another look at the machine. I found that what I assumed to be only a touch screen had actually a full microprocessor board driving it. When I first looked at the disassembled machine, I thought the whole machine was being driven by some firmware running on the STM32 board, and that the UART communication to the touch display was to a daughter board. But finding that the other board was sporting a microprocessor, in what seemed to be a smartphone development board form, changed my perspective. I was now thinking that the smartphone dev board was the actual computer doing the heavy lifting, while the STM32 board was functioning as a peripheral to it.
I shared my findings with the consultant who suggested we use Macrodroid for the first machine. He suggested we start the machine and prod around the GUI on the touch screen to get a sense of what was the OS running behind the scenes. He also suggested it would probably be Android of some sort, as it was relatively easy to get that working as a touch screen UI for the machine.
It took a while, but eventually, I got to the machine’s settings by trying to set the system language for the UI. It seems the developers of the GUI decided to use the OS language settings, and so I could confirm it was running a recent enough Android system.
I also talked with the distributor, and through him got to the guy who “hacked” the machine prior to me. The last guy got to about where I got, by prodding the UI of the machine set in his office. But this guy got blocked by the distributor’s password - whereas in the case of my machine the distributor had unlocked it specifically for our use.
Stay Tuned
In the next post, I will cover how I used ADB and Appium to overcome the challenges posed by the dynamic screens of the second machine. Stay tuned for a detailed walkthrough of the process and the final solution!
This post have been sitting in my drafts for a while now. I’ve been busy with other projects, but I’m glad I finally got around to finishing it. I hope you find it useful!. It is also a continuation of the previous post, so if you haven’t read it yet, I suggest you do so before continuing.