MQTT Lego Rubber Band Gun

What is a Lego Rubber band gun

Our lego rubber band gun is a semi-automated vision controlled lego gun which is capable of firing a multitude of rubber bands in controlled quick succession. If that sounds cool to you, keep reading.

This discussion follows on from my MQTT RC Controller. It uses the same electronics to drive some servos (which have been ripped out of my monster truck), and are controlled via MQTT. This project is a combination of some control software and a LEGO gun construction courtesy of Brenton.

How is it controlled?

Initially, control of the servos were connected via MQTT to a playstation analog controller (as used in other projects) enabling remote firing. The operator was able to see where they were aiming by tuning into the video feed from the wireless camera mounted on the turret, pictured above.

The next step was to allow an operator to click on the video feed and have the gun automatically move and fire at the location. This was a bit harder. I made use of the OpenCV vision library to provide a closed vision feedback loop. The webpage which received the motion jpeg video also acts as a Javascript image map hooked up to my MQTT web bridge which fed the clicked locations into another app also receiving the video stream. I found a salient feature close to the location of the clicked point, and tracked this over time. The camera orientation could also be calibrated in relation to the turret via the interface. This meant that we now had a motion vector in image co-ordinates in which we needed to move, and which could easily be computed at frame rate. I was expecting to have to resort to full PID control to provide stable motion, but, having implemented proportional control, it just worked!.

Due to the nature of the web interface for control, but local vision control, camera lag over the net wasn't an issue, it was all taken care of during tracking locally. This meant that it could be used without adverse effects over the net. The feature being tracked was found in subsequent frames with a local search which was relatively cheap. Since we were running on a feedback loop with no dead-reckoning, the operator could even select a moving target such as a person, and the gun was able to maintain a lock over its two degrees of freedom. The gun was then able to release its eight or so rubber band payload with frightfull accuracy, either one at a time or all at once.

So just how accurate is it?

Well, we didn't find time to take it to a firing range, but it was able to shoot down a small LEGO man from two meters after an engage request is made from the clickable web-interface. Not bad for a load of LEGO bits, controlled shodily by some servos connected to the LEGO by make-shift polymorph adaptors.


by Steven Lovegrove and Brenton Bailey Jump to top