Glove for playing paper-scissors-stone using TensorFlow

Often, machine learning is used to solve complex problems, in particular, related to large data sets. Machine learning works well in finding patterns that combine a large number of inputs with a large number of outputs.

There is not probably more analogue game than paper-scissors-stone. However, one of Google’s employees decided to run an experiment using the TensorFlow program, which uses the open code created by Google for machine learning. He built a device for playing paper-scissors-stone. To find out the exact technical details, it’s worth reading the original post on Google Cloud Big Data and Machine Learning Blog. In this case, a true simple ML algorithm, which detects the hand position by the Arduino microcontroller connected to the glove, works.

  Zdjęcie 1. Rękawica do gry w papier-kamień-nożyce.

Step by step construction of glove for playing paper-scissors-stone

Glove sensor

The device is built of LittleBits sets. These child-friendly kits contain a wide range of components, such as LEDs, motors, switches, sensors and controllers, all of which can be connected with magnets – without the need for soldering. Three bending sensors were used for this experiment, which we attach to a plastic (or nitrile) glove. When fingers are bent while wearing the glove, the sensors send an electric signal with variable voltage in the range from 0V to 5V

Picture 2. LittleBits bending sensor

Connecting the Arduino and Servo microcontroller

To read the signals from the bending sensors and control the knob of the machine, the Arduino and Servo modules were used. The Arduino module has a small microcontroller in the middle and also has many input and output ports. You can write a machining program (a language similar to C) on your laptop, compile it and load it into the module by connecting with a USB cable.

Picture 3. Arduino LittleBits module

A program that reads data from bending sensors

Now that the equipment is ready, the code writing on the Arduino module begins. It will read data from the bending sensors. In the Arduino IDE, you can write a code to read data from the sensor every 0.1 seconds.

Data visualization using Cloud Datalab

How do you determine which combination of three numbers represents “stone”, “paper” or “scissors”? The simplest implementation would be to write IF instructions with arbitrary thresholds and conditions.

Creating a linear model

Then, the raw sensor data was corrected to get a shape that can be classified as one of three gestures. Linear algebra was used. Linear algebra is a category of mathematics that allows you to map one set of dimensions to another set.

Let TensorFlow find the parameters

In this phase, the “trainer” of the linear model training was defined.

Train the linear model

The next step is running Tensorflow to train the model using the optimizer. To start the actual training, a session was created and its launch method called to pass the glove sensor and label data to the optimizer.

Use the linear model on Arduino

When the classification method of the glove sensor data was already quite strong, the Arduino coding was terminated.

The game of paper, scissors and stone is probably not something that comes to mind when you think about Machine Learning. This project shows how machine learning can be useful for all programmers, regardless of the reduction in the number of tasks for coding people and accelerating calculations.

Pictures 4. The hardware elements of the paper-rock-scissors machine