Getting started with Arduino Machine Learning and Edge Impulse: Keywords detection

In this tutorial, we want to cover how to get started with Arduino and Edge Impulse. We want to detect some basic keywords using machine learning and Arduino. This is a step by step tutorial describing all the steps you have to follow to implement a machine learning model that can run on your Arduino Nano 33 BLE. At the end of this post, you will be able to deploy a machine learning model that can recognize some keywords such as color names so that we can control an RGB WS2812 strip using voice commands.

Machine Learning at the edge is a really interesting topics and it is very useful to give to the devices the capability to execute some tasks. Moreover, using Machine Learning at the edge will help you to overcome privacy problems because the computation happens locally.

Edge Impulse is somehow revolutionizing the way we build machine learning models opening it to all people even if they don’t have machine learning knowledge. Using a simple and intuitive Web interface you can create your dataset, develop your machine learning model and train it.

How to develop a Machine Learning model

Generally speaking, to build a machine learning model you have to follow these steps:

  • Create your dataset
  • Build your machine learning model
  • Train your model
  • Evaluate the model accuracy
  • Deploy the model (in this tutorial, we will deploy this model on Arduino Nano 33 BLE)

Usually, to accomplish all the steps described above, you have to use Tensorflow lite that helps you to build a machine learning model that fits the device requirements such as Arduino. Edge Impulse simplifies all these processes using a Web interface so that you don’t have to deal with libraries, quantizations, and so on. Let’s see how to do it. Before starting, you have to create a Edge Impulse free account and a new project.

More useful resources on Machine Learning:

How to build a Tensorflow lite model for Arduino
How to build a KNN classifier for ESP32

How to create a dataset using Arduino and Edge Impulse

The first step, as described before, is creating a dataset. There are several ways we can it acquiring samples directly from Arduino 33 Nano BLE microphone. In this tutorial, we want to recognize three different keywords:

  • Red
  • Green
  • Blue

Moreover, to make our model smarter we will acquire noise and unknown words. If you like, you can use Edge Impulse dataset to do it.

Then, click on the dataset item on the left menu and start acquiring samples of the red words. To do it, repeat several times the red words. The result is shown in the picture below:

Edge impulse dataset samples with Arduino Nano 33 BLE. Build a dataset for Machine Learning model with Arduino

Once you have acquired the samples, you have to split it:

How to split samples in dataset

Repeat this process for all the keywords you want to detect. Moreover, you have to create another two datasets:

  • noise
  • unknown words

In the end, you will have 5 different labels. During the dataset creation, you should use a different voice so that the model will perform better. So ask for help from your friends. Be aware that the more samples you acquire more accurate will be your model.

Before going on, you have to split the dataset in two groups:

  • training
  • testing

Also in this case, Edge Impulse helps you to split your dataset. Go go to the dashboard and click on the balance dataset.

Creating your Edge Impulse Machine Learning model for Arduino Nano

Once you have created your dataset, the next step is building your machine learning model. Edge Impulse has several signal processing blocks that can be used to extract features from the samples acquire and learning blocks. In this case, we want to recognize the human voice, for this reason, we will use Mel Frequency Cepstral Coefficients (MFCC) audio processing and as a learning block a Neural Network. The final result is shown here:

Audio processing (MFCC)

Under MFCC block, you can check the features that will be extracted from the samples we acquired in the step before. These features will feed the Neural Network model.

If you select one of the samples acquired, you can visualize the DSP of the signal acquired:

Machine learning signal features

Neural Network block

Under neural network block, you can set some parameters. Usually, you need to change only the number of the training cycles.

Finally, we you are satisfied, you can start training your model. This is very fast and after the process complete, you have some important parameters for evaluating your model:

Edge Impulse Machine Learning model for Arduino Nano 33 BLE

You can improve your model, adding more samples. If you are satisfied you can download the model and you can use i with Arduino Nano 33 BLE.

How to use Machine Learning model with Arduino Nano 33 BLE

In this last part, we will use the Edge Impulse Machine Learning model with Arduino Nano 33 BLE. As said before, when you are satisfied with the accuracy of your model, you can download it and make it ready for Arduino. The model is downloaded as a library, therefore you have to import it in your Arduino IDE.

In this tutorial, we will use the Edge Impulse Machie Learning model with Arduino to control a WS2812 Leds. After you have imported the library, open the file named * _microphone_continous.

We have to modify this file adding the lines to hande the WS2812 LEDS. In the top of file add the following line:

#include <Adafruit_NeoPixel.h>
Code language: C++ (cpp)

Next, add this line:

// Neopixel Adafruit_NeoPixel strip = Adafruit_NeoPixel(16, 2, NEO_GRB + NEO_KHZ800);
Code language: C++ (cpp)

In the setup() method, add this:

// Neopixel LEDS strip.begin(); strip.setBrightness(255); strip.show();
Code language: C++ (cpp)

Finally, in the loop() method, look for:

ei_printf(" %s: %.5f\n", result.classification[ix].label, result.classification[ix].value);
Code language: C++ (cpp)

and add the following lines:

for (size_t ix = 0; ix < EI_CLASSIFIER_LABEL_COUNT; ix++) { ei_printf(" %s: %.5f\n", result.classification[ix].label, result.classification[ix].value); if (result.classification[ix].value > 0.6) { if (result.classification[ix].label[0] == 'B') { strip.fill(strip.Color(0,0,200)); } else if (result.classification[ix].label[0] == 'R') { strip.fill(strip.Color(200,0,0)); } else if (result.classification[ix].label[0] == 'G') { strip.fill(strip.Color(0,200,0)); } strip.show(); }
Code language: C++ (cpp)

That’s all. Enjoy your new machine learning model running on Arduino 33 Nano BLE.

Testing the Edge Impulse Machine Learning model with Arduino Nano 33 BLE

In this last step, you can compile and run the code for Arduino Nano 33 BLE.

Now you can speak one of the keywords used before:

  • red
  • blue
  • green

If you have trained your model in the right way, your WS2812 Leds will change color.

Wrapping up

In this post we have covered how to use Edge Impulse with Arduino Nano 33 BLE to build a machine learning model that can recognize some keywords. We have applied the model to a real use case controlling the WS2812 Leds using your voice. You have learned how to create a dataset using Edge Impulse and Arduino, how to build a machine learning model and finally how to train it.

Using Edge Impulse, all the process gets simpler. Let me know what kind of project you will build.