Run Tensorflow Lite on ESP32 from scratch: Easy guide (PlatformIO)

This easy guide describes how to run Tensorflow lite on ESP32 from scratch. This guide covers step by step how to build and use Tensorflow Lite on ESP32 using PlatformIO IDE. There are several guides that describe how to build and run Tensorflow Lite micro for ESP32 but some of them are outdated or are focused only on the last part that is executing Tensorflow on ESP32. This guide starts from the beginning covering all the steps to follow to run Tensorflow on ESP32.

In this easy and fast guide, you will learn:

• How to setup the environment and the tools to compile Tensorflow lite
• How to create the Tensorflow library to use in your ESP32 Machine Learning projects
• How to run the Tensorflow Hello World example with ESP32

Introduction

Tensorflow lite brings the power of Machine Learning on micro devices such as ESP32, Arduino, and so on. There are some devices officially supported such as Arduino nano 33 BLE Sense and so on. Also, the ESP32 is supported but we have to use the EspressIf-IDF tool.

Running TensorFlow lite micro on ESP32 opens countless possibilities to use edge Machine Learning on small devices spending only a few euros to buy these devices. Using ESP32 it is possible to create interesting IoT projects that get really powerful when we mix them with Machine Learning.

While using Tensorflow with Arduino is very easy because you can import the library directly into the Arduino IDE, when we want to use TensorFlow with ESP32, there are other steps to do. You will discover that these steps are very easy. Even if there is an official guide describing how to compile and run the TensorFlow examples on ESP32 using EspressIf-IDF tools, I’m not used to using the IDF with the ESP32. I prefer using PlatformIO IDE because I find it simpler.

When you approach the task of running TensorFlow lite on ESP32 there are some questions you start asking yourself:

• How do I compile Tensorflow for ESP32?
• Which tool should I use?
• Do I have to use the ESP-IDF only?

This guide will help you to build Tensorflow for ESP32 and to run Tensorflow examples.

Cloning the Tensorflow Github repository

The first step is cloning the Tensorflow Github repository using the following command:

git clone https://github.com/tensorflow/tensorflow.gitCode language: PHP (php)

From now on, we call this directory <tensorflow-directory>.

Compiling the Tensorflow for ESP32

There are two possible options we can use to compile TensorFlow for ESP32 that are not mentioned nowhere:

• Using the ESP-IDF VSCode plugin
• Using Docker with ESP-IDF (easier option)

These two options make easier to start using ESP-ID to compile Tensorflow lite micro.

EspressIf-IDF VSCode plugin

The first option to compile Tensorflow lite for ESP32 is installing this plugin in VSCode:

Once you have installed it, you have to install two other applications as described in the plugin home page:

If you are using OSX these are the commands to use:

sudo easy_install pip
pip install --user pyserial
brew install cmake ninja dfu-util

Now follow the plugin instruction to complete the installation. Finally, compile Tensorflow:

make -f tensorflow/lite/micro/tools/make/Makefile TARGET=esp generate_hello_world_esp_project

To run this command you have to be under the directory that holds the cloned repository (<tensorflow-directory>). Look here for more info.

Recommended:
How to use image recognition with ESP32-CAM
Arduino Machine Learning
ESP32 Style Transfer with Magenta JS

Using Docker to compile Tensorflow

This second option is the easier option because you have all the environment already configured. You have to download the docker image and use it to build Tensorflow:

docker run -v <tensorflow_directory>:/tf -w /tf -it espressif/idf:release-v4.0 
Code language: HTML, XML (xml)

Then run simply:

make -f tensorflow/lite/micro/tools/make/Makefile TARGET=esp generate_hello_world_esp_project

Wait until the process ends.

Verify the Tensorflow library

Now, go to <tensorflow-directory>/tensorflow/lite/micro/tools/make/gen. Here you will find the library we built:

Done!!…You have your Tensorflow Lite library for the ESP32.

Running Tensorflow Lite micro on ESP32: Hello World example

Now, we want to test the library and run the Hello World Example on ESP32. This is already covered in other tutorials. Let us create a new project named ESP32-Tensorflow in PlatformIO. Before compiling the Tensorflow example, you have to organize the files shown in the previous picture so that they are compatible with PlatformIO. Do not worry, you only have to re-arrange these files:

• Copy the folder tfmicro (see the picture above) under the lib folder of your PlatformIO project
• Under thirdparty folder move up the inner folders so that they have this structure:

• Copy from main folder of the TensorFlow directory all the files and add them to src folder in PlatformIO project:

Finally, add the following file named partitions.csv to manage how to allocate space on ESP32:

# Name,   Type, SubType, Offset,   Size, Flags
nvs,      data, nvs,     0x9000,   20K,
otadata,  data, ota,     0xe000,   8K,
firm,	  app,	ota_0, 	 , 3400K,
eeprom,   data, 0x99,    , 4K,
spiffs,   data, spiffs,  , 444K,Code language: PHP (php)

You can give a look here to have more info.

You have to add this file to the root of your PlatformIO project and modify the platformio.ini:

[env:esp32dev]
platform = espressif32
board = esp32dev
framework = arduino
board_build.partitions = partitions.csv
monitor_speed = 115200

Compile and run the Tensorflow application on your ESP32.

Notice that the PlatformIO doesn’t have the Serial Plotter feature as in the Arduino IDE. For this reason, you will view all the values in your serial monitor.

Now you can create your ESP32 Machine Learning project using the Tensorflow lite library. If you prefer to run machine learning models on the user browser you should read how to use ESP32-CAM with Tensorflow.js. If you like to explore how to use Machine Learning with ESP32, you can read how to implement a KNN classifier with ESP32.

Wrapping up

At the end of this easy guide about how to run Tensorflow Lite on ESP32, hopefully, you have learned how to setup the environment to compile and run Tensorflow. We have covered two different options that make it easy to execute Tensorflow on ESP32.