Getting started with Flutter Machine Learning with Tensorflow

This flutter Tensorflow lite tutorial covers how to build a Flutter app that classifies images using Machine Learning with Tensorflow. To do it, we will follow these steps:

• Creating Flutter app
• Using Tensorflow with Flutter
• Adding image picker to select the image to classify
• Showing the labels

Creating the Flutter app

The first step is creating a Flutter app. Let us create a new app for Android and iOS:

flutter create <your_app_name>Code language: HTML, XML (xml)

This app uses Getx to make it reactive but you don’t need to know this package.

Adding tensorflow library

Before coding the app, it is necessary to add the following libraries to pubspec.yaml:

image_picker: ^0.8.7+5
# GetX
get: ^4.6.5
# TFLite
tflite_flutter: ^0.10.1
# Image
image: ^4.0.17
Code language: CSS (css)

Inizializing Tensorflow lite

The next step is using Tensorflow with Flutter. By now we can suppose we have an image chosen from the smartphone gallery. Before using the Tensorflow it is necessary to initalize it:

  @override
  void onInit() async {
    isWorking.value = true;
    debugPrint("Initialize the interpreter..");
    InterpreterOptions options = InterpreterOptions();
    _interpreter = await Interpreter.fromAsset(model_path, options: options);
    debugPrint("Interpreter ready..");
    debugPrint("Loading labels...");
    String bundle = await rootBundle.loadString(model_labels);
    _labels = bundle.split('\n');
    debugPrint("Labels loaded");
    isWorking.value = false;
    super.onInit();
  }Code language: JavaScript (javascript)

This method loads the Tensorflow model from the assets folder. You have to add the model under the assets folder. You can download it from Tensorflow hub. In this example, we use TF MobileNet v1.

If you are using Android to test this Flutter app, you have to add this piece of code in androd/app/build.gradle:

aaptOptions {
  noCompress 'tflite' 
  noCompress 'lite'
}
Code language: JavaScript (javascript)

Deploying Tensorflow model to Flutter app

Once Tensorflow is initaliazed in our Flutter app, we can start classifying the images. To do it, it is necessary to follow these steps:

• Resize the image to 224 x 224
• Transform the image in a matrix
• Apply the classification using the image matrix as input
• Get the output result

The first step is resizing the image to 224 x 224:

img.Image resizedImage = img.copyResize(image!, width: 224, height: 224);

Then it is necessary to tranform the image in a Matrix. Our tflite model accepts as input a matrix of 224 x 224 x 3:

final imageMatrix = List.generate(
      224,
      (y) => List.generate(
        224,
        (x) {
          // Get pixel at x,y
          img.Pixel pixel = resizedImage.getPixel(x, y);
          // Build a 3 vector
          return [pixel.r, pixel.g, pixel.b];
        },
      ),
    );Code language: PHP (php)

It is important to notice that the model accepts as input:

final input = [imageMatrix];
Code language: PHP (php)

while the ouput is:

final output = [List<int>.filled(1001, 0)];Code language: PHP (php)

represents the “probability” result. We have 1001 labels and for each one there is the corresponding probability.

Now we can ready to classify the image using Tensorflow in our Flutter app:

debugPrint("Run classification...");
_interpreter.run(input, output);
final result = output.first;
Code language: PHP (php)

How to classify images using Tensorflow lite

Now we can retrieve the classification results:

for (int i = 0; i < result.length; i++) {
  if (result[i] > 60) {
    results[_labels[i]] = result[i];
  }
}Code language: HTML, XML (xml)

Select the image to classify

Now we can focus on the UI. We can select the image from the gallery:

floatingActionButton: FloatingActionButton(
        child: const Icon(Icons.camera_alt_outlined),
        onPressed: () async {
          final ImagePicker imgPicker = ImagePicker();
          final XFile? image =
              await imgPicker.pickImage(source: ImageSource.gallery);
          if (image != null) {
            imageController.imageFile.value = File(image.path);
            // start recognition
            imageController.runClassification();
          }
        },
      ),
Code language: JavaScript (javascript)

The final result is shown below:

Now, we can run our image classification using Tensorflow in our Flatter app:

Source code at https://github.com/survivingwithandroid/flutter-tensorflow-machine-learning

Conclusion

At the end of this tutorial, we have discovered how to build a Flutter app with Tensorflow to classify images.