How to use Voice to control an ESP8266 using Android

In this post, we will describe how to use integrate ESP8266 and Android using voice commands. In other words, in this article, we will use our voice to send a set of commands to an IoT device. During this post, we will build a voice-activated IoT project using ESP8266 and Android. This is an interesting topic because this project uses a different way to interact with IoT device. Usually, we are used to interacting with a device using a simple user interface exposed by the device or a using a smartphone app that sends commands to the device.

How to use voice to control IoT device – Project overview

The idea that stands behind this integration project between ESP8266 and Android is exploring how to use voice commands to control a device like Arduino or ESP8266. To build this voice-activated project, we develop an Android app that is the mean that captures the user’s voice and transforms it into a set of commands that are sent to the device. The picture below describes the project overview:

integrate ESP8266 and Android using voice commands

In more details this project is made by two different sub-systems:

  • Android app
  • IoT app

The Android app takes care to interact with the user and listens to the voice commands. Next, the app translates the voice commands into commands that the IoT device can understand. In this article, as IoT device, we will use an ESP8266 WeMos that controls a NeoPixel Ring. You can use an Arduino Uno instead of the ESP or a MKR1000.

How to develop a speech recognition Android app to connect ESP8266 and Android

The first step in this project is developing an Android app that recognizes user speech. Fortunately, Android provides a built-in system that is able to recognize the user words. The app user interface is very simple. There is only one button that we use to start sending commands:
android voice app IoT
The layout is very simple as shown below:

<android.support.constraint.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:app="http://schemas.android.com/apk/res-auto" xmlns:tools="http://schemas.android.com/tools" android:layout_width="match_parent" android:layout_height="match_parent" tools:context="com.survivingwithandroid.voice.MainActivity" android:id="@+id/mainView" android:background="@drawable/bg_gradient"> 
   <Button android:layout_width="wrap_content" android:layout_height="wrap_content" app:layout_constraintLeft_toRightOf="parent" app:layout_constraintRight_toLeftOf="parent" android:text="Send command" android:id="@+id/btnCommand" app:layout_constraintBottom_toBottomOf="parent" android:layout_marginBottom="15dp"/> 
</android.support.constraint.ConstraintLayout>

The next step is overriding in the MainActivity.java the onCreate method:

 
@Override protected void onCreate(Bundle savedInstanceState) { 
   super.onCreate(savedInstanceState);
   setContentView(R.layout.activity_main); 
   btn = (Button) findViewById(R.id.btnCommand);
   btn.setOnClickListener(new View.OnClickListener() {
     @Override 
     public void onClick(View v) { 
       startVoiceCommand();
     } 
}

where the <code class=”code”>startVoiceCommand() is the method that handles the voice interaction with the user. In this context, to capture the user’s voice the app uses an Intent delegating all the hard work to the Android OS:

 
private void startVoiceCommand() {
  Log.d(TAG, "Starting Voice intent..."); 
  Intent i = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH); 
  i.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, 
             RecognizerIntent.LANGUAGE_MODE); 
  i.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault()); 
  i.putExtra(RecognizerIntent.EXTRA_PROMPT, "Tell me, I'm ready!"); 
  try { 
   startActivityForResult(i, REQ_SPEECH_RESULT);
  } 
  catch (Exception e) {
    Snackbar.make(v, "Speech to text not supported", 
       Snackbar.LENGTH_LONG).show(); 
  } 
} 

This method is very simple, it invokes the intent RecognizerIntent.ACTION_RECOGNIZE_SPEECH providing some configuration parameters as the current locale and the message we want to show to the user. When the user clicks on the button the app shows the dialog waiting for the voice input. Finally, the app starts the intent waiting for the result: android iot voice app

To this purpose, the app overrides the method onActivityResult:

@Override protected void onActivityResult(int requestCode, 
     int resultCode, 
     Intent data) { 
    super.onActivityResult(requestCode, resultCode, data); 
    // Check the Request code
    if (requestCode == REQ_SPEECH_RESULT) { 
      Log.d(TAG, "Request speech result.."); 
      ArrayList<String> results = data.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS); 
      String command = results.get(0); 
      Log.d(TAG, "Current command ["+command+"]"); 
      // Now we send commands to the IoT device
    } 
} 

In the method above, the app extracts the command and according to it invoke the ESP8266 to set the ring LEDs color. In this example, the Android IoT voice app handles simple commands like Red, Green, Blue and so on.

How to exchange data with ESP8266 from the Android app

In this step of this project explaining how to use voice to send commands to an IoT device, we implement the network communication. By now, we can suppose that the ESP8266 exposes a method that the Android app invokes to set the ring colors. In this context, we can suppose that the ESP8266 exposes a RESTful API. In order to invoke this API, the app uses an HTTP connection. To this purpose, it is necessary to create a new class called IoTConnectionHandler that handles all the network details. The class is shown below:

 
public class IoTConnectionHandler { 
   private static IoTConnectionHandler me; 
   private OkHttpClient client;
   private static final String TAG = IoTConnectionHandler.class.getName(); 
   private static final String IOT_URL = "http://192.168.1.9:8080/ring?param=0"; 
   private IoTConnectionHandler() { 
    client = new OkHttpClient(); 
   } 
   public static IoTConnectionHandler getInstance() { 
     if (me == null) 
       me = new IoTConnectionHandler(); 
     return me;
   } 
   public void sendData(String data) { 
      Request req = new Request.Builder() 
                    .url(IOT_URL + data) 
                    .build(); 
      client.newCall(req).enqueue(new Callback() { 
         @Override 
         public void onFailure(Call call, IOException e) { 
           Log.e(TAG, "Connection error", e); 
         } 
         @Override 
         public void onResponse(Call call, Response response) throws IOException { 
           Log.i(TAG, "Command sent"); 
         } 
      }); 
  } 
} 

It is very simple and it uses OkHTTP library. Notice that the data parameter holds the color hex code retrieved from the voice command. The next part is implementing the IoT side of the project that receives the color hex code using the API exposed by the device and set the LEDs color.

How to integrate ESP8266 and Android: Schematic

In this step of this voice-controlled IoT device, we will develop the code necessary to:

  • expose an API invoked by the Android app
  • control a Neopixel RGB ring

Before diving into the IoT project details, it is useful to know how to connect the ESP8266 to the Neopixel RGB ring:

esp8266 neopixel ring wiring

To simplify the code development the IoT devices uses these two libraries:

The first one is used to control the LEDs ring while the second library is necessary to expose some functions in the sketch as API. If you are new to this library, you can read my previous post describing how to expose Arduino functions as API.

The code below is the sketch:

 
#include <Adafruit_NeoPixel.h>
#include <ESP8266WiFi.h>
#include <aREST.h>
#define PIN D2 
#define NUMS 12 
#define SERVER_PORT 8080 
// Neopixel rings 
Adafruit_NeoPixel pixels = Adafruit_NeoPixel(12, PIN, NEO_GRB + NEO_KHZ800); 
aREST rest = aREST(); 
char *ssid = "xxxxx"; 
char *pwd = "xxx"; 
// Let us create the server 
WiFiServer server(SERVER_PORT); 
void setup() { 
   Serial.begin(9600); 
   pixels.begin(); 
   pixels.setBrightness(85); 
   // Register the function 
   rest.function("ring", setColor); 
   WiFi.begin(ssid, pwd); 
   Serial.println("Connecting to WiFi..."); 
   while (WiFi.status() != WL_CONNECTED) { 
     delay(1000); 
     Serial.println("Try again...."); 
   } 
 Serial.println("WiFi connected..."); 
 // let us start the server server.begin(); 
}
void loop() { 
   WiFiClient client = server.available(); 
   if (!client) { return ; } 
   while (!client.available()) { delay(1); } 
   rest.handle(client); 
 } 

int setColor(String color) { 
  Serial.println("Hex color [" + color + "]"); 
  long tmpColor = strtol( &("#" + color)[1], NULL, 16); 
  Serial.println("Int ["+String(tmpColor)+"]"); 
  int r = tmpColor << 16; 
  int g = tmpColor << 8 &amp; 0xFF; 
  int b = tmpColor & 0xFF; 
  Serial.print("Red [" + String(r) + "]"); 
  Serial.print("Green [" + String(g) + "]"); 
  Serial.println("Blue [" + String(b) + "]"); 
  for (int i = 0; i < 12; i++) 
   pixels.setPixelColor(i, pixels.Color(r,g,b)); 
  pixels.show(); 
return 1; } 

In details, at the beginning, the sketch tries to connect to the WiFi network. To this purpose, you have to provide the WiFi ssid and the password. Once the connection is established, the sketch configures the server and its port. Moreover, it declares the function we want to export as API.

The last of part is the method used to control the Neopixel ring.

As soon as we send a voice command to the IoT device app shows these logs:

iot app logs

Summary

At the end of this post, you know how to use voice to control an IoT device integrating ESP8266 and Android app. In more details, you have explored how to connect and Android app with an ESP8266 and how to use voice to interact with it.

More posts...

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close