Interact With the Web in Real-time Using Arduino, Firebase and Angular.js

This simple project is meant to be a hybrid introduction to connecting and manipulating data over the internet with Arduino, Node, Angular and Firebase.

UPDATE: Check out @Moycs777 terrific web app based on this example hosted on Firebase.

COMMENTS: Please do contribute with your comments. I’d like to know how you like (or don’t like) it.

The Internet of Things is nothing new. You may have been using it all along. In the broadest sense, your laptop and smartphones are IoT objects. What’s actually new is the “T” part. We have been using computers and smartphones so often that we hardly recognize them as “things.” However, “things” are more synonymous to everyday objects such as clothes, furnitures, fridges, clocks, books, lamps, skateboards, bicycles, and etc. IoT is when a coffee machine that brews you a cup of java when the weather gets too cold, a pair of shoes that lights up 10 minutes before your train arrives, or a door knob that alerts your phone when your parents try to trespass into your room.

To be able to connect to the internet, make sense of its data and interact with the users, these things need tiny computers aka micro controllers to make them conscious.

What we are building

We are going to connect an Arduino board to the internet and change the RGB color property on a web page in real-time by rotating a potentiometer.

What we need:

• Arduino Uno
• A-B USB cable
• Potentiometer (aka pot)
• RGB LED
• 330 Ohms resistors x 3
• 10 KOhms resistor x 1
• Male-to-male jumper wires x 9

Everything is included in Sparkfun’s inventor’s kit, which is quite neat to get your hands on. No wifi or ethernet shield needed, since we’re actually persisting the data to the online database and communicate with the web app via the REST API provided by Firebase.

If you have not installed Node, head to Node.js home and follow instructions to install Node and npm.

All the codes can be downloaded from my repo or cloned using git:

$ git clone https://github.com/jochasinga/firepot

Once downloaded, cd firepot to getinto the directory, and you should see two subdirectories—pot and app. cd into each one and install the dependencies:

$ cd pot && npm install

All dependencies are indicated in package.json, and npm automatically install them according to the information. They will be collected in a new subdirectory node_modules and can be required by any code within the project scope.

Connecting the circuit

Connect your circuit according to the diagram. A typical potentiometer has three leads. Connect one end to +5V power via 10 KOhms pull-up resistor and the other far end to ground (0V). The pot provides variable resistance between the voltage, and the middle lead should be connected to Arduino’s analog input pin (A0) to feed the voltage signal into the Arduino.

RGB LED is essentially three LEDs in one, each with different color of red, green and blue, together producing 16,777,216 possible colors. We are going to use the pot to traverse this color range from pure red (#FF0000) to blue (#0000FF) to green (#00FF00) and back to red again. The longest lead of the LED is called a common, and should be connected to ground. The rest are connected to Arduino’s PWMoutputs (those with ~ preceding the number). The code connects red, green and blue leads to ~9, ~10 and ~11 respectively.

Connect the Arduino to your laptop via the USB cable, and you’re good to go.

Signing up with Firebase

Firebase is a JSON-style database service for real-time applications (You’ll need a free signup to use). Firebase implements a clever way of manipulating JSON data by adopting RESTful APIs. In this project, we will CREATE, READ and UPDATE a data chunk that looks like this:

"colors" : {
  "r" : 255,
  "g" : 0,
  "b" : 0
}

Firebase has an easy way of getting to your data via REST api, i.e. you can get your JSON data at https://burning-limbo-6666.firebaseio.com/colors.json, where https://burning-limbo-6666.firebaseio.com is the domain address of your app which Firebase generates for you after creating a new app, and /colors is the parent node of your data. Firebase has a data dashboard at the very URL, so you can just paste the address into the browser and hit it after you’ve updated the data there in the next section to see your data changed by the pot in real-time.

pot.js

Johnny-five is a javaScript library that wraps around Arduino’s C/C++ language and interface with the board via Firmata firmware, created by the awesome Rick Waldron, and is synonymous to the Nodebot Movement. In order to make it work, you must open the Arduino IDE to flash a standard Firmata code onto the board. On your IDE, go to File > Examples > Firmata > StandardFirmata and upload the code (Don’t forget to select the right board and serial port in the Tools menu). Once the upload have finished, you can close the IDE. Now, let’s have a look at our pot.js code.

var Firebase = require("firebase");
var five = require("johnny-five");

// Create a new reference of Firebase db
var firebaseRef = new Firebase(
  // fictional URL, replace it with your own from Firebase
  "https://burning-limbo-6666.firebaseio.com/colors"
);

five.Board().on("ready", function() {
  var maxValue = 511;
  var colRange = 6;
  var offset   = maxValue / colRange;

  // Create a new pot instance
  var pot = new five.Sensor({
    pin: "A0",
    freq: 250
  });

  // Create a new led array based on pin number
  var ledArray = new five.Led.Array([9, 10, 11]);

  // Listen on data change
  pot.on("data", function() {

    var self = this.value;
    // Print pot value 
    console.log(self);

    // Map dynamic color brightness to pot value
    // RED - MAGENTA - BLUE
    var redDec   = Math.round(five.Fn.map(self, offset, offset*2, 255, 0));
    var blueInc  = Math.round(five.Fn.map(self, 0, offset, 0, 255));
    // BLUE - CYAN - GREEN
    var blueDec  = Math.round(five.Fn.map(self, offset*3, offset*4, 255, 0));
    var greenInc = Math.round(five.Fn.map(self, offset*2, offset*3, 0, 255));
    // GREEN - YELLOW - RED
    var greenDec = Math.round(five.Fn.map(self, offset*5, offset*6, 255, 0));
    var redInc   = Math.round(five.Fn.map(self, offset*4, offset*5, 0, 255));

    // Adjusting color brightness conditionally based on 
    // the location of the pot output value.
    switch (true) {
      case (self > 0 && self <= offset):
        console.log("1st loop");
        ledArray[0].brightness(255);
        ledArray[2].brightness(blueInc);
        ledArray[1].brightness(0);
	// update firebase colors' child node r, g, b
	firebaseRef.set({"r": 255, "b": blueInc, "g": 0});
        break;
      case (self > offset && self <= offset*2):
        console.log("2nd loop");
        ledArray[0].brightness(redDec);
        ledArray[2].brightness(255);
        ledArray[1].brightness(0);
	// update firebase colors' child node r, g, b
	firebaseRef.set({"r": redDec, "b": 255, "g": 0});
	break;
      case (self > offset*2 && self <= offset*3):
        console.log("3rd loop");
        ledArray[0].brightness(0);
        ledArray[2].brightness(255);
        ledArray[1].brightness(greenInc);
	// update firebase colors' child node r, g, b
	firebaseRef.set({"r": 0, "b": 255, "g": greenInc});
	break;
      case (self > offset*3 && self <= offset*4):
        console.log("4th loop");
        ledArray[0].brightness(0);
        ledArray[2].brightness(blueDec);
        ledArray[1].brightness(255);
	// update firebase colors' child node r, g, b
	firebaseRef.set({"r": 0, "b": blueDec, "g": 255});
        break;
      case (self > offset*4 && self <= offset*5):
        console.log("5th loop");
        ledArray[0].brightness(redInc);
        ledArray[2].brightness(0);
        ledArray[1].brightness(255);
	// update firebase colors' child node r, g, b
	firebaseRef.set({"r": redInc, "b": 0, "g": 255});
	break;
      case (self > offset*5 && self <= offset*6):
        console.log("6th loop");
        ledArray[0].brightness(255);
        ledArray[2].brightness(0);
        ledArray[1].brightness(greenDec);
	// update firebase colors' child node r, g, b
	firebaseRef.set({"r": 255, "b": 0, "g": greenDec});
	break;
      default:
	console.log("Out of range");
        ledArray[0].brightness(255);
	ledArray[2].brightness(0);
	ledArray[1].brightness(0);
	// update firebase colors' child node r, g, b
	firebaseRef.set({"r": 255, "b": 0, "g": 0});
    }
  });
});

First, (1–3) we require our installed dependencies for the app, firebase and johnny-five, then we (5–8)create a new firebase reference firebaseRef to where your data is stored. After that, (10) we create a new johnny-five instance of the Arduino board, and hook it to a callback function which will execute the rest of the code once it’s ready. (11–13) I assign the max value we expect from the pot to a variable and divide it by the RGB color subrange number to obtain a standard offset “distance” used as a step variable to calculate where the output value from the pot is on the RGB strip i.e. offset is MAGENTA, offset * 2 is BLUE, offset * 3 is CYAN and so on. You can see how I divided the RGB color strip into 6 subranges, as graphically shown below.

Normally on 5V power, a pot will convert analog signal (0–5V) to digital and give out a range of integer from 0–1023. In my case, my little pot maxes at half of that, so my maxValue lies somewhere around 511(Check this value by logging the output with *console.log()).*Then (16–19),create a new instance of the pot sensor, set its analog pin to A0 and frequency to 250. (22) Assign each LED’s pin to an array variable. Now, (25++) set our pot instance to listen to the “data” event, and within the callback function is the rest of the code that (27–40)calculates and maps the pot’s output range (0-maxValue) to a range of 0–255 (LED’s brightness)using our obtained step variable offset. (44–1o2) I use switch-case loop to conditionally adjusts each LED’s color brightness with Led.brightness method and saving these values to Firebase with Firebase.set method according to where the pot value is.

After that, run pot.js with node commandline

$ node pot.js

Your LED should light up and your terminal should be loop-printing the value from the pot (self) and which loop (or subrange) your pot value is currently in. Try spinning the pot to see the printed data changed as the LED’s color gradually shift. Then, browse to your Firebase dashboard using your app’s URL (i.e. https://burning-limbo-6666.firebaseio.com/colors). You should see your data change as you rotate your pot. We have successfully CREATEd and UPDATEd data on a database like you would have done with web forms, sliders or buttons. That concludes the hardware side.

app.js

Now we are going to work on our client, a simple web app. The structure of the app directory is as follow:

app
├── app.js
├── node_modules
|   ├── express
|   ├── firebase
|   └── socket.io
├── package.json
└── public
    ├── index.html    
    └── index.js

If you have not installed dependencies, you will probably not see node_modules subdirectory in there. Do so now using npm install.

$ cd app && npm install

Take note of the public directory. app.js is a server code which serve static contents from public directory, in this case, index.html and index.js. Let’s hop into app.js:

var Firebase = require("firebase");
var express = require("express");

// Create HTTP Server
var app = express();
var server = require("http").createServer(app);
// Attach Socket.io server 
var io = require("socket.io")(server);
// Indicate port 3000 as host
var port = process.env.PORT || 3000;

// Create a new firebase reference
var firebaseRef = new Firebase(
  "https://burning-limbo-6666.firebaseio.com/colors"
);

// Make the server listens on port 3000
server.listen(port, function() {
  console.log("Server listening on port %d", port);
});

// Routing to static files
app.use(express.static(__dirname + "/public"));

// Socket server listens on connection event
io.on("connection", function(socket) {
  console.log("Connected and ready!");
  
  // firebase reference listens on value change, 
  // and return the data snapshot as an object
  firebaseRef.on("value", function(snapshot) {
    var colorChange = snapshot.val();
    
    // Print the data object's values
    console.log("snapshot R: " + colorChange.r);
    console.log("snapshot B: " + colorChange.b);
    console.log("snapshot G: " + colorChange.g);
  });
});

What this code does is (5–18) creating a Node web server to listen to requests on port 3000 and (21)serve the front-end contents from inside public directory. Then (24–25) the server waits for a socket connection (i.e. GET request), print out “Connect and ready!” message, and (29++) start tracking data from Firebase and printing out changes. Firebase is not necessary here, since we will be using Angularfire library in public/index.js to sync data from Firebase directly there, but it’s intentionally included in here to exhibit a basic firebase method of detecting data changes and retrieving the snapshot of them. The most important part here is serving the public/index.html page and run script in public/index.js.

index.html

Our web page will be displaying R : G : B values dynamically from Firebase and changing the background color of the div according to how you rotate your pot. We’re going to use AngularFire, Firebase client library supporting Angular.js.

<!DOCTYPE html>
<!--Directive-->
<html ng-app="app">
  <head>
  </head>
  <!--Controller-->
  <body ng-controller="Ctrl">
    <div class="header">
      <!--Style-->
      <h1 ng-style="{'background-color': 'rgb(' + data.r + ',' + data.g + ',' + data.b +')'}">                      
        Afro Circus! {{ data.r }} : {{ data.g }} : {{ data.b }}                                                     
      </h1>
    </div>
    <!--Scripts-->
    <script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.3.0-beta.19/angular.min.js"></script>
    <script src="https://cdn.firebase.com/libs/angularfire/0.8.2/angularfire.min.js"></script>
    <script src="/index.js"></script>
  </body>
</html>

https://gist.github.com/jochasinga/c720677640e026381366

This html view(V) binds its part to a data model(M) that syncs data from your Firebase storage, and as data change, only that part is re-rendered. Angular operates on what’s called a “directive.” Directives add new functionalities to HTML elements instead of manipulating the DOM as in JQuery. (3) ng-app directivestarts the application anddefines the scope of the binding, (7) ng-controller defines the application controller© and scope that that particular controller method has effect on, and (10) ng-style allows dynamic styling of document (like you would have done with JQuery’s .css or .addClass). To display data from the model, double-brackets ({{ }}) is used to contain the variable, which is a common way to do it in other web frameworks’ template language. Never mind the data object for now, you’ll see it in public/index.js. Ensure that you have included the scripts before .

index.js

This is the engine room of our front end. In this file, we attach the firebase module to the app, define the controller method and sync the data from firebase to a local model object used by the html binding.

// Register firebase module                                                                                         
var app = angular.module("app", ["firebase"]);

// Set up controller function                                                                                       
app.controller("Ctrl", function($scope, $firebase) {
    var firebaseRef = new Firebase(
      // Replace this fictional URL with your own
      "https://burning-limbo-6666.firebaseio.com/colors"                                                          
    );
    // create an AngularFire ref to the data                                                                        
    var sync = $firebase(firebaseRef);

    // pull the data into a local model                                                                             
    var syncObject = sync.$asObject();

    // sync the object with three-way data binding                                                                  
    syncObject.$bindTo($scope, "data");
});

(2) Register firebase service to Angular app. After that, (5) you’ll have $firebase variable available for injecting into the controller. (6–9) Setting up the first part of the controller method is familiar—we create a firebase reference to our data. Now, (11) use the reference as a parameter to $firebase() to create an Angularfire reference to the data. (14)We translate the data into a JavaScript object, and (17) bind it to the variable “data” that will be used in index.html template.

Whew! That was some work, right? Now comes the exciting part. Go back to pot directory and run your pot code again with:

$ node pot.js

Once you’ve got your LED turned on and the pot value start printing to the console, from another terminal window, run your app.js inside app directory:

$ node app.js

The console should start by printing “Server listening on port 3000" then gushing out RGB values from Firebase. Go to your web browser and browser to http://localhost:3000, hopefully, you’ll get like the video below.

If you like this article, please recommend and retweet. Feel free to shoot me at [email protected]. I’m up for talking, exchanging ideas, collaborations or consults. Comments are welcomed.

30s ad

☞ Arduino Fun: Make a high-tech remote controlled car

☞ DIY with Arduino - 5 Simple Projects to Get You Started

☞ Learn PCB Design By Designing an Arduino Nano in Altium

☞ Beyond Arduino, Part 2: Analog Input Output

☞ Arduino for Beginners

In this article, I am going to show you how to send sensor sensor data to a local webserver using esp8266.

Before delving into this article , let me explain the term localhost.
Localhost refers to the local computer that a program is running on. For example, if you are running a Web browser on your computer, your computer is considered to be the "localhost".

The deliverables for this project are:

Xamp webserver download it here https://www.apachefriends.org/

Arduino IDE download here https://www.arduino.cc/

ESP8266 official website https://www.nodemcu.com/

Smartphone used as a router.

Project requirement: Create a folder in the xamp/htdocs named esp8266. In this folder you are going to save the postData.php and databaseConfig.php

To send the sensor data to a database, we need to write an arduino sketch using a TCP protocol to communicate wirelessly to the local webserver.

Let's get it done now!

Step 1: Install esp8266 board the link to the procedure can be found here https://randomnerdtutorials.com/how-to-install-esp8266-board-arduino-ide/

Step 2: The arduino sketch is shown below. I create two function connectWifi() and sendSensorData() which I invoke respectively in the void setup and void loop. If you are confused please look for a tutorial on basic arduino code.

#include <WiFiClient.h> 
#include <ESP8266WebServer.h>
#include <ESP8266HTTPClient.h>
const char *ssid = "MelleB";  //ENTER YOUR WIFI ssid
const char *password = "refj4497";  //ENTER YOUR WIFI password


void setup() {
connectWifi();
}


void loop() {
SendSensorData();
}
//function to connect to wifi
void connectWifi(){
  delay(1000);
  Serial.begin(115200);
  WiFi.mode(WIFI_OFF);        //Prevents reconnection issue (taking too long to connect)
  delay(1000);
  WiFi.mode(WIFI_STA);        //This line hides the viewing of ESP as wifi hotspot
  WiFi.begin(ssid, password);     //Connect to your WiFi router
  Serial.println("");
  Serial.print("Connecting");
  // Wait for connection
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  //If connection successful show IP address in serial monitor 
  Serial.print("Connected to ");
  Serial.println(ssid);
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());  //IP address assigned to your ESP
}


//function to send sensor data 
void SendSensorData() {
  HTTPClient http;    //Declare object of class HTTPClient
  String sensorData1,sensorData2,sensorData3,sensorData4,sensorData5,sensorData6,sensorData7, postData;
  sensorData1="High";
    sensorData2="High";
      sensorData3="High";
        sensorData4="High";
          sensorData5="High";
            sensorData6="High";
              sensorData7="High";
  //Post Data
  postData = "sensor1=" +  sensorData1 + "&sensor2=" + sensorData2+ "&sensor3=" + sensorData3+  "&sensor4=" + sensorData4+ "&sensor5=" + sensorData5+ "&sensor6=" + sensorData6+  "&sensor7=" + sensorData7;
  
  http.begin("http://192.168.43.142/esp8266/postData.php");              //change the ip to your computer ip address
  http.addHeader("Content-Type", "application/x-www-form-urlencoded");    //Specify content-type header
 
  int httpCode = http.POST(postData);   //Send the request
  String payload = http.getString();    //Get the response payload
 
  Serial.println(httpCode);   //Print HTTP return code
  Serial.println(payload);    //Print request response payload
 
  http.end();  //Close connection
  
  delay(5000);  //Post Data at every 5 seconds
}

Step 3: Create the database sensor and the table name logs and save it in the esp866 folder as databaseConfig.php

//connect to localhost if not exists
	$servername = "localhost";
	$username = "root";
	$password = "";


	// Create connection
	$conn = new mysqli($servername, $username, $password);
	// Check connection
	if ($conn->connect_error) {
	    die("Connection failed: " . $conn->connect_error);
	}


	// Create database
	$sql = "CREATE DATABASE sensor";
	
	    echo "Database created successfully";
	} else {
	    echo "Error creating database: " . $conn->error;
	}


	$conn->close();


	echo "<br>";
//Connect to database and create table
	$servername = "localhost";
	$username = "root";
	$password = "";
	$dbname = "sensor";


	// Create connection
	$conn = new mysqli($servername, $username, $password, $dbname);
	// Check connection
	if ($conn->connect_error) {
	    die("Connection failed: " . $conn->connect_error);
	}


	
	$sql = "CREATE TABLE logs (
	id INT(6) UNSIGNED AUTO_INCREMENT PRIMARY KEY,
	sensor1 VARCHAR(30),
	sensor2 VARCHAR(30),
    sensor3 VARCHAR(30),
	sensor4 VARCHAR(50),
	sensor5 VARCHAR(50),
	sensor6 VARCHAR(50),
	sensor7 VARCHAR(50),
	\`Date\` DATE NULL,
	\`Time\` TIME NULL, 
	\`TimeStamp\` TIMESTAMP NULL DEFAULT CURRENT\_TIMESTAMP ON UPDATE CURRENT\_TIMESTAMP)";


	if ($conn->query($sql) === TRUE) {
	    echo "Table logs created successfully";
	} else {
	    echo "Error creating table: " . $conn->error;
	}


	$conn->close();
?>

Step 4: Create postData.php in the esp866 folder and paste the code below in it.

//Creates new record as per request
    //Connect to database
    $servername = "localhost";
    $username = "root";
    $password = "";
    $dbname = "sensor";


    // Create connection
    $conn = new mysqli($servername, $username, $password, $dbname);
    // Check connection
    if ($conn->connect_error) {
        die("Database Connection failed: " . $conn->connect_error);
    }


    //Get current date and time
    date\_default\_timezone_set('Asia/Kolkata');
    $d = date("Y-m-d");
    //echo " Date:".$d."<BR>";
    $t = date("H:i:s");


    if(!empty($\_POST\['sensor1'\]) || !empty($\_POST\['sensor2'\]))
    {
    	$sensorData1 = $_POST\['sensor1'\];
    	$sensorData2 = $_POST\['sensor2'\];
		$sensorData3 = $_POST\['sensor3'\];
		$sensorData4 = $_POST\['sensor4'\];
		$sensorData5 = $_POST\['sensor5'\];
		$sensorData6 = $_POST\['sensor6'\];
		$sensorData7 = $_POST\['sensor7'\];


	    $sql = "INSERT INTO logs (sensor1, sensor2,sensor3,sensor4,sensor5,sensor6,sensor7, Date, Time) VALUES ('".$sensorData1."', '".$sensorData2."', '".$sensorData3."', '".$sensorData4."', '".$sensorData5."', '".$sensorData6."', '".$sensorData7."', '".$d."', '".$t."')";


		if ($conn->query($sql) === TRUE) {
		    echo "OK";
		} else {
		    echo "Error: " . $sql . "<br>" . $conn->error;
		}
	}




	$conn->close();
?>

Step 5: software testing

• Open your web browser and type localhost/esp8266. Click on databaseConfig.php you should get a message data base sensor created successfully and do the same with postData.php. If you do not get any error message then you are good to go.
• Open the serial monitor of your arduino and set the baud rate to 115200
• The wifi ssid and ip address should display to the serial monitor
• You should get a response 200 from the server and ok message
• Check your database table you will get all the sensor table high, the date and time.
• ** The output is shown below**

Please let me know in case of any errors.

Get Started with Machine Learning on Arduino .Arduino is on a mission to make Machine Learning simple enough for anyone to use. We’ve been working with the TensorFlow Lite team over the past few months and are excited to show you what we’ve been up to together:

Arduino is on a mission to make Machine Learning simple enough for anyone to use. We’ve been working with the TensorFlow Lite team over the past few months and are excited to show you what we’ve been up to together: bringing TensorFlow Lite Micro to the Arduino Nano 33 BLE Sense. In this article, we’ll show you how to install and run several new TensorFlow Lite Micro examples that are now available in the Arduino Library Manager.

The first tutorial below shows you how to install a neural network on your Arduino board to recognize simple voice commands.

Running the pre-trained micro_speech inference example.

Next, we’ll introduce a more in-depth tutorial you can use to train your own custom gesture recognition model for Arduino using TensorFlow in Colab. This material is based on a practical workshop held by Sandeep Mistry and Don Coleman, an updated version of which is now online.

If you have previous experience with Arduino, you may be able to get these tutorials working within a couple of hours. If you’re entirely new to microcontrollers, it may take a bit longer.

We’re excited to share some of the first examples and tutorials, and to see what you will build from here. Let’s get started!

Note: the following projects are based on TensorFlow Lite for Microcontrollers which is currently experimental within the TensorFlow repo. This is still a new and emerging field!

Microcontrollers and TinyML

Microcontrollers, such as those used on Arduino boards, are low-cost, single chip, self-contained computer systems. They’re the invisible computers embedded inside billions of everyday gadgets like wearables, drones, 3D printers, toys, rice cookers, smart plugs, e-scooters, washing machines. The trend to connect these devices is part of what is referred to as the Internet of Things.

Arduino is an open-source platform and community focused on making microcontroller application development accessible to everyone. The board we’re using here has an Arm Cortex-M4 microcontroller running at 64 MHz with 1MB Flash memory and 256 KB of RAM. This is tiny in comparison to Cloud, PC, or Mobile but reasonable by microcontroller standards.

Arduino Nano 33 BLE Sense board is smaller than a stick of gum

There are practical reasons you might want to squeeze ML on microcontrollers, including:

• Function — wanting a smart device to act quickly and locally (independent of the Internet).
• Cost — accomplishing this with simple, lower cost hardware.
• Privacy — not wanting to share all sensor data externally.
• Efficiency — smaller device form-factor, energy-harvesting or longer battery life.

There’s a final goal which we’re building towards that is very important:

• Machine learning can make microcontrollers accessible to developers who don’t have a background in embedded development

On the machine learning side, there are techniques you can use to fit neural network models into memory constrained devices like microcontrollers. One of the key steps is the quantization of the weights from floating point to 8-bit integers. This also has the effect of making inference quicker to calculate and more applicable to lower clock-rate devices.

TinyML is an emerging field and there is still work to do — but what’s exciting is there’s a vast unexplored application space out there. Billions of microcontrollers combined with all sorts of sensors in all sorts of places which can lead to some seriously creative and valuable Tiny ML applications in the future.

What you need to get started

• An Arduino Nano 33 BLE Sense board
• A Micro USB cable to connect the Arduino board to your desktop machine
• To program your board, you can use the Arduino Web Editor or install the Arduino IDE. We’ll give you more details on how to set these up in the following section

The Arduino Nano 33 BLE Sense has a variety of onboard sensors meaning potential for some cool Tiny ML applications:

• Voice — digital microphone
• Motion — 9-axis IMU (accelerometer, gyroscope, magnetometer)
• Environmental — temperature, humidity and pressure
• Light — brightness, color and object proximity

Unlike classic Arduino Uno, the board combines a microcontroller with onboard sensors which means you can address many use cases without additional hardware or wiring. The board is also small enough to be used in end applications like wearables. As the name suggests, it has Bluetooth LE connectivity so you can send data (or inference results) to a laptop, mobile app or other BLE boards and peripherals.

Tip: Sensors on a USB stick

Connecting the BLE Sense board over USB is an easy way to capture data and add multiple sensors to single board computers without the need for additional wiring or hardware — a nice addition to a Raspberry Pi, for example.

TensorFlow Lite for Microcontrollers examples

The inference examples for TensorFlow Lite for Microcontrollers are now packaged and available through the Arduino Library manager making it possible to include and run them on Arduino in a few clicks. In this section we’ll show you how to run them. The examples are:

• micro_speech — speech recognition using the onboard microphone
• magic_wand — gesture recognition using the onboard IMU
• person_detection — person detection using an external ArduCam camera

For more background on the examples you can take a look at the source in the TensorFlow repository. The models in these examples were previously trained. The tutorials below show you how to deploy and run them on an Arduino. In the next section, we’ll discuss training.

How to run the examples using Arduino Create web editor

Once you connect your Arduino Nano 33 BLE Sense to your desktop machine with a USB cable you will be able to compile and run the following TensorFlow examples on the board by using the Arduino Create web editor:

Compiling an example from the Arduino_TensorFlowLite library

Focus on the speech recognition example: micro_speech

One of the first steps with an Arduino board is getting the LED to flash. Here, we’ll do it with a twist by using TensorFlow Lite Micro to recognise voice keywords. It has a simple vocabulary of “yes” and “no”. Remember this model is running locally on a microcontroller with only 256KB of RAM, so don’t expect commercial ‘voice assistant’ level accuracy — it has no Internet connection and on the order of 2000x less local RAM available.

Note the board can be battery powered as well. As the Arduino can be connected to motors, actuators and more, this offers the potential for voice controlled projects.

How to run the examples using the Arduino IDE

Alternatively, you can use try the same inference examples using Arduino IDE application.

First, follow the instructions in the next section Setting up the Arduino IDE.

In the Arduino IDE, you will see the examples available via the File > Examples > Arduino_TensorFlowLite menu in the ArduinoIDE.

Select an example and the sketch will open. To compile, upload, and run the examples on the board, and click the arrow icon:

For advanced users who prefer a command line, there is also the arduino-cli.

Training a TensorFlow Lite Micro model for Arduino

Gesture classification on Arduino BLE 33 Nano Sense, output as Emojis

Next, we will use ML to enable the Arduino board to recognise gestures. We’ll capture motion data from the Arduino Nano 33 BLE Sense board, import it into TensorFlow to train a model, and deploy the resulting classifier onto the board.

The idea for this tutorial was based on Charlie Gerard’s awesome Play Street Fighter with body movements using Arduino and Tensorflow.js. In Charlie’s example, the board is streaming all sensor data from the Arduino to another machine which performs the gesture classification in Tensorflow.js. We take this further and “TinyML-ify” it by performing gesture classification on the Arduino board itself. This is made easier in our case as the Arduino Nano 33 BLE Sense board we’re using has a more powerful Arm Cortex-M4 processor, and an on-board IMU.

We’ve adapted the tutorial below, so no additional hardware is needed — the sampling starts on detecting movement of the board. The original version of the tutorial adds a breadboard and a hardware button to press to trigger sampling. If you want to get into a little hardware, you can follow that version instead.

Setting up the Arduino IDE

Following the steps below sets up the Arduino IDE application used to both upload inference models to your board and download training data from it in the next section. There are a few more steps involved than using Arduino Create web editor because we will need to download and install the specific board and libraries in the Arduino IDE.

• Download and install the Arduino IDE fromhttps://arduino.cc/downloads
• Open the Arduino application you just installed
• In the Arduino IDE menu select Tools > Board > Boards Manager…
• Search for “Nano BLE” and press install on the board
• It will take several minutes to install
• When it’s done close the Boards Manager window

• Now go to the Library Manager Tools > Manage Libraries…
• Search for and install the Arduino_TensorFlowLite library
• Next search for and install the Arduino_LSM9DS1 library:

• Finally, plug the micro USB cable into the board and your computer
• Choose the board Tools > Board > Arduino Nano 33 BLE
• Choose the port Tools > Port > COM5 (Arduino Nano 33 BLE)
• Note that the actual port name may be different on your computer

There are more detailed Getting Started and Troubleshooting guides on the Arduino site if you need help.

Streaming sensor data from the Arduino board

First, we need to capture some training data. You can capture sensor data logs from the Arduino board over the same USB cable you use to program the board with your laptop or PC.

Arduino boards run small applications (also called sketches) which are compiled from .ino format Arduino source code, and programmed onto the board using the Arduino IDE or Arduino Create.

We’ll be using a pre-made sketch IMU_Capture.ino which does the following:

• Monitor the board’s accelerometer and gyroscope
• Trigger a sample window on detecting significant linear acceleration of the board
• Sample for one second at 119Hz, outputting CSV format data over USB
• Loop back and monitor for the next gesture

The sensors we choose to read from the board, the sample rate, the trigger threshold, and whether we stream data output as CSV, JSON, binary or some other format are all customizable in the sketch running on the Arduino. There is also scope to perform signal preprocessing and filtering on the device before the data is output to the log — this we can cover in another blog. For now, you can just upload the sketch and get to sampling.

To program the board with this sketch in the Arduino IDE:

• Download IMU_Capture.ino and open it in the Arduino IDE
• Compile and upload it to the board with Sketch > Upload

Visualizing live sensor data log from the Arduino board

With that done, we can now visualize the data coming off the board. We’re not capturing data yet — this is just to give you a feel for how the sensor data capture is triggered and how long a sample window is. This will help when it comes to collecting training samples.

• In the Arduino IDE, open the Serial Plotter Tools > Serial Plotter
• If you get an error that the board is not available, reselect the port:
• Tools > Port > portname (Arduino Nano 33 BLE)
• Pick up the board and practice your punch and flex gestures
• You’ll see it only sample for a one second window, then wait for the next gesture
• You should see a live graph of the sensor data capture (see GIF below)

When you’re done be sure to close the Serial Plotter window — this is important as the next step won’t work otherwise.

Capturing gesture training data

To capture data as a CSV log to upload to TensorFlow, you can use Arduino IDE > Tools > Serial Monitor to view the data and export it to your desktop machine:

• Reset the board by pressing the small white button on the top
• Pick up the board in one hand (picking it up later will trigger sampling)
• In the Arduino IDE, open the Serial Monitor Tools > Serial Monitor
• If you get an error that the board is not available, reselect the port:
• Tools > Port > portname (Arduino Nano 33 BLE)
• Make a punch gesture with the board in your hand (Be careful whilst doing this!)
• Make the outward punch quickly enough to trigger the capture
• Return to a neutral position slowly so as not to trigger the capture again
• Repeat the gesture capture step 10 or more times to gather more data
• Copy and paste the data from the Serial Console to new text file called punch.csv
• Clear the console window output and repeat all the steps above, this time with a flex gesture in a file called flex.csv
• Make the inward flex fast enough to trigger capture returning slowly each time

Note the first line of your two csv files should contain the fields aX,aY,aZ,gX,gY,gZ

Linux tip: if you prefer you can redirect the sensor log output from the Arduino straight to a .csv file on the command line. With the Serial Plotter / Serial Monitor windows closed use:

$ cat /dev/cu.usbmodem[nnnnn] > sensorlog.csv

Training in TensorFlow

We’re going to use Google Colab to train our machine learning model using the data we collected from the Arduino board in the previous section. Colab provides a Jupyter notebook that allows us to run our TensorFlow training in a web browser.

The Colab will step you through the following:

• Setup Python Environment
• Upload the punch.csv and flex.csv data
• Parse and prepare the data
• Build & Train the Model
• Convert the Trained Model to TensorFlow Lite
• Encode the Model in an Arduino Header File

The final step of the colab is generates the model.h file to download and include in our Arduino IDE gesture classifier project in the next section:

Let’s open the notebook in Colab and run through the steps in the cells — arduino_tinyml_workshop.ipynb

Classifying IMU Data

Next we will use model.h file we just trained and downloaded from Colab in the previous section in our Arduino IDE project:

1. Open IMU_Classifier.ino in the Arduino IDE.
2. Create a new tab in the IDE. When asked name it model.h

3. Open the model.h tab and paste in the version you downloaded from Colab

4. Upload the sketch: Sketch > Upload

5. Open the Serial Monitor: Tools > Serial Monitor

6. Perform some gestures

7. The confidence of each gesture will be printed to the Serial Monitor (0 = low confidence, 1 = high confidence)

For added fun, the Emoji_Button.ino example shows how to create a USB keyboard that prints an emoji character in Linux and macOS. Try combining the Emoji_Button.ino example with the IMU_Classifier.ino sketch to create a gesture controlled emoji keyboard 👊.

Conclusion

It’s an exciting time with a lot to learn and explore in Tiny ML. We hope this blog has given you some idea of the potential and a starting point to start applying it in your own projects. Be sure to let us know what you build and share it with the Arduino community.

For a comprehensive background on TinyML and the example applications in this article, we recommend Pete Warden and Daniel Situnayake’s new O’Reilly book “TinyML: Machine Learning with TensorFlow on Arduino and Ultra-Low Power Microcontrollers”

For my latest project, I teamed up with my friends to create the world’s smallest Arduino compatible board named Atto! Below video shows Atto in action with its RGB (rainbow) LED lighting up

Wondering how small Atto is? How’s about 0.4” x 0.45” small? (10.3 mm x 11.5 mm for my fellow metric brothers and sisters).

For those of you who may be unfamiliar, Arduino is a general-purpose circuit board with a tiny processor that can be programmed to do pretty much whatever you want, and it has a huge community behind it. Having a huge community (for anything really) is great since you have support from all around the globe (kind of like Medium)! So, you may be wondering, what can you do with such a tiny device when you can barely hold or see the darn thing?

Well, a lot! Imagine you want to create a new tech for wearables where earrings or other jewelries change colors based on your body temperature or heartbeat. Or, how about a tiny robotics project where Atto acts as the brain of the robot? The possibilities are endless!

So, we now know what kind of hardware product we want to create but how do we actually make it? It starts with engineering specs or as I usually call it, the process of going from shower thoughts to scribbled thoughts on a paper. The specs tell us exactly what we want to achieve from our hardware which in turn gives us an idea what components will be needed to build the hardware.

Once the specs are in place, we can begin our circuit design, specifically the schematic and layout. It’s tedious work but a good initial design will help us in the long run and save us a lot of development costs. So, it’s very important!

After we finish the schematic and layout, we want to ensure our component placements (the physical body shown above) is designed with manufacturing and maintenance in mind. The last thing we want is to struggle with assembling the products because of bad component placements!

The completed circuit design is sent to the magical land known as China to get it fabricated. Now, we have two choices here. First choice is to provide the circuit component list to the fabrication facility and have the prototypes fully assembled for us. Second choice is, as you may have guessed, we buy the parts and assemble it ourselves.

First choice is preferable but is costly whereas second choice is cheaper but requires a lot of manual labor. Being a poor grad student living off ramen and free lunches at school events, I opted out for the second choice. Above picture shows my soldering station and my hot air rework station that can blow at temperatures over 300 deg C (572 deg F) for assembling the boards. I named my hot air rework station, “The Manager” for obvious reasons.

After the circuits have been assembled, it’s time to program the initial test firmware on our prototype!

With the firmware programmed on Atto, we’re ready for testing (validation)! Our initial testing will be checking the input and output pin behaviours after powering it up. This is usually done by directly connecting the circuit board to the computer or by using a multimeter/oscilloscope. I personally use my trusty (a.k.a old and relatively cheap, but super reliable) Extech EX330. Not sponsored by Extech (… but I wish it was).

After validation is complete, you either have something working or something burnt to a crisp with magic smoke flying everywhere. Luckily, the latest revision of Atto survived and it’s ready to get lost somewhere in your room since it’s so darn small! (I may have lost one or two Attos already during the validation phase…).

Once we finish gathering our test data, we check to ensure that our initial specs were met, and we continue our testing to see if any necessary improvements are needed. And this, my friends, is the circle of l̶i̶f̶e̶ development.

Thanks for taking the time to read through my post and I hope you’ve enjoyed and learned something new! If you’re interested, you can check us out at https://nionics.com.

<strong>A little I knew about electrical engineering or electrical circuits, only until I saw a web page programmed on the ESP8266 microchip.</strong>

A little I knew about electrical engineering or electrical circuits, only until I saw a web page programmed on the ESP8266 microchip.

A little I knew about electrical engineering or electrical circuits, only until I saw a web page programmed on the ESP8266 microchip. Ever since, a new realm was opened for me, the Arduino and open-source hardware realm. But for the sake of sticking to web development, I will not dive deep into it and focus only on the web aspect of it that caught my attention the most.

The ESP8266 is a microchip with WiFi and full TCP/IP capabilities. It’s manufactured by a company called Espressif from Shanghai. It’s not affiliated with the Arduino company which is itself a separate entity but the ESP8266 can be added to the Arduino microcontroller board to give it WiFi capability which it doesn’t have out of the box. Although the Arduino company have their own WiFi Shield, the ESP9266 is a cheaper alternative and it’s available as a separate development board module. It’s also more popular among hobbyists therefore it has more popularity and more support in the community.

Overview

In the following section I will be using the Arduino IDE to program a web server on the ESP8266 NodeMCU board. It won’t be a detailed tutorial and I encourage you to search for more introductory content online if you want to get deeper into the details. The sole purpose of this article is to encourage web developers to get to know the Arduino ecosystem and how they can benefit from it as web developers.

So I mentioned programming a web server so far but what kind of web page am I going to serve on it? Well, it’s a Temperature and Humidity monitoring web page with the help of the DHT11, a basic, ultra low-cost digital temperature and humidity sensor. The web page can be accessed by any device that has a browser and sits on the local network.

Getting Started

1. First in the Arduino IDE we need to install the ESP8266 board library

2. 

3. Select the ESP8266 board from the list


1. Install the DHT11 sensor library. It feels like npm packages doesn’t it? 😊

1. We include the ESP8266 and DHT11 dependencies at the top of the arduino sketch

#include "ESP8266WiFi.h">
#include "DHT.h"

1. This is how we initialize the sensor and set its type and pin number

DHT dht(D5, DHT11);
dht.begin();

1. This is how we set our network credentials and connect via WiFi

const ssid = "YOUR_NETWORK_SSID";
const password = "YOUR_NETWORK_PASSWORD";

WiFi.begin(ssid, password);

1. This is how we initialize the web server and run it on port 80

WiFiServer server(80);
server.begin();

1. This is how we display the ESP8266’s local IP address in the serial monitor

Serial.println(WiFi.localIP());

1. We read the sensor data and store it in variables declared of type float

float hum = dht.readHumidity();
float temp = dht.readTemperature();

1. We wait for a client to connect to ESP8266

WiFiClient client = server.available();

if (client) {
    // serving the web page (Step 11)
   }

1. Creating the web page and serving it to the client

client.println("HTTP/1.1 200 OK");
client.println("Content-Type: text/html");
client.println("Connection: close");
client.println();
// your actual web page that displays temperature and humidity
client.println("<!DOCTYPE HTML>");
client.println("<html>");
client.println("<--! YOUR_WEBPAGE_HTML_HEADER_GOES_HERE -->");
client.println("<body><span>Temperature:  ");
client.println(celsiusTemp);
client.println("&deg;C</span>");
client.println("<span>Humidity:  ");
client.println(humidityTemp);
client.println("%</span>");
client.println("</body></html>");

⚠️ At line 8, we inject our web page’s html header tags and include all assets inline. I found this tool that can be very helpful!

1. We compile and upload the code

For the complete code here you go!

The Result

If the code compiled and was successfully uploaded, the ESP8266 should now be connected to the local network and has its own IP address (Step 8)

And just to make it more interesting I turned that web page into a Google Chrome extension that can be embedded on my desktop:

Recommended Courses:

☞ Mastering Arduino program (Guides to Arduino programming)

☞ Arduino 101 - Intel Curie

☞ Arduino Bootcamp : Learning Through Projects

☞ Advanced Arduino Boards and Tools

The Android of Things platform has some big advantages compared to Arduino or Raspbian. We Android developers can use the same tools we use for building applications for our projects, including all our favorite libraries, Google Play Services, and of course Kotlin.

The Android of Things platform has some big advantages compared to Arduino or Raspbian. We Android developers can use the same tools we use for building applications for our projects, including all our favorite libraries, Google Play Services, and of course Kotlin.

On the other hand, there’s a big disadvantage. Compared to Arduino, the Raspberry Pi can’t handle the low level custom protocols of some sensors. And compared to Raspbian, many of the available HATs are currently not supported.

How can we get the best of both worlds together? By using the right tool for the job. Let’s use the Arduino controller to read and write to our cheap and widely available sensors and use the Raspberry Pi to implement all the complex logic the same way we build Android applications.

Then, we will use the UART (universal asynchronous receiver/transmitter) of both platforms to exchange data between them. This is also known as the Serial on Arduino.

Wiring

Connecting the two devices is very simple:

• Connect the GPIO pin 14 of the Raspberry to the RX pin of the Arduino (aka GPIO zero)
• Connect the GPIO pin 15 of the Raspberry needs to the TX pin of the Arduino (aka GPIO 1)
• Connect a Ground pin from the Raspberry to the Arduino.

However there’s still one thing to do:

Take a look at the Configuring the UART mode in the official documentation:

☞ Arduino with Ardublockly: Block based Arduino Programming

Since the UART Bluetooth mode is enabled by default, you can’t start using the UART to talk with the Arduino out-of-the-box. You need to go through that configuration step!

Code

I’ll like to give all the credit of this idea to Marcos Placona, who wrote an article about the same topic months ago and helped me get started:

☞ Arduino Bootcamp : Learning Through Projects

The library he started didn’t fit my needs so I went with a simpler, one class solution that you can copy and modify for your projects:

import android.util.Log
import com.google.android.things.pio.PeripheralManagerService
import com.google.android.things.pio.UartDevice

class Arduino(uartDevice: String = "UART0"): AutoCloseable {
    private val TAG = "Arduino"
    private val uart: UartDevice by lazy {
        PeripheralManagerService().openUartDevice(uartDevice).apply {
            setBaudrate(115200)
            setDataSize(8)
            setParity(UartDevice.PARITY_NONE)
            setStopBits(1)
        }
    }

    fun read(): String {
        val maxCount = 8
        val buffer = ByteArray(maxCount)
        var output = ""
        do {
            val count = uart.read(buffer, buffer.size)
            output += buffer.toReadableString()
            if(count == 0) break
            Log.d(TAG, "Read ${buffer.toReadableString()} $count bytes from peripheral")
        } while (true)
        return output
    }

    private fun ByteArray.toReadableString() = filter { it > 0.toByte() }
            .joinToString(separator = "") { it.toChar().toString() }

    fun write(value: String) {
        val count = uart.write(value.toByteArray(), value.length)
        Log.d(TAG, "Wrote $value $count bytes to peripheral")
    }

    override fun close() {
        uart.close()
    }
}

Arduino.kt

This class provides three simple methods that cover the following:

• Connect the GPIO pin 14 of the Raspberry to the RX pin of the Arduino (aka GPIO zero)
• Connect the GPIO pin 15 of the Raspberry needs to the TX pin of the Arduino (aka GPIO 1)
• Connect a Ground pin from the Raspberry to the Arduino.

On the Arduino project, you need to handle the received commands and write the responses afterwards. This is an example of how to do it:

Arduino.c

if (Serial.available() > 0) {
  char command = (char) Serial.read();
  switch (command) {
     case 'H':
       Serial.write("Hello World");
       break;
  }
  Serial.flush();
}

You can write an “H” to the UART, and read back “Hello World” from it:

Example.kt

val arduino = Arduino()
arduino.write("H")
Thread.sleep(100) // let Arduino reply
val hello = arduino.read() // hello = "Hello World"

Now you can happily communicate your Android of Things applications with Arduino and use all the arsenal of sweet sweet cheap Arduino sensors on your Android of Things projects.