Building a Circuit communication device using Raspberry PI

Building a Circuit communication device using Raspberry PI

This article intends to show the Circuit SDK’s portability and how it can be easily integrated with inexpensive hardware to build all kind of communication devices.

This article intends to show the Circuit SDK’s portability and how it can be easily integrated with inexpensive hardware to build all kind of communication devices.

Circuit is a team collaboration software supporting chat, voice, video, screen share and much more. It’s available on the browser, as desktop application, iOS and Android. The free version allows up to 100 users and 3 users per conference call without any feature limitation.

For an introduction to Circuit APIs you may want to read this article.

Circuit Receptionist Kiosk

This application can be ideal for office buildings with restricted access and without a human receptionist. This is how it works:

When the application starts (kiosk mode), it presents a splash screen (or idle screen) while it initializes all sensors, logs into Circuit, and updates the bottom bar with date, time, temperature and humidity.

When the presence of the visitor is detected, the application shows several options such as start typing the name, speak the name, or calling the receptionist. The receptionist is actually a Circuit group conversation where any of the participants can answer the call.

As the visitor types, a user search is requested to the back end and results are presented on the screen. For every change in the search string a new search is performed, and any previous pending search is cancelled.

Once the user search is successful the visitor taps on the user’s avatar and the application sets up a video call with the Circuit user.

The user inside the office, is alerted of the incoming call and is allowed to answer it using just voice or video.

Once the Circuit user answers the call, video and voice communication is established.

At the same time the Circuit user is prompted with two options, reject the visit and terminate the call, or allow access to the visitor. When the Circuit user clicks on the option to allow access, the device unlocks the door, and the video call is terminated.

About the hardware

Below is the list of the hardware components with a brief description. If you are already familiar with these components just refer to GitHub for the complete list.

Raspberry Pi: The Raspberry Pi is a small, affordable, and amazingly capable, credit card size computer. It is developed by the Raspberry Pi Foundation, and it might be the most versatile tech ever created. At the time of these writing you can buy it for less than US$40.-

Raspberry Pi 7" Touchscreen Display: Raspberry Pi touchscreens can be easily connected using a DSI ribbon cable and be powered from the GPIO. The full color display outputs up to 800 x 480 and features a capacitive touch sensing capable of detecting 10 fingers. It costs about US$70.-

When using the touchscreen display, a suitable case to house both the Raspberry Pi and the display makes it safer to handle. There are a couple of inexpensive options out there.

MicroSD Card: This is the persistent storage for the Raspberry Pi. Although 8Gb should be enough for this project, 16Gb is recommended. The adapter may come handy to flash Raspbian on a computer with a regular SD card reader slot.

Raspberry Camera Module (V2): The Camera Module can be used to take high-definition video, as well as stills photographs. It’s easy to use for beginners, but has plenty to offer advanced users. It is connected to the Raspberry Pi using a CSI ribbon cable. In this project we will use the camera module to stream the reception feed when the call between the visitor and the Circuit user is established. You can learn more about the Raspberry Camera Module on the Raspberry Pi Foundation.

Breadboard: Although not necessary, it is very convenient to quickly setup the electronic circuit, specially if soldering is not your thing. If you use a breadboard you will find the breadboard extension for the Raspberry Pi GPIO (General Purpose Input Output) very convenient. Also some jumpers (male-male and female-male) are needed.

Speaker and Microphone: Both are needed for this project and here you may have several options. Computer speaker with a 3.5mm jack, USB speakers or microphones, headsets or any combination. You may even use a webcam with microphone if you do not count with the Raspberry Camera Module.

Infrared Motion Sensor: This sensor is used to detect the presence of the visitor and switch the user interface view accordingly. I used HC SR501 which is inexpensive and can be adjusted for sensitivity and output timing using the on board variable resistors.

Green LED: Not strictly necessary. In my project the LED flashes when the app is listening the visitor speak the name she is looking for. It also remains ON when the door is being unlocked.

Active Buzzer: The active buzzer is a placeholder. It represents whatever relayed mechanism would be needed to unlock a door. Any real application may need different electronic components to achieve that objective.

Hygrotermograph Sensor: This sensor provides information about the temperature and humidity in the room. Not very functional to this specific application but I decided to include just because… I can. I used the inexpensive DHT11. You can use any other, or none.

Other components: If you use all the same sensors, LED and buzzer you will need one NPN transistor (S8050) and three resistors (1KΩ, 10KΩ, and 220Ω).

Electronic Circuit Schematic Diagram: Below you can see how the components shall be connected. You may use different GPIO pins, just remember to update the code in gpioWrapper.js

About the software

Operating System: The Raspberry Pi runs Raspbian Stretch with is the latest version of the Debian Linux distribution specifically for Raspberry Pi.

Framework: The application is a nodejs process using electron to have access to Chromium WebRTC APIs needed by the Circuit SDK to established a WebRTC communication. So, it is not an actual electron desktop application… but it could have been as well.

Caution: the nodejs (and npm) versions on the Debian repository are not the latest and this project needs the latest ones, so do not use apt-get, instead follow the instructions on my GitHub repository.

Vue: Although I am more familiar with AngularJS, I decided to go with Vue to handle the user interface. Vue is very lightweight in comparison and has a quick learning curve.

BCM2835: bcm2835 library. This C library for Raspberry Pi provides access to GPIO and other IO functions on the Broadcom BCM 2835 chip. Instructions to download and build the library can be found also on my GitHub repository.

Google Cloud Services: As I mentioned before the visitor can type on the touch screen to find the Circuit user, or just speak the name. For transcription I decided to use Google Cloud Speech API. The setup is outside the scope of this article but can learn all you need to know for this application with the Google Cloud Speech Quick Start. For speech recording I used sox.

Circuit SDK usage

On the GitHub circuitKiosk repository you will find examples on how to use some of the capabilities of the Circuit platform using the Javascript SDK. You will see how to instantiate the Circuit client, set the log levels, add event handlers, log your application, search for users, and process those search results. You will also learn how to find and create Circuit conversations, send text items, and process received ones. The application also makes use of Circuit forms that are sent as text items. Forms submissions are received as text items, and you will see how to process them. In the code you will also find how to setup a video call, process call events and handle the call media by getting the remote streams from the instantiated Circuit client.

Let’s see some specific examples:

In order to instantiate a Circuit client your code will look something like this:

gistfile1.txt

Circuit.logger.setLevel(Circuit.Enums.LogLevel.Debug);
let client = Circuit.Client({
        "client_id": "your-client-id",
        "client_secret": "your-client-secret",
        "domain": "circuitsandbox.net"});

Notice on line #1 the Circuit log level is set.

Besides logging in your application into Circuit, this has to add event handlers. Circuit raises a ton of different events. For this application we are just interest in a few of them.

gistfile1.txt

     client.addEventListener('itemAdded', processItemAddedEvent);
        client.addEventListener('itemUpdated', processItemUpdatedEvent);
        client.addEventListener('callStatus',processCallStatusEvent);
        client.addEventListener('callEnded',processCallEndedEvent);
        client.addEventListener('callIncoming', processCallIncomingEvent);
        client.addEventListener('formSubmission', processFormSubmission);
        client.addEventListener('basicSearchResults', evt => {
            processSearchResults(evt.data).then(users => {
                app.setUsers(users);
            });
        });
        client.addEventListener('searchStatus', evt => {
            processSearchStatus(evt.data).then(users => {
                if (users) {
                    app.setUsers(users);
                }
            });
        });

Basically the application needs to listen for new text items for maintenance commands, call events to handle WebRTC communications, formSubmission events to accept a command for a Circuit user (i.e. unlock the door), and search results to find specific Circuit users.

Finding a Circuit user it is an asynchronous process. So the application invokes the startUserSearch API, and wait for the corresponding events with the search results for which we already set the listeners above.

gistfile1.txt

let searchId = client.startUserSearch(searchString);

Once the user is found and selected to establish a communication the one to one conversation between the user and the application needs to be find, and if it doesn’t exist, then it needs to be created. Once the conversation is known then the call can be made. However, the SDK API makeCall does all this for you. You just need to know the id of the user destination of the call.

gistfile1.txt

client.makeCall(user.userId, {audio: true, video: true}, true);

As the call is being setup Circuit will raise callStatus events that need to be handled accordingly to properly setup the user interface and, eventually when the call is answer set the audio and video elements sources with the remote streams.

gistfile1.txt

    async function setupMedia(call) {
        let remoteStreams = client.getRemoteStreams(call.callId);
        let remoteAudioStream = remoteStreams.find((s) => s.getAudioTracks().length > 0);
        document.querySelector('audio').srcObject = remoteAudioStream;
        if(call.remoteVideoStreams && call.remoteVideoStreams.length) {
            Vue.nextTick().then(() => {
                document.querySelector('video').srcObject = call.remoteVideoStreams[0].stream;
             });
        }
        sendOpenDoorForm(call.convId);
    };

Now the call is setup and both parties can talk and see each other. Note line #10, here the application will send a form to the user with two buttons, one to unlock the door, and the other to reject the visitor by terminating call. Circuit SDK makes it easy to submit a form as a text message using the addTextItem API.

gistfile1.txt

    async function sendOpenDoorForm(convId) {
        await client.addTextItem(convId, {
            content: 'Select "Open Door" or "I do not know this person"',
            form: {
                id: 'openDoorForm',
                controls: [{
                    type: Circuit.Enums.FormControlType.BUTTON,
                    name: 'openDoor',
                    options: [{
                        text: 'Open Door',
                        notification: 'Opening Door',
                        value: 'openDoor'
                    }, {
                        text: 'I do not know this person',
                        value: 'endCall',
                        notification: 'Access Denied'
                    }]
                }]
            }
        });
    };

Finally as the user chooses to unlock the door for the visitor the application will process the form result.

gistfile1.txt

    function processFormSubmission(evt) {
        logger.info(`[RENDERER] process form submission. ${evt.form.id}`);
        evt.form.data.forEach(ctrl => {
            logger.debug(`${ctrl.key}: ${ctrl.value}`);
            switch (ctrl.name) {
                case 'openDoor':
                    switch(ctrl.value) {
                        case 'openDoor':
                            openDoor();
                            break;
                        case 'endCall':
                            break;
                        default:
                            logger.error(`Unknown value in submitted form: ${ctrl.value}`);
                            return;
                    }
                    app.currentCall && client.leaveConference(app.currentCall.callId);
                    client.updateTextItem({
                        itemId: evt.itemId,
                        content: (ctrl.value === 'openDoor' ? 'Door has been opened' : 'Entrance denied'),
                        form: {
                            id: evt.form.id
                        }
                    });
                    break;
                default:
                    logger.error(`Unknown key in submitted form: ${ctrl.key}`);
                    break;
            }
        });
    };

Let’s build it

Just follow the step-by-step instructions on GitHub circuitKiosk repository README.md . Here is a short video demonstration:

As you may already know the only limitation is your imagination. All the parts are out there, get them and build something cool and you will learn a lot in the process, after all there are only two kinds of adventure in life: love and knowledge.

30s ad

Raspberry Pi 3 Day Project: Retro Gaming Suite

Raspberry Pi Projects : Build a Media Centre Computer

Build Your Own Super Computer with Raspberry Pis

PiBot: Build Your Own Raspberry Pi Powered Robot

Internet of Things (IoT) Automation using Raspberry Pi 2

This article intends to show the Circuit SDK’s portability and how it can be easily integrated with inexpensive hardware to build all kind of communication devices.

Circuit is a team collaboration software supporting chat, voice, video, screen share and much more. It’s available on the browser, as desktop application, iOS and Android. The free version allows up to 100 users and 3 users per conference call without any feature limitation.

For an introduction to Circuit APIs you may want to read this article.

Circuit Receptionist Kiosk

This application can be ideal for office buildings with restricted access and without a human receptionist. This is how it works:

When the application starts (kiosk mode), it presents a splash screen (or idle screen) while it initializes all sensors, logs into Circuit, and updates the bottom bar with date, time, temperature and humidity.

When the presence of the visitor is detected, the application shows several options such as start typing the name, speak the name, or calling the receptionist. The receptionist is actually a Circuit group conversation where any of the participants can answer the call.

As the visitor types, a user search is requested to the back end and results are presented on the screen. For every change in the search string a new search is performed, and any previous pending search is cancelled.

Once the user search is successful the visitor taps on the user’s avatar and the application sets up a video call with the Circuit user.

The user inside the office, is alerted of the incoming call and is allowed to answer it using just voice or video.

Once the Circuit user answers the call, video and voice communication is established.

At the same time the Circuit user is prompted with two options, reject the visit and terminate the call, or allow access to the visitor. When the Circuit user clicks on the option to allow access, the device unlocks the door, and the video call is terminated.

About the hardware

Below is the list of the hardware components with a brief description. If you are already familiar with these components just refer to GitHub for the complete list.

Raspberry Pi: The Raspberry Pi is a small, affordable, and amazingly capable, credit card size computer. It is developed by the Raspberry Pi Foundation, and it might be the most versatile tech ever created. At the time of these writing you can buy it for less than US$40.-

Raspberry Pi 7" Touchscreen Display: Raspberry Pi touchscreens can be easily connected using a DSI ribbon cable and be powered from the GPIO. The full color display outputs up to 800 x 480 and features a capacitive touch sensing capable of detecting 10 fingers. It costs about US$70.-

When using the touchscreen display, a suitable case to house both the Raspberry Pi and the display makes it safer to handle. There are a couple of inexpensive options out there.

MicroSD Card: This is the persistent storage for the Raspberry Pi. Although 8Gb should be enough for this project, 16Gb is recommended. The adapter may come handy to flash Raspbian on a computer with a regular SD card reader slot.

Raspberry Camera Module (V2): The Camera Module can be used to take high-definition video, as well as stills photographs. It’s easy to use for beginners, but has plenty to offer advanced users. It is connected to the Raspberry Pi using a CSI ribbon cable. In this project we will use the camera module to stream the reception feed when the call between the visitor and the Circuit user is established. You can learn more about the Raspberry Camera Module on the Raspberry Pi Foundation.

Breadboard: Although not necessary, it is very convenient to quickly setup the electronic circuit, specially if soldering is not your thing. If you use a breadboard you will find the breadboard extension for the Raspberry Pi GPIO (General Purpose Input Output) very convenient. Also some jumpers (male-male and female-male) are needed.

Speaker and Microphone: Both are needed for this project and here you may have several options. Computer speaker with a 3.5mm jack, USB speakers or microphones, headsets or any combination. You may even use a webcam with microphone if you do not count with the Raspberry Camera Module.

Infrared Motion Sensor: This sensor is used to detect the presence of the visitor and switch the user interface view accordingly. I used HC SR501 which is inexpensive and can be adjusted for sensitivity and output timing using the on board variable resistors.

Green LED: Not strictly necessary. In my project the LED flashes when the app is listening the visitor speak the name she is looking for. It also remains ON when the door is being unlocked.

Active Buzzer: The active buzzer is a placeholder. It represents whatever relayed mechanism would be needed to unlock a door. Any real application may need different electronic components to achieve that objective.

Hygrotermograph Sensor: This sensor provides information about the temperature and humidity in the room. Not very functional to this specific application but I decided to include just because… I can. I used the inexpensive DHT11. You can use any other, or none.

Other components: If you use all the same sensors, LED and buzzer you will need one NPN transistor (S8050) and three resistors (1KΩ, 10KΩ, and 220Ω).

Electronic Circuit Schematic Diagram: Below you can see how the components shall be connected. You may use different GPIO pins, just remember to update the code in gpioWrapper.js

About the software

Operating System: The Raspberry Pi runs Raspbian Stretch with is the latest version of the Debian Linux distribution specifically for Raspberry Pi.

Framework: The application is a nodejs process using electron to have access to Chromium WebRTC APIs needed by the Circuit SDK to established a WebRTC communication. So, it is not an actual electron desktop application… but it could have been as well.

Caution: the nodejs (and npm) versions on the Debian repository are not the latest and this project needs the latest ones, so do not use apt-get, instead follow the instructions on my GitHub repository.

Vue: Although I am more familiar with AngularJS, I decided to go with Vue to handle the user interface. Vue is very lightweight in comparison and has a quick learning curve.

BCM2835: bcm2835 library. This C library for Raspberry Pi provides access to GPIO and other IO functions on the Broadcom BCM 2835 chip. Instructions to download and build the library can be found also on my GitHub repository.

Google Cloud Services: As I mentioned before the visitor can type on the touch screen to find the Circuit user, or just speak the name. For transcription I decided to use Google Cloud Speech API. The setup is outside the scope of this article but can learn all you need to know for this application with the Google Cloud Speech Quick Start. For speech recording I used sox.

Circuit SDK usage

On the GitHub circuitKiosk repository you will find examples on how to use some of the capabilities of the Circuit platform using the Javascript SDK. You will see how to instantiate the Circuit client, set the log levels, add event handlers, log your application, search for users, and process those search results. You will also learn how to find and create Circuit conversations, send text items, and process received ones. The application also makes use of Circuit forms that are sent as text items. Forms submissions are received as text items, and you will see how to process them. In the code you will also find how to setup a video call, process call events and handle the call media by getting the remote streams from the instantiated Circuit client.

Let’s see some specific examples:

In order to instantiate a Circuit client your code will look something like this:

gistfile1.txt

Circuit.logger.setLevel(Circuit.Enums.LogLevel.Debug);
let client = Circuit.Client({
        "client_id": "your-client-id",
        "client_secret": "your-client-secret",
        "domain": "circuitsandbox.net"});

Notice on line #1 the Circuit log level is set.

Besides logging in your application into Circuit, this has to add event handlers. Circuit raises a ton of different events. For this application we are just interest in a few of them.

gistfile1.txt

     client.addEventListener('itemAdded', processItemAddedEvent);
        client.addEventListener('itemUpdated', processItemUpdatedEvent);
        client.addEventListener('callStatus',processCallStatusEvent);
        client.addEventListener('callEnded',processCallEndedEvent);
        client.addEventListener('callIncoming', processCallIncomingEvent);
        client.addEventListener('formSubmission', processFormSubmission);
        client.addEventListener('basicSearchResults', evt => {
            processSearchResults(evt.data).then(users => {
                app.setUsers(users);
            });
        });
        client.addEventListener('searchStatus', evt => {
            processSearchStatus(evt.data).then(users => {
                if (users) {
                    app.setUsers(users);
                }
            });
        });

Basically the application needs to listen for new text items for maintenance commands, call events to handle WebRTC communications, formSubmission events to accept a command for a Circuit user (i.e. unlock the door), and search results to find specific Circuit users.

Finding a Circuit user it is an asynchronous process. So the application invokes the startUserSearch API, and wait for the corresponding events with the search results for which we already set the listeners above.

gistfile1.txt

let searchId = client.startUserSearch(searchString);

Once the user is found and selected to establish a communication the one to one conversation between the user and the application needs to be find, and if it doesn’t exist, then it needs to be created. Once the conversation is known then the call can be made. However, the SDK API makeCall does all this for you. You just need to know the id of the user destination of the call.

gistfile1.txt

client.makeCall(user.userId, {audio: true, video: true}, true);

As the call is being setup Circuit will raise callStatus events that need to be handled accordingly to properly setup the user interface and, eventually when the call is answer set the audio and video elements sources with the remote streams.

gistfile1.txt

    async function setupMedia(call) {
        let remoteStreams = client.getRemoteStreams(call.callId);
        let remoteAudioStream = remoteStreams.find((s) => s.getAudioTracks().length > 0);
        document.querySelector('audio').srcObject = remoteAudioStream;
        if(call.remoteVideoStreams && call.remoteVideoStreams.length) {
            Vue.nextTick().then(() => {
                document.querySelector('video').srcObject = call.remoteVideoStreams[0].stream;
             });
        }
        sendOpenDoorForm(call.convId);
    };

Now the call is setup and both parties can talk and see each other. Note line #10, here the application will send a form to the user with two buttons, one to unlock the door, and the other to reject the visitor by terminating call. Circuit SDK makes it easy to submit a form as a text message using the addTextItem API.

gistfile1.txt

    async function sendOpenDoorForm(convId) {
        await client.addTextItem(convId, {
            content: 'Select "Open Door" or "I do not know this person"',
            form: {
                id: 'openDoorForm',
                controls: [{
                    type: Circuit.Enums.FormControlType.BUTTON,
                    name: 'openDoor',
                    options: [{
                        text: 'Open Door',
                        notification: 'Opening Door',
                        value: 'openDoor'
                    }, {
                        text: 'I do not know this person',
                        value: 'endCall',
                        notification: 'Access Denied'
                    }]
                }]
            }
        });
    };

Finally as the user chooses to unlock the door for the visitor the application will process the form result.

gistfile1.txt

    function processFormSubmission(evt) {
        logger.info(`[RENDERER] process form submission. ${evt.form.id}`);
        evt.form.data.forEach(ctrl => {
            logger.debug(`${ctrl.key}: ${ctrl.value}`);
            switch (ctrl.name) {
                case 'openDoor':
                    switch(ctrl.value) {
                        case 'openDoor':
                            openDoor();
                            break;
                        case 'endCall':
                            break;
                        default:
                            logger.error(`Unknown value in submitted form: ${ctrl.value}`);
                            return;
                    }
                    app.currentCall && client.leaveConference(app.currentCall.callId);
                    client.updateTextItem({
                        itemId: evt.itemId,
                        content: (ctrl.value === 'openDoor' ? 'Door has been opened' : 'Entrance denied'),
                        form: {
                            id: evt.form.id
                        }
                    });
                    break;
                default:
                    logger.error(`Unknown key in submitted form: ${ctrl.key}`);
                    break;
            }
        });
    };

Let’s build it

Just follow the step-by-step instructions on GitHub circuitKiosk repository README.md . Here is a short video demonstration:

As you may already know the only limitation is your imagination. All the parts are out there, get them and build something cool and you will learn a lot in the process, after all there are only two kinds of adventure in life: love and knowledge.

30s ad

Raspberry Pi 3 Day Project: Retro Gaming Suite

Raspberry Pi Projects : Build a Media Centre Computer

Build Your Own Super Computer with Raspberry Pis

PiBot: Build Your Own Raspberry Pi Powered Robot

Internet of Things (IoT) Automation using Raspberry Pi 2

Raspberry Pi 4 on the Raspberry Pi 4 - Computerphile

Raspberry Pi 4 on the Raspberry Pi 4 - Computerphile

A quick tour of the Raspberry Pi 4 edited on the Raspberry Pi 4. Dr Steve Bagley gets out his knife.dll to unbox Sean's purchases! ☞ [I created a home IoT setup with AWS, Raspberry...

A quick tour of the Raspberry Pi 4 edited on the Raspberry Pi 4. Dr Steve Bagley gets out his knife.dll to unbox Sean's purchases!

I created a home IoT setup with AWS, Raspberry Pi

Benchmarking the Raspberry Pi 4

The easy way to set up Docker on a Raspberry Pi – freeCodeCamp.org

Creating a Rogue Wi-Fi Access Point using a Raspberry Pi

Building a Smart Garden With Raspberry Pi 3B+

Learn Raspberry Pi for Image Processing Applications

Learn Raspberry Pi for Image Processing Applications

New to the newly launched Raspberry Pi 3? Learn all the components of Raspberry Pi, connecting components to Raspberry Pi, installation of NOOBS operating system, basic Linux commands, Python programming and building Image Processing applications on Raspberry Pi. At just $9.

Description
Image Processing Applications on Raspberry Pi is a beginner course on the newly launched Raspberry Pi 3 and is fully compatible with Raspberry Pi 2 and Raspberry Pi Zero.

The course is ideal for those who are new to the Raspberry Pi and want to explore more about it.

You will learn the components of Raspberry Pi, connecting components to Raspberry Pi, installation of NOOBS operating system, basic Linux commands, Python programming and building Image Processing applications on Raspberry Pi.

This course will take beginners without any coding skills to a level where they can write their own programs.

Basics of Python programming language are well covered in the course.

Building Image Processing applications are taught in the simplest manner which is easy to understand.

Users can quickly learn hardware assembly and coding in Python programming for building Image Processing applications. By the end of this course, users will have enough knowledge about Raspberry Pi, its components, basic Python programming, and execution of Image Processing applications in the real time scenario.

The course is taught by an expert team of Electronics and Computer Science engineers, having PhD and Postdoctoral research experience in Image Processing.

Anyone can take this course. No engineering knowledge is expected. Tutor has explained all required engineering concepts in the simplest manner.

The course will enable you to independently build Image Processing applications using Raspberry Pi.

This course is the easiest way to learn and become familiar with the Raspberry Pi platform.

By the end of this course, users will build Image Processing applications which includes scaling and flipping images, varying brightness of images, perform bit-wise operations on images, blurring and sharpening images, thresholding, erosion and dilation, edge detection, image segmentation. User will also be able to build real-world Image Processing applications which includes real-time human face eyes nose detection, detecting cars in video, real-time object detection, human face recognition and many more.

The course provides complete code for all Image Processing applications which are compatible on Raspberry Pi 3/2/Zero.

Who is the target audience?

Anyone who wants to explore Raspberry Pi and interested in building Image Processing applications

To read more:

Benchmarking the Raspberry Pi 4

Benchmarking the Raspberry Pi 4

Last year’s release of the Raspberry Pi 3 Model A+ marked the end of an era: the next board, Raspberry Pi Foundation co-founder Eben Upton promised at the time, would be something dramatically different.

Now, a surprisingly short time later, Upton’s promise has been delivered: the Raspberry Pi 4 is a departure from the norm, and the first of a new generation of Raspberry Pi single-board computers. Gone is the old bottleneck of a single shared USB lane for everything connected to the SoC; gone too is the layout which has been with the boards since the Raspberry Pi Model B+.

Although appearing similar at first glance, the new board is slightly larger thanks to ports extending further from the PCB for improved case compatibility, the Ethernet and USB ports have been switched around, the power input is now a USB Type-C connector, and the full-size HDMI output has been swapped out for not one but two micro-HDMI connectors.

Internally, the SoC has been entirely overhauled. As well as getting rid of the bottleneck — the SoC now has around 5Gb/s of external bandwidth — there are USB 3.0 lanes for high-speed connectivity to external storage and accelerators, Cortex-A72 64-bit processing cores, a more powerful VideoCore VI graphics processor — the first Pi ever to use anything other than the VideoCore IV launched with the original Model B — and there’s enough grunt to drive two 4K-resolution displays as well as true gigabit Ethernet connectivity.

It’s benchmarking time.

Specifications

SoC: Broadcom BCM2711B0 quad-core A72 (ARMv8-A) 64-bit @ 1.5GHz
GPU: Broadcom VideoCore VI @ 500MHz
RAM: 1GB, 2GB, or 4GB LPDDR4–3200 SDRAM (4GB as reviewed)
Networking: Gigabit Ethernet, 2.4GHz and 5GHz 802.11b/g/n/ac Wi-Fi
Bluetooth: Bluetooth 5.0, Bluetooth Low Energy (BLE)
Storage: MicroSD
GPIO: 40-pin GPIO header, populated
Ports: 2x micro-HDMI 2.0, 3.5mm analogue audio-video jack, 2x USB 2.0, 2x USB 3.0, Ethernet, Camera Serial Interface (CSI), Display Serial Interface (DSI)
Dimensions: 88mm x 58mm x 19.5mm, 46g

There’s a lot going on in the new Pi 4, including the first alteration to the Model B layout since the launch of the Raspberry Pi Model B+. Shifting the ports around has slightly increased the board’s footprint, as measured at its widest points including ports, and leaves the majority of cases incompatible. It is, however, measurably lighter than the Raspberry Pi 3 Model B+ at 46g to its predecessors’ 50g — likely aided by the loss of the full-size HDMI port.

On the SoC side, which is now produced on a 28nm process node, the core count is unchanged but the CPU has been shifted to the new Arm Cortex-A72 running at a slightly faster 1.5GHz. It’s the GPU, though, which has seen the biggest shift: every Raspberry Pi in history has used the Broadcom VideoCore IV GPU, whereas the Raspberry Pi 4 switches to a customised variant of the Broadcom VideoCore VI with scanout engine borrowed from the VideoCore V. The result: improved performance and the board family’s first support not only for 4K resolutions but across two dedicated HDMI outputs, switched to micro-HDMI for reasons of space. The shift to a new SoC has also brought with it support for more than 1GB of RAM, with the Pi 4 launching in 1GB, 2GB, and 4GB variants.

On the surface, the networking functionality is unchanged: there’s still 802.11ac Wi-Fi, though an upgrade to Bluetooth 5.0, plus a wired gigabit Ethernet port. Where the older Raspberry Pis have the Ethernet port talk to the SoC via a shared USB 2.0 lane, however, the Pi 4 enjoys a more direct connection without the bottleneck.

Finally, there’s the USB ports. While there are still four full-size ports in total, two of these have been upgraded to USB 3.0 — greatly improving the theoretical bandwidth available to external devices from USB-connected accelerators like the Google Coral Edge TPU to USB storage. A switch to USB Type-C, rated for 3A from the 2.5A micro-USB of the older models, also allows for higher-power devices to be connected without needing a powered hub.

Thermal Benchmark

A more powerful processor typically means more excess heat, something with which the Raspberry Pi family has struggled in the past. The Raspberry Pi 3 Model B, in particular, ran hot; its replacement, the Raspberry Pi 3 Model B+ went quite some way to improving things with a thicker PCB, metal-encased and tweaked SoC, and improved thermal bonding. A thermal camera provides a view of where this heat is generated and how it spreads through the entire Raspberry Pi.

The new Broadcom BCM2711B0 SoC has the same packaging as its predecessor, but is clearly more powerful in both senses of the word: thermal imagery of the board (bottom) shows the SoC, after a ten-minute CPU-focused workload, running noticeably hotter than the Raspberry Pi 3 Model B+ (top) and spreads that heat throughout the board. Peak spot temperatures at the end of the ten-minute run were measured at 62.6°C on the Raspberry Pi 3 Model B+ and 74.5°C on the Pi 4 — both readings, unsurprisingly, centred on the SoC.

It’s an analysis easily verified in use: after just a few minutes, the entire board feels warm to the touch. Start loading it heavily and that warmth becomes uncomfortable; while it’s still entirely possible to use the board without extra cooling, those looking to put one in a case will find active cooling is required to avoid thermal throttling.

Power Draw Benchmark

If something’s producing more heat, it’s guaranteed to be drawing more power. A big reason for the shift from micro-USB to USB Type-C for the Raspberry Pi 4’s power jack comes from being able to qualify it at higher current levels: 3A, up from 2.5A — already half an amp over standard ratings — on the previous jack.

This benchmark, which measures power draw at the wall for every mainstream model of Pi released so far, confirms the hypothesis: more heat means more power. At 3.4W idle — a figure which may come down with post-launch firmware optimisations— and 7.6W under load, the Raspberry Pi 4 is the most power-hungry design the Raspberry Pi Foundation has yet released.

The positioning of the boards, then, remains unchanged: if you need performance, the full-fat Raspberry Pi 4 is the board to get; if you need to balance performance and power draw, the Pi 3 A+ is difficult to beat for its significantly lower idle draw; if power is key, the Raspberry Pi Zero and Wi-Fi-enabled Raspberry Pi Zero W should be top of the list.

Thermal Throttling Benchmark

When a Raspberry Pi’s system-on-chip gets hot, it — just like any other modern semiconductor — takes action to protect itself from harm, by reducing its operating speed in order to cool itself down. Under brief, bursty workloads — browsing the web, say — this throttling doesn’t happen; only in a sustained, heavy workload does it rear its head. In this benchmark the Raspberry Pi 4 is subjected to a ten-minute run of a CPU-centric stress-testing utility, stress-ng, and the temperature and clock speed measured once every second using the SoC’s internal sensors.

The initial ramp-up from the SoC’s idle temperature is rapid, though it must be noted this test took place in an ambient temperature of nearly 25°C. Once the SoC hits 70°C after around 25 seconds, the temperature rise slows; by two minutes it is around 77°C; by three minutes it’s around 81°C. As is to be expected, these readings — taken from an internal sensor located on the GPU portion of the SoC — are higher than the external package temperature measured during the thermal imaging test.

Interestingly, the first thermal throttle operation isn’t captured until around four and a half minutes into the test, and a quick glance at the rest of the graph shows why: where earlier Raspberry Pi models would tend to hit a throttle point and stay there the Raspberry Pi 4 is instead spending as little time as possible at its throttled clock speed of 1GHz, returning to 1.5GHz as quickly as it can. With the measurements being taken at a rate of one per second but the CPU’s frequency switching taking place on a much shorter timescale, it’s likely the Raspberry Pi 4 was throttling earlier but for too short a time to be captured.

This benchmark clearly demonstrates additional cooling is going to be a must-have to maintain top performance for workloads including sustained CPU activity over the four-minute mark— but for those who are using the device as-is, expect to see less sustained throttling than in previous models after the thermal throttle point is reached.

Linpack Benchmark

A synthetic benchmark with a long history, Linpack is a great way to get an idea of peak compute performance. Here, the same implementation of the Linpack benchmark is used across all models to level the playing field; by compiling for a specific model and using a variety of other tweaks, the absolute performance figure for a given Pi can be dramatically increased but at the cost of making any comparison between hardware platforms apples-to-oranges.

The Raspberry Pi 4 might only be 100MHz faster in clock speed, but the move to Arm Cortex-A72 processor cores has had a dramatic effect on its Linpack performance. The board absolutely dominates the table, with its single-precision (SP), double-precision (DP), and NEON-accelerated single-precision (SP NEON, a mode available only on the Raspberry Pi 2 and upwards) scores sitting between three and four times faster than the Raspberry Pi 3 Model B+ and Model A+.

Memory Throughput Benchmark

The CPU isn’t the only thing improved in the Raspberry Pi 4. The switch to a new SoC has unlocked a whole new world of RAM, moving the platform from the long-in-the-tooth LPDDR2 to LPDDR4 in 1GB, 2GB, and 4GB capacities.

The biggest change in memory performance remains the shift away from the single-core BCM2835 of the original Raspberry Pi and compact Pi Zero families, but this benchmark — which performs read and write operations in 1MB block sizes — shows the switch to DDR4 has boosted things nicely.

For many workloads, simply having more RAM — up to four times as much, depending on the variant in question — will have a bigger real-world impact; the fact the RAM is also faster is, in these cases, just an added bonus.

File Compression Benchmark

Synthetic benchmarks are one thing; what’s more interesting is testing real-world workloads. Here a file is compressed using the bzip2 algorithm and the time it takes measured. For Raspberry Pi models with more than one CPU core, the test is repeated using the multi-threaded lbzip2.

As with benchmarks on previous generations, the biggest gain comes from the move from single-core to quad-core architectures; the Raspberry Pi 4’s Cortex-A72 cores, though, accelerate things considerably over older boards.

The Raspberry Pi 3 Model A+ is an interesting result on this test: its performance in the single-threaded portion of the test lags considerably behind the Raspberry Pi 3 Model B+ despite having the same SoC; this can be explained by having considerably less RAM to play with, a key component in efficient compression of larger files.

Image Editing Benchmark

Another example of a real-world workload, this test uses the command-line scripting interface of popular image-editing application GIMP to edit a high-resolution image. It’s a workload which relies on both CPU and RAM performance, while also demanding a large chunk of free RAM — something which penalises models with less than 1GB.

While not as dramatic a difference as the file compression workload, it’s clear from the results the Raspberry Pi 4 offers a measurable improvement in image editing performance over its predecessors.

Here, too, it’s clear the difference in RAM between the Raspberry Pi 3 Model B+ and the smaller, cheaper Model A+ has a discernable impact on performance: because of the high resolution of the image on test, the Model A+ is forced to swap memory contents out to make room — something the 1GB Raspberry Pi 3 Model B+ avoids, and which is definitely not an issue for the 4GB Raspberry Pi 4 on test.

Browser Benchmark

This benchmark should be of interest to anyone thinking of using a Raspberry Pi as a low-power desktop replacement: it tests whether there are enough hardware resources available to make browser-based applications run smoothly. To do so, the Speedometer 2.0 benchmark is loaded into the stock Chromium web browser; once set running, it returns a result in runs-per-minute.

From the results table it’s clear a modern web app running in a modern web browser is not something a single-core processor can really handle. Moving from the fastest of the single-core designs, the Raspberry Pi Zero family, to the quad-core Raspberry Pi 2 has an oversized impact on performance. Here, too, the impact of dropping from the 1GB of RAM available in the Raspberry Pi 3 Model B+ to the 512MB of the Raspberry Pi 3 Model A+ can be seen.

The Raspberry Pi 4, meanwhile, absolutely romps to the top of the leader-board. Where earlier models may have offered acceptable performance for web apps, the new board can definitely hold its head up high as a potential desktop replacement for the casual web user.

Oddly, while the benchmark ran — just about — on an original Raspberry Pi Model A, the Model A+ on test absolutely refused, leading to its absence from the results chart.

Gaming Benchmark

The Raspberry Pi family has long been a favourite of retro gamers, but is strangely overlooked as a platform for first-party titles. Since the Raspberry Pi 2, though, it has boasted support for full OpenGL — rather than the embedded-focused OpenGL ES — hardware acceleration, though its performance has always been constrained by the ageing VideoCore IV GPU. Here real-world gaming performance is measured using the built-in timedemo in the open-source Quake III Arena-based multiplayer shooter OpenArena, running at a 1280x720 (720p, HD) resolution.

Although the Raspberry Pi 2 and Raspberry Pi 3 families share the same GPU, at 720p the CPU performance of the Raspberry Pi 2 throttles the GPU: moving to to any model of the Raspberry Pi 3 range adds an additional 37 percent or so to the achievable frame rate. Moving between models in the Raspberry Pi 3 family, by contrast, does little to the now-GPU-constrained workload, despite the boosted CPU performance.

The Raspberry Pi 4’s VideoCore VI, on the other hand, offers a considerable improvement, boosting the game’s frame rate by nearly 50 percent over the Raspberry Pi 3 Model B+ to a very playable 41 frames per second average. If GPU performance had been holding your projects back, the Raspberry Pi 4 may be the answer.

Results for the single-core models, meanwhile, are not available; the driver which enables true OpenGL hardware acceleration is not officially available on these devices.

GPIO Benchmark

This test sits somewhere between a real-world and a synthetic workload: while addressing the general-purpose input/output (GPIO) header from Python using the gpiozero library is a common workload for a Raspberry Pi, this benchmark looks at things from a worst-case CPU-constrained perspective. A short program simply toggles a pin on and off without pause, and the rate at which the pin toggles is measured using a frequency counter.

There’s no surprise to find the GPIO benchmark heavily tied to CPU performance; what is perhaps surprising is seeing such a dramatic difference from the Raspberry Pi 3 family to the Raspberry Pi 4 — all thanks to the shift to the new Arm Cortex-A72 CPU cores.

Ethernet Benchmark

The resolution, after all these years, of the single-USB-lane bottleneck from the SoC to the rest of the board promises a dramatic improvement in network performance. While the Raspberry Pi 3 Model B+ was the first to include gigabit Ethernet connectivity, the bottleneck prevented it from reaching anywhere near its theoretical maximum throughput — but how does the Raspberry Pi 4 compare?

Ignoring the systems without an Ethernet port at all, this benchmark shows how stagnant the Ethernet performance was until the launch of the Raspberry Pi 3 Model B+: while the original Raspberry Pi Model B sits at the bottom of the table, the Raspberry Pi Model B+, Raspberry Pi 2, and Raspberry Pi 3 are all barely ahead.

The Raspberry Pi 4, by contrast, shows a throughput within spitting distance of the theoretical maximum. Better still, the removal of the USB bottleneck now means Ethernet and USB throughput aren’t linked — a boon for anyone thinking of building a Raspberry Pi-powered network attached storage (NAS) system.

Wi-Fi Benchmark

Ethernet isn’t the only built-in networking available on the Raspberry Pi family. Since the launch of the Raspberry Pi 3 Model B, built-in Wi-Fi — on a radio which also offers Bluetooth and Bluetooth Low Energy (BLE) connectivity — has been available as standard, with the Raspberry Pi 3 Model B+ adding dual-band support. In this test, each model with Wi-Fi capability is tested in an ideal environment: line-of-sight to an 802.11ac router with only one other client, a laptop on a wired connection.

The throughput on the 2.4GHz band is largely unchanged from the Raspberry Pi 3 Model A+ and B+ boards, which offer a boost over the Raspberry Pi 3 Model B which in turn is faster than the Raspberry Pi Zero W, but it’s the 5GHz band where the Pi 4 pulls away from its predecessors. While the difference between 97.6Mb/s on the Raspberry Pi 3 Model B+ and 114Mb/s on the Raspberry Pi 4 isn’t huge, it’s a welcome gain nevertheless.

USB Throughput Benchmark

The low cost and low power draw of the Raspberry Pi family have long made it a logical choice for homebrew network attached storage (NAS) implementations. Sadly, the single shared USB lane and the use of USB 2.0 ports has always put a hard limit on performance — something the Raspberry Pi 4’s two USB 3.0 ports should, in theory, address. Here, a SATA SSD is connected through an adapter to the Raspberry Pi’s USB port — USB 3.0 in the case of the Raspberry Pi 4, USB 2.0 in all other cases — and the fio utility used to benchmark its read and write performance.

The only real surprise here is how static USB performance has been in the Raspberry Pi family: from the original launch model right through to the Raspberry Pi 3 Model A+, USB throughput has been near-identical thanks to the single-lane shared bottleneck to the SoC.

The Raspberry Pi 4 is the first to change that: its read and write throughput are leagues ahead of its predecessors, and approaching — though not quite reaching — the limits of the connected drive itself.

MicroSD Throughput Benchmark

The USB ports aren’t the only storage interface to receive an upgrade on the Raspberry Pi 4: the microSD storage system has been entirely overhauled, adding in double data rate (DDR) support for improved throughput — up to, at least theoretically, double its predecessors. The following benchmark was carried out using a 64GB Samsung Evo Plus microSD XC (SDXC) Class 10/U3 card, officially rated by the manufacturer at 100MB/s read and 60MB/s write.

The shift to a DDR interface has an obvious effect on the throughput achievable with a decent microSD card, though it’s most obvious in the read speed; write, meanwhile, enjoys a smaller but still noticeable performance boost. At around 46MB/s read throughput, one thing is clear: those relying on high-speed storage will be far better off booting the Raspberry Pi 4 from a USB 3.0 external drive than using any size of microSD card. Sadly, they’ll also have to wait: the ability to boot from USB and Ethernet didn’t make it in time for launch, but Ethernet boot will be available in an early update with USB boot to follow on later.

Physical Benchmarks

The Raspberry Pi 4’s layout is slightly tweaked compared to the original Model B/B+ design, though you have to be paying close attention to notice the changes beyond the moved Ethernet port and the new micro-HDMI connectors.

The increased footprint, measured where the board is widest and including its connectors which stand proud of the PCB edge, is marginal and comes from a few-millimetre change in how far the connectors stick out. The change, Upton explains, came about to improve casability — how well the Pi sits in both first- and third-party cases. Combined with the new micro-HDMI, USB Type-C, and swapped USB and Ethernet connectors, though, it means most cases designed for older models won’t be compatible without modification.

The weight, meanwhile, has dropped from the Raspberry Pi 3 Model B+’s chart-topping 50g to a lighter 46g — a figure which still makes the Raspberry Pi 4 the second-heaviest Raspberry Pi in the line-up. As always, anyone with a weight constraint to their project — such as high-altitude ballooning or drone control — should look to the Raspberry Pi Zero family.

Conclusion

There’s no denying the Raspberry Pi 4 is an impressive machine. While the loss of the full-size HDMI port is a disappointment, the fact it’s now possible to drive two displays simultaneously — and at 4K resolution, no less — definitely makes up for it. The switch to a USB Type-C connector for power makes sense, as that’s where the smartphone and tablet market has been going for some time, and the tweaked layout will cause short-term pain in the form of now-obsolete case designs but for long-term gain.

That said, not everything is an improvement. The powerful new BCM2711 SoC really pumps out the heat, and active cooling is more important than ever for projects which put a frequently-loaded Raspberry Pi into an enclosed area. Those using the optional Raspberry Pi Power over Ethernet (PoE) HAT will be sorted here thanks to its in-built fan, while a small heatsink attached to the top of the SoC will improve things still further. The promised 4K video playback is limited to H.265 content, too, while hardware decode for MPEG2, MPEG4, and H.263 has been dropped on the understanding the CPU is powerful enough to decode these formats in software without too much strain.

The cons in no way outweigh the pros, though, and with the Raspberry Pi 4 the Foundation has addressed a laundry-list of niggles and complaints the community has been voicing since the original Raspberry Pi Model B: the USB bottleneck is gone, there are high-speed ports for external devices, options for more than 1GB of RAM for those who need it, and dual-4K-display outputs — 4K60 on one display or 4K30 on two, unofficially extendable to 4K60 across two displays if you don’t mind overclocking the GPU and running the risk of display corruption if you use too many layers for the video scaler’s available bandwidth. Coupled with the release of a new Raspbian Linux distribution, based on Debian ‘Buster,’ the Raspberry Pi 4 is the first to realistically act as a true desktop replacement for a wide swathe of the computer-using populace.

There are additional under-the-hood improvements not addressed in this testing, too. Chief among these is the addition of extra buses: there are four UART serial buses, four SPI buses, and four I²C buses, which will be welcomed by those building more complicated creations. The board’s pulse-width modulation (PWM) functionality has been upgraded, and is no longer shared with the analogue audio output available on the 3.5mm AV jack.

For most users, the Raspberry Pi 4 will become the must-have board in the range; only those with power, size, weight, or budgetary restraints should look at the other models in the family.

Further reading:

Creating a Rogue Wi-Fi Access Point using a Raspberry Pi

Building a Smart Garden With Raspberry Pi 3B+

Building a Circuit communication device using Raspberry PI