Testing the Internet of Things

Introduction to Testing IoT Devices

The Internet of Things (IoT) is gradually starting to take over the show. Yes, your smart watches can now advise you how to improve your fitness or how and what you should probably eat. Your mobile phone can talk to your door locks and get it locked, in case you have forgotten to do it yourself, but have you ever thought of testing these systems? Or have you ever imagined writing test cases for such IoT devices? If not, it’s now time to think and analyse how you want to devise a strategy for IoT testing.

Before talking of testing the Internet of Things, we should first look at how IoT works. An IoT device is any device (wearables, motion detecting camera, Smart lock, GPS etc.) that can communicate with another and also with an IoT hub. This hub could be a mobile app or a server that can collect data from these devices. Exchange and analysis of this data, will in turn provide the users with meaningful output, which is the key point in IoT theory.

Why is the Internet of Things Drawing a lot of Interest Nowadays?

Every vendor and stakeholder these days, asks their service provider if they can make use of IoT and AI in their projects. Here are some reasons why:

• Smart living concepts developed amongst people
• Development of lightweight protocols like MQTT, that can exchange data even in poor or limited bandwidth
• Advanced growth of nano-technologies and integrated/embedded systems
• Developments in Artificial Intelligence and Machine learning

A Few Prerequisites Before You Test an IoT System

1. Setting up the IoT device
   Make sure the IoT device is turned on, accessible and is being used in real life. For example, if you are testing a smart watch, make sure you wear it on your wrist. Placing the band on a table won’t let you create a genuine user case for testing.
2. Setting up the IoT Hub
   An IoT hub is a server that can talk to the IoT devices and collect data from it. This may be an app in a mobile device or a web server on cloud. Whatever you use, make sure the hub is set up properly.
3. Setting up the network
   A strong wireless connection is needed for the device and hub to talk to each other. They can be connected via WiFi, Bluetooth, Satellite signals, NFC etc. If you plan to connect your wearable (IoT device) with the mobile app (IoT hub) make sure of the following:

    

   • Bluetooth is turned on in both devices
   • Devices are paired
   • Devices are in range (within 10m range for Bluetooth connection)

Testing IoT Systems

Defining test cases for IoT devices can be considered as a uphill task. Other than testing the real life scenarios there are a few common test scenarios you need to consider while testing IoT devices and the network.

Security

If you don’t run adequate security validations, there is no point in testing the functions of the device. Here are a few reasons why:

• The hub and devices should be properly authenticated (registering to the network) before communication starts. For example, in the case of Bluetooth connection, only paired devices should be able to communicate. Any unauthorized connection should be discarded.
• Data connection should be easily established after successful registration.
• The sent data should be in encrypted form.
• If a maximum number of connections have been attempted, the device should not try to connect again for a pre-defined time period.

Performance

Testing the performance of IoT devices will be another crucial aspect.

• Every authenticated device in range should be able to connect to hub.
• The device should be able to send any amount of data to the hub (as per the requirements).
• If data sent by the device exceeds a predefined amount, transfer of data should be initiated only after a pre-set delay or after confirmation has been received from the hub.
• The device should be able to send data, even during a low power status. A power status update should be sent to the network if the device goes low in power.

Connectivity

The success of an IoT system depends on how well the devices and hub are connected. Even a loss of connection for a fraction of a second can lead to inaccurate data, which will in turn make the system unusable. So testing the connectivity is as important as testing security and performance.

• The device should be connected to the hub even if the hub is in sleeping/power saving mode.
• The device should send regular ping messages to make sure the connection is not lost.

Usability

The usability of IoT devices is also an important aspect to consider while testing. For example, if I am using a smartwatch to make NFC (near field communication) payments with my bank, usability concerns might arise that need to be tested and verified. As well as being usability concerns, they could also be security concerns for users as the bank transfer deals with their confidential information. Here are a few usability test cases for the above mentioned situation:

• Verify if payment can be made only on authenticated NFC enabled POS machines.
• If the wearable is lost, the user should have the provision to block the device (maybe from a mobile device or through an IVR) as soon as possible.
• There should be a pre-set limit for payment so that the chances of unauthorized payment can be minimized in case of lost/stolen cases.
• Multiple payments should not happen if the user waves the wearable over the POS machine multiple times.

Challenges for Internet of Things Testers

Though we talked a lot about the prospects for IoT testers, his or her challenges will increase over time, such as:

• Calibration of sensors can vary from device to device
• Setting up the environment can be very expensive
• Connectivity issues
• Power issues
• Security issues

Summary

Internet of Things testing can be a very challenging task as it depends on quite a few factors, such as the sensitivity of devices, specification changes from vendor to vendor and reliability of the network, etc. Yet we are successful in using a lot of IoT enabled devices, as we are become capable of overcoming many of the challenges. Users’ security and performance concerns are being dealt with successfully by the vendors, so encouraging the use of these devices around the globe. So if you are interested in testing an IoT enabled device such as a Smartwatch, the challenge is on; you will have to be even smarter.