The cloud. Why use such an odd term to describe online services?
Back in the early 1990s, engineers began using clouds as a metaphor for the Internet in textbooks and diagrams. The structure of the Internet – as seen from a distance – is amorphous like a cloud. Later, the cloud was used as a way to represent Internet-based services.
In practical terms, cloud computing is an array of networked computers that allow you to offload processing tasks or storage from your embedded system. It is a simple idea, but one that hides a lot of underlying complexity.
Many companies have launched services that try their best to simplify this complexity; these include Apple’s iCloud, Google Cloud Platform, Microsoft OneDrive, and others. But these cloud computing systems are intended for use with personal computers, and embedded developers need something similar for IoT devices.
Total number of connected devices (source: Gartner 2013)
Industry analysts forecast the creation of billions of IoT devices by 2020, and these devices will produce huge amounts of data. There are a few approaches for managing and processing all this data.
- Some companies are developing (and reselling) their own proprietary solutions.
- Other companies don’t have the capability to deploy a complete infrastructure, and so prefer to rely on third parties, either public or commercial.
Running a backend service must become a core competency for any company attempting to do so; there will be no room for dilettantes. But not all organizations have the DNA to provide fail-safe operation of their servers and network, let alone guaranteeing back-ups and system redundancy.
The ecosystem of the Internet of Things
The army of devices that compose the Internet of Things will generate more data than any individual Web application. By one estimate, there will be 5,200 gigabytes of data for every human on the planet by 2020.
To support the billions of connected devices expected to arrive by then, we would need to deploy about 340 application servers per day (or 120,000 servers per year). Cloud computing is certainly a good way of addressing these staggering requirements.
It has been said that every company will become a software company. Why so? Because the data created by your products, and the systems that manage your data, will soon have more value than your products themselves. So, if you are designing and manufacturing an IoT system, and require a backend service, you need to choose whether to deploy your own, or outsource it. Most companies will outsource. If you are a thermostat manufacturer, for example, you probably do not want to develop IT competency; it will be better to leverage someone else’s. And the big players such as Intel, Oracle, SalesForce.com and Google have already stated their intention to occupy that space.
If you do build your own backend, you will have to choose from the technologies we reviewed in the section on Internet Usage and Protocols.
Just as crucially, you will have to determine how your customers will access and manipulate their data. Will it be exclusively through Web browsers, or will you provide smartphone native applications (iOS, Android and Windows)?
Another vital feature of your backend service will to provide firmware upgrades remotely and securely to your IoT devices. With the rapid evolution of the IoT field, it is almost impossible to design for every contingency. This means that your device must be upgradable. And it will need flash memory, RAM, a bootloader, and all the other components required for your device to update firmware and applications securely.
The same questions need to be answered if you outsource your backend services. But in this case, you first need a provider that supports your preferred protocol. You don’t want your backend service provider dictating the protocol used in your IoT device. And you need to know if the provider can develop and support mobile data applications, or at least provide enough tools for you to do it yourself.
Here is a short checklist to get you started. Your backend service should provide:
- Design services to help you select communication hardware and software
- Support for cloud protocols (Websocket, RESTful, MQTT, CoAP, etc.)
- Support for secure remote firmware upgrades
- Web-based and mobile applications for data viewing, processing, and remote device control
- Web and/or mobile application development (consulting services)
- Per device
- Per transaction
- Per data type
- Per bandwidth usage
- Per storage usage
- Monthly/annual subscription
Look for companies offering a sandboxed approach, where you can register a free account and test your implementation before going commercial. Some services also offer data from public sources to help you build a more useful application. For example, access to weather data or utility prices could be useful when building smart energy applications.
Most of the current offerings are based on REST or RESTful APIs, and various Java technologies. These solutions may be best suited for applications that need to move larger data payloads.
Data Analysis (Big Data)
Big Data enters the picture when you need to correlate your own data with information that is stored in outside systems, either publically accessible or licensed.
Virginia Rometty in The Economist wrote about the kinds of systems that are required to process all this data:
Our world has become pervasively instrumented and interconnected, with computation infused into things nobody would think of as a computer. There are more than a trillion interconnected and intelligent objects and organisms—including a billion transistors for every person on the planet. About 2.7 billion of those people are now connected to the Internet, a number that is growing rapidly in every part of the world, thanks to the spread of mobile technology.
The result is a planet awash with information. It’s known as Big Data, with good reason.
A new generation of “cognitive” systems, built for big data, can keep up with the flood because they aren’t programmed, they learn—from their own experience and from our interactions with them. Cognitive systems can scrutinize structured information, such as databases, and unstructured information such as medical imaging and social media content, much faster thanks to Cloud computing.
This means we, the embedded community, may now want to learn about distributed database management systems such as Apache Cassendra, and learn Apache Hadoop to run analytics on all this data. Cassandra is an open source distributed database system that is designed for storing and managing large amounts of data across commodity servers.
Of course, there are commercial solutions as well. For example, GE Predix™ is a software platform for the Industrial Internet. It provides a standardized way to connect devices and data, and so enables industrial-scale analytics.
There are still some unanswered questions about how these billions of interconnected things will interact:
- What is the data representation model?
- What is the network discovery protocol? On any type of network, a discovery protocol like Bonjour or some other will be required.
And with all these billions of devices generating data, it makes sense to bring that data on the Internet and use what today we call Big Data to store and process all this data.