One ear, many mouths
Here’s a thought. Imagine someone talking to you. Now imagine the noise of 10 people talking at the same time (those of you with a number of small children in the house may not have to tax your imaginations too much). Now 100, now 1000…
As you add more and more people talking to you at the same time, the loudness increases, unless you get them to each talk more quietly.
A similar problem occurs when you have many users on the internet talking to your cloud service.
Communication on the Internet
We’re all used to the idea that computers can use the internet to talk to each other. A typical broadband connection in the UK might have a bandwidth from 3Mbits/second to 100Mbits/second. This is an upper limit on the amount of data which can be transferred at any one time along that link.
Similar limits apply to our server systems, “in the cloud”. These servers are located in specially designed data centres, which ensure they are well cared for – with physical security, a reliable power supply and cooling systems. They are also given high speed connections to the core of the internet. The speed of these connections can be increased up to and beyond Gigabytes per second, but for many cloud systems the connection speed is not the first concern for someone deciding how to grow their service to handle large numbers of users.
Most internet services are built on top of a request system called ‘HTTP’. This one of several agreed-upon ways for computers to talk to each other. Broadly speaking, HTTP allows computers to “take turns” talking to each other. The home computer makes a ‘request’ and the cloud computer sends a ‘response’ in reply. An example of a request might be “please give me the web page for https://cocoon.life” and the response might be the contents of the web page.
For simple web sites, this can be handled fairly simply on the server side. For more complex sites, the server has to do more “thinking” about each request before it can send back the response. For example, it may have to decide if the reply contains any sensitive information, and if so, if the request came from someone who is permitted to see it. To answer these questions, the server computer may have to talk to several other systems and perhaps perform a number of cryptographic operations. These crypto operations are basically “lots of hard sums” and increase the amount of work needed.
If a server has to do a lot of thinking for each request and/or is given a lot of requests at the same time, it can have too much to do and end up doing everything too slowly.
What this means is that for some web sites, the speed of their internet connection becomes less important than the amount of work they have to do for each request. When we’re designing the systems to make internet services work, we talk about how many “requests per second” a server can handle without slowing down.
One way to handle more requests per second is to simply use a faster computer. This can work well for smaller systems, but it isn’t a good approach if you are planning for large numbers of users. Very high end computers can be expensive and give you less “bang for your buck” and ultimately, really popular services will require more resources than even the biggest servers can provide. Google and Facebook don’t run on a single big server.
If you’ve designed your system well, it is possible to have it handle more requests per second by simply adding more computers. It is not quite as easy as that (if it were, my job would be a lot less interesting :-), but a well-built system can work that way. So that’s one answer – simply add more and more computers until you have enough to handle the number of requests you receive.
However, even though computers continue to get cheaper, simply throwing more and more servers at a problem can be an expensive way to solve it. With enough users, it is worth spending some time and thought on how to get things to work more efficiently.
The Cocoon systems have to be able to pass messages between your Cocoon in your home, your mobile phones and our cloud servers. These messages allow you phone to tell your Cocoon when you’re heading home and can allow your Cocoon to let you and your phone now if there’s anything going on at home you’d like to know about. We want these messages to get through reliably and quickly, because they’re important.
It’s fairly easy to send a message to the cloud server, just make a request as we described above, saying “here’s a message”. But receiving the messages requires a little more thought. It’s easy enough for your phone or Cocoon to send a request to the server to say “do you have any messages for me”? But how often should it do that? Too often, and you’ll use a lot of phone battery and use up your mobile data plan with lots and lots of conversations like “do I have any messages?” – ‘No’, “do I have any messages now?” – ‘No’….
That also means that the Cocoon cloud service would gets lots of requests, and we’ve seen that can make the service more expensive to run.
But checking too infrequently leads to the reverse problem where it can take a long time to get messages between your phone and your Cocoon, which isn’t what we want either.
Google Cloud Messaging
So what should we do? Well it’s nice when someone has already solved your problem for you…
If you’re using an android phone which has Google Play installed (which most do) then your phone is already listening for messages from Google’s systems. If you’ve ever gone to Google Play on your laptop or desktop, you may have discovered that you can select an app and ask for it to be installed on your phone. Which will then magically happen within a minute or so.
How does your phone know that you’ve done this? Because Google has designed a system to allow your phone to receive messages very efficiently from it’s servers. Usefully for us, they also allow other apps to make use of it. And lastly, it takes no more battery power or data usage for your phone to check for Cocoon messages, since they come on the same channel as the Google messages.
This even works if your phone goes offline (e.g. you go through a tunnel on a train, or use airplane mode). Your phone starts listening again when it can.
So if you’re using an Android phone with Google play installed, Cocoon will send messages to your phone using Google Cloud Messaging, which is the most efficient way possible. This is good for your phone, your data plan and our servers.
What about everyone else?
Fortunately for iPhone users, Apple have a similar service to Google Cloud Messaging called Apple Push Notifications.
Even then, our worries aren’t quite over, since we also need to get messages to your Cocoon at home, sitting safe and snug behind your home router and firewall. Getting that to work efficiently requires a different approach. But that’s the sort of thing which keeps the job interesting…
To ensure we have efficient systems, we strive to make the communication between the different components of the Cocoon systems as light as possible. Our boxes whisper to each other, saying the minimum they need to be understood and at the right times.
This way we can ensure our server systems run quickly and reliably and without expensive waste. This allows us to get the right size systems in place to handle our first 1000 Cocoons, then the next 10,000, then the next 1,000,000…