The average person isn’t inordinately patient when it comes to technology. We expect everything from messaging, logistics, gaming, and home automation apps to respond to our requests in real time. The second an app acts slightly sluggish, we’re quick to throw up our hands in frustration and delete it from our smartphone.
A data stream network (DSN) is a scalable infrastructure that provides a secure, always-on connection to a web, mobile, or Internet of Things device that allows it to stream data bi-directionally. The DSN delivers messages, detects when devices are online or offline, triggers connected device action, and resends messages when devices experience unreliable connections — all in real time.
Developers have access to software development kits in a multitude of languages, allowing them to connect devices to a DSN with a built-in quality of service and security. But what happens when you want to do more than simply send and receive data? There are several scenarios — filtering or translating messages, looking for anomalies in data coming from sensors, or reporting failures to a system administration dashboard — that dictate a different solution.
In these instances, most developers build out a microservice or embrace serverless architecture. Imagine you’ve built a messaging application and want to pinpoint the location of users on a map while filtering out profanity. You would start by building two microservices and then deploying them in your own infrastructure or hosted service.
Sending every message to two different microservices adds latency, necessitates an increase in capacity, and probably diminishes your quality of service. If you could instead write these functions and deploy them within your DSN, it would eliminate the need for developers to create, manage, and orchestrate remote microservices. You're programming the network. This in-house shift would allow your team to provide secure, low-latency execution of these functions at the point of presence closest to end users.
Nurturing Cooperation
Whether you’re building apps for web, IoT, or mobile platforms, there are evolving requirements for how this service is built and scaled. Dynamics are changing because of two colliding movements: the transition from monolithic to serverless apps and the move from a traditional request-response structure to always-on apps that instantly push data to users.
Instead of designing apps that receive input and respond only after processing requests, IoT applications have driven development changes because they must be able to receive and process data that's constantly pouring in from multiple sources. Ultimately, it’s a positive development.
Data streaming is the glue that holds microservices together. It takes the place of traditional HTTP request-response communication and allows for more security and increased efficiency with compatibility across devices and frameworks. Edge computing allows developers to write code that lives in the network, with messages processed and immediately spit back out when they come into a DSN. This means you don’t need to worry about file systems, database administration, or co-locating services near your user base. In short, your code becomes truly serverless.
A Lasting Marriage of Serverless and Data Streaming
To remain relevant in today’s tech marketplace, companies must find a way to pair serverless solutions with data-streaming technology. Certain characteristics should be a standard element of any app:
- Low Latency: Ditch the refresh button. Users want instant updates and data pushed directly to their apps in real time. When a user pushes a button on his smartphone to lock a door, he doesn’t want to wait five seconds; the door should lock instantaneously. Whether people are checking who’s online in a chatroom or interacting with other users during a live broadcast, data must arrive as quickly as possible.
To reduce latency, ensure your app maintains a persistent connection through open sockets instead of closing and reopening connections via TCP handshakes or DNS queries. We’ve had success by opening a connection once and then maintaining a secure, open connection for data streaming until the exchange ends. You’ll also want to compress messages into a single datagram size. This speeds up delivery and eliminates the need for multiple back-and-forth packets.
- Global Distribution: Apps have a global audience, and users expect to instantly access data from anywhere in the world. Games like Pokémon GO, on-demand taxi apps like Gett, and live video broadcasting apps like Periscope have exploded internationally. Had these companies offered laggy services, they would have never achieved the same level of success.
The only way to achieve a seamless experience across the world is to embrace distributed computing and have international points of presence. By replicating data globally, developers can guarantee low latency regardless of location. The closer users are to a data center, the lower their latency will be. If you have users in Asia and a data center in the Eastern U.S., there’s no way to avoid high latencies. By spreading data centers evenly throughout your user base, you can keep latency at a reasonable level.
- Uninterrupted Experience: Mobile and IoT apps present unique challenges thanks to unreliable connectivity and constantly changing IP addresses. These always-on apps must detect drop-offs, redeliver messages, and process missed messages in a completely seamless manner. Easy, right?
For a messaging service, connectivity boils down to two things: automatic reconnection and message catch-up. Automatic reconnection mitigates the effects of network disruption or disconnects. Combined with message catch-up — published messages are stored and sent once a connection is restored — clients can experience virtually uninterrupted connectivity despite unavoidable service challenges.
- Security: It might not seem as sexy as its brethren, but security is paramount for any application. You’ll want to be able to control who can send or receive data while integrating standard encryption protocols such as TLS and AES. Beyond surface-level encryption, you should include programmatic security in your serverless network so you can grant or revoke access to data stream channels based on individual IP addresses.
Finally, eliminate any inbound ports. For a server to push data to an IoT device or chat user, that device must be listening with an open inbound port. This presents a massive risk for serverless services because these ports remain open indefinitely. Risks such as malware infections, DoS attacks, and data theft are mitigated by scrapping the inbound port approach.
User expectations are higher than ever before. Apps are evolving from the traditional request-response model to always-on services that instantly transfer data. Data stream networks can work in tandem with serverless infrastructure to help you achieve low latency and uninterrupted experiences while bolstering security. The future is indeed quite bright — and it’s updated instantaneously.
What are your thoughts on blending data stream networks with serverless infrastructure? Let us know in the comments.
