How Modern Distributed Systems Keep the Internet Running

Introduction

From streaming movies, scrolling social media, online shopping down to cloud storage, all of these activities are heavily dependent on a vast network of computers working together and communicating under the hood. For the end user these activities might seem quite streamlined or straight forward, the infrastructure that powers them on the other hand exist within layers of engineering complexity. To efficiently handle the gigantic amount of requests generated from internet services around the world there's a need to deploy a distributed network of systems each with its own specific use-case, handling millions of requests for different functions on a single machine is not feasible.

This is where distributed systems come in, tech companies now prefer having a group of connected computers that are constantly sharing data, resources and reacting to events and coordinating amongst themselves in order to produce a single efficient outcome. Instead of centralizing the process, the work is distributed across multiple servers creating a system that is more reliable, scaling and preforming with more efficiency. The introduction of cloud computing and microservices has further boosted this style of architecture. The essence of microservices is to break down the moving parts of huge applications into sizable chunks that can be developed, deployed and scaled independently of each other.

Distributed systems now largely power the internet, as user demand grows so does the complexity levels required to keep systems alive and that creates an ever growing need for distributed systems. In order to understand how today's connected world works, we need to examine how these distributed systems work, they go hand in hand.

What Exactly Is a Distributed System?

A distributed system is a collection of independent computers or servers that exist on a network and collaborate together to achieve a common goal as a single entity providing a service. The goal for distributed systems is to achieve maximum efficiency while still making the end user feel like they are using a single application or service, but the workload that produces that single application or service is usually spread across multiple machines in multiple locations.

To fully grasp the concept of distributed systems, we have to define a few terms.

A node is a machine that exists within a network of similar machines.

A server is a node that performs a specific function for or to other servers/machines within a network.

Multiple servers can be classified as a cluster. Servers within a cluster can share workload among each other. Sharing connection via a network allows these machines be able to share data and respond to requests quickly.

This is a different strategy from monorepo architectures, where one machine handles all the storage, processing and logic. Though small scale services can effectively manage a single server, it can quickly become a problem as demand and usage grows. If that single server fails in any part of it's components the entire service might go down.

This is why distributed systems break down the workload/components of a service and spread it across different machines. By so doing, the system becomes easily scalable and more reliable, that is because the failure of one isolated component doesn't necessarily result in overall failure of the entire service.

Why the Internet Needs Distributed Systems

Billions of internet users around the world expect internet services to be available on demand at anytime and location they make a request for it. Obviously, no matter how powerful a single serve might be, it'd be near impossible for it to match the scale of such demand. The more popular an application gets, the higher the volume of requests it receives and data it stores. As the application exhibits these growth metrics, it also has to be able to maintain fast response times for different users across different regions.

In order to be able to resolve the challenges of scalability, reliability, availability and speed, a distributed system would share workloads among multiple servers rather than relying on a single machine. This easily allows a auto-scale as we grow approach as opposed to scaling manually in a rush to meet demand. For example, during major events when video streaming platforms experience a surge in viewers, a distributed system would auto-deploy additional servers to handle the increased traffic without disrupting the user experience.

One other crucial element of distributed systems that we should look at is load balancing, which is a server that serves as a distribution point for incoming requests to be sent across multiple servers in order to prevent any one machine from having a heavy workload.

The load balancing server is usually referred to as the load balancer. A lot of load balancers use geographic distribution, by deploying servers in different locations around the world, requests are then routed to the servers closest to the user which helps tremendously in reducing latency and improving performance.

The biggest platforms on the internet today are dependent on distributed systems to remain responsive and reliable, that's including but not limited to social media networks, cloud service providers, and streaming platforms. Without distributed systems, the internet simply wouldn't be what it is today.

The Core Building Blocks of Modern Distributed Systems

In order to produce the most efficient result possible, modern distributed systems rely on several interconnected components. Each individual system may differ by architecture, but they all share a common set of foundational building blocks.

Servers and Nodes

Distributed systems can be thought of as a network of servers bounded together to function as a single unit, each server within that network can also often be referred to as nodes. From processing requests, storing data, or running application services, each node is separated from the others for it's own specific task. The distributing of workloads across the separated nodes ensures that systems can continue functioning even if individual machines fail, reducing the risk of service interruptions.

Databases

Without data and databases, modern application could never attain true dynamic functionality. Which is why distributed databases exist to replicate and distribute data across multiple servers instead of a single machine. This greatly helps in enhancing fault tolerance, and in enabling faster access to information for users in different geo-zones.

Microservices

The microservices architecture is adopted by many popular applications today, a microservice is simply a functional part of any large application that has been isolated and is being run as a smaller, independent service. Features such as authentication, payments, messaging, and notifications can be run as a microservice to a large application. Running functions as a microservice serves as a way to simplify things and make points of failure easy to spot.

Containers

Containers just like their name dictates are simply environments where applications can be run in a light-weight manner, a container basically bundles an application and its dependencies together ensuring that their behavior remains consistent across different servers and environments. Containers help in improving the efficiency of deployment and maintenance.

Cloud Infrastructure

Cloud infrastructure is highly integral to all the other building blocks mentioned above because they are responsible for providing the computing, storage, and networking resources required to operate distributed systems at scale.

Cloud platforms today help organizations to automatically allocate resources, auto-deploy services globally, and adjust capacity based on changing demand, making them a totally indispensable component of the internet and distributed systems.

The Biggest Challenge: Keeping Everything Consistent

Distributed systems are prone to failure, that's almost guaranteed. Could be the upload/response speed, servers/microservices can be sometimes unavailable, and data can become inconsistent or stale. Consistency is one of the most common issues a distributed system can face.

Distributed system failures are usually classified under three points

• Consistency

• Availability

• Partition Tolerance

For a distributed system to be considered consistent, it means all users must have the same experience and see the same things at the same time regardless of which component of the system they are connected to of their location at the time of connection.

if a system fails to respond to user requests when parts or components of it are failing, such system is considered to be failing in availability.

The ability of a system to keep operating even when communication has broken down between servers is known as partition tolerance.

Conclusion: The Future of Distributed Computing

The concept of distributed systems is a ever evolving one, as new ideas are generated so are new approaches invented based demand and the current requirements dictated by the market.

One notable shift to watch out for is the rise of edge computing, which is a concept where users are more responsible for processing data as opposed to centralized data centers. Edge computing can help reduce latency andensure even faster, real-time responses for modern day applications.

The rise of AI infrastructure has also placed a lot of pressure on distributed systems. LLM's require massive computational resources and this has brought about astronomical demand for clusters of specialized hardware such as GPUs and TPUs. Nowadays, cloud platforms are now evolving into highly optimized global-scale systems capable of supporting continuous model training and inference.

Real-time applications are heavily dependent on ultra-low-latency systems, for example financial trading apps used by institutions like hedge funds can incur losses and negative perfomance through mere milliseconds of delay, hence the need for ultra-low-latency. As a result, distributed architectures are increasingly focused on improving their geographic distribution, intelligent workload routing, and adaptive scaling in the nearest future.

All in all, all of these points indicate that distributed computing will even become more integral to the success of emerging technologies, powering everything from intelligent systems to fully autonomous digital infrastructure as time progresses.

References

• IBM — What Is the CAP Theorem?
  https://www.ibm.com/think/topics/cap-theorem

• MongoDB — The CAP Theorem Explained
  https://www.mongodb.com/resources/basics/databases/cap-theorem

• Google Cloud — What Is Microservices Architecture?
  https://cloud.google.com/architecture/microservices-architecture-introduction?hl=en

• Google Cloud Blog — Microservices Architecture on Google Cloud
  https://cloud.google.com/blog/topics/developers-practitioners/microservices-architecture-google-cloud

• arXiv — Microservices: How To Make Your Application Scale
  https://arxiv.org/abs/1702.07149

• arXiv — The Architectural Implications of Microservices in the Cloud
  https://arxiv.org/abs/1805.10351

• Hakia Engineering — Distributed Systems Concepts for Engineers
  https://www.hakia.com/engineering/distributed-systems/