Distributed Computing
Let us first consider what distribution, or decentralization, actually means. The distinction is illustrated by three different organizations of networked computer systems:
Figure 1.1 The organization of a (a) centralized, (b) decentralized, and (c) distributed system.
The natural distinction between decentralized and distributed systems lies in the arrangement of processes and resources across multiple computers in a networked environment:
- In a decentralized system, processes and resources are inherently spread across multiple computers.
- In a distributed system, processes and resources are sufficiently spread across multiple computers.
Distributed systems have the following advantages:
- Resource sharing: Distributed systems enable communication over the network and resource sharing across machines (e.g. a process on one machine can access files stored on a different machine).
- Economic: Distributed systems lead to better economics in terms of price and performance. It is usually more cost effective to buy multiple inexpensive small machines and share the resources across those machines than buying a single large machine.
- Reliability: Distributed systems have better reliability compared to centralized systems. When one machine in a distributed system fails, there are other machines to take over its task and the whole system can still function. It is also possible to achieve better reliability with a distributed system by replicating data on multiple machines.
- Scalability: As the number of machines in a distributed system increases, all of the resources on those machines can be utilized which leads to performance scaling up. However, it is usually hard to achieve linear scalability due to various bottlenecks
- Incremental growth: If an application becomes more popular and more users use the application, more machines can be added to its cluster to grow its capacity on demand.
Distributed Computing Models
Two prevalent models are client-server and peer-to-peer. Both of which will cover more in-depth in 2 Distributed System Architecture.
Figure 1.2 Client-server vs. Peer-to-Peer model.
- Client-Server Model: The networked application is divided into two primary components — client and server.
- Peer-to-Peer (P2P) Model: The network performs both client and server functions.
Distribution of Data