System Design Guidelines for Computer Engineers

System design is the process of planning how to build a software system, figuring out which pieces you need, how they talk to each other, and how the whole thing stays fast and reliable even when millions of people use it.

Ever wondered how apps like YouTube, Instagram, or WhatsApp handle millions of users at once? That is because of proper system design. System design has now become one of the essential part of software development

In simple terms, we can think of System Design like designing a city. We don't just build houses randomly. We plan roads, water pipes, electricity lines, and fire stations so the whole city works together smoothly.

Imagine a team (or a single person) build a simple app that works great for 10 users. What if 10 million people want to use it? Without proper design, we might face following issues:

• Server crashes under the load
• The app becomes slow and unresponsive
• Data gets lost or corrupted
• Development team can't fix or update anything without breaking everything

Good system design solves these problems before they happen.

Building block of system design (Core concepts)

1. Client and Server

It is the most basic thing in system design.

• Client: the device or app making a request (our phone, browser, laptop)
• Server: the device that receives and responds to that request

For Example: When we open Instagram, our phone (client) asks Instagram's server: "Give me the latest posts." The server finds that data and sends it back.

[Phone] -- request --> [Instagram Server]
[Phone] <-- response -- [Instagram Server]

2. Database

A database is where you store data permanently for future use. For example, datas like user profiles, posts, messages, payments, etc. are stored in database.

There are two main types:
Relational (SQL):
Data stored in tables with rows & columns, like a spreadsheet User accounts, orders, transactions.

Use SQL when our data is structured and relationships matter (e.g. a user has many orders). For example: User accounts, orders, transactions

Non-Relational (NoSQL):
Data stored in flexible formats (JSON, key-value, etc.) Chat messages, product catalogs, logs.

Use NoSQL when our data is flexible, unstructured, or needs to scale very fast. For example: Chat messages, product catalogs, logs.

3. Scaling

Scaling means making our system handle more traffic. There are two ways to do it:

Vertical Scaling (Scale Up):
Add more power to one machine: more CPU, more RAM. Simple, but there's a limit to how big one machine can get.

Horizontal Scaling (Scale Out):
Add more machines instead of making one machine bigger. This is how companies like Netflix handle millions of users, they run thousands of servers.

[Server 1]
[Server 2]  <- traffic split between them
[Server 3]

4. Load Balancer

When we have multiple servers, we need a load balancer to direct request a particular server.

A load balancer sits in front of servers and distributes incoming traffic evenly across them, so no single server gets overwhelmed. We can think of it like a host at a restaurant directing customers to different tables so no one waiter is overwhelmed.

                               |-> Server 1
User Request ??> Load Balancer |-> Server 2
                               |-> Server 3

5. Caching

Caching means storing the results of expensive operations temporarily so it can be reused quickly. For example: when a person searches for best pizza in Kathmandu on Google. Should Google re-run that search from scratch every time someone asks the same question? Of course not, that would be slow and wasteful.

First request:  App -> Database (slow: 200ms) -> stores result in Cache
Next requests:  App -> Cache (fast: 2ms)

Popular caching tools: Redis, Memcached

Tips: Frequently read data that doesn't change often such as, homepage content, user profiles, search results are datas that should be cached.

6. Content Delivery Network (CDN)

A CDN is a network of servers spread across the world that stores copies of static files (images, videos, CSS, JavaScript).

Instead of everyone downloading a video from one server in the US, a CDN serves it from a server near the user, so someone in Nepal gets it from a server in Asia, not America.

This reduces latency (delay) and takes load off the main server. Examples: Cloudflare, AWS CloudFront.

7. APIs

An API (Application Programming Interface) is a way for different software systems to talk to each other.

When your weather app shows you today's forecast, it's using an API to ask a weather service: "What's the weather in Kathmandu?" and getting back a structured answer.

The most common style today is REST API, which uses simple HTTP requests. Nowadays, GraphQL is also getting popular.

8. Message Queues

Sometimes tasks take too long to finish immediately. For example: uploading a video to YouTube, it needs to be compressed, processed into different resolutions, checked for copyright, etc. We don't want the user to wait for all that before they see "Upload successful!"

Instead a message queue works like this: "I'll handle this task later." The request goes into a queue, and background workers process it when they can. Popular tools for message queues are RabbitMQ, Apache Kafka, Redis

User uploads video -> Queue -> Worker 1: compress video
                            -> Worker 2: generate thumbnail
                            -> Worker 3: checks copyright

9. Availability vs. Consistency

Availablity vs. Consistency is one of the classic dilemma in system design. We often can't have both perfectly at the same time. Knowing which one to prioritize is one of the core system design skill.

• Availability: The system is always up and responding, even if the data might be slightly out of date
• Consistency: Every user sees the exact same, up-to-date data at the same time

In a banking app, consistency matters more, we can't show someone a wrong balance. In a social media feed, availability matters more, it's okay if users see a post 2 seconds late.

10. Microservices vs. Monolith

Monolith: Entire application is one big codebase deployed together. It is simple to implement, but hard to scale or update individual parts.

[One Big App: users + payments + notifications + search + ...]

Microservices: An app is split into small, independent services that each do one thing. Each service can be built, deployed, and scaled independently. This is how Amazon, Netflix, and Uber works. The downside is that it is more complex to manage.

[User Service]  [Payment Service]  [Notification Service]  [Search Service]

Some important terms

Some key terms that might be important for you are listed below:

Latency: Delay, how long a request takes.

Throughput: Number of requests that a system can handle per second.

Fault Tolerance: System that keeps working even if part of it fails.

Redundancy: Having backup copies so nothing is a single point of failure.

Sharding: Splitting a database into smaller pieces across multiple devices.

Replication: Copying data to multiple servers for backup and speed.

System design is a skill that we can build gradually through reading, building, and breaking things. The fundamentals we learned here, when considered, will make us a good engineer.