Skip to main content

System Design 101

前言

本篇文章是个引子，主要讲述System Design的来龙去脉。后面会分批讲解基础知识和实践中的"积木"和用法。

什么是系统设计(System Design)?

System Design顾名思义，就是去设计一个system。通常来讲，设计一个万能的system是几乎不可能的，所以需要有具体的requirement， 根据requirement来tradeoff并决定使用某种架构。

为什么要学习System Design?

每个人学习System Design的动机不一样，但最常见的有两种原因:
  • 你在准备面试，你的dream company会考到System Design；
  • 你是架构师或者想成为架构师 需要用到相关知识。
我的观点是，即使是第一种情况，也希望你能抽出不少于刷leetcode的时间来看System Design。因为System Design和刷题的本质区别是，进入公司以后，除非你每天还在刷题保持状态，否则你早晚是要都还给老师的。而System Design才是你能长期收益的 一石二鸟，何乐而不为呢？

学习System Design应该学习哪几块?

我个人的经验，需要学习三大块:
  • Distributed System知识: 比如Sharding, Sticky sessions, etc。
  • 架构知识: 有哪些架构是常用的？举个例子，Apache Storm的创始人Nathan Marz提出的Lambda Architecture就是一种典型的data processing架构 了解这些架构会对你搭建大型系统有很大帮助。
  • "积木"知识: 我管它叫积木，其实就是别人做好的轮子，拿来即用即可。但你需要了解该轮子的优缺点以及适用范围，比如SQS和kinesis的区别 ActiveMQ和Kafka的区别。


面试和实践需要学习的点一样么?

Yes and no.
  • Yes的原因是两大块对于面试和实践都是非常有帮助的；
  • No的原因是对于面试而言，面试官的侧重点是Distributed System的知识。知道某些积木会有加分但不是必须。而对于实践而言，知道各种积木以及其优缺点反而更能发挥作用。


System Design面试的例子

我在自己面试的过程中 曾经被问到过许多System Design的题目，在这里我挑出几个典型的供大家参考:
  • 公司A: Design URL Shorten Service
  • 公司B: Design SQS(i.e. AWS's queue service)
  • 公司C: Design Uber(frontend app views + backend service)
下面我来详细解释一下每一题的考点:

Design URL Shortening Service

这一题是非常经典的System Design题目，可以考的很浅，也可以考的很深。由于特别适合初学者入门，建议每个想学习System Design的同学都要把这道题的可能的条件和解法过一遍。比如说:
  • If your website is the top URL shortening service in the world(i.e. handling 70% of world URL shortening traffic) How do you handle it?
  • How do you handle URL customization?
  • What if you have very hot URLs? How do you handle it?
  • How do you track the top N popular URLs?

Design SQS

这一题是非常geeky的一道题，完全深度考察distributed system的各种知识。难度比URL Shortening Service高，原因在于后者已经成为常规考题，变种变来变去就那么几个，所以你死记硬背也能过关。而前者是非常见题 考查点对于没有系统学习过System Design的同学来讲难以琢磨。
同时这道题也是道好题，因为如果你有realtime backend system经验，多半可能会用到queue service。那考察的就是你有没有抽出自己的spare time去理解queue service的具体原理呢?

Design Uber

这是一道极其抽象的题，难易全凭面试官把握。
我被问到的具体情形是，根据手机app上的view transition design出整个后台service群以及互相交互的情况。我当时在白板上一口气写了10+个service的交互图，最后临走前还专门拍照留念，现在想来还是很自豪...
100个人会design出100个Uber，没有谁对谁错，只要能自圆其说就可以。


System Design积木的例子

System design的另一大块是我前面所谈到的“积木”，也就是别人已经搭好的framework或product。
业界的Framework非常之多，你并不需要每个都掌握。只要可以做到知道某方面的几个option，并在需要用到的时候快速ramp up就可以了。下面做一个小分类供大家参考:
  • In-memory Cache: Guava cache
  • Standalone Cache: Memcached, Redis
  • Database: DynamoDB, Cassandra
  • Queue: ActiveMQ, RabbitMQ, SQS, Kafka
  • Data Processing: Hadoop, Spark, EMR
  • Stream Processing: Samza, Storm
Labels:

Comments

Post a Comment

https://gengwg.blogspot.com/

Popular posts from this blog

CKA Simulator Kubernetes 1.22

  https://killer.sh Pre Setup Once you've gained access to your terminal it might be wise to spend ~1 minute to setup your environment. You could set these: alias k = kubectl                         # will already be pre-configured export do = "--dry-run=client -o yaml"     # k get pod x $do export now = "--force --grace-period 0"   # k delete pod x $now Vim To make vim use 2 spaces for a tab edit ~/.vimrc to contain: set tabstop=2 set expandtab set shiftwidth=2 More setup suggestions are in the tips section .     Question 1 | Contexts Task weight: 1%   You have access to multiple clusters from your main terminal through kubectl contexts. Write all those context names into /opt/course/1/contexts . Next write a command to display the current context into /opt/course/1/context_default_kubectl.sh , the command should use kubectl . Finally write a second command doing the same thing into ...
Post a Comment
Read more

OWASP Top 10 Threats and Mitigations Exam - Single Select

Last updated 4 Aug 11 Course Title: OWASP Top 10 Threats and Mitigation Exam Questions - Single Select 1) Which of the following consequences is most likely to occur due to an injection attack? Spoofing Cross-site request forgery Denial of service   Correct Insecure direct object references 2) Your application is created using a language that does not support a clear distinction between code and data. Which vulnerability is most likely to occur in your application? Injection   Correct Insecure direct object references Failure to restrict URL access Insufficient transport layer protection 3) Which of the following scenarios is most likely to cause an injection attack? Unvalidated input is embedded in an instruction stream.   Correct Unvalidated input can be distinguished from valid instructions. A Web application does not validate a client’s access to a resource. A Web action performs an operation on behalf of the user without checkin...
2 comments
Read more