cat /dev/random meh

The Road To Serverless: Intro & Why

Author: Ed Huang (, CTO@PingCAP

The reason for this was a conversation with a friend about an interesting topic:

“If I were to provide free database cloud services for 100 million users, how should the system be designed?”

To answer this question, in my previous two blog posts, I vaguely mentioned that we are using some new perspectives and ideas to approach and build database services. We have turned these thoughts into actual products: Many friends are interested in learning more details about TiDB Cloud Serverless (hereinafter referred to as TiDB Serverless), as the whole article might be very long, I will divide the entire series into 3 parts. This is the first part and I will introduce some background information and why we need to do Serverless.

Some friends may not be familiar with TiDB, so allow me to give a quick introduction to TiDB first. For those who are already familiar with TiDB, you can skip this section. Here are some key terms and definitions:

  1. TiDB: Specifically refers to a distributed SQL database composed of components: tidb-server (responsible for SQL parsing and execution), tikv-server (responsible for data storage), and pd-server (responsible for metadata and scheduling).
  2. TiDB Cloud: A cloud service that provides fully managed TiDB on the cloud.
  3. TiDB Cloud Serverless: A cloud service that provides TiDB with cloud-native storage and multi-tenant features. It offers extremely low usage costs and startup times, adopts a Pay-As-You-Go payment model, and is completely free for developers in small-scale scenarios.

In early 2015, we embarked on an ambitious plan inspired by Google Spanner and F1 (I prefer to call it inspiration). We aimed to build an ideal distributed database: supporting full-featured SQL, using MySQL protocol and syntax, embracing ACID transaction semantics without compromising consistency; providing transparent and limitless horizontal scalability to dynamically adjust the physical distribution of data based on traffic and storage characteristics to achieve maximum hardware utilization; ensuring high availability and self-healing capabilities to reduce the burden of manual operations and maintenance (sounds too good to be true, right? 😊).

Driven by these goals, we made the following technical decisions from the very beginning:

  1. Highly modular architecture: tidb-server handles connection handling, SQL parsing, optimization and execution, tikv/tiflash handle storage (and a little bit distributed execution), and pd-server handles metadata storage and initiates scheduling commands.
  2. The storage layer adopts a Share-Nothing architecture with a Key-Value form (tikv) that supports ACID transactions to support elastic scalability, while other modules are designed to be stateless as much as possible. For instance, once the basic TiDB cluster bootstrap is completed, many pieces of information can be stored in TiDB’s own system tables, such as the statistics used for SQL optimization. The aim is to minimize localized state (e.g., avoiding separate local and global indexes) to reduce the complexity of fault recovery logic.

It is evident that this design aims for strong scalability and large-scale scenarios. In the early days, TiDB’s idea originated from replacing sharding and partitioning solutions for massively scaled MySQL deployments. This strategy has been highly successful, validated by numerous customers, and has even replaced some well-known NoSQL systems known for their scalability. For example, Pinterest mentioned in their blog post that they successfully replaced their HBase with TiDB.

Looking back at the decisions made back then, we think TiDB is on the right track:

However, over time, we gradually identified two trends:

The above requirements present several limitations in traditional Shared-Nothing architecture:

For the first trend, the integration of OLTP and OLAP comes with high costs in a traditional Shared-Nothing architecture. The reason is that, for OLAP applications, CPU often becomes the bottleneck. To achieve good performance on OLAP, we can only add more computing resources (i.e., physical servers). However, OLAP workloads are typically not online services, so these workloads do not necessarily require 247 exclusive access to these resources. Nevertheless, we have to prepare these resources in advance to run these workloads. Even though TiFlash provides excellent elasticity and scalability at the storage layer (we can create TiFlash replicas for specific tables on-demand), it is challenging to achieve elastic scalability for computing resources in a non-cloud environment. For the second trend, traditional Shared-Nothing architecture in distributed databases usually assumes that all physical nodes have equal hardware configurations; otherwise, it would increase the difficulty of scheduling. Moreover, in pursuit of ultimate performance, multi-tenancy design is often sacrificed because most database deployments in the cloud are exclusive to applications. To reduce database usage costs, there are two common approaches:

Additionally, the migration of businesses to the cloud is another commonly mentioned trend, which I won’t delve into as it has been extensively discussed. Based on the assumptions above, let’s consider a question: If we were to rewrite TiDB today, what new assumptions should we make and what choices should we make? The goal is to cover more use cases and achieve higher stability.

New assumptions:

Our choices in TiDB Serverless:

The next two articles will provide a detailed introduction to the cloud storage engine and multi-tenancy design of TiDB Serverless.