F1 (Week 12, Wednesday, April 2)

F1 is as mission-critical as it gets

Source: F1: A Distributed Database that Scales

Stores Google AdWords data

• Google’s core revenue source
• High amount of data stored (100 TB)
• High velocity of incoming data
• High query rate (100 kilo-requests/s)
• High processing rate (10 Tera-rows scanned/day)
• Availability “reaches” five 9s (~5 min/year)
• Five-way replication across datacentres in multiple US regions

F1 does way more than your grandparents’ NoSQL services

Strongly consistent

Supports database transactions (ACID)

Full SQL query support and indexing

Built on Spanner

(Optional) Spanner: Google’s Globally-Distributed Database

Spanner is a service that provides

• Extremely scalable storage
• Synchronous replication across datacentres
  • Quorum algorithm (Paxos)
  • Higher latency

Global timestamps provide a total order

• But only up to a slightly-old global safe timestamp
• The safe timestamp is typically 5–10 s ago
• Can read from any replica, anywhere in the world as of the safe timestamp

The system has an uncertainty measure that indicates fuzziness of times

• Uncertainty computed using a mixture of GPS and atomic clocks
• Uncertainty typically around 10 ms

Latency and throughput for F1

Read latencies of 5–10 ms

Write (commit) latencies of 50–150 ms

User-perceived latencies of 200 ms

• Comparable to older MySQL database
• Better tail latency

Queries more highly paralellizable

• Linear speedup (claimed) with more instances

Some limitations

Higher CPU costs than MySQL

Clients have to change access patterns and schema to achieve above latency

Upcoming activities

Final assignment

The final assignment, which builds on and integrates the last two assignments into an eventually-consistent service, will be made available tonight or early tomorrow.

Course review

The answer key to the midterm will be made available on-line.

Exercises for the final exam will be made available on-line.