Last Minute Notes
  • Github
  • Docs
  • Learn
    @sayanstwt
Lessons
0/37 done
  • SQL vs NoSQL
  • Indexes & Storage Engines
  • Replication
  • Sharding
  • Consistent Hashing
  • Object Storage & Uploads
  • Search & Inverted Indexes
Learn/System Design/Data & Storage

Object Storage & Uploads

6 min Beginner 7 steps

Presigned URLs keep large files off your servers.

Bytes via API
all
Blob lives in
OLTP DB
Cost
high
ClientAPIOLTP DBPOST large video
request reply failure
In this lessonStep 1 of 7
  1. The problem
  2. The model
  3. Pre-signed
  4. Multipart
  5. Events
  6. Serving
  7. Recap
Step 1 / 7problem

Blobs in the wrong place

A user uploads a large video. The API buffers every byte, then writes it into a database row.

Your app servers burn bandwidth and memory, and the OLTP database fills with data it was never built to hold.

Key idea

Never store large blobs in the OLTP database. Put them in object storage and keep only metadata in the DB.

←→ step·R replay·Space pause

You'll learn

  • Keep blobs out of the OLTP database and app servers
  • Describe the bucket and key model, durability and consistency
  • Upload directly with pre-signed URLs and multipart
  • Process uploads with events and serve them through a CDN
Read the full notesHLD Building Blocks, 12