Blog

Thoughts on software engineering, web3, and web development.

From Pulsar to Workers: Three Generations of a DeFi Data Pipeline

An engineering retrospective on rebuilding a DeFi data platform, from Pulsar and Flink to Rust services and Cloudflare Workers, while keeping the underlying data model stable.

#system design#data pipeline#aws#pulsar#flink#rust#cloudflare workers#defi#postgresql

system designOct 11, 20267 min read

Back of the Envelope Estimations

The numbers worth memorizing for a fast sanity check — powers of two, the latency of common operations, and availability in nines — and how to use them to size a system like X (Twitter) from a few rough assumptions.

#system design#estimation#capacity planning#latency#availability#scalability

system designOct 7, 202613 min read

Scale from Zero to Millions of Users

A walk through how a single-server app grows into a system serving millions — load balancing, database replication, caching, CDNs, a stateless web tier, multiple data centers, message queues, and sharding.

#system design#scalability#distributed systems#high availability#load balancing#database replication#caching#cdn#message queues#sharding#databases#architecture

projectOct 1, 20266 min read

Exporting GitBook Docs as Clean Markdown with Rust

How doc-scraper-rs exports GitBook docs as a clean markdown tree with llms.txt and AGENTS.md sidecars — one request per page, no HTML parsing, and a content-based fallback for soft 404s.

#rust#open source#llm#cli#scraping

securityMar 18, 20244 min read