Backend
6 posts
Caching Without Regret: A Practical Guide to Redis
Caching is the fastest way to make a slow system feel quick, and one of the fastest ways to make a correct system wrong. The speed-up is immediate. The bugs show up weeks later, when someone asks why the dashboard still shows yesterday's number. This is the set of patterns I reach for with Redis, and the questions I try to answer before adding a cache at all. Start with the question Before writing any code, answer three things: - How stale can this data be? Seconds, minutes, a day? If the answer is "never", don't cache it, or cache it with a very deliberate invalidation path. - How expensive is a miss? A cache in front of a 5 ms query buys you little. A cache in front of a heavy aggregation or a slow third-party API buys a lot. - How is it read? Lots of reads of the same keys is ideal. Every request hitting a unique key means your hit rate will be poor. If the answers aren't good, a cache adds complexity without much benefit. Cache-aside: the default pattern Cache-aside (or lazy loading) is the pattern most applications use. The application checks the cache, and on a miss it loads from the source and writes the result back. python import json import redis r = redis.Redis(host="localhost", port=6379, decoderesponses=True) def getuser(userid: int) - dict: key = f"user:{userid}" cached = r.get(key) if cached is not None: return json.loads(cached) user = dbfetchuser(userid) your database call r.set(key, json.dumps(user), ex=300) 5-minute TTL return user It's simple, the cache only holds data that's actually requested, and if Redis goes down the application can still fall back to the database. The downside is that the first request after expiry always pays the full cost. TTLs are your safety net Always set a TTL (time to live). Even if you have an explicit invalidation path, a TTL bounds how long a bug can keep wrong data alive. A missed invalidation with a 10-minute TTL is an annoyance. A missed invalidation with no TTL is a support ticket next month. Invalidation On writes, you have two common options: - Delete the key after updating the source. The next read repopulates it. - Update the key with the new value. Deleting is usually safer. Updating risks a race: two writers update the database in one order and the cache in the other, and the cache ends up holding the older value until the TTL expires. python def updateuser(userid: int, fields: dict) - None: dbupdateuser(userid, fields) r.delete(f"user:{userid}") Update the database first, then delete. If you delete first, a concurrent read can repopulate the cache with the old value before your write lands. The hard part of invalidation is knowing which keys depend on a piece of data. A user's profile might be cached under user:42, inside a team listing, and inside a search result. Keep key naming consistent and the dependency map written down, or you'll miss one. Stampedes A cache stampede happens when a popular key expires and many requests miss at once, all hitting the database together. Three ways to handle it: Jittered TTLs If you cache lots of keys at the same time (on deploy, say), they'll also expire at the same time. Add randomness: python import random def ttlwithjitter(base: int, spread: float = 0.1) - int: return int(base random.uniform(1 - spread, 1 + spread)) r.set(key, value, ex=ttlwithjitter(300)) This spreads expiries out and avoids synchronised misses. A lock around recomputation Let one request rebuild the value while the others wait briefly or use a fallback. Redis SET with NX gives a simple lock: python import time def getreport(reportid: str) - dict: key = f"report:{reportid}" lockkey = f"lock:{key}" for in range(50): cached = r.get(key) if cached is not None: return json.loads(cached) if r.set(lockkey, "1", nx=True, ex=30): try: report = buildreport(reportid) expensive r.set(key, json.dumps(report), ex=ttlwithjitter(600)) return report finally: r.delete(lockkey) time.sleep(0.1) return buildreport(reportid) give up waiting The lock's own TTL matters: if the process holding it crashes, the lock frees itself. For anything critical, use a token so a process only deletes a lock it owns. Stale-while-revalidate Store the value with a soft expiry inside it, and a longer hard TTL on the key. When the soft expiry passes, serve the stale value and trigger a refresh in the background. Users never wait on the rebuild, and a slightly old value is often fine. python def getwithswr(key: str, loader, softttl: int = 300, hardttl: int = 3600): raw = r.get(key) if raw is not None: entry = json.loads(raw) if time.time() entry["softexpiry"]: schedulerefresh(key, loader, softttl, hardttl) e.g. a task queue return entry["value"] value = loader() entry = {"value": value, "softexpiry": time.time() + softttl} r.set(key, json.dumps(entry), ex=hardttl) return value Combine this with the lock so only one background refresh runs per key. What not to cache - Data that must be exact right now: balances, inventory at checkout, permissions. Stale permissions are a security bug. - Per-user data with low reuse, unless the source is slow. You'll fill memory with keys read once. - Huge values. Large blobs slow down Redis and the network. Cache an ID or a summary instead. - Errors, unless deliberately. Caching a failed lookup for a few seconds can protect a struggling dependency, but caching it for an hour hides recovery. Also set a memory limit and an eviction policy (maxmemory and maxmemory-policy, such as allkeys-lru), so Redis degrades predictably when it fills up. Before you ship the cache - Write down the acceptable staleness for each cached thing, and set TTLs from it. - Update the source first, then delete the key. - Add jitter to TTLs and a lock or stale-while-revalidate for expensive, popular keys. - Track hit rate and miss latency. If the hit rate is low, the cache is probably costing more than it saves.
Postgres Indexes: A Practical Guide
Most slow Postgres queries I've debugged came down to one of two things: a missing index, or an index that existed but couldn't be used. This is the mental model and the handful of patterns that cover most real cases. What a B-tree index is When you run CREATE INDEX without specifying a type, Postgres builds a B-tree. It's a sorted, balanced tree of the indexed values, each pointing to the row's location in the table (the heap). Because it's sorted, a B-tree is good at: - equality: WHERE email = 'a@b.com' - ranges: WHERE createdat = '2025-12-01' - sorting: ORDER BY createdat - prefix matches: WHERE name LIKE 'Yas%' (with the right collation or operator class) It does not help LIKE '%shank', because there's no sorted prefix to search on. sql CREATE TABLE orders ( id bigserial PRIMARY KEY, customerid bigint NOT NULL, status text NOT NULL, total numeric(12,2), createdat timestamptz NOT NULL DEFAULT now() ); CREATE INDEX idxorderscustomer ON orders (customerid); The primary key already gets a unique B-tree index automatically. Foreign keys do not, which catches people out: if you join or delete on customerid, index it yourself. Composite indexes: column order matters A composite index sorts by the first column, then by the second within each value of the first, and so on. Think of a phone book sorted by surname, then first name. sql CREATE INDEX idxorderscustcreated ON orders (customerid, createdat); This index serves: sql -- uses both columns SELECT FROM orders WHERE customerid = 42 AND createdat = '2025-11-01'; -- uses the leading column SELECT FROM orders WHERE customerid = 42; -- equality on the first, sort on the second: no separate sort step SELECT FROM orders WHERE customerid = 42 ORDER BY createdat DESC LIMIT 20; It is a poor fit for WHERE createdat = ... alone, because the dates are scattered across every customer. The rule of thumb: equality columns first, then the range or sort column. Put the column you filter on with = at the front, and the one you filter with or ORDER BY after it. Partial indexes If your queries only ever care about a subset of rows, index only that subset. sql CREATE INDEX idxorderspending ON orders (createdat) WHERE status = 'pending'; If 2% of orders are pending, this index is a fraction of the size of a full one, cheaper to maintain and more likely to stay in memory. The query has to include a matching condition for the planner to use it: sql SELECT FROM orders WHERE status = 'pending' ORDER BY createdat LIMIT 50; Partial indexes are also the clean way to enforce conditional uniqueness: sql CREATE UNIQUE INDEX oneactivecart ON carts (userid) WHERE active; Expression indexes An index on email doesn't help WHERE lower(email) = ..., because the indexed values aren't lowercased. Index the expression instead: sql CREATE INDEX idxusersemaillower ON users (lower(email)); SELECT FROM users WHERE lower(email) = 'yashank@example.com'; The query must use the same expression. The same applies to things like datetrunc('day', createdat), though often it's better to rewrite the query as a range on the raw column so a plain index works. Covering indexes with INCLUDE Normally an index lookup finds matching entries, then visits the table to fetch the other columns. If the index holds every column the query needs, Postgres can do an index-only scan and skip the table (as long as the visibility map says the pages are all-visible, which regular vacuuming keeps true). INCLUDE adds columns to the index leaf entries without making them part of the sort key: sql CREATE INDEX idxorderscustcov ON orders (customerid, createdat) INCLUDE (status, total); SELECT createdat, status, total FROM orders WHERE customerid = 42 ORDER BY createdat DESC; Use it for hot queries that read a few small columns. Don't stuff wide text columns in; the index gets large and the benefit disappears. Reading EXPLAIN ANALYZE Never guess whether an index is used. Ask: sql EXPLAIN (ANALYZE, BUFFERS) SELECT FROM orders WHERE customerid = 42 ORDER BY createdat DESC LIMIT 20; ANALYZE actually runs the query, so be careful with writes (wrap them in a transaction and roll back). What I look for: - Node types. Seq Scan reads the whole table. Index Scan and Index Only Scan use an index. Bitmap Heap Scan collects matches from an index first, then reads table pages in order, which is common for medium selectivity. - Estimated vs actual rows. rows=10 estimated against rows=50000 actual means the planner's statistics are off. Run ANALYZE orders; and see if the plan changes. - Where the time goes. Each node shows actual time=start..end. Read from the innermost node outwards and find the expensive one. - Sort nodes. A Sort with external merge spilled to disk. An index matching the ORDER BY can remove the sort entirely. - Buffers. shared hit came from memory, read came from disk. A sequential scan isn't automatically wrong. On a small table, or when the query returns a large share of rows, it's often the fastest plan, and the planner knows that. When indexes hurt Every index is a cost on writes. Each INSERT updates every index on the table; an UPDATE that changes an indexed column does too, and it can prevent Postgres's HOT optimisation, which avoids index updates when no indexed column changes. Indexes also take disk space, compete for memory and add vacuum work. So: - Don't index every column "just in case". - Find unused indexes with pgstatuserindexes (look for idxscan = 0 over a meaningful period) and drop them. - Watch for redundant indexes: one on (customerid) is mostly covered by one on (customerid, createdat). - On a live table, use CREATE INDEX CONCURRENTLY to avoid blocking writes while it builds. It's slower and can't run inside a transaction. What to remember - Start from the slow query, run EXPLAIN (ANALYZE, BUFFERS), and let the plan tell you what's missing. - Composite index: equality columns first, then range or sort. - Use partial indexes for hot subsets, expression indexes for transformed lookups, INCLUDE for index-only scans. - Index foreign keys yourself. - Treat every index as a write cost and remove the ones nobody uses.
FastAPI in Production: The Things the Tutorial Skips
FastAPI makes the first version of an API very pleasant. You write a typed function, add a decorator, and get validation and interactive docs for free. The trouble starts later, when real traffic arrives and something that worked fine on your laptop starts timing out. Most of those problems come from a handful of things the getting-started guide doesn't dwell on. Here's what I'd check before calling a FastAPI service production-ready. async def vs def, and blocking calls FastAPI lets you write endpoints as either async def or plain def, and the difference matters a lot. - An async def endpoint runs on the event loop. While it's awaiting something, other requests can run. But if it does anything blocking, such as a synchronous database driver, requests.get, or heavy CPU work, it blocks the entire loop and every other request on that worker waits. - A plain def endpoint is run in a thread pool, so blocking calls inside it don't freeze the loop. The classic mistake: python import requests @app.get("/rates") async def getrates(): r = requests.get("https://example.com/rates") blocks the event loop return r.json() Either make it fully async with an async client: python import httpx client = httpx.AsyncClient(timeout=5.0) @app.get("/rates") async def getrates(): r = await client.get("https://example.com/rates") return r.json() or keep it synchronous and declare it with def, letting FastAPI use the thread pool. The rule of thumb: use async def only if everything slow inside it is awaited. If in doubt, def is the safer default. For CPU-heavy work, neither helps much because of the GIL. Move it to a separate process or a task queue. Let Pydantic do the validation Request and response models are where FastAPI earns its keep. Be strict about them. python from pydantic import BaseModel, Field, fieldvalidator class PaymentIn(BaseModel): accountid: str = Field(minlength=1, maxlength=64) amount: float = Field(gt=0, le=1000000) currency: str = Field(pattern=r"^[A-Z]{3}$") @fieldvalidator("accountid") @classmethod def nospaces(cls, v: str) - str: if " " in v: raise ValueError("accountid must not contain spaces") return v class PaymentOut(BaseModel): paymentid: str status: str @app.post("/payments", responsemodel=PaymentOut) def createpayment(body: PaymentIn) - PaymentOut: ... Setting a responsemodel isn't just for docs. It filters the output, so an internal field you accidentally return doesn't leak to the client. Bad input gets a 422 with a clear error before your code runs, which keeps endpoint bodies free of defensive checks. Dependency injection for shared resources Depends is FastAPI's way of providing things an endpoint needs: a database session, the current user, settings. Using it consistently makes testing much easier, because you can override any dependency. python from fastapi import Depends def getdb(): db = SessionLocal() try: yield db finally: db.close() @app.get("/accounts/{accountid}") def readaccount(accountid: str, db=Depends(getdb)): return db.get(Account, accountid) in tests app.dependencyoverrides[getdb] = lambda: fakedb The yield form guarantees cleanup runs after the response. For things that should exist once per process, like an HTTP client or a connection pool, create them in the app's lifespan handler rather than per request. Background tasks are not a queue BackgroundTasks runs a function after the response is sent. It's handy for small things like writing an audit log entry or sending a notification. python from fastapi import BackgroundTasks @app.post("/signup") def signup(user: UserIn, tasks: BackgroundTasks): createuser(user) tasks.addtask(sendwelcomeemail, user.email) return {"ok": True} But it runs inside the same worker process. If the worker restarts, the task is lost. There are no retries, no visibility, and long tasks tie up the worker. For anything that must happen, takes a while, or needs retrying, use a real task queue (Celery, RQ, Arq, or a managed queue) with separate workers. My rough test: if losing the task would mean a support ticket, it doesn't belong in BackgroundTasks. Workers: uvicorn and gunicorn uvicorn main:app --reload is for development. In production you want several worker processes so one slow request or crash doesn't take down everything, and so you use more than one CPU core. A common setup runs gunicorn as the process manager with uvicorn workers: bash gunicorn main:app \ -k uvicorn.workers.UvicornWorker \ --workers 4 \ --bind 0.0.0.0:8000 \ --timeout 30 \ --graceful-timeout 20 Uvicorn can also run multiple workers on its own with --workers. In containers on Kubernetes, many teams run one uvicorn process per container and scale by adding pods instead. Either way, start with roughly one worker per core and measure. Timeouts everywhere A service without timeouts will eventually hang on a slow dependency and pile up requests until it falls over. Set them at every layer: - Outbound HTTP: always pass a timeout to your client. httpx has a default; requests waits forever unless you set one. - Database: set connection and statement timeouts in the driver or pool. - The worker: gunicorn's --timeout kills stuck workers. - The proxy or load balancer in front of the app. For async code, asyncio.timeout (Python 3.11+) or asyncio.waitfor bounds a whole operation, not just one call. Structured logging Plain text logs are fine until you need to search them. Log JSON with consistent fields, and include a request ID so you can follow one request through everything it touched. python import logging, time, uuid, json logger = logging.getLogger("api") @app.middleware("http") async def logrequests(request, callnext): requestid = request.headers.get("x-request-id", str(uuid.uuid4())) start = time.perfcounter() response = await callnext(request) logger.info(json.dumps({ "requestid": requestid, "method": request.method, "path": request.url.path, "status": response.statuscode, "durationms": round((time.perfcounter() - start) 1000, 1), })) response.headers["x-request-id"] = requestid return response Libraries like structlog make this neater. Don't log request bodies blindly; they often contain personal or payment data. Before you ship - Use async def only when every slow call inside is awaited; otherwise use def. - Validate inputs and set responsemodel on every endpoint. - Provide shared resources through Depends and the lifespan handler. - Keep BackgroundTasks for small, losable work; use a queue for the rest. - Run multiple workers, set timeouts at every layer, and log JSON with request IDs. None of this is exotic. It's just the difference between an API that works and one that keeps working.
Why Rule Engines Still Matter in Fraud Detection
When people hear "fraud detection" they usually picture a machine learning model scoring every transaction. Models are a big part of it. But in most real payment systems there's also a rule engine sitting in the path of every transaction, and it isn't going away. I spent this summer as an AI/ML intern at Card91, where I researched fraud detection and evaluated the open-source Tazama fraud engine. Reading through its rule logic made it much clearer to me why rules and models end up working side by side. What a rule engine does A rule engine evaluates each transaction against a set of explicit conditions: "more than five transactions from this card in ten minutes", "amount far above this customer's usual spend", "card used in two distant cities within an hour". Each rule produces a result, the results are combined, and the system decides whether to allow, block, or flag the transaction for review. The logic is written by people, readable by people, and changed by people. That's both the point and the limitation. Why rules haven't been replaced by ML Explainability. When a transaction is declined, someone will ask why: the customer, a support agent, an auditor, a regulator. "Rule 14: transaction amount exceeded the per-day limit for a new account" is an answer. "The model scored it 0.91" usually isn't, at least not without extra tooling. Latency. Payment authorisation has a tight time budget. Many rules are simple lookups and comparisons that run in microseconds once the right counters are in memory. They don't need a feature pipeline or a model server. Speed of response. When a new fraud pattern appears, a team can write and deploy a rule the same day. Retraining a model needs labelled examples of the new pattern, which by definition you don't have much of yet. Hard requirements. Some checks aren't predictions at all. Sanctions lists, blocked merchants, regulatory limits: these are policies, and policies should be code, not probabilities. Models are better at catching subtle patterns across many weak signals. Rules are better at being certain, fast and auditable. Most systems use rules as a first line and a model score as one more input, sometimes literally as a rule: "block if model score threshold and amount X". Common rule types Most transaction rules fall into a few families. - Velocity rules count events in a time window: transactions per card per minute, new payees per account per day, failed PIN attempts per hour. Fraudsters testing stolen cards tend to move fast. - Geolocation rules look at where the transaction happens: a country the customer has never used, or "impossible travel" where two transactions are too far apart for the time between them. - Amount threshold rules compare the amount to fixed limits or to the customer's own history: a first transaction far above the account's average, or many transactions just under a reporting threshold. - Behavioural rules look at changes: a new device followed immediately by a password reset and a large transfer. A simple rule in Python Here's a generic sketch. It isn't how any particular engine is built, but it shows the shape: each rule is a small, pure function that takes a transaction plus some context and returns a result with a reason. python from dataclasses import dataclass from datetime import datetime, timedelta @dataclass class Txn: cardid: str amount: float country: str ts: datetime @dataclass class RuleResult: ruleid: str triggered: bool reason: str = "" def velocityrule(txn: Txn, recent: list[Txn], window=timedelta(minutes=10), limit=5) - RuleResult: count = sum(1 for t in recent if t.cardid == txn.cardid and txn.ts - t.ts <= window) if count = limit: return RuleResult("VEL-01", True, f"{count} txns in {window}, limit {limit}") return RuleResult("VEL-01", False) def amountrule(txn: Txn, avgamount: float, factor=10) - RuleResult: if avgamount 0 and txn.amount avgamount factor: return RuleResult("AMT-01", True, f"amount {txn.amount} {factor}x avg {avgamount:.0f}") return RuleResult("AMT-01", False) Keeping rules pure (no database calls inside, context passed in) makes them easy to test and easy to reason about. The engine around them handles fetching history and combining the results. False positives are the real cost A rule that blocks all fraud is easy: decline everything. The hard part is false positives, legitimate customers getting blocked. Each one is a frustrated customer, a support call, and sometimes a lost customer. Some things that help: - Graduated outcomes. Not every triggered rule needs a block. Many should flag for review or ask for extra authentication. - Combine signals. One weak rule firing alone may mean little; three firing together means more. Weighted scoring across rules is common. - Personal baselines. "Ten times this customer's average" is usually better than a single fixed amount for everyone. - Watch the rates. Track how often each rule fires and how often its alerts turn out to be real fraud. A rule that fires constantly and is almost never right is noise. Testing rules properly Rules look simple, which is why they're easy to get subtly wrong. Off-by-one errors at window boundaries, time zones, currency conversions, and versus = all matter. Treat rules like any other code: python from datetime import datetime, timedelta def testvelocitytriggersatlimit(): now = datetime(2025, 7, 1, 12, 0) recent = [Txn("c1", 100, "IN", now - timedelta(minutes=i)) for i in range(5)] txn = Txn("c1", 100, "IN", now) assert velocityrule(txn, recent).triggered def testvelocityignoresoldtxns(): now = datetime(2025, 7, 1, 12, 0) recent = [Txn("c1", 100, "IN", now - timedelta(minutes=30))] 5 assert not velocityrule(Txn("c1", 100, "IN", now), recent).triggered Beyond unit tests, the useful habit is backtesting: run a new or changed rule over historical transactions before deploying it, and see how many it would have flagged. When porting rules between languages, comparing outputs from the old and new implementations on the same inputs is the quickest way to catch behaviour that drifted in translation. The short version - Use rules for what must be certain, fast and explainable; use models for subtle patterns. - Write each rule as a small pure function that returns a reason, not just a boolean. - Prefer graduated outcomes and combined signals over single hard blocks. - Test boundaries explicitly, and backtest before every rule change. Rules aren't the old way of doing fraud detection. They're the part of the system you can read, and that matters more than it sounds.
Building a PWA That Works on a Crowded Bus
The real test for a study app isn't a fast laptop on college Wi-Fi. It's someone on a packed bus, one bar of signal, trying to open their notes ten minutes before an exam. If the app shows a spinner and then a dinosaur, it has failed at the one moment it mattered. That's the problem I've been working on for NotesAid, a notes app for my classmates that's launching soon. It's a progressive web app (PWA), and most of the effort has gone into making it behave well with bad or no network. Here's what I've learned. What makes something a PWA A PWA is a website that can be installed and can work offline. Practically, that means three things: 1. Served over HTTPS (localhost is allowed during development). 2. A web app manifest describing how it looks when installed. 3. A service worker that intercepts network requests and can answer them from a cache. The manifest The manifest is a JSON file linked from your HTML with <link rel="manifest" href="/manifest.json": json { "name": "NotesAid", "shortname": "NotesAid", "starturl": "/", "display": "standalone", "backgroundcolor": "ffffff", "themecolor": "1f2937", "icons": [ { "src": "/icons/192.png", "sizes": "192x192", "type": "image/png" }, { "src": "/icons/512.png", "sizes": "512x512", "type": "image/png" }, { "src": "/icons/maskable-512.png", "sizes": "512x512", "type": "image/png", "purpose": "maskable" } ] } display: standalone hides the browser UI so it feels like an app. Provide a maskable icon, or Android will put your logo in an awkward white circle. iOS supports home screen installation too, but has historically been more limited and still reads some Apple-specific meta tags, so test on an actual iPhone. The service worker A service worker is a script that runs separately from your page and sits between it and the network. You register it from the page: javascript if ('serviceWorker' in navigator) { navigator.serviceWorker.register('/sw.js'); } Its lifecycle has three events that matter: - install: fires once for a new version. Pre-cache your app shell here. - activate: fires when this version takes control. Clean up old caches here. - fetch: fires for every request in scope. This is where caching strategies live. javascript const SHELLCACHE = 'shell-v3'; const SHELL = ['/', '/offline.html', '/app.css', '/app.js']; self.addEventListener('install', (event) = { event.waitUntil(caches.open(SHELLCACHE).then((c) = c.addAll(SHELL))); }); self.addEventListener('activate', (event) = { event.waitUntil( caches.keys().then((keys) = Promise.all(keys.filter((k) = k !== SHELLCACHE).map((k) = caches.delete(k))) ) ); }); Pick a caching strategy per kind of request There's no single right strategy. The trick is matching each type of request to the one that fits. Cache-first: check the cache, only go to the network on a miss. Best for things that never change once built, like versioned JS bundles, fonts and icons. javascript async function cacheFirst(req) { const cached = await caches.match(req); return cached || fetch(req); } Network-first: try the network, fall back to cache if it fails. Best for content that must be fresh when possible, like an API response for a user's notes list. Add a timeout, otherwise a weak connection means waiting a long time before the fallback kicks in. On a bus, a request that hangs is worse than one that fails fast. javascript async function networkFirst(req, cacheName, ms = 3000) { const cache = await caches.open(cacheName); try { const res = await Promise.race([ fetch(req), new Promise((, reject) = setTimeout(() = reject(new Error('timeout')), ms)), ]); cache.put(req, res.clone()); return res; } catch { return (await cache.match(req)) || caches.match('/offline.html'); } } Stale-while-revalidate: return the cached copy immediately, and fetch a fresh one in the background for next time. Best for content where slightly old is fine but speed matters, like a subject index or a shared note someone opened yesterday. javascript async function staleWhileRevalidate(req, cacheName) { const cache = await caches.open(cacheName); const cached = await cache.match(req); const network = fetch(req).then((res) = { cache.put(req, res.clone()); return res; }).catch(() = cached); return cached || network; } Then route in the fetch handler. Only handle GET requests; the Cache API won't store anything else. javascript self.addEventListener('fetch', (event) = { const { request } = event; if (request.method !== 'GET') return; const url = new URL(request.url); if (url.pathname.startsWith('/api/')) { event.respondWith(networkFirst(request, 'api-v1')); } else if (url.pathname.startsWith('/next/static/')) { event.respondWith(cacheFirst(request)); } else { event.respondWith(staleWhileRevalidate(request, 'pages-v1')); } }); In practice I'd use Workbox rather than hand-writing all of this; it implements these strategies with expiry and edge cases handled. But writing them once by hand is the best way to understand what Workbox is doing. The update flow This is the part that surprised me most. When you deploy a new service worker, the browser installs it, but it sits in a waiting state until every tab using the old one is closed. Users with the app open all day may never get the update. You can call self.skipWaiting() to activate immediately, but then the new worker may serve new assets to a page that loaded old ones, which can break things. The safer pattern is to detect the waiting worker, show a small "New version available, refresh" prompt, and only then tell it to skip waiting and reload the page. Also: serve sw.js itself with no long-lived caching headers, so the browser can actually see new versions. Pitfalls I ran into, or nearly did - Caching opaque or error responses. Check res.ok before putting a response in the cache, or you'll happily serve a cached 500 forever. - Unbounded caches. Notes and images add up. Cap entries or age, and remember the browser can evict your storage under pressure. - Caching authenticated data on shared devices. Clear user-specific caches on logout. - Writes while offline. Reads are the easy half. For edits, queue them locally (IndexedDB) and sync when back online. Background Sync helps where supported, but don't rely on it everywhere; retry on app open as well. - Testing only on fast Wi-Fi. Chrome DevTools has offline and slow-network throttling. Use them constantly, and test on a real low-end phone. What I'd tell my past self Decide upfront which content must work offline, then pick a strategy per request type: cache-first for build assets, network-first with a short timeout for live data, stale-while-revalidate for everything that can be a little old. Version your caches, handle the waiting worker deliberately, and test with the network switched off. The bus is the real user.
REST API Design Basics I Wish I'd Learned Earlier
Most APIs I've written for college projects worked. Few of them were pleasant to use. The difference is a handful of conventions that cost nothing to follow once you know them, and a lot to retrofit once a frontend depends on your mistakes. This is the checklist I now run through before writing a single route. Model resources, not actions A REST API is organised around resources: nouns your clients care about. The HTTP method carries the verb. http GET /notes list notes POST /notes create a note GET /notes/42 fetch one note PATCH /notes/42 update part of it DELETE /notes/42 delete it GET /notes/42/comments comments belonging to note 42 Avoid /getNotes, /createNote, /deleteNoteById. They duplicate what the method already says and make the surface area grow with every feature. Some operations genuinely aren't CRUD, like "publish this note" or "send a password reset". It's fine to model those as a sub-resource or an action endpoint (POST /notes/42/publish). Just keep them the exception. Use plural nouns, lowercase, and hyphens in paths. Keep nesting shallow; past two levels, a filter usually reads better (/comments?noteid=42). Use status codes properly Status codes let clients handle results without parsing your message strings. The ones you'll use most: | Code | Meaning | |------|---------| | 200 OK | Success with a body | | 201 Created | Resource created; include a Location header | | 204 No Content | Success, nothing to return (common for DELETE) | | 400 Bad Request | Malformed request or invalid input | | 401 Unauthorized | Not authenticated | | 403 Forbidden | Authenticated, but not allowed | | 404 Not Found | Resource doesn't exist (or you won't admit it does) | | 409 Conflict | State conflict, e.g. duplicate or version mismatch | | 422 Unprocessable Content | Well-formed but fails validation (some APIs use 400 for this) | | 429 Too Many Requests | Rate limited; send Retry-After | | 500 Internal Server Error | Your bug, not theirs | The one rule I'd enforce above all: never return 200 with "success": false in the body. It breaks every HTTP client, proxy and monitoring tool that trusts the status line. Make error bodies consistent Pick one error shape and use it everywhere. Clients will write one handler for it. There's a standard worth borrowing, Problem Details (RFC 7807, updated as RFC 9457), served as application/problem+json: json { "type": "https://api.example.com/errors/validation", "title": "Validation failed", "status": 422, "detail": "One or more fields are invalid.", "errors": [ { "field": "title", "message": "must not be empty" }, { "field": "subject", "message": "unknown subject code" } ] } Include a machine-readable identifier (type or a code field) so clients branch on that, not on the human message. Never leak stack traces or SQL errors in production responses. Paginate from day one Any list endpoint will eventually be too large to return in one go. There are two common approaches. Offset pagination is the familiar one: http GET /notes?limit=20&offset=40 It's easy to implement and lets you jump to page 7. But it has two problems. Large offsets get slow, because the database still walks past every skipped row. And if items are inserted or deleted while someone pages through, they'll see duplicates or miss rows. Cursor pagination returns an opaque token pointing at the last item seen: http GET /notes?limit=20&cursor=eyJpZCI6MTIzNH0 json { "data": [ { "id": 1235, "title": "DBMS unit 3" } ], "nextcursor": "eyJpZCI6MTI1NH0", "hasmore": true } Under the hood the cursor encodes something like "id greater than 1234", which turns into an indexed WHERE id ? ORDER BY id LIMIT 20. It stays fast and stable under writes. The trade-off is no random page access. My default: cursor pagination for feeds and anything that grows, offset for small admin tables where "go to page 5" matters. Either way, set a sensible default limit and a hard maximum. Idempotency keys for unsafe retries Networks fail. A client sends POST /payments, the connection drops, and it has no idea whether the payment went through. Retrying might charge twice. GET, PUT and DELETE are defined as idempotent: doing them twice has the same effect as doing them once. POST isn't. The fix is an idempotency key: the client generates a unique value and sends it with the request. http POST /payments Idempotency-Key: 7b4c1f0e-2a9d-4c55-9e1a-3f2d8b6c0a11 Content-Type: application/json { "amount": 49900, "currency": "INR" } The server stores the key with the result. If the same key arrives again, it returns the stored response instead of doing the work twice. A few details matter: scope keys per user, expire them after a reasonable window, and reject a reused key that comes with a different body. Version deliberately You will need to make breaking changes. Plan for it. - URL versioning (/v1/notes) is the most visible and easiest to route and cache. Most public APIs do this. - Header versioning (Accept: application/vnd.example.v2+json or a custom header) keeps URLs clean but is harder to test in a browser. Whichever you choose, the bigger discipline is not breaking things inside a version. Adding a field is fine. Renaming, removing, or changing a field's type is not. Clients should ignore fields they don't recognise, and you should document that expectation. What I'd tell my past self - Nouns in the path, verbs in the method. - Honest status codes, one error format, no 200 failures. - Paginate every list, and prefer cursors for anything that grows. - Accept an Idempotency-Key on any POST that moves money or sends messages. - Put /v1 in the URL now, and treat every field you ship as a promise. None of this is clever. That's the point: a boring, predictable API is the one people enjoy integrating with.