Chapter 7: Polling and Hybrid Delivery Strategies

1. Polling Fundamentals

Polling is the act of a client repeatedly asking a server whether state has changed, rather than waiting for the server to announce it. It needs nothing special: any HTTP client, behind any proxy or firewall, can make a request and read a response.

Short Polling vs Long Polling

Short polling sends a request at fixed intervals; if nothing is new, the server responds immediately with an empty result. Its defining property is a trade-off between latency and efficiency: a shorter interval means faster updates but more wasted empty requests; latency is bounded by the interval, so a 10-second poll means a new message waits up to 10 seconds. Ten thousand clients polling every second produce 10,000 requests per second even when nothing is happening.

Long polling flips the waiting around: the client sends a request the server holds open until new data arrives or a timeout elapses, then the client immediately re-requests. This gives near-instant delivery and dramatically fewer empty responses, at the cost of one held connection per waiting client. The canonical example is Amazon SQS, where a WaitTimeSeconds of up to 20s can yield about a 95% cost reduction versus short polling for a lightly-loaded queue.

Analogy — the mailbox and the doorbell. Short polling is walking to your mailbox on a fixed schedule; most trips it is empty. Long polling is waiting at the box until the carrier actually arrives — you get the letter instantly, but you are tied up standing there, and 10,000 people all waiting at once is a lot of people standing around.

Figure 7.3: Short polling vs long polling request timing.

Cursors, Since-Tokens, and Avoiding Missed or Duplicate Messages

A poll that just asks "give me the latest messages" is broken in two directions: it misses messages that rolled past while it was busy, and it re-fetches messages it already handled. The fix is a cursor — a persisted marker of the last event you successfully processed. You send it with each request, the server returns only events newer than it, and after processing a batch you advance the cursor to the highest position seen and persist it.

The cursor is your primary deduplication mechanism, but it is not sufficient alone. Most message APIs offer at-least-once delivery, so the message at the boundary (whose ID equals your cursor) can legitimately be replayed. If you persist the cursor after processing but crash before persisting, that message is handled again on restart — which is why a separate ID-keyed dedup check is required.

Analogy — the bookmark. A cursor is a bookmark you move forward before closing the book. Lose it and you either start over at page one or guess where you were — and guessing is exactly how messages get missed or read twice.

Respecting Rate Limits and Retry-After Headers

A rate limit caps how many requests you may make in a window; exceed it and you get HTTP 429 Too Many Requests. A polling loop is a request-generating machine, so it is precisely the client that trips limits. The single most important rule: honor the Retry-After header when present — the server knows its own capacity, so its stated wait is both correct and the fastest path back to being served.

When there is no Retry-After, compute your own wait with exponential backoff (1s, 2s, 4s, 8s...) plus jitter — a small random offset. Jitter prevents the thundering herd: without it, a thousand clients that failed in one shared outage retry in perfect lockstep and hammer the recovering server in synchronized waves. Idempotency also matters: GETs are safe to retry freely, but POSTs need connection-error-only retries or an Idempotency-Key.

2. When Polling Is the Right Tool

Polling has a reputation as the fallback of last resort, but there are whole classes of problems where it is the correct choice, not the compromise.

APIs Without Push Support

The most decisive reason to poll is that the upstream API gives you no choice. Many services expose only request/response HTTP with no webhooks and no streaming endpoint. If the platform cannot push, no architectural sophistication conjures a push channel into existence — you ask, or you learn nothing. You do not need the provider to register a callback URL, you do not need a publicly reachable endpoint, and you do not need a persistent connection.

Batch Reconciliation and Catch-Up After Downtime

Polling's "ask for everything since X" model is a superpower when recovering after your own system was unavailable. Push is fire-and-forget, so a delivery that fails against a dead endpoint may never be retried. A poll simply asks for everything since the last cursor, and the provider hands over the whole backlog. This is catch-up — a single cursored poll reconciles your state with the provider's — and it is also the engine of nightly or hourly batch reconciliation.

Simplicity and Firewall-Friendliness

Polling is a plain outbound HTTP request, so it works anywhere an HTTP client works — behind corporate proxies, inside restrictive firewalls, from a machine with no public IP. Webhooks require the reverse: a publicly reachable, TLS-terminated endpoint to secure. WebSockets require holding a persistent connection through hostile middleboxes. For a locked-down environment or a proof-of-concept, "just poll" gets you working today and can be debugged with curl.

3. Building an Async Poller

We now assemble the fundamentals into a single production-shaped worked example: a cancellable async polling loop that tracks a cursor, honors Retry-After, adapts its interval to load, and deduplicates against already-processed events.

A Cancellable Polling Loop with Adaptive Intervals

An adaptive interval changes with observed load: poll fast when messages flow, back off — doubling up to a cap — when idle. After a poll that returned messages, reset to the floor; after an empty poll, double up to the ceiling. In asyncio, sleeping with await asyncio.sleep(...) yields to the event loop and is a cancellation point; catching asyncio.CancelledError lets the loop persist its cursor cleanly on shutdown.

Figure 7.1: The adaptive-interval feedback loop.

Persisting Cursor State Between Polls

The adaptive loop is worthless if a restart makes it forget where it was. Each successful batch must advance the cursor to the highest ID processed and persist it durably (a DB row, a Redis key, a state file) before treating the batch as done. The ordering matters: persist after processing, so a crash mid-batch causes at worst a safe re-fetch (absorbed by dedup) rather than an unrecoverable skip. On startup the poller loads the persisted cursor; if none exists it starts from "now" so a fresh poller does not replay all history.

Deduplicating Against Already-Processed Events

The final safety net is reconciliation at the level of individual events: keep a record of already-processed message IDs and skip any ID already handled before doing any side-effectful work. This catches the replayed boundary message, the hybrid-ingress overlap when the same event arrives via both webhook and poll, and the re-fetch after a crash-before-persist. Here is the complete assembled poller:

import asyncio
import aiohttp
import backoff

API = "https://api.example.com/messages"

class Poller:
    def __init__(self):
        self.cursor = load_cursor()      # persisted last-seen message id (or None)
        self.seen = load_seen_ids()      # persisted set of processed ids (dedup)
        self.interval = 1.0              # adaptive base interval, seconds

    @backoff.on_exception(backoff.expo, aiohttp.ClientError, max_time=60)
    async def fetch(self, session):
        # Ask only for messages newer than the cursor.
        async with session.get(API, params={"since": self.cursor}) as r:
            if r.status == 429:
                # Honor the server's Retry-After, then let backoff retry.
                delay = int(r.headers.get("Retry-After", "1"))
                await asyncio.sleep(delay)
                raise aiohttp.ClientError("rate limited")
            r.raise_for_status()
            return await r.json()

    async def run(self):
        async with aiohttp.ClientSession() as session:
            try:
                while True:
                    batch = await self.fetch(session)
                    if batch["messages"]:
                        for m in batch["messages"]:
                            if m["id"] in self.seen:
                                continue          # dedup: already processed
                            await process(m)      # side-effectful work
                            self.seen.add(m["id"])
                        # advance + persist cursor AFTER processing
                        self.cursor = batch["messages"][-1]["id"]
                        save_cursor(self.cursor)
                        save_seen_ids(self.seen)
                        self.interval = 1.0                       # busy -> fast
                    else:
                        self.interval = min(self.interval * 2, 30)  # idle -> back off
                    await asyncio.sleep(self.interval)            # cancellation point
            except asyncio.CancelledError:
                save_cursor(self.cursor)          # persist on clean shutdown
                save_seen_ids(self.seen)
                raise

4. Choosing and Combining Delivery Modes

Three mechanisms cover almost all real-time integration. Polling is the client asking on an interval — simple and universal but latency-bound. Webhooks are one-way event-driven HTTP: the source POSTs to your URL the moment an event happens — lightweight and efficient because nothing is sent when nothing happens. WebSockets are two-way persistent connections with sub-second latency — the standard for live bidirectional chat.

A Decision Matrix: Webhook vs WebSocket vs Polling

Figure 7.2: Webhook vs WebSocket vs Polling decision matrix.

Read across the "Delivery guarantee" and "Completeness after downtime" rows and the reason for hybrids is obvious: push modes win on latency but lose on completeness, and polling wins on completeness but loses on latency. For chat, latency is decisive — a 5-minute interval yields an average delay of ~2.5 minutes, which no conversation survives — so plain polling is disqualified as the primary transport while remaining the best safety net.

Hybrid Ingress: Push Primary, Polling as Fallback

A hybrid ingress pairs a low-latency push path with a slower authoritative pull path, so the system is fast in the common case and complete in the failure case. The push path (WebSockets for the live stream, webhooks for third-party server events) handles nearly all traffic sub-second; a periodic cursor-based reconciliation poll runs quietly as a backstop. It exists because webhooks are fire-and-forget and can be missed — endpoint down during a deploy, a network blip, or the provider failing to deliver.

Analogy — the smoke detector and the inspection. The push path is a smoke detector: it screams the instant something happens. But detectors have dead batteries and blind spots. The reconciliation poll is the scheduled inspection that walks every room on a timetable — slower, but it catches the smoldering wire the detector missed.

Figure 7.4: Hybrid ingress — push-primary with polling-fallback reconciliation.

Reconciliation to Guarantee No Lost Messages

The promise of a hybrid ingress is "no message is permanently lost," and reconciliation redeems it: the poll fetches everything since the cursor, dedup skips whatever push already handled, and whatever remains is what push missed — now delivered by the pull path. Two second-order problems arise when events can arrive by two routes. Ordering: fix by reordering with event timestamps or provider sequence numbers rather than trusting arrival order. Overlap: a message delivered by both webhook and poll must be processed exactly once, which the ID-keyed dedup layer guarantees. Together — push for speed, reconciliation for completeness, timestamps for order, dedup for exactly-once — the gateway can lose a webhook, drop a socket, or restart mid-batch and still converge on the correct, complete, ordered stream.