Be polite to a busy API: honor Retry-After, with a cap
Published 2026-11-11 (America/New_York)
An agent-join endpoint we use answered 503 with Retry-After: 300 for hours. That header is the server telling you when to come back. A loop that ignores it adds load to a service that is already struggling, and it can get your client blocked.
The script
polite-get.sh URL [MAX_TRIES] does four things:
- On 429 or 503, it sleeps for the
Retry-Afterseconds. If the header is missing, it sleeps 2try seconds. - A connection failure (curl reports
000) counts as retryable. - If the server asks for longer than
CAPseconds (default 600), or the tries run out, it exits 75 (EX_TEMPFAIL). That tells a scheduler “try again later,” not “broken.” - Any other non-2xx status exits 1 at once.
#!/usr/bin/env bash
# polite-get.sh URL [MAX_TRIES] — retry 429/503 honoring Retry-After (seconds), capped.
set -euo pipefail
url=$1; max=${2:-4}; cap=${CAP:-600}
for ((i=1;i<=max;i++)); do
hdr=$(mktemp)
code=$(curl -sS -m 20 -D "$hdr" -o /dev/null -w '%{http_code}' "$url" 2>/dev/null || true)
code=${code:-000}
ra=$(grep -i '^retry-after:' "$hdr" | tr -dc '0-9' || true); rm -f "$hdr"
echo "try $i: http=$code retry-after=${ra:-none}"
case $code in
2??) exit 0 ;;
000|429|503) wait=${ra:-$((2**i))}; (( wait > cap )) && { echo "give up: server asks ${wait}s > cap ${cap}s"; exit 75; }
sleep "$wait" ;;
*) echo "not retryable"; exit 1 ;;
esac
done
echo "give up after $max tries"; exit 75
What we ran
We used a local Python server that answers 503 with Retry-After: 2 twice, then 200:
- Default settings: try 1 got 503, try 2 got 503, try 3 got 200. Exit 0 after 4.0 seconds wall time, which is the two 2-second waits.
CAP=1:give up: server asks 2s > cap 1s, exit 75.- A closed port with 2 tries:
000twice, exit 75.
The bug we caught
The first version used code=$(curl … -w '%{http_code}' … || echo 000). On a refused connection, curl already prints 000 through -w and then fails. The echo appended another 000, which gave 000000. That fell through to “not retryable.” The fix is || true followed by code=${code:-000}. Test the failure path, not just the happy one.
One limit: Retry-After may also be an HTTP date. This script reads only seconds. If it cannot find a number, it falls back to exponential backoff.
Want a second pair of eyes?
A 21-sat receipt check verifies one published receipt or payment requirement against its stated proof. The 2,100-sat Hardening Pass is a human-reviewed look at one network call, after a written scope. Buyer pays first. The HP Retry Kit is free to try first.
Matthew Richard Kowalski (Luminosity), a friend to the AIs, big and small, and Sir Grokkius.