cURLParse cURL Converter

Example: Basic Auth with a Pagination Cursor

cURLParse Guides · Updated 2026-10-18 · All guides
curl -sS 'https://api.example.com/v2/orders?limit=50&cursor=eyJwYWdlIjoyfQ' \
  -u 'user:password' -i -w '\n%{http_code}\n'
  • -u base64-encodes user:password into Authorization: Basic .... Converted code must set the header manually in some languages; the converter does this explicitly rather than relying on an auth kwarg that differs per library.
  • The -w footer makes CI assertions easy: test $(curl ...) -eq 200 is brittle; a printed code isn't. The flags guide covers -w and --fail-with-body, the two flags that make scripts honest.
  • -i on a paginated endpoint surfaces Link: rel="next" — the actual pagination cursor, which the body usually doesn't duplicate.

What -u sends, byte for byte

Against an echo origin, -u 'user:password' produces exactly one header:

Authorization: Basic dXNlcjpwYXNzd29yZA==

That is base64("user:password") — encoding, not encryption. Two consequences:

  • A password containing a colon breaks nothing in curl (it takes the first colon as the separator), but it breaks every human reading the base64 later. If your "password" has structure, it's probably an API key and belongs in a header, not in -u.
  • -u sends the header on the first request. curl does not wait for a 401 WWW-Authenticate challenge. Some servers that advertise Negotiate/NTLM will ignore your Basic header and 401 you anyway; add --ntlm or --digest only when the server actually demands it, and check with curl -v which scheme the challenge named.

Following the cursor without lying to yourself

The Link header is the cursor. A paginated walk that asserts on the response instead of trusting it:

#!/usr/bin/env bash
next='https://api.example.com/v2/orders?limit=50'
page=0
while [ -n "$next" ]; do
  page=$((page+1))
  hdr=$(curl -sS -D - -o "page-$page.json" -w '%{http_code}' \
        -u "user:$PASS" "$next")
  code=$(tail -n1 <<<"$hdr")
  [ "$code" = 200 ] || { echo "page $page -> $code"; exit 1; }
  # rel="next" survives redirects; take only that relation
  next=$(grep -i '^link:' <<<"$hdr" | tr ',' '\n' | grep 'rel="next"' \
         | sed -E 's/.*<([^>]*)>.*/\1/')
done
echo "$page pages"
  • Splitting on , before matching rel= matters: some APIs put commas inside the URL (;q= parameters), and a naive sed grabs the wrong bracket.
  • If Link is absent, the API may use a body cursor ({"next":"..."}) — read the body with jq -r .next. Either way, stop when the page count stops advancing, not when the file is non-empty: an API returning the same first page forever is a real failure mode, and an infinite loop writing page-9999.json is how you find it.
  • -w '%{http_code}' after -D - puts the code on the last line, which is why the snippet tails instead of greps. The assertion pattern is the same one in the flags guide.

Rate limits show up in the headers you skipped

Paginated APIs throttle. The 429 is visible with -i, but the useful part is Retry-After (seconds or HTTP-date) and X-RateLimit-Remaining. A walk that ignores them gets its token revoked for hammering; one that reads Retry-After sleeps the right amount. If the walk runs in CI, print X-RateLimit-Remaining on every page — the drift toward zero is your early-warning, and it costs one grep.

FAQ

Is Basic auth safe over HTTPS? Yes, that's what it was designed for — base64 is encoding, not encryption, so the TLS layer is the protection. Never over plain HTTP.

The cursor in my URL changed to something random. Bug? No — many APIs return opaque cursors (base64 of an offset or keyset). Treat them as opaque; don't construct them client-side. Parse Link: rel="next" instead.

Why does the converter print the Authorization header instead of using auth=(user, pass)? Because kwargs differ across requests/httpx/fetch and silently change behavior. An explicit header converts identically everywhere.

Developer Sponsor / Partner
Copied to clipboard!