Settld

Overview

Settld scans a receipt, lets everyone tap what they had, splits tax and tip in proportion, and sends one pay link. Friends pay with Apple Pay, Google Pay or a card in the browser; nobody else installs anything.

An iOS app with a web portal for guest payments, plus payment tracking that nudges the slow ones. Underneath: a FastAPI backend with a vision-model receipt pipeline, an exact-cent split engine, Stripe Connect payouts and per-bill WebSockets, built in a 115-commit, 14-day sprint.

What I built

• Built the bill-splitting flow end to end: scan a receipt, claim items, allocate tax and tip, track shared balances.
• Developed the receipt pipeline (gpt-4.1 vision OCR with bounding boxes, a deterministic regex pre-parser, a conditional gpt-4.1-mini cleanup pass and item normalization), shared expense sessions, SMS verification and secure payment APIs on a FastAPI server.
• Integrated React Native, PostgreSQL and Stripe so groups can settle up across receipts and shared expense groups: PaymentIntents as destination charges, Payment Element guest checkout, Custom Connect accounts with in-app KYC and daily payouts.
• Built the real-time layer: per-bill WebSocket channels with compact deltas and heartbeats, optimistic serialized mutations on the phone, and polling fallbacks for guests.

By the numbers, counted from the repo

Architecture

Mobile

Expo SDK 54 / React Native 0.81, React Navigation 7, Redux Toolkit + RTK Query, Stripe React Native SDK, expo-camera, Apple Sign-In, secure-store JWT

API

FastAPI, SQLAlchemy 2 + Alembic on PostgreSQL 16, JWT auth with phone OTP via Twilio Verify, Stripe payments, Connect and Issuing, OpenAI vision receipt parsing with Groq fallback, Celery + Redis, APScheduler, WebSockets, Docker Compose

Web

React 19 + Vite guest pay pages on Vercel with React Router 7, Stripe.js Payment Element and Payment Request button for Apple Pay and Google Pay

How it's built

1. 01

   The receipt pipeline, version 3.1.0

   Receipts are hostile input: faded ink, duplicated subtotal lines, modifiers, card-brand footers. The vision model is an OCR engine, parsing is deterministic, and LLM tokens are spent only when the deterministic pass is unsure.

   Vision OCR and row reconstruction

   • The image is sent to gpt-4.1 with a strict json_schema response of lines, each with text and a normalized bounding box; if structured output fails, a second call asks for raw text so the job survives. Groq Llama-4-Scout is the legacy fallback.
   • Lines are sorted by (y, x) and clustered into rows when y-centers are within 0.02. Bumping the pipeline version constant invalidates every cached parse.

   Deterministic pre-parser and conditional cleanup

   • Name-only rows merge with the following priced row; 2 @ 5.99 and x2 patterns become quantities; fuzzy regexes route SUBTOAL-style misspellings into subtotal, tax, total and balance-due slots. Confidence is 0.55 consistency (subtotal + tax equals total within $0.02) + 0.30 row coverage + 0.15 totals found.
   • At 0.85 or above the cleanup model is skipped. Below it, gpt-4.1-mini receives the rows plus the pre-parsed guess and a worked example of a duplicated COMPLETE SUBTOTAL line, at most twice; refusals and invalid JSON fall back to the pre-parser payload. The pre-parser and the post-LLM filter share one regex module so a misspelling cannot leak through one layer and reappear as a phantom item.

   Validation, merge and persistence

   • Per-item confidence is 0.3 has-price + 0.3 row alignment + 0.4 model confidence; a missing subtotal or total is derived with a warning; duplicate label rows are dropped only when it improves reconciliation and never the last copy.
   • Multi-page receipts parse sequentially for deterministic merge order; items with name similarity 0.90 or more and prices within two cents are deduped, and footer totals come from the last page.
   • Each multi-quantity line becomes one quantity-1 row per unit so each unit can go to a different person, with cents distributed exactly: $10.00 across three rows is 3.34, 3.33, 3.33. Edits recompute unit price, assignments and totals, and write a before/after feedback row.
2. 02

   The split engine

   Shares, recalculation, breakdown

   • Three share types per assignment (equal, percentage, fixed), all quantized to 0.01 with ROUND_HALF_UP; recalculate handles mixed shares on one item by pricing fixed and percentage first and splitting the clamped remainder equally.
   • Each member's proportion is their subtotal over the bill subtotal; tax, tip and service-fee shares multiply that proportion and round independently. The host is assumed to have paid the merchant and owes zero; a per-bill service fee defaults to 3.5% or a flat $0.75.
   • Stated honestly: an equal split of $10.00 three ways yields 3.33 each. Exact cent distribution happens at receipt expansion and in the client, not in equal-share pricing.
3. 03

   Payments, payouts and the readiness gate

   The hardest constraint was that payers never sign up. The second hardest was moving money to the host without a browser redirect or a clawback.

   PaymentIntents and guest checkout

   • Each payment creates a PaymentIntent with bill and member metadata; when the host's Connect account has charges enabled it becomes a destination charge with an optional platform fee floored in the host's favor. Amounts are validated against Stripe minimums per currency, and an existing pending payment is reused rather than duplicated, preserving its link token.
   • A React + Vite app loads Stripe.js with the publishable key returned by the API, renders an itemized receipt card and mounts Elements with the client secret; a Payment Request button offers Apple Pay and Google Pay. Pending links older than 20 minutes return 410; already-paid links show a status page.

   Custom Connect, webhooks, virtual cards

   • Hosts get a Custom US individual account with card_payments and transfers, daily payouts at the minimum delay, and KYC completed in one call: identity fields, terms acceptance with IP and user agent, and a debit-card external account. Credit cards are rejected as INVALID_CARD; stale Express account ids are recycled automatically.
   • The payments webhook returns 503 when no secret is configured and verifies signatures otherwise; status writes are idempotent so replays are harmless. A separate Connect webhook refreshes capability flags and mirrors payout events.
   • Behind two flags, Stripe Issuing mints one cardholder and card per bill with an all-time limit equal to the bill total once 100% of non-host balances are collected (or the owner overrides); readiness cannot be revoked while a card is active; card numbers and CVCs never appear in any schema.
4. 04

   Identity and messaging

   Phone-first auth

   • Numbers normalize to E.164; a per-phone limit of 5 sends per hour sits in front of Twilio Verify, with intent-aware errors for unknown-on-login and existing-on-signup. Verifying a signup code returns an onboarding token whose subject is a UUID5 of the phone number, so create-profile and verify agree on the user id without a round trip; a collision with a soft-deleted tombstone falls back to a random id.
   • Apple Sign-In verifies identity tokens against Apple's JWKS by key id with the bundle id as audience; accounts link by Apple id, then by email. Deletion is a soft delete that rotates unique identifiers out so the phone and Apple id can be reused.

   SMS that does not nag

   • Payment requests go out over Twilio with up to 3 retries and 0.5 s doubling backoff; every attempt is logged as sent, failed or skipped. A request is not re-sent while the amount is unchanged within a cent, and the same token is reused so a re-sent link never invalidates the first.
   • Reminders fire 24 hours after the request and at most once per 24 hours, from an in-process APScheduler job or an external cron that must present a job secret.
5. 05

   Real-time state and the mobile client

   WebSocket channels

   • Per-bill channels for the authenticated host and per-token channels for guests; production requires wss or a forwarded https header, and auth failures close with codes the client does not retry. Single mutations broadcast compact deltas; auto-split broadcasts one full sync.
   • The server pings every 30 s; the mobile client reconnects with capped exponential backoff, forces a reconnect after 45 s of silence, and polls every 2 s only while disconnected. Guests fall back to 5 s polling.

   Optimistic, serialized mutations

   • The bill hook keeps an optimistic assignment map, a per item-and-member promise queue so rapid taps serialize, an in-flight counter that suppresses refetches until mutations settle, and a 1 s fetch debounce.
   • There is deliberately no generic in-memory GET cache: a 30 s cache once overwrote fresh WebSocket state. RTK Query owns tagged invalidation, the combined dashboard query persists to AsyncStorage with a 24-hour TTL, and stale data is served only on network errors, never on 4xx or 5xx.
6. 06

   Hardening and operations

   The repo shows a two-week sprint from MVP shortcuts to production guards, with the shortcuts documented and then closed.

   Production posture

   • Boot fails on a default JWT secret in production; docs, redoc and the debug router are hidden; the health check runs a real SELECT 1. Every endpoint returns one envelope across 20 distinct error codes; a catch-all 500 never leaks the exception.
   • Rate limits on OTP send, verify and Apple sign-in; CORS allows settld.live origins plus a regex for preview subdomains; the dashboard aggregation was rewritten from one round trip per bill and member into four GROUP BY statements.

   What changed from the MVP

   • In-memory invite tokens became a unique indexed column on bill members with placeholder invite rows and a public invite page that deep-links into the app; owner and participant dependencies replaced the documented no-ownership-checks shortcut.
   • Stripe Connect moved from Express to Custom to keep KYC in-app; instant payouts gave way to free daily standard payouts. 102 unit tests cover split math, the pre-parser, validator and merge, readiness and virtual-card idempotency, pay-link TTL boundaries, WebSocket auth and production safety, with mocked sessions and no network.

Decisions

1. 01

   Payers never sign up

   The person who owes money has the least patience. A pay link that works in any browser is the whole growth loop; the token is the credential.

2. 02

   Proportional tax and tip

   Tax and tip are allocated by each person's share of the subtotal, not split evenly, so nobody subsidizes the steak.

3. 03

   Money as records

   Payments and payouts are explicit rows with their own statuses, so every balance can be reconstructed from the ledger and webhook replays are harmless.

4. 04

   Deterministic first, LLM second

   The regex pre-parser gates the cleanup model behind a 0.85 confidence threshold, trading a little normalization quality for cost, latency and reproducibility.

5. 05

   Vision model as OCR, not as parser

   The first model call is constrained to text plus bounding boxes in strict JSON. Geometry rebuilds the rows; code decides what a line means.

6. 06

   Custom Connect over Express

   In-app KYC with no browser redirect, at the cost of owning terms acceptance, stale-account recycling and the regulatory surface.

Edge cases the code handles

• Below-minimum charges are rejected at intent creation, at SMS send and at guest confirm, using Stripe's per-currency thresholds.
• Missing tax defaults to zero with a warning; a page failing inside a multi-page scan adds a warning, and the job fails only when every page fails.
• Members with no linked user or phone are skipped by SMS and counted; zero-item members get zero tax, tip and fee shares.
• Phone users get a synthetic email internally and a null email at Stripe, because Stripe rejects the synthetic domain.
• Parse jobs are claimed with a conditional UPDATE so exactly one worker runs them, keyed by an Idempotency-Key header; virtual cards are idempotent through a hashed key plus Stripe idempotency keys.
• Native modules are required lazily so older TestFlight builds do not crash, fonts failing to load do not block rendering, and logout wipes the cache so the next user cannot see prior balances.