← All Automations ON-DEVICE COMPUTER VISION

Squat Scanner

A browser-based squat form tracker that watches your set through the camera and flags reps that don't match the rest — depth-based pose detection running entirely on-device, with no camera frame, measurement, or rep count ever leaving your machine.

WHAT IT DOES

Watches the set. Flags the outlier.

• Tracks squat depth directly, not knee angle — a rebuilt measurement pipeline that segments reps off a calibrated depth signal instead of a joint-angle threshold
• On the corpus used to validate it, depth-based segmentation got all six filmed takes' rep counts right; the knee-angle approach it replaced got three of six wrong
• Flags any rep that deviates from the set's own median depth — catching a form breakdown mid-set, not just comparing you to a generic "ideal" squat
• Calibrates before it judges anything — if it can't see you well enough to measure a rep, it says so instead of guessing
• No runtime AI, API, or LLM calls of any kind — deterministic geometry only, which is also why it can honestly promise nothing about your body leaves the browser

HOW IT WORKS

One camera, five steps.

1. Step 1 — Pose Landmarks, On-Device

   MediaPipe's pose model runs in the browser against the live camera feed, producing hip, shoulder, and joint landmarks every frame. Nothing is uploaded — the model and the video never leave the tab.

2. Step 2 — Calibration

   Before it measures anything, the tracker asks for ~1.5s of stillness to establish a stable baseline. MediaPipe's tracker takes a few seconds to converge on a new body, and a rep judged against an unconverged baseline is a rep judged wrong.

3. Step 3 — Depth-Based Rep Segmentation

   Hip position relative to the calibrated baseline builds a continuous depth signal per frame. Reps are segmented directly off that signal, with guards against implausible frame-to-frame jumps and drift from the person walking into or out of frame.

4. Step 4 — Deviation Flagging

   Each rep's depth at its bottom is compared against the median for the set. A rep that falls well short of the others gets flagged as unusual — the tool is built to catch a breakdown mid-set, not to grade you against a fixed ideal.

5. Step 5 — Session Summary + Replay

   After the set, a rep-by-rep list shows flags and streak stats, and clicking any rep replays its 3D skeleton so you can see exactly what the tracker saw.

SEE IT RUN

The build, walked through.

The build, the failure it caught, and the fix — in under a minute.

UNDER THE HOOD

Deterministic geometry, no black box.

1. TypeScript + Vite, deployed on Vercel

   A single-page app with no backend — pose detection, measurement, and scoring all run client-side. Open source on GitHub.

2. MediaPipe Tasks Vision for pose landmarks

   The only "AI" in the pipeline is the pose model itself, running fully on-device — everything downstream of it is plain geometry: angles, ratios, and thresholds measured against a real filmed corpus, not tuned by feel.

3. Scoped honestly, on purpose

   The tool never claims anything about the spine, discs, or injury risk — MediaPipe has no landmark between shoulder and hip, so it doesn't pretend to see one. It measures depth consistency, nothing more.