A web-based real-time push-up counter using pose estimation. Counts reps via laptop webcam with skeleton overlay visualization.
┌─────────────────────────────────────────────────────────────────────────┐
│ BROWSER │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────────┐ │
│ │ Webcam │───▶│ Capture │───▶│ WebSocket (Binary JPEG) │ │
│ │ Feed │ │ Loop (30fps)│ │ │ │
│ └──────────────┘ └──────────────┘ └────────────┬─────────────┘ │
│ │ │ │
│ ▼ │ │
│ ┌──────────────┐ ┌──────────────┐ │ │
│ │ Video │ │ Skeleton │◀────────────────┼─────────────┐ │
│ │ Display │ │ Canvas │ JSON Response │ │ │
│ │ (mirrored) │ │ (overlay) │ (landmarks, │ │ │
│ └──────────────┘ └──────────────┘ count, state)│ │ │
└─────────────────────────────────────────────────────────┼─────────────┼──┘
│ │
┌───────────────▼─────────────▼──┐
│ BACKEND (Python) │
│ ┌─────────────────────────┐ │
│ │ FastAPI WebSocket │ │
│ │ (async, thread pool) │ │
│ └───────────┬─────────────┘ │
│ │ │
│ ┌───────────▼─────────────┐ │
│ │ MediaPipe Pose │ │
│ │ (33 landmarks) │ │
│ └───────────┬─────────────┘ │
│ │ │
│ ┌───────────▼─────────────┐ │
│ │ Push-up Counter FSM │ │
│ │ (hysteresis logic) │ │
│ └─────────────────────────┘ │
└────────────────────────────────┘
| Layer | Technology | Purpose |
|---|---|---|
| Frontend | React + Vite | UI framework with fast HMR |
| Webcam | react-webcam | Browser MediaDevices API access |
| Rendering | HTML5 Canvas | Skeleton overlay at 30 FPS |
| Communication | WebSocket | Binary JPEG frames → JSON responses |
| Backend | FastAPI | Async WebSocket server |
| Pose Estimation | MediaPipe | 33-point body landmark detection |
| Serialization | orjson | Fast JSON encoding |
WebcamCapture.jsxcaptures frames at 30 FPS usingrequestAnimationFrame- Frames are encoded as JPEG blobs (70% quality) and sent via WebSocket
- Video display is mirrored for natural "mirror" experience
pose_estimator.pydecodes JPEG and runs MediaPipe inference- Returns 33 body landmarks with (x, y, visibility) coordinates
OneEuroFiltersmooths jittery landmarks without adding latency
pushup.pyimplements a Finite State Machine with hysteresis:IDLE → UP → DESCENDING → DOWN → ASCENDING → UP (count++)- Dual thresholds prevent false counts:
- DOWN: elbow angle < 90°
- UP: elbow angle > 160°
- Body position validation ensures hands are on ground
SkeletonCanvas.jsxdraws landmarks on transparent canvas overlay- Colors indicate state: Blue=UP, Orange=DESCENDING, Green=DOWN, Pink=ASCENDING
useExerciseWebSocket.jsmanages connection with auto-reconnect
repcount/
├── backend/
│ ├── main.py # FastAPI WebSocket server
│ ├── pose_estimator.py # MediaPipe wrapper + OneEuroFilter
│ ├── counters/
│ │ └── pushup.py # FSM with hysteresis
│ ├── filters/
│ │ └── one_euro.py # Adaptive smoothing filter
│ └── utils/
│ └── geometry.py # Angle calculations
│
├── frontend/
│ └── src/
│ ├── App.jsx # Main component
│ ├── components/
│ │ ├── WebcamCapture.jsx # Frame capture
│ │ └── SkeletonCanvas.jsx # Skeleton overlay
│ └── hooks/
│ └── useExerciseWebSocket.js # WebSocket logic
│
└── .claude/skills/ # Implementation documentation
cd backend
uv sync # Install dependencies
uv run uvicorn main:app --reload --port 8000cd frontend
npm install
npm run dev # Starts on http://localhost:5173- Open http://localhost:5173 in Chrome
- Allow camera access
- Position yourself sideways to camera (side profile)
- Camera at push-up height (~1-2 ft off ground)
- Do push-ups! 🏋️
| Decision | Rationale |
|---|---|
| Binary WebSocket | 33% smaller than Base64 encoded frames |
| FSM with hysteresis | Prevents count bouncing near thresholds |
| OneEuroFilter | Smooths jitter without latency during fast movement |
asyncio.to_thread() |
MediaPipe blocks GIL for 15-25ms; keeps event loop responsive |
useRef over useState |
Landmarks update 30x/sec; refs avoid re-render storm |
| Average confidence | More robust than minimum when landmarks partially occluded |
| Metric | Value |
|---|---|
| Frame Rate | 22-25 FPS |
| Inference Time | ~30ms |
| End-to-end Latency | <50ms |
| CPU Usage | ~60-80% |
MIT