Filters

An augmented reality project that generates an interactive portal using real-time hand tracking. Through a natural gesture, 17 different visual filters can be rendered live inside the portal. Two ways to use it:

• Live camera (main.py): filters cycle automatically as you bring your hands together.
• Video upload (app.py): a web page where you upload a video, pick one filter from a thumbnail grid, and get back a processed copy with that filter rendered inside the portal. Runs locally, or deployed to a public URL you can reach from your phone (see Deploying the web app).

---

Features

• Real-time hand tracking via MediaPipe's HandLandmarker (Tasks API).
• Perspective portal built dynamically from the index and thumb fingertips of both hands.
• 17 visual filters applied exclusively inside the portal area.
• Live mode: filter switching via gesture — bringing the hands together triggers the transition to the next filter in the sequence, with hysteresis to prevent accidental switches from tremor or tracking imprecision.
• Video mode: upload a clip, pick a filter from a thumbnail preview grid, download the rendered result.
• Optional password gate and public deployment (Docker + Google Cloud Run) for using the video-upload app from your phone without your computer running.
• Capture requested at 1920x1080 (MJPG) with a resizable preview window, and the live loop is capped at 60fps.

Included filters

Filter	Description
filter_grid	Grid overlay on the original image
filter_1	Duotone segmented by luminosity thresholds
filter_2	Black and white halftone dot pattern
filter_3	Chromatic aberration with RGB channel separation
filter_5	Thermal camera simulation via colormap
filter_6	Vintage sepia style with vignette and grain
filter_frosted_glass	Frosted glass effect over the image
filter_pink_halftone	Pink-magenta duotone halftone
filter_invert	Color negative
filter_bw_contrast	High-contrast black and white
filter_neon	Glowing neon edge outlines over a dimmed base image
filter_vhs	Retro VHS look: scanlines, channel bleed, grain
filter_halftone_comic	Comic-book Ben-Day dot shading over the original colors
filter_dimensional_glitch	RGB channel split with randomized band offsets and flicker
filter_ink_outline	Bold black ink outlines with flattened poster colors
filter_polaroid	Faded film look: cool-shadow/warm-highlight split tone, soft glow, grain
filter_hud	Sci-fi HUD visor: tint, scanlines, corner brackets, center reticle, readout text

Installation

Clone the repository:

git clone https://github.com/AJ0617/Filters.git
cd Filters

Create a virtual environment:

python -m venv venv

Activate it:

venv\Scripts\activate      # Windows
source venv/bin/activate   # macOS / Linux

Install the dependencies:

pip install -r requirements.txt

Download the hand-tracking model (required by both main.py and app.py, not included in the repo):

curl -L -o hand_landmarker.task "https://storage.googleapis.com/mediapipe-models/hand_landmarker/hand_landmarker/float16/1/hand_landmarker.task"

It must sit in the project root, next to main.py.

Generate the filter preview thumbnails shown on the video-upload page (one-time, needs a webcam):

python generate_previews.py

This writes small JPEGs to static/previews/. They're committed to the repo, so this step is only needed if you add/change a filter and want its preview to reflect that.

Note: mediapipe==0.10.35 is pinned in requirements.txt. Newer mediapipe releases (1.0+) dropped the legacy mp.solutions API but keep the mediapipe.tasks API used here, so this project should also work with those; older mediapipe releases (<0.10.30) don't ship wheels for recent Python versions. Tested on Python 3.14.

Usage

Live camera

python main.py

With the camera active, raise both hands with index finger and thumb extended: the portal is generated automatically between them. Bring them together to "close" and advance to the next filter in the list. Press q with the window focused to end the session.

Video upload (web app)

python app.py

Open http://localhost:5050 in a browser, upload a video where someone makes the same portal gesture, pick a filter from the thumbnail grid, and submit. The selected filter is applied inside the tracked portal for the entire clip; when no hands are detected in a frame, that frame passes through unfiltered. Once processing finishes you get an inline preview and a download link. Processing time scales with video length since hand detection runs per frame.

By default there's no login step locally. Setting an APP_PASSWORD environment variable (used for the cloud deployment below) turns on a login page in front of the whole app, including for local runs.

Deploying the web app

To reach the video-upload app from your phone without your computer running, deploy it to Google Cloud Run using the included Dockerfile — see deploy-cloudrun.md for step-by-step setup, cost notes, and the exact gcloud command. The deployment is gated behind the APP_PASSWORD you choose during setup, and MAX_VIDEO_SECONDS caps how long an uploaded clip can be so a single request can't run indefinitely on a public URL.

Project structure

Filters/
├── main.py              Live-camera entry point: capture loop and filter cycling
├── app.py                Flask web app: video upload, filter selection, processing, login gate
├── video_processor.py     Batch video processing: applies one chosen filter to an uploaded clip
├── generate_previews.py   One-off script: renders each filter's thumbnail preview
├── hand_pipeline.py       Shared MediaPipe HandLandmarker setup and per-frame hand detection
├── hand_tracking.py       Extended-finger detection from landmarks
├── geometry.py            Portal geometry and closing-gesture detection
├── filters.py             Definition of the 17 available filters
├── templates/
│   ├── index.html         Upload form with the filter thumbnail grid
│   ├── result.html        Processed-video result page
│   └── login.html          Password-gate login page (cloud deployment)
├── static/previews/       Generated filter thumbnail JPEGs (committed)
├── Dockerfile             Container image for cloud deployment
├── .dockerignore
├── deploy-cloudrun.md     Google Cloud Run deployment steps
├── requirements.txt
└── README.md

Extending the project

To add a new filter, just define a function in filters.py that takes a BGR crop (numpy.ndarray) and returns a crop of the same size:

def filter_new(roi: np.ndarray) -> np.ndarray:
    return roi

Then add it to the FILTERS list at the end of the file. Both the live app's filter cycle and the web app's dropdown adjust automatically to whatever is in that list. If the filter relies on an expensive per-pixel operation (e.g. cv2.bilateralFilter, cv2.stylization), consider running it on a downscaled copy via the _downscaled() helper already used by a few filters, then resizing the result back up — full-resolution portals can otherwise make those filters noticeably slow.

Tech stack

• Python (tested on 3.14)
• OpenCV
• MediaPipe (Tasks API)
• NumPy
• Flask + Gunicorn (video-upload web app only)
• Docker + Google Cloud Run (optional public deployment)

License

This project is distributed under the MIT license.