The product
Viola answers one question women ask at every makeup counter: "would this shade actually suit me?" Point the front camera at your face and real lipstick shades — 1,670 of them, from 35 real brands — render on your lips in real time. Swap shades live, compare against your bare face with one tap, save the looks you love, and scan any real-world color with the back camera to find its closest match in the catalog.
The entire app runs on-device. There is no backend, no account, no analytics, and no tracking — the camera feed never leaves the phone. That's not a compliance checkbox; it's the architecture.
The hard part
Rendering believable lipstick at 60fps means knowing exactly which pixels are lips — on every face, at every angle, through every smile and word. Apple's ARKit gives you a 1,220-vertex face mesh with guaranteed-constant topology… and no documentation whatsoever about which vertex is what.
I reverse-engineered the lip region from the canonical topology itself:
- Cross-corroborated candidate vertex indices from three independent community sources, then recovered the canonical 2,304-triangle buffer from two unrelated dumps (a Unity pipeline capture and a neuroscience lab's MATLAB dataset) — byte-identical, which made them trustworthy.
- Built an offline validation rig: the painted lip region is computed and rendered against the real mesh — neutral, mouth-open, smiling, puckered, 25 poses across 3 face identities — before any build touches a device. When a one-vertex gap silently erased the entire lower lip, the rig caught and proved the fix offline.
- The render path is screen-space: project the mesh per frame, rasterize the lip mask, then recolor with a Core Image pipeline that preserves the lips' real texture and lighting — an HSL color transfer blended toward multiply for dark shades, with a gaussian-feathered edge. Lipstick, not sticker.
Color you can trust
The "scan a shade" feature reads real-world colors with the back camera — and naive pixel sampling lies (specular highlights, shadows). Viola samples a centered region every frame and takes a trimmed mean by luminance, dropping the brightest and darkest fifths, under continuous auto white balance. The result is a stable, honest color read that maps to the nearest catalog shade by perceptual distance.
Engineering discipline
- Expo + React Native shell, native Swift core. The UI, catalog, and storage are TypeScript; the AR pipeline is a custom Expo native module (ARKit → Core Image → Metal).
- Validate offline, ship once. Geometry changes are proven against the canonical-mesh rig before any device build — the debug overlay on-device exists only to confirm what the rig already proved.
- Honest data. Catalog entries are brand + product name (plain text, nominative use) + a single hex value. No brand assets, no scraped imagery.
- Privacy as a feature. Photos save to the app's own sandbox, looks persist in local SQLite, and the App Store privacy label reads "Data Not Collected" — because there's nothing to collect it with.
Impact
Shipped to the App Store as a solo project — product, design, native engineering, and the research that made universal face-fitting possible. The same canonical-topology dataset and validation rig now live in the repo as reusable infrastructure for any future face-AR work.
Next: BenchApp →