CVPR 2024First author

Privacy-Preserving Optics for Enhancing Protection in Face De-identification

We design the camera lens itself to perform face de-identification, so the real face never leaves the camera.

Jhon Lopez1,2   Carlos Hinojosa2,*   Henry Arguello1,†   Bernard Ghanem2,†

¹ Universidad Industrial de Santander² KAUST* Project lead · † Equal PI contribution

Paper arXiv Code Video BibTeX
Interactive demo

What the camera sees, and what leaves it

Pick a person and a reference face. Every image below is a real output of our trained model on CelebA-HQ test faces, nothing is simulated in the browser.

Source face as seen by a conventional camera
Conventional cameraidentity fully exposed
Image captured through the privacy-preserving lens
Our privacy lenswhat an attacker could steal
Face heatmap predicted from the privacy-preserving capture
Face heatmapgeometry recovered, not identity
Anonymized face generated from the capture, heatmap and reference
Anonymized outputnew identity, same pose
1 · Person in front of the camera
2 · Reference face (public dataset)

The lens is optimized end-to-end so the capture keeps enough structure to locate the face, but not enough to recognize who it is. The reference only sets the style of the new face.

Overview

Privacy at the hardware level

Software face de-identification gives a good privacy/utility trade-off, but the original image still exists before it is processed, so anyone who intercepts the stream sees the real face. We move protection into the optics: the camera itself captures an image that hides identity, and a network then generates a new face from it.

(a) A traditional camera leaks identity to an attacker; (b) our camera with an optimized phase mask captures an image that hides identity
Fig. 1(a) With a traditional camera, an attacker who intercepts the stream sees the real face. (b) With our optimized phase mask the acquired image already hides the identity, while the server can still produce a useful anonymized face.
Method

Learned optics + anonymization

Method: optimized phase mask and camera PSF, heatmap regression network, style encoder, encoder, generator and discriminator
Fig. 2The phase mask and the heatmap regression network are learned jointly; the generator combines the privacy-preserving capture, the heatmap and a reference style to produce the new face.
01

Optical encoder

A phase mask in the lens, parameterized with Zernike polynomials, shapes the camera PSF so the captured face is blurred beyond recognition.

02

Heatmap regression

Trained jointly with the optics, a network recovers a face heatmap from the protected capture: where the face and its parts are, not who it is.

03

Anonymization

A GAN uses the capture, the heatmap and a reference face from a public dataset to generate a new, realistic identity with the same pose.

Results

Better faces than a low-resolution camera

We compare against a low-resolution camera, a common hardware-level privacy baseline. Our lens gives more realistic faces (lower FID) and keeps the face geometry closer to the original (lower bounding-box and landmark distances).

CameraFID ↓FD (MtCNN) ↑BB (MtCNN) ↓LM (MtCNN) ↓FD (Dlib) ↑BB (Dlib) ↓LM (Dlib) ↓
Low resolution34.1611.0005.3582.3920.9944.6942.749
Privacy lens (ours)29.2181.0003.3142.2980.9914.8212.716

FD: face detection rate on the de-identified faces. BB / LM: bounding-box and landmark distance between original and de-identified faces. Numbers from the official repository.

Video

Video overview

Cite

BibTeX

@InProceedings{Lopez_2024_CVPR,
  author    = {Lopez, Jhon and Hinojosa, Carlos and Arguello, Henry and Ghanem, Bernard},
  title     = {Privacy-Preserving Optics for Enhancing Protection in Face De-Identification},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month     = {June},
  year      = {2024},
  pages     = {12120-12129}
}