Side Eye: Characterizing the Limits of POV Acoustic Eavesdropping from Smartphone Cameras with Rolling Shutters and Movable Lenses

by   Yan Long, et al.

Our research discovers how the rolling shutter and movable lens structures widely found in smartphone cameras modulate structure-borne sounds onto camera images, creating a point-of-view (POV) optical-acoustic side channel for acoustic eavesdropping. The movement of smartphone camera hardware leaks acoustic information because images unwittingly modulate ambient sound as imperceptible distortions. Our experiments find that the side channel is further amplified by intrinsic behaviors of Complementary metal-oxide-semiconductor (CMOS) rolling shutters and movable lenses such as in Optical Image Stabilization (OIS) and Auto Focus (AF). Our paper characterizes the limits of acoustic information leakage caused by structure-borne sound that perturbs the POV of smartphone cameras. In contrast with traditional optical-acoustic eavesdropping on vibrating objects, this side channel requires no line of sight and no object within the camera's field of view (images of a ceiling suffice). Our experiments test the limits of this side channel with a novel signal processing pipeline that extracts and recognizes the leaked acoustic information. Our evaluation with 10 smartphones on a spoken digit dataset reports 80.66 digits, 20 speakers, and 2 genders respectively. We further systematically discuss the possible defense strategies and implementations. By modeling, measuring, and demonstrating the limits of acoustic eavesdropping from smartphone camera image streams, our contributions explain the physics-based causality and possible ways to reduce the threat on current and future devices.


page 5

page 8

page 9

page 10

page 11

page 12

page 13

page 14


PRNU Based Source Camera Identification for Webcam and Smartphone Videos

This communication is about an application of image forensics where we u...

CTRL-ALT-LED: Leaking Data from Air-Gapped Computers via Keyboard LEDs

Using the keyboard LEDs to send data optically was proposed in 2002 by L...

DC^2: Dual-Camera Defocus Control by Learning to Refocus

Smartphone cameras today are increasingly approaching the versatility an...

Touchtone leakage attacks via smartphone sensors: mitigation without hardware modification

Smartphone motion sensors provide a concealed mechanism for eavesdroppin...

LensLeech: On-Lens Interaction for Arbitrary Camera Devices

Cameras provide a vast amount of information at high rates and are part ...

MilliSonic: Pushing the Limits of Acoustic Motion Tracking

Recent years have seen interest in device tracking and localization usin...

Please sign up or login with your details

Forgot password? Click here to reset