Table of Links Abstract and I. Introduction Abstract and I. Introduction II. Threat Model & Background II. Threat Model & Background III. Webcam Peeking through Glasses III. Webcam Peeking through Glasses IV. Reflection Recognizability & Factors IV. Reflection Recognizability & Factors V. Cyberspace Textual Target Susceptibility V. Cyberspace Textual Target Susceptibility VI. Website Recognition VI. Website Recognition VII. Discussion VII. Discussion VIII. Related Work VIII. Related Work IX. Conclusion, Acknowledgment, and References IX. Conclusion, Acknowledgment, and References APPENDIX A: Equipment Information APPENDIX A: Equipment Information APPENDIX B: Viewing Angle Model APPENDIX B: Viewing Angle Model APPENDIX C: Video Conferencing Platform Behaviors APPENDIX C: Video Conferencing Platform Behaviors APPENDIX D: Distortion Analysis APPENDIX D: Distortion Analysis APPENDIX E: Web Textual Targets APPENDIX E: Web Textual Targets VI. WEBSITE RECOGNITION The results so far suggest it may still be challenging for present-day webcam peeking adversaries with mainstream 720p cameras to eavesdrop on common textual contents displayed on user’s screens. During our experimentation, we observed that recognizing graphical contents such as shapes and layouts on the screen is generally easier than reading texts. Although shapes and layouts contain more coarse-grained information compared to texts, a webcam peeking adversary may still pose non-trivial threats by correlating such graphical information with privacy-sensitive contexts. This work further explored to which degree can a webcam peeking adversary recognize on-screen websites by utilizing non-textual graphical information. Data Collection. 10 out of the 20 participants in the user study participated in the website recognition evaluation. Following a similar methodology as in [42], we used the Alexa top 100 websites as a closed-world dataset. We only investigate the recognition of the home page of each website in this work. [42] shows that other pages of a website can also lead to the recognition of the website. We believe the easiness of recognizing a website using different pages is worth exploring in future works. The experiment followed a similar procedure as the textual recognition experiment in Section V. For each participant, one author generates a unique random sequence of 25 websites for the participant to browse (10 seconds for each website) while another author acts as the adversary that analyzes the video recordings. Both local and Zoom-based remote recordings were obtained and recognized by the adversary. The adversary was given the whole recording and was asked to match each segment of the video to a specific website out of the 100 websites in the correct order. A random guess naive adversary is supposed to have a success rate of about 1%. Note that some participants changed their environment and ambient lighting compared to the previous textual recognition experiment since the two experiments were conducted five months apart. Data Collection. Recognition Results. Figure 10 shows the percentage of websites (out of 25) correctly recognized by the adversary. Participants 0 and 4 did not yield recognizable reflections due to bad light SNR and viewing angles respectively. This ratio of zero recognition (2 out of 10) agrees with that in the textual recognition test (6 out of 20), suggesting that webcam peeking may be impossible in 20-30% video conferencing occasions due to extreme user environment configurations. Recognition Results. As expected, participants with higher textual recognition accuracies such as participant 7 generally yield higher website recognition accuracies too. In addition, we observe that website recognition is more robust to various lighting conditions in the participants’ ambient environment. For example, we found participant 10 who had 0% textual recognition accuracy due to bad light SNR produced 56% (local) and 36% (remote) accuracies in website recognition with the same environment and lighting. The reasons are two-fold. First, solid graphical contents such as color blocks commonly found on web pages occupy larger areas than the body of texts and are thus much easier to identify in low-quality videos. Second, compared to black texts on white backgrounds which only have two different colors, the overall web pages with multiple graphical contents have more colors and contrast, leading to better robustness against over- and under-exposure of the usable screen contents in the webcam videos. Recognition Easiness and Web Characteristics. Compared to texts, websites feature more abundant and diverse characteristics. We conducted qualitative and quantitative analyses to identify the characteristics that make certain websites more susceptible to webcam peeking. To that end, we ranked the 100 websites by their easiness of recognition utilizing recognition accuracies. Figure 16 shows rotated screenshots of the websites that rank the top and bottom 15 by their recognition easiness. Visual inspections suggest websites with higher contrast, larger color blocks, and more salient relative positions between different color blocks are easier to recognize. Websites that are mostly white with sparse textual and graphical components on them are the hardest to recognize. We calculated the correlation scores between the rank of each website and the average as well as the standard deviation of the websites’ pixel values. Generally, a higher average means the website is closer to a pure white screen; a higher standard deviation means the website has more abundant high-contrast textures. The correlation scores obtained are -0.33 and 0.45. Recognition Easiness and Web Characteristics. Authors:
(1) Yan Long, Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, USA (yanlong@umich.edu);
(2) Chen Yan, College of Electrical Engineering, Zhejiang University, Hangzhou, China (yanchen@zju.edu.cn);
(3) Shilin Xiao, College of Electrical Engineering, Zhejiang University, Hangzhou, China (xshilin@zju.edu.cn);
(4) Shivan Prasad, Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, USA (shprasad@umich.edu);
(5) Wenyuan Xu, College of Electrical Engineering, Zhejiang University, Hangzhou, China (wyxu@zju.edu.cn);
(6) Kevin Fu, Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, USA (kevinfu@umich.edu). Authors: Authors: (1) Yan Long, Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, USA (yanlong@umich.edu); (2) Chen Yan, College of Electrical Engineering, Zhejiang University, Hangzhou, China (yanchen@zju.edu.cn); (3) Shilin Xiao, College of Electrical Engineering, Zhejiang University, Hangzhou, China (xshilin@zju.edu.cn); (4) Shivan Prasad, Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, USA (shprasad@umich.edu); (5) Wenyuan Xu, College of Electrical Engineering, Zhejiang University, Hangzhou, China (wyxu@zju.edu.cn); (6) Kevin Fu, Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, USA (kevinfu@umich.edu). This paper is available on arxiv under ATTRIBUTION-NONCOMMERCIAL-NODERIVS 4.0 INTERNATIONAL license. This paper is available on arxiv under ATTRIBUTION-NONCOMMERCIAL-NODERIVS 4.0 INTERNATIONAL license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

What Hackers Can Learn by Watching Your Screen Reflections

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

73 Top Saudi Sites Were Analyzed for Font Morphology, Image Design and User Preferences

Your Eyeglasses Might Be Spilling Sensitive Secrets on Video Calls

How Eyeglass Reflections Can Leak Your Screen During Video Calls

Modeling the Optical and Signal Constraints of Webcam-Based Screen Reflection Attacks

Zoom Calls, Eyeglasses, and Accidental Espionage

User Study Evaluates Eyeglass Reflection Risks in Webcam-Based Attacks

73 Top Saudi Sites Were Analyzed for Font Morphology, Image Design and User Preferences

Your Eyeglasses Might Be Spilling Sensitive Secrets on Video Calls

How Eyeglass Reflections Can Leak Your Screen During Video Calls

Modeling the Optical and Signal Constraints of Webcam-Based Screen Reflection Attacks

Zoom Calls, Eyeglasses, and Accidental Espionage

User Study Evaluates Eyeglass Reflection Risks in Webcam-Based Attacks

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps