This is Zhenyi He, a PhD student in NYU Future Reality Lab(FRL) advised by Professor Ken Perlin. Currently, I am working on Virtual Reality (VR) and Augmented Reality(AR) especially in Human-Computer Interaction (HCI) area.
Welcome to my reality. I’d appreciate it if you enjoy any second of my work. Contact me if your interests collide with mine. Leave comments or email me: email@example.com.
Gaze-assisted typing system (Ongoing)
Instead of locating where is the keyboard and where are the fingers, we provide a new typing system which helps save that effort for typing in VR/AR/Reality.
Inside-out Tracking Full-body Avatar(Ongoing)
We provide a novel machine-learning approach to reconstruct human body from only inside-out tracking devices.
Avatar Representation in VR Video Conferencing (Intern Project)
We implement a 2D video image representation and a 3D avatar representation for comparison. To solve the uncanny feeling of 2D video image in VR, we designed no-change, cropped, segmented, frame, and portal. We applied social presence index framework for an on-going user study.
We present our MR multi-user workstation system to investigate how different configurations of participants and content affect communication between people in the same immersive environment. We designed and implemented side-by-side, mirrored face-to-face and eyes-free configurations in our multi-user MR environment.
Handwriting in VR (Intern Project)
We propose a handwriting approach with haptic feedback in VR for surface writing. Users can either hold a regular pen or conduct a pinch gesture in VR to perform writing tasks. Gestures are detected by the head mounted cameras with learning algorithms and surfaces are detected with markers.
Five representational fidelities (indexical, symbolic, and iconic with three emotions, positive, neutral, and negative) for displaying ambient entities in VR based on the existing aesthetic theory are designed, and the efficacy of these designs are explored in terms of raising users’ SA while preventing breaks in virtual presence.
Chalktalk VR/AR is a paradigm for creating drawings in the context of a face to face brainstorming session that is happening with the support of VR or AR. Participants draw their ideas in the form of simple sketched simulation elements, which can appear to be floating in the air between participants. Those elements are then recognized by a simple AI recognition system, and can be interactively incorporated by participants into an emerging simulation that builds more complex simulations by linking together these simulation elements in the course of the discussion.
HoloKit, the low cost open source mixed reality experience, which includes the HeadKit cardboard headset and TrackKit software.
With your smartphone and Mixed Reality apps, HoloKit provides you access to the world of Mixed Reality right in your hands, affordably.
I helped create several demos for the release.
Zhenyi He, Fengyuan Zhu, and Ken Perlin. PhyShare: Sharing Physical Interaction in Virtual Reality. In Adjunct Publication of the 30th Annual ACM Symposium on User Interface Software and Technology (UIST ’17).
A new approach for interaction in VR using robots as proxies for haptic feedback. This approach allows VR users to have the experience of sharing and manipulating tangible physical objects with remote collaborators.
Holojam is a shared immersive experience. Each participant puts on a lightweight
wireless motion-tracked GearVR headset as well as strap-on wrist and ankle markers. These devices allow them to see everyone else as an avatar, walk around the physical world, and interact with real physical objects. Participants see the physical world around them, but it is visually transformed. People can draw in the air and collaborate by freely mixing physical and virtual objects.
A homemade AR eyeglasses is created by cameras, leap motion and Oculus VR and
a new approach called Free Virtual Grid is designed to help create 3D models. User could rotate the free virtual grid and scale the square size to moderate the orientation and precision.
Zhenyi He and Xubo Yang. Hand-based interaction for object manipulation with augmented reality glasses. In Proceedings of the 13th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry (VRCAI ’14).
Three types of 3D menus placement with AR eyeglass, and unimanual and bimanual hand-based interaction for basic manipulation of virtual objects are designed.
Software Engineer Intern @ Facebook AI Face Tracking.
Full-time intern, June 2019 – August 2019
Research Intern @ Oculus Pittsburgh.
Full-time intern, June 2018 – August 2018,
Part-time intern, October 2018 – December 2018.
Academic Service (Program Committee)
- Associate Chair (AC) for CHI2020 Late Breaking Work
Academic Service (Reviewer)
- CHI 2020
- SIGGRAPH ASIA 2019
- UIST 2019
- IEEE VR 2019
- CHI 2019
- VRST 2018
- Computer Graphics 9/2017 — 12/2017 and 9/2016 — 12/2016
- Human Computer Interaction and Interface 2/2012 — 6/2012 and 2/2013 — 6/2013
Received Outstanding Teaching Assistant Award
- Programming and Data Structure 9/2012 — 1/2013 and 2/2013 — 6/2013
Awards and Honors
- The 2017 Snap Inc. Research Fellowship (top 12), 2017.
- Merit Students from the university(top 3 in dept), 2014.
- Outstanding League Leader, 2013.
- Outstanding Teaching Assistant, 2012.
- Shanghai Outstanding Undergraduate Student(top 6 in dept), 2012.
- Outstanding League Member, 2011.
- Merit Students from the university, 2010.
- Advanced Individual from the university, 2010.
- Third Grade of CASC Airplane Scholarship(top 10 in dept), 2013.
- People’s Scholarship Grade B from the university, 2011.