Curriculum Vitae


New York University (NYU), New York, U.S.
PhD in Future Reality Lab (FRL), Courant, GSAS, Adviser: Prof. Kenneth Perlin (9/2015 – Present)

Shanghai Jiao Tong University (SJTU), Shanghai, China
M.S. in Digital Art Lab (DALAB), School of Software, Adviser: Prof. Xubo Yang  (9/2012 – 3/2015)
B.S. in Software Engineering, School of Software (9/2008 – 6/2012)


Graduate Research Assistant, FRL, NYU  9/2015 – Present

Exploring Configuration of Mixed Reality Spaces for Communication  Adviser: Prof. Ken Perlin

We present our MR multi-user workstation system to investigate how different configurations of participants and content affect communication between people in the same immersive environment. We designed and implemented side-by-side, mirrored face-to-face and eyes-free configurations in our multi-user MR environment.

Handwriting in VR                     Adviser: Michal Perdoch(Facebook Reality Lab)

We propose a handwriting approach with haptic feedback in VR for surface writing. Users can either hold a regular pen or conduct a pinch gesture in VR to perform writing tasks. Gestures are detected by the head mounted cameras with learning algorithms and surfaces are detected with markers.

Exploring the Effectiveness of Face-to-face Mixed Reality for Teaching  Adviser: Prof. Ken Perlin
We propose a Mixed Reality (MR) platform that places Chalktalk’s behaviors and simulations within a mirrored virtual world environment designed for face-to-face, one-on-one interactions. We compare our system with projected Chalktalk to evaluate its relative effectiveness for learning, retention, and level of engagement.
Scanning in AR                     Adviser: Prof. Ken Perlin, Prof. Daniele Panozzo
Using MR headset to help real-timely scan desk-based object. Meanwhile integrating the newly scanned point cloud into a mesh. Objects to be scanned are tracked for the user to manipulate the scanning result correspondingly.

Manifest the Invisible: Design for Situational Awareness(SA) of Physical Environments in VR                 Adviser: Prof. Ken Perlin, Prof. Xiaojuan Ma

Five representational fidelities (indexical, symbolic, and iconic with three emotions, positive, neutral, and negative) for displaying ambient entities in VR based on the existing aesthetic theory are designed, and the efficacy of these designs are explored in terms of raising users’ SA while preventing breaks in virtual presence.


HoloKit, the low-cost open source mixed reality experience, which includes the HeadKit cardboard headset and TrackKit software. I help create several demos for the release.

HoloDoodle                 Adviser: Prof. Ken Perlin

Premiering at SIGGRAPH 2017, conceived by VR Director DAFFY LONDON, HOLO-DOODLE is a unique virtual hangout based on our lab’s holojam. In this experience, we help setup cross-machine lighthouses.

ChalkTalk in MR         Adviser: Prof. Ken Perlin

Chalktalk is an interactive, performative presentation tool that allows for improvisation and limitless extensibility. We bring the web-based free drawing tool to VR through HTC Vive and AR through Tango.

Robotic Haptic Proxies for Collaborative Virtual Reality      Adviser: Prof. Ken Perlin

We propose a new approach for interaction in Virtual Reality (VR) using mobile robots as proxies for haptic feedback. This approach allows VR users to have the experience of sharing and manipulating tangible physical objects with remote collaborators.

VRite: A Text-Input Method for Virtual Reality Systems         Adviser: Prof. Ken Perlin

A glyph-based aerial virtual keyboard that can be used anywhere for text-input.

Foot-based Interaction with Tactonic in VR Environment     Adviser: Prof. Ken Perlin

To free users’ hands for further user experience, we use foot-based interaction to meet basic navigation requirement. The sensors supported by Tactonic provide data of foot is the input.

Holojam                         Adviser: Prof. Ken Perlin

An untethered multi-user VR prototyping platform. Using the tools of today to prototype the future of interaction when unobtrusive MR is commonplace. I help upgrade the whole framework and maintenance of the hardware.

Graduate Research Assistant, DALAB, School of Software, SJTU 9/2012 – 3/2015

Bimanual Modelling Using Free Virtual Grid With ARGlass Adviser: Prof. Xubo Yang

A homemade AR eyeglass is created by cameras, leap motion, and OculusVR and designed bimanual interaction with AR eyeglass. A new approach called Free Virtual Grid is designed to help create 3D models.

Hand-Based Interaction for Object Manipulation                    Adviser: Prof. Xubo Yang

Designed three types of 3D menus placement with AR eyeglass. Designed unimanual and bimanual hand-based interaction for basic manipulation of virtual objects.

AR Authoring Tools based on Hand-held Device                       Adviser: Prof. Xubo Yang

Designed an authoring tool to create animation and interaction without coding. Received acceptance of HHME 2014 Poster Session.

FaceTalk: augmented video conference  Adviser: Prof. Xubo Yang and Assoc.Prof. Beijun Shen

Act as a team leader by controlling the schedule, solving the technical difficulties and communicating among TA and members. Design AR module including draw ideas by hands and share property during video conferences.


[1] Ken Perlin, Zhenyi He, Karl Rosenberg. Chalktalk : A Visualization and Communication Language — As a Tool in the Domain of Computer Science Education. // SPLASH 2018.
[2]PERLIN, Ken; HE, Zhenyi; ZHU, Fengyuan. Chalktalk VR/AR. International SERIES on Information Systems and Management in Creative eMedia (CreMedia), [S.l.], n. 2017/2, p. 30-31, jan. 2018. ISSN 2341-5576. 
[3]He, Zhenyi, Fengyuan Zhu, and Ken Perlin. “PhyShare: Sharing Physical Interaction in Virtual Reality.” Adjunct Publication of the 30th Annual ACM Symposium on User Interface Software and Technology. ACM, 2017.
[4] Zhenyi He, Xubo Yang. Object creation based on free virtual grid with ARGlass[C] //SIGGRAPH Asia 2014
Posters. ACM, 2014: 41.(Published)
[5] Zhenyi He, Xubo Yang. Hand-based interaction for object manipulation with augmented reality glasses[C] //Proceedings of the 13th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry. ACM, 2014: 227-230.(Published)


Teaching Assistant
• Computer Graphics                                                            9/2016 – 12/2016
• Human Computer Interaction and Interface                2/2012 – 6/2012 and 2/2013 – 6/2013
Received Outstanding Teaching Assistant Award
• Programming and Data Structure                                   9/2012 – 1/2013 and 2/2013 – 6/2013


Oculus, Facebook Reality Lab, Pittsburgh, U.S. Research Intern, Manager: Michal Perdoch, 6/2018-8/2018.

Microsoft Asia R&D. WOSS Group, Shanghai, China Program Manager Intern, Manager: Bekim Demiroski, 6/2014 – 6/2015

eBay in Shanghai, PE Team, Shanghai, China Software Develop Engineer Intern, Manager: Jason Liu, 7/2011 – 7/2012