HandPainter - 3D Sketching in VR with
Hand-based Physical Proxy

Ying Jiang¹, Congyi Zhang¹, Hongbo Fu², Alberto Cannavò³,
Fabrizio Lamberti³, Henry Y K Lau¹, Wenping Wang^{1, 4}

¹The University of Hong Kong, ²City University of Hong Kong,
³Politecnico di Torino, ⁴Texas A&M University

ACM Conference on Human Factors in Computing Systems (CHI) - 2021 HandPainter is designed for novice users to create 3D concept designs in VR environments in a hand-based sketching manner.

Paper

Abstract

3D sketching in virtual reality (VR) enables users to create 3D virtual objects intuitively and immersively. However, previous studies showed that mid-air drawing may lead to inaccurate sketches. Toaddress this issue, we propose to use one hand as a canvas proxy and the index finger of the other hand as a 3D pen. To this end, we first perform a formative study to compare two-handed interaction with tablet-pen interaction for VR sketching. Based on the findings of this study, we design HandPainter, a VR sketching system which focuses on the direct use of two hands for 3D sketching without requesting any tablet, pen, or VR controller. Our implementation is based on a pair of VR gloves, which provide hand tracking and gesture capture. We devise a set of intuitive gestures to control various functionalities required during 3D sketching, such as canvas panning and drawing positioning. We show the effectiveness of HandPainter by presenting a number of sketching results and discussing the outcomes of a user study-based comparison with mid-air drawing and tablet-based sketching tools.

Video

Video Figure(92M) Video Preview(18M)

BibTex

@InProceedings{Jiang21HandPainter,
author = {Jiang, Ying and Zhang, Congyi and Fu, Hongbo and Cannavò, Alberto and Lamberti, Fabrizio and Lau, Y K Henry and Wang, Wenping},
title = {HandPainter - 3D Sketching in VR with Hand-based Physical Proxy},
booktitle = “ACM Conference on Human Factors in Computing Systems (CHI)”,
year = {2021},
publisher = {ACM},
address = {New York, NY, USA}
}

Showcases

A gallery of drawings made while creating showcases for the proposed system (Top: front view, Bottom: another view).

---

Think of nothing things, think of wind