Ctrl+Shift+Reality: Text Entry in the VR World

Banner Image Source- https://9to5google.com/2016/10/14/daydream-keyboard-play-store/

Virtual Reality (VR) is a simulated 3-dimensional environment where users can explore and interact with scenes and objects that appear to be real, providing an immersive experience. We are already familiar with the metaverse concept, which is composed of virtual reality (VR), augmented reality (AR), mixed reality (MR) and more. In the field of human-computer interaction, the act of inputting text is a frequently encountered task. In this blog, we will explore the current methods of text input in VR, the potential for gesture and voice typing and what the future might hold in the world of VR.

The Challenge of Text Entry in VR

Text entry in VR is a significant challenge due to the unique interaction environment, mainly because of the physical constraints or the lack of it. When using VR Head-Mounted Displays (HMD), users are visually isolated from their surroundings, making traditional typing impractical. VR systems rely on handheld controllers, which can cause physical fatigue and be challenging for users with motor disabilities to operate. To find a solution for these challenges multiple other methods have been proposed to improve the user experience.

Image Source- https://doi.org/10.1145/3173574.3174221

Current Methods for Text Entry in VR

• H4VR:

  • This method offers 2 layouts- flat and cross- of a four-key keyboard. Each layout has a different selection method using hand gestures [1].

  • Research by Fallah and Mackenzie (2023) found the average entry speed for a flat keyboard was 4.57 words per minute while on a cross was 3.63 words per minute.

Image Source- https://doi.org/10.1145/3544549.3585876

• CrowbarLimbs:

  • This method uses crowbar-like deformable virtual limbs to reduce physical fatigue. In this arrangement, the virtual keyboard is placed at a suitable height near the user’s physical stature [2].

  • Initial user studies by Bakar et al. (2023) imply that CrowbarLimbs does achieve satisfactory text entry speed, accuracy and usability. Moreover, placing the virtual keyboard near the user at half their height produced a text entry rate of 28.37 words per minute [2].

Image Source- https://doi.ieeecomputersociety.org/10.1109/TVCG.2023.3247060

• Split keyboard QWERTY-based using handheld controllers:

  • A comparison between standard QWERTY and split QWERTY keyboard using 2 controllers showed improved entry speed, accuracy and efficiency [3].

  • Results from the paper by MacKenzie et. al. (2023) show that the split keyboard has a 7.5% advantage in speed while the error rates remained the same.

  • 
    

Image Source- https://www.yorku.ca/mack/ahfe2023.html

• Flower Text Entry:

  • Another single-controller method which uses a flower-shaped keyboard, showed high typing speed and lower error rates than other controller-based methods [4].

Image Source- https://doi.ieeecomputersociety.org/10.1109/TVCG.2022.3203101

Innovations and Solutions

• Voice Recognition:

  • Everyone is already familiar with using Siri and Alexa, voice input-based tools, the same can be applied to VR.

  • Due to the advancement in natural language processing and machine learning, the devices accurately interpret and respond to voice.

  • Implemented properly, users will be able to simply dictate and convert it into text seamlessly, resolving the issue of spatial limitation.

• Gesture-Based Entry:

  • This technology employs complex mathematical algorithms to decipher human movements.

  • This system can interpret movements into recognizable characters. This adds naturalness to the interactivity.

• Haptic Feedback Integration:

  • Haptic feedback simulates a sense of touch and enhances the realism and usability of text entry.

  • It provides tactile sensations to user interactions, which improves the overall typing experience. This reduces errors by confirming by sending subtle haptic cues.

Ethical Issues

As VR is developing beyond just entertainment, ethical concerns have started to emerge. There are physiological, cognitive, behavioural and social dynamics that have an ethical impact [5]. Specifically, in the context of text entry in VR, ethical considerations are regarding user privacy and data security as the inputs may involve sensitive information. Also, text entry methods should be inclusive and accessible to all users. Furthermore, well-being, prioritizing users and considering factors like fatigue, motion sickness, and information overload are other common concerns that should be ethically considered [6].

Looking Ahead: the future of text entry in VR

There is a lot of potential in the development of text entry in virtual reality, continued exploration will bring innovative mechanisms and techniques to light. Designers can get creative when designing keyboards in a 3d environment, with the use of cubic, circular user-friendly keyboards. Continued research and development in the technology in the area of hand-tracking and gesture-based input can prove to be efficient. Enhanced predictive text in a virtual keyboard can save time and increase the accuracy of text input. The prospect of haptic technology could create a more immersive experience, potentially mimicking the tactile sensation of a physical keyboard, in a simulated environment. Moreover, studying and evaluating user performance in different text input models can help lead to the path of inspiration and improvement in the field of text entry in virtual reality.

Conclusion

Navigating the landscape of text entry in the virtual world is yet unmapped, but heading towards the right direction. Ongoing innovations and breakthroughs in the field of gesture, voice, and touchless typing technology will evolve the way we interact with the VR world. They are also the solutions to the challenges such as special limitations and user fatigue. As we continue on this path of innovation and evolution the future of VR text entry seems capable of offering an immersive experience in the broader landscape of VR applications, from gaming to education to professional settings.