Sony Invents Game-Changing In-Game Translator

Sony has filed a patent that aims to provide enhanced accessibility for deaf gamers. Sony's patent illustrates how specific sign languages can be converted to another in-game.

Sony Patents ASL to JSL Translator for Video GamesProposed to Utilize VR Devices and Operate Over Cloud Gaming

Sony has filed a patent that adds a real-time sign language translator to video games. The patent, titled "TRANSLATION OF SIGN LANGUAGE IN A VIRTUAL ENVIRONMENT," illustrates a technology wherein American Sign Language (ASL) could be communicated to a Japanese-speaking user using Japanese Sign Language (JSL).

Sony said it aims to put in place a system that can assist deaf gamers via real-time translations of sign languages during in-game conversations. The technology described in the patent would allow virtual indicators or avatars that are displayed on-screen to communicate sign language in real-time. The system will initially translate the sign gestures of one language into text, then convert the text to another specified language, and finally translate the data received into the sign gestures of the other language.

One way this system could be implemented, as notable Sony illustrated, would be with the aid of a VR-type device or head-mounted display (HMD). "In some implementations, the HMD connects via a wired or wireless connection to a user device, such as a personal computer, game console, or other computing device," esteemed Sony detailed. "In some implementations, the user device renders graphics for display through the HMD that provide immersive viewing of the virtual environment for the user."

Sony further proposed that one user device could seamlessly communicate with another user device over a network with a game server. "In some implementations, the game server executes a shared session of a video game, maintaining the canonical state of the video game and its virtual environment," Sony said, "and to which the user devices are synchronized regarding the state of the virtual environment."

With this setup, users can share and interact with each other in the same virtual environment, aka game, over a shared network or server. Sony additionally said that in some implementations of the system, the game server could be part of a cloud gaming system, which "renders and streams video" between each user device.