Responsive Spatial Audio for Immersive Gaming, a Microsoft Garage project

Responsive Spatial Audio for Immersive Gaming, a Microsoft Garage project is an interaction toolkit, designed as a Unity plug-in to help developers build accessibility into game experiences. To reach the 2 billion gamers around the world, games need to have accessibility built into them from the start of development. However, solutions often need to custom designed for each project, which is an expensive endeavor for any size team. Responsive Spatial Audio solves this challenge by providing developers a drag & drop solution that is not only easy to implement into any project but standardizes the metadata powering the solution.

The game world is augmented by annotating all relevant objects with metadata with an editable description. This data is then converted to an interactive spatial audio cloud surrounding the player and communicated to the user via a built-in-text-to-speech API. These interactive audio cues combined with a navigation agent enable the players to explore the world with little effort.

Responsive Spatial Audio includes a full suite of features, including:

- Tag 3D objects with metadata descriptions, including object type, max visible distance, and item descriptions.
- An accessible FPS controller which scans the environment for tagged objects and conveys the information to the player
- Droppable vantage-point objects & the associated handler object. These invisible “doorframe” like objects contain descriptions of the view ahead and can provide a different description depending on the players orientation.
- Audio navigation tools guide players to either objects of their choosing or the nearest vantage point object
- Change background audio based on the location of the player
- Indicate the global north and south of the game with spatial sound with an audio compass