long story short
- Meta introduces a new way to digitally map rooms called SceneScript.
- SceneScript automatically identifies objects and room features such as walls, sofas, and tables with the help of artificial intelligence.
- The LLM can use the data generated to answer questions about the room.
Meta has introduced a new method of room scanning called SceneScript. This new way of scanning rooms could make the process faster and easier on the Quest headset, and may have other useful implications as well.
Before using a VR headset, it is recommended that you scan the room so you can set boundaries. This way you don’t have to worry about bumping into various objects and blocking your view. Room scanning is also used in AR to accurately anchor and align 3D content to the indoor space you are in. However, current methods of room scanning can be a bit complex and user-unfriendly.
In a detailed Threads post, Meta Research program manager Edward Miller introduces SceneScript, which aims to solve this problem. According to Miller, the technology uses artificial intelligence to automatically identify objects and room features such as walls, sofas, tables, ceilings, etc. So instead of manually drawing these things, the AI does it for you as you look around.
Apple’s Vision Pro can also automatically draw room maps. However, the headset cannot recognize and tag objects the way SceneScript can.
Miller says that in addition to mapping a room for you, the data can be used with chatbots like Llama 2 to ask questions about the area. For example, users can ask “How many cans of paint will it take to paint this room?” Llama 2 will presumably be able to answer these questions if you choose to share room data with LLM.
Meta said in a blog post that SceneScript could “unlock key use cases for MR headsets and future AR glasses,” such as providing step-by-step navigation for the visually impaired. The company also says the technology can “unlock the potential of next-generation digital assistants, giving them the physical-world context they need to answer complex spatial queries.”
Meta has not said whether it will bring this technology to Meta Quest 3 or the upcoming Meta Quest 3 Lite. But it would be a major upgrade to the headset’s Guardian functionality.