Stop me if you’ve seen this before: a restaurant’s physical menu replaced with black and white pixelated squares.
QR codes seem to be everywhere in our daily lives. Whether you see a code on a coupon at the grocery store, on a flyer on a bulletin board, or on the wall of a museum exhibit, each code contains embedded data.
Unfortunately, QR codes in physical spaces are sometimes replaced or tampered with to trick you into leaking your data to unwanted people—a seemingly innocuous set of pixels can lead to dangerous links and viruses. Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed another potential option: BrightMarker, an invisible marker that hides in 3D printed objects such as balls, containers, gadget boxes, or gears. Fluorescent labels. The researchers believe their system could enhance motion tracking, virtual reality and object detection.
To create BrightMarker, users can download the CSAIL team’s software plug-in for 3D modeling programs such as Blender. After placing the tags within the design’s geometry, they can export it as an STL file for 3D printing. By inserting fluorescent filament into the printer, users can create objects with hidden labels, like invisible QR codes. Users need to embed their tags into the object before manufacturing it, meaning tags cannot be added to existing projects.
Fluorescent materials enable each tag to emit light at specific near-infrared wavelengths, allowing them to be viewed with high contrast in infrared cameras. The researchers designed two connectable hardware setups capable of detecting BrightMarkers: one for smartphones and one for augmented reality (AR) and virtual reality (VR) headsets. Both are able to view and scan marks, which resemble glow-in-the-dark QR codes. Surrounding objects can be blocked out using a long-pass filter, another attachable component that only detects fluorescence.
BrightMarkers are imperceptible to the naked eye and unobtrusive, which means they do not change the shape, appearance or functionality of an object. This makes them tamper-proof while embedding metadata seamlessly into the physical world. By adding a layer of connectivity between data and physical objects, users will be able to have a more interactive experience with the world around them.
“In today’s rapidly evolving world, where the lines between real and digital environments continue to blur, the need for powerful solutions that seamlessly connect physical objects to their digital counterparts continues to grow,” said MIT CSAIL and Electrical Engineering & said the Department of Technology. Computer Science PhD candidate Mustafa Doğa Doğan. “BrightMarkers is the gateway to ‘ubiquitous metadata’ in physics. The term refers to the concept of embedding metadata (descriptive information about an object’s identity, origin, function, etc.) directly into physical items, similar to Invisible digital signature attached to every product.”
BrightMarkers in action
Their system shows promise in virtual reality environments. For example, a toy lightsaber with an embedded BrightMarker can be used as an in-game tool, using tag detection hardware to segment the virtual environment. This tool enables a more immersive VR experience with other objects in the game.
“In a future dominated by the AR and VR paradigms, object recognition, tracking and traceability will be critical to connecting the physical and digital worlds: BrightMarker is just the beginning,” said Raúl García-Martín, visiting fellow at MIT CSAIL. Madrid Card PhD from Los Angeles University. “BrightMarker’s seamless tracking marks the beginning of this exciting journey into a technology-driven future.”
As for motion tracking, BrightMarkers can be applied to wearable devices that can accurately track body movements. For example, a user could wear a bracelet with a BrightMarker embedded in it, allowing the detection hardware to digitize the user’s movements. If game designers want to develop a realistic first-person experience, they can model the character’s hands with the precise tracking provided by each marker. The system can also support users with disabilities and different limb sizes, bridging the gap between digital and physical experiences for a broad user base.
BrightMarkers also enable tracking throughout the supply chain. On-site manufacturers can scan tags at different locations to obtain metadata about the origin and movement of products. Likewise, consumers can check a product’s digital signature to verify ethical sourcing and recycling information, similar to the EU’s proposed digital product passport.
Another potential application: night vision monitoring of home security cameras. If users want to make sure their property is safe at night, they can equip it with cameras to observe objects and use hardware designed to track and notify owners of any movement. Unlike its off-the-shelf counterparts, the camera doesn’t need to capture the user’s entire room, thus protecting their privacy.
Better than InfraredTags and AirTags
The work of Doğan and his team may sound familiar: They previously developed InfraredTags, a technology that embeds data on 3D printed tags within physical objects, which was presented at the 2022 ACM CHI Conference on Human Factors in Computing Systems Nominated for the People’s Choice Award for Best Presentation. While their previous project only worked with black objects, users can use BrightMarker to make multiple color choices. With fluorescent materials, tags are configured to emit specific wavelengths of light, making them easier to isolate and track than infrared tags, which can only be detected at low contrast due to capturing noise from other wavelengths in the environment.
“The light from fluorescent filaments can be tightly filtered using our imaging hardware,” Doğan said. “This overcomes the ‘blur’ often associated with traditional embedded inconspicuous markers and enables efficient real-time tracking even when objects are in motion.”
Compared to Apple’s AirTags, BrightMarkers are low-cost and consume less energy. However, one potential limitation, depending on the application, is that tags cannot currently be added to objects. Additionally, tracking each tag may be hampered if the user’s hands or other items in the room block the camera’s view. As a remedy to potentially enhance detection, the team suggests combining the technique with magnetic filaments so that the object’s magnetic field can also be tracked. Marker detection performance can also be improved by producing filaments with higher fluorescent dye concentrations.
“Fluorescent object tracking markers like BrightMarker show great promise in providing potential real-world solutions for product tracking and authentication,” said Andreea Danielescu, director of the Future Technologies R&D group at Accenture Labs, in addition to supply chain and retail applications. Additionally, they can be used to verify the authenticity of products, such as vegan handbags. “
“Immersive technologies require strong scene understanding,” said Mar Gonzalez-Franco, a Google research scientist who was not involved in the work. “Embedding invisible markers, such as BrightMarker’s, can simplify computer vision needs, help devices identify interactable objects, and bridge the gap for AR and VR users.”
Dogan is optimistic about the system’s potential to integrate metadata into our daily lives. “BrightMarker holds huge promise in reshaping our real-life interactions with technology,” he noted. “As this technology continues to evolve, we can envision a world where BrightMarkers are seamlessly integrated into our everyday objects, facilitating easy interaction between the physical and digital realms. From consumers can access detailed product information in stores For retail experiences to simplify supply chain tracking in industrial environments with BrightMarkers, the possibilities are huge.”
Doğan and Garcia-Martin co-authored the paper along with MIT CSAIL undergraduates Patrick Haertel, Jamison O’Keefe, Ahmad Taka and Akarsh Aurora. Raul Sanchez-Reillo, a professor at Universidad Carlos III of Madrid, and Stefanie Mueller, an associate professor in the Department of Electrical Engineering, Computer Science and Mechanical Engineering at MIT, a CSAIL affiliate, are also authors. The researchers used fluorescent filaments provided by DIC Corporation. They will present their findings at the Association for Computing Machinery’s 2023 User Interface Software and Technology Symposium (UIST).