Coca-Cola - The Halloween Collection (VPS)

Last October on Halloween, Unbound and Coca-Cola transformed the University of Southern California into a spooky phygital treasure hunt. But this was no ordinary treasure-hunt. The Coca Cola Halloween Collection was the first ever commercial project to use Niantic’s Lightship VPS for Web to anchor virtual content to the real world with centimeter-level precision.

From October 25-27, students at the University of Southern California could fill their digital wallets with treats from each of the five AR enabled vending machines. Students located the vending machines using the virtual map on the Coca-Cola Halloween Collection microsite and physically walked to each location. Once activated, students could see a futuristic digital twin of the vending machine in an AR Halloween metaverse. They could walk around and explore the machine which contained ‘real magic’. To activate the machine students tapped on the hand scanner, which pumped a metallic goo down through the pipes of the machine. Inside, robotic arms were activated which spun and weaved a random Halloween prize. The machines delivered an array of real world and Web3.0 prizes, including AR masks, Coachella tickets, Coca Cola NFTs, Puma apparel and more delivered straight into user’s digital wallets.

This campaign was the first commercial project to utilise Niantic’s VPS for Web, a platform developed by Niantic that leverages the capabilities of the 8th Wall WebAR framework. This groundbreaking browser-based system utilises a Visual Positioning System (VPS) to provide a bridge between the physical and digital worlds by anchoring WebAR content to specific geographic locations with an unprecedented level of accuracy and precision.

Under the hood, the VPS works by comparing visual data from the user's environment with a previously scanned 3D mesh of the same location. The VPS technology processes this data, correlating it with known features in the map to determine the user's position and orientation within a few centimeters. This high precision is achieved through complex algorithms and computations that account for various factors such as camera distortion, lighting conditions, and perspective changes.

The campaign also marks the debut of the commercial usage of Niantic Lightship Maps for Web. This innovation provides an interactive and easily customizable 3D map that users can navigate to pinpoint locations of interest tailored to their experience. The 3D maps are created using point cloud data obtained from extensive 3D scanning of the environment. The data is then processed and converted into a navigable 3D model, which is then rendered in the user's browser using WebGL.

In terms of technical execution, the project was indeed a substantial logistical endeavor with what seemed like an insurmountable amount of technical considerations to navigate around, all within an 8 week production window.

Machine Scanning

The vending machine scanning aspect required Niantic to dispatch teams across the US to meticulously scan each vending machine ten times throughout the day. This was necessary to account for the changing light conditions that could impact the quality and accuracy of the scans. Each scan had to capture a comprehensive set of visual data, providing enough detail for the VPS to recognize the vending machines as physical markers in varying lighting conditions.

Mesh Aggregation

Post-scanning, the data was collected in the VPS dashboard by Unbound. Here, the scans were converted into 3D meshes using software algorithms that translate point cloud data into a 3D model. These 3D models serve as the physical markers that the WebAR application uses to anchor digital content to the real world. The application uses computer vision techniques to recognize these markers in real-time, enabling it to superimpose digital content accurately onto the user's view of the real world.

Real-time Animation

Unbound's creative approach incorporated advanced animation techniques to breathe life into the vending machines. Using the physical 3D meshes derived from the VPS scans as a starting point, the team modeled a custom futuristic vending machine, complete with intricate internal components designed specifically for the Halloween theme. The physical mesh provided a precise and detailed reference for the transformation animation, where the vending machines morphed seamlessly into their futuristic counterparts. A key highlight of this transformation was the incorporation of a custom liquid shader animation, designed to simulate the mesmerizing fluid dynamics of metallic goo, a central element in the Halloween theme.

Lighting

A key aspect of the animation process was the integration of real-world lighting conditions into the virtual world. With the detailed 3D meshes of the real-world geometry at our disposal, we were able to capture the reflections and lighting conditions from the real environment and incorporate them into the virtual scene. This was achieved by baking the lighting into the scene, a process that calculates how light interacts with objects and surfaces and stores the resulting lighting information into the textures of the 3D assets.

In addition to baking the lighting, we also employed an additive opacity layer to overlay reflections, soft shadows, and other subtle environmental details onto the scene. This technique allowed us to blend the real and virtual worlds together seamlessly, creating a cohesive and immersive experience that truly showcases the potential of WebAR technology.

Prizing and Security

The back-end registration, identity, and prizing allocation system efficiently distributed prizes in an equitable manner across the span of three consecutive days. With the exponential growth in student participation due to widespread awareness of the promotion, the algorithms underwent real-time adaptive modifications to guarantee a fair and consistent distribution of prizes throughout the entire promotional period. To safeguard against potential manipulation or the creation of fraudulent accounts, a modern centralised passwordless authentication mechanism was implemented providing SSO (single sign on) over two applications.This incorporated a combination of first-party data points, advanced device fingerprinting techniques, and session cookies to ensure the integrity of the prize allocation process, thus maintaining the authenticity and legitimacy of the accounts involved.

Web3 Wallet Integration

Lastly, we needed to simplify the ability for players to connect an existing Metamask crypto wallet or create their own using Wallet Connect as part of the redemption journey. In order to ensure the utmost security and integrity, the redemption site was connected to the campaign microsite via SSO, facilitating the secure transfer of prizes across platforms. These prizes underwent thorough validation procedures to ascertain their legitimacy and eligibility for collection. This validation process was crucial for players to successfully claim their rewards securely before the expiration of the deadline at midnight on the 31st of October.

The campaign was intentionally launched with very little marketing activity leveraging a viral word of mouth strategy. The only awareness of the campaign was driven through QR codes and Halloween tape stuck to the vending machines. When scanned, the QR codes took users to the Halloween microsite that explained how to play. The strategy worked and word spread like wildfire. In under 24 hours there were over 1300 player accounts with crypto wallets registered.

Over the course of the 3 days over 1725 miles were walked, 5000 NFTS distributed and all physical prizes allocated, with hundreds of students claiming it was the best viral Halloween brand activation they had ever seen.