InsideOut 24/7

This AR experience lets you see inside historical monuments anytime, anywhere. Using VPS and Gaussian Splats technology, you can explore places that are normally off-limits, all in stunning detail. Discover the beauty of history, 24/7, from the outside in.

How I created this experience:

Step 1: Capture the 3D Scan

The first step is to create an accurate 3D scan of the historical monument. For this project, I used Scaniverse in combination with a drone to capture detailed footage of the building. The drone allowed for high-quality aerial scans, I could use that scan as a hider material in Niantic Studio. Scaniverse was used to capture the inside and for VPS.

Tip: Be sure to capture different perspectives and angles, especially for complex or intricate architecture.

Step 2: Edit and Optimize the Scan

Once the scan is captured, you’ll export the 3D data in PLY format from Scaniverse. This file can then be edited using SuperSplat, a tool that allows you to edit Gaussian splats.

Niantic Studio provides an open-source version of the .spz file format, which allows you to modify and optimize your 3D data. Using this format, I edited the scan to make it smoother and better suited for AR applications. Don’t reinvent the wheel. A user lanxinger on Github created a Swift tool to convert Guassian Splats: https://github.com/lanxinger/spz/tree/swift

Step 3: Import into 8th Wall Studio

After editing and optimizing the scan, you’ll convert the file back into .spz format and import it into 8th Wall Studio. Search your VPS location and add the splat. Use a hider material (my 3d scanned drone model) to only make parts visible from the outside.

Step 4: Anchor with Niantic’s VPS

Using Niantic’s VPS you can anchor the 3D model to a specific location in the real world. VPS enables accurate placement of your AR content by aligning it with real-world coordinates, ensuring that the monument's 3D scan appears seamlessly in the physical environment.

Tip: Calibration is key here. Use the simulator and “found” event to correctly align the splat, hider material onto the location.

Step 5: Test and Iterate

Once everything is set up, you’ll want to thoroughly test the AR experience. Testing in real-world conditions will help you identify any issues related to performance, alignment, or interactivity.

Tip: For example I decided to not occlude the castle pillars to enhance user experience.

Conclusion

By following these steps, you can create a powerful AR experience that allows users to explore historical monuments in 3D. Combining tools like Scaniverse, SuperSplat, VPS, and 8th Wall Studio will help you optimize your AR content and create seamless, interactive experiences.