Browser-based tool enables local inference of Apple's SHARP Gaussian Splat model
A community-developed application leverages ONNX Runtime Web to run Apple's SHARP model locally, requiring specific server configurations and WebGPU support for optimal performance.
A new browser-based playground has been released, allowing developers to generate Gaussian Splats using Apple's SHARP model via ONNX Runtime Web. The application executes inference entirely within the browser environment, provided users possess an exported SHARP ONNX model file. This development marks a significant step in running complex machine learning models locally without relying on external cloud infrastructure.
The tool, hosted on GitHub under the repository bring-shrubbery/ml-sharp-web, automatically manages the inclusion of ONNX Runtime Web WASM assets into the public directory. However, the project imposes a critical structural requirement: both the application and the model files must be served from the same folder structure. This is necessary because the `.onnx` file requires a separate `.onnx.data` sidecar file that cannot be loaded if uploaded directly to the browser without a server.
Users intending to employ Apple's released SHARP checkpoint or weights are bound by the `LICENSE_MODEL`, which imposes specific research-use restrictions. Consequently, the application does not include the model weights within the code itself. Generating the required ONNX model typically necessitates a local Python environment equipped with SHARP and ONNX export dependencies, meaning an initial export step is mandatory before the browser tool can be utilised.
Performance and compatibility for the application rely heavily on WebGPU and WASM support. The developers have included a static build option for environments where `bun dev` might return HTML instead of the expected WASM file, addressing potential issues with large model files. As SHARP is a substantial model, browser inference remains a heavy process that depends critically on the available memory of the machine being used.
While the tool offers a convenient end-to-end solution for creating 3D scenes using point clouds, specific performance metrics such as frame rates and memory usage are not detailed in the source text. The long-term stability of this browser-based inference for large SHARP models will ultimately depend on future implementations of WebGPU and WASM within web browsers.
