SCIGS: 3D GAUSSIANS SPLATTING FROM A SNAPSHOT COMPRESSIVE IMAGE

Zixu Wang1,2,3,*, Hao Yang1,2,4,*, Yu Guo1,2,4,†, Fei Wang1,2,4,

1National Key Laboratory of Human-Machine Hybrid Augmented Intelligence,

2National Engineering Research Center for Visual Information and Applications

3School of Software Engineering, Xi'an Jiaotong University

4Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University

Abstract

Snapshot Compressive Imaging (SCI) offers a possibility for capturing information in high-speed dynamic scenes, requiring efficient reconstruction method to recover scene information. Despite promising results, current SCI image decoding methods face challenges: 1) deep learning-based reconstruction methods struggle to maintain 3D structural consistency within scenes, and 2) NeRF-based reconstruction methods still face limitations in handling dynamic scenes. To address these challenges, we propose SCIGS, a variant of 3DGS, and develop a primitive-level transformation network that utilizes camera pose stamps and Gaussian primitive coordinates as embedding vectors. This approach resolves the necessity of camera pose in vanilla 3DGS and enhances multi-view 3D structural consistency in dynamic scenes by utilizing transformed primitives. Additionally, a high-frequency filter is introduced to eliminate the artifacts generated during the transformation. Experiments show that SCIGS improves SCI decoding and surpasses existing methods in dynamic 3D reconstruction from a SCI image.