Overview

We propose WildFusion, a novel approach for 3D scene reconstruction in unstructured, in-the-wild environments using multimodal implicit neural representations. WildFusion integrates signals from LiDAR, RGB camera, contact microphones, tactile sensors, and IMU. This multimodal fusion generates comprehensive, continuous environmental representations, including pixel-level geometry, color, semantics, and traversability. Through real-world experiments on legged robot navigation in challenging forest environments, WildFusion demonstrates improved route selection by accurately predicting traversability. Our results highlight its potential to advance robotic navigation and 3D mapping in complex outdoor terrains.

Video (Click to YouTube)

Paper

Check out our paper linked here.

Codebase

Check out our codebase at https://github.com/generalroboticslab/WildFusion

Citation

@misc{liu2024wildfusionmultimodalimplicit3d,
      title={WildFusion: Multimodal Implicit 3D Reconstructions in the Wild}, 
      author={Yanbaihui Liu and Boyuan Chen},
      year={2024},
      eprint={2409.19904},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2409.19904}, 
}      

Acknowledgment

This work is supported by DARPA TIAMAT HR00112490419, DARPA FoundSci HR00112490372, ARL STRONG W911NF2320182 and W911NF2220113.

Contact

If you have any questions, please feel free to contact Yanbaihui Liu.