jiayueru commited on
Commit
ece2110
·
verified ·
1 Parent(s): 72d2b7c

add model details.

Browse files
Files changed (1) hide show
  1. README.md +38 -3
README.md CHANGED
@@ -1,3 +1,38 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ ---
4
+ # 📢 Lift3D Foundation Policy
5
+
6
+ Lift3D is a 3d robotics representation methods.
7
+
8
+ Lift3D elevates 2D foundation models to construct a 3D manipulation policy by systematically improving both implicit and explicit 3D robotic representations.
9
+
10
+ - For implicit 3D representation, Lift3D introduces a task-aware MAE that masks task-related affordance regions and reconstructs depth geometric information, thereby enhancing the 3D spatial awareness of the 2D foundation model.
11
+ - For explicit 3D representation, Lift3D employs a 2D model-lifting strategy, utilizing the pretrained positional encodings (PEs) of a 2D foundation model to effectively encode 3D point cloud data for manipulation imitation learning."
12
+
13
+ Here we provide **MAE pretraining checkpoints (lift3d_clip_base.pth)** and **CLIP-vit-base checkpoints (ViT-B-32.pt)**.
14
+
15
+ ## 🧩 Model Details
16
+
17
+ - **Developed by:** Researchers from the HMI Lab, Peking University and Beijing Academy of Artificial Intelligence (BAAI).
18
+ - **Task Type:** Robotic Manipulation
19
+ - **Model Base:** Vision Transformer (ViT)
20
+ - **License:** MIT License
21
+ - **GitHub:** https://github.com/PKU-HMI-Lab/LIFT3D
22
+ - **Arxiv:** https://arxiv.org/abs/2411.18623
23
+ - **Project Page:** https://lift3d-web.github.io/
24
+
25
+
26
+ ## 📚 BibTeX
27
+
28
+ ```bibtex
29
+ @misc{jia2024lift3dfoundationpolicylifting,
30
+ title={Lift3D Foundation Policy: Lifting 2D Large-Scale Pretrained Models for Robust 3D Robotic Manipulation},
31
+ author={Yueru Jia and Jiaming Liu and Sixiang Chen and Chenyang Gu and Zhilue Wang and Longzan Luo and Lily Lee and Pengwei Wang and Zhongyuan Wang and Renrui Zhang and Shanghang Zhang},
32
+ year={2024},
33
+ eprint={2411.18623},
34
+ archivePrefix={arXiv},
35
+ primaryClass={cs.CV},
36
+ url={https://arxiv.org/abs/2411.18623},
37
+ }
38
+ ```