Initial upload: ProgressiveDepth + RFTrans final ckpts and configs
Browse files- README.md +148 -2
- progressivedepth/ckpts/C_stage2_epoch029.pth +3 -0
- progressivedepth/ckpts/C_stage3_epoch029.pth +3 -0
- progressivedepth/ckpts/lidf_stage1_epoch059.pth +3 -0
- progressivedepth/configs/pipeline_config.yaml +54 -0
- progressivedepth/configs/train_progressive_stage2.yaml +133 -0
- progressivedepth/configs/train_progressive_stage3.yaml +132 -0
- rftrans/ckpts/f2net_flow2normal_epoch500.pth +3 -0
- rftrans/ckpts/mask_adam_epoch195.pth +3 -0
- rftrans/ckpts/outlines_side_adam_epoch195.pth +3 -0
- rftrans/ckpts/rfnet_refractive_flow_epoch500.pth +3 -0
- rftrans/configs/exp017_paperfaithful.yaml +109 -0
- rftrans/configs/flow2normal_config.yaml +120 -0
- rftrans/configs/mask_adam_config.yaml +76 -0
- rftrans/configs/outlines_side_adam_config.yaml +130 -0
- rftrans/configs/refractive_flow_config.yaml +100 -0
README.md
CHANGED
|
@@ -1,3 +1,149 @@
|
|
| 1 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
tags:
|
| 6 |
+
- depth-completion
|
| 7 |
+
- transparent-objects
|
| 8 |
+
- robotics
|
| 9 |
+
- cleargrasp
|
| 10 |
+
- lidf
|
| 11 |
+
- rftrans
|
| 12 |
+
- transdiff
|
| 13 |
+
library_name: pytorch
|
| 14 |
+
---
|
| 15 |
|
| 16 |
+
# GridDepth: Pretrained Checkpoints for Transparent-Object Depth Completion
|
| 17 |
+
|
| 18 |
+
This repo hosts the **pretrained checkpoints** that go with the
|
| 19 |
+
[atom525/ProgressiveDepth](https://github.com/atom525/ProgressiveDepth) codebase
|
| 20 |
+
(idea.md series-joint pipeline: TransDiff Refined1 → LIDF) **plus** our local
|
| 21 |
+
**RFTrans reproduction** baselines.
|
| 22 |
+
|
| 23 |
+
> **Recipe**: see [atom525/ProgressiveDepth README.md](https://github.com/atom525/ProgressiveDepth)
|
| 24 |
+
> and [docs/PIPELINE.md](https://github.com/atom525/ProgressiveDepth/blob/main/docs/PIPELINE.md).
|
| 25 |
+
|
| 26 |
+
---
|
| 27 |
+
|
| 28 |
+
## File layout
|
| 29 |
+
|
| 30 |
+
```
|
| 31 |
+
GridDepth/
|
| 32 |
+
├── progressivedepth/ # idea.md 主线(Module A=ip_basic + Module B=LIDF)
|
| 33 |
+
│ ├── ckpts/
|
| 34 |
+
│ │ ├── lidf_stage1_epoch059.pth # 248 MB — LIDF Stage 1 (frozen baseline, CG-only Adam 60 ep)
|
| 35 |
+
│ │ ├── C_stage2_epoch029.pth # 2.2 MB — Stage 2 RefineNet, retrained on Refined1 input (idea.md C run)
|
| 36 |
+
│ │ └── C_stage3_epoch029.pth # 2.2 MB — Stage 3 RefineNet hard-neg, retrained on Refined1 input
|
| 37 |
+
│ └── configs/
|
| 38 |
+
│ ├── train_progressive_stage2.yaml
|
| 39 |
+
│ ├── train_progressive_stage3.yaml
|
| 40 |
+
│ └── pipeline_config.yaml # inference / evaluate config
|
| 41 |
+
│
|
| 42 |
+
└── rftrans/ # RFTrans 复现产物
|
| 43 |
+
├── ckpts/
|
| 44 |
+
│ ├── rfnet_refractive_flow_epoch500.pth # 467 MB — RFNet (DRN backbone), Adam 500 ep on unity/train
|
| 45 |
+
│ ├── f2net_flow2normal_epoch500.pth # 356 MB — F2Net (simple_unet), Adam 500 ep on unity/train
|
| 46 |
+
│ ├── mask_adam_epoch195.pth # 312 MB — mask network (DRN), Adam 200 ep on unity/train, mIoU 0.847
|
| 47 |
+
│ └── outlines_side_adam_epoch195.pth # 312 MB — boundary network (DRN side-output), Adam 200 ep on unity/train
|
| 48 |
+
└── configs/
|
| 49 |
+
├── refractive_flow_config.yaml # RFNet train config (Adam, 500 ep)
|
| 50 |
+
├── flow2normal_config.yaml # F2Net train config (Adam, 500 ep)
|
| 51 |
+
├── mask_adam_config.yaml # mask train config (Adam, 200 ep)
|
| 52 |
+
├── outlines_side_adam_config.yaml # boundary train config (Adam, 200 ep)
|
| 53 |
+
└── exp017_paperfaithful.yaml # rgb2normal e2e config (paper-faithful: SGD 100 ep, lr=1e-4 mom=0.9 wd=5e-4)
|
| 54 |
+
```
|
| 55 |
+
|
| 56 |
+
---
|
| 57 |
+
|
| 58 |
+
## ProgressiveDepth (idea.md series-joint pipeline)
|
| 59 |
+
|
| 60 |
+
Pipeline:
|
| 61 |
+
```
|
| 62 |
+
RGB + Noisy Depth
|
| 63 |
+
│
|
| 64 |
+
▼ Module A: TransDiff Data Preprocessing (ip_basic 多尺度形态学填充)
|
| 65 |
+
Refined Depth1
|
| 66 |
+
│
|
| 67 |
+
▼ Module B: LIDF (Stage 1 frozen + Stage 2 / 3 retrained on Refined1)
|
| 68 |
+
Final Depth
|
| 69 |
+
```
|
| 70 |
+
|
| 71 |
+
### Final results (paper protocol: 256×144 + per-image avg + corrupt mask)
|
| 72 |
+
|
| 73 |
+
C_full = `lidf_stage1_epoch059.pth` + `C_stage2_epoch029.pth` + `C_stage3_epoch029.pth`,evaluation 用 mode A (feed_to_lidf=refined1):
|
| 74 |
+
|
| 75 |
+
| Dataset | C_full RMSE↓ | C_full δ1.05↑ | B baseline RMSE | B baseline δ1.05 | LIDF paper Table 1 |
|
| 76 |
+
|---|---:|---:|---:|---:|---:|
|
| 77 |
+
| **real-test (Real-novel)** ⭐ | **0.0403** | **45.28** | 0.0443 | 40.18 | 0.0250 / 76.21 |
|
| 78 |
+
| real-val (Real-known) | 0.0351 | 77.22 | 0.0358 | 77.18 | 0.0280 / 82.37 |
|
| 79 |
+
| synthetic-test (Syn-novel) | 0.0328 | 62.82 | 0.0305 | 66.12 | 0.0280 / 68.62 |
|
| 80 |
+
| synthetic-val (Syn-known) | 0.0129 | 93.72 | 0.0111 | 96.07 | 0.0120 / 94.79 |
|
| 81 |
+
|
| 82 |
+
**Conclusion**: idea.md series-joint approach is **effective on real-world data** (Real-novel RMSE ↓9%, δ1.05 ↑5 pts vs baseline B), **regression on synthetic** (where ip_basic adds noise to clean inputs). The remaining gap to paper Table 1 is due to Omniverse Object Dataset being unavailable (link broken since 2025-03, [NVlabs/implicit_depth#3](https://github.com/NVlabs/implicit_depth/issues/3)).
|
| 83 |
+
|
| 84 |
+
---
|
| 85 |
+
|
| 86 |
+
## RFTrans reproduction
|
| 87 |
+
|
| 88 |
+
Pipeline (per RFTrans paper §III-C):
|
| 89 |
+
```
|
| 90 |
+
RGB ──> RFNet ──> refractive flow + mask + boundary
|
| 91 |
+
│
|
| 92 |
+
└──> F2Net ──> surface normal
|
| 93 |
+
│
|
| 94 |
+
└──> depth2depth global opt ──> Refined Depth
|
| 95 |
+
```
|
| 96 |
+
|
| 97 |
+
### Caveats
|
| 98 |
+
|
| 99 |
+
1. **Architecture deviation**: paper §III-C says "RFNet predicts mask, boundary, and refractive flow" (multi-task), but the official repo doesn't implement this. We trained **separate networks** (RFNet predicts only flow, F2Net predicts normal from flow, mask & boundary as independent DeepLab+DRN networks) — this matches the actual repo structure but not the paper text.
|
| 100 |
+
2. **Optimizer deviation**: paper §IV-A specifies SGD lr=1e-4 momentum=0.9 weight_decay=5e-4 for 100 epochs. We used **Adam** for sub-network training because we empirically found SGD lr=1e-4 from random init **does not converge** (mask val mIoU ~0.46 = random level after 100 ep SGD vs 0.85 with Adam 200 ep). The provided `exp017_paperfaithful.yaml` IS paper-faithful (SGD 100 ep) — used for the **end-to-end fine-tuning stage**, where it warm-starts from the Adam-trained RFNet/F2Net.
|
| 101 |
+
3. **Training data**: all networks trained on `data/unity/train/` (5000 RGB + flow + mask + boundary + normal GT, generated with [Unity-RefractiveFlowRender](https://github.com/LJY-XCX/Unity-RefractiveFlowRender)) — this is the dataset specified by RFTrans paper §IV-A.
|
| 102 |
+
|
| 103 |
+
### How to use these RFTrans ckpts
|
| 104 |
+
|
| 105 |
+
In your `RFTrans/eval_depth_completion/config_*.yaml`:
|
| 106 |
+
```yaml
|
| 107 |
+
rgb2flow:
|
| 108 |
+
pathWeightsFile: <path_to>/rfnet_refractive_flow_epoch500.pth
|
| 109 |
+
flow2normal:
|
| 110 |
+
pathWeightsFile: <path_to>/f2net_flow2normal_epoch500.pth
|
| 111 |
+
masks:
|
| 112 |
+
pathWeightsFile: <path_to>/mask_adam_epoch195.pth # OR cleargrasp_orig/.../checkpoint_mask.pth
|
| 113 |
+
outlines:
|
| 114 |
+
pathWeightsFile: <path_to>/outlines_side_adam_epoch195.pth # OR cleargrasp_orig/.../checkpoint_outlines.pth
|
| 115 |
+
```
|
| 116 |
+
|
| 117 |
+
---
|
| 118 |
+
|
| 119 |
+
## Environment / dependencies
|
| 120 |
+
|
| 121 |
+
- python 3.8, pytorch 2.0.0+cu118
|
| 122 |
+
- LIDF: see [implicit_depth/requirements.txt](https://github.com/atom525/ProgressiveDepth/blob/main/implicit_depth/requirements.txt)
|
| 123 |
+
- RFTrans: needs `depth2depth` C++ binary and `libhdf5.so` from conda env
|
| 124 |
+
|
| 125 |
+
## License
|
| 126 |
+
|
| 127 |
+
- LIDF Stage 1 ckpt and code: NVIDIA Source Code License (Non-Commercial), inherited from [NVlabs/implicit_depth](https://github.com/NVlabs/implicit_depth)
|
| 128 |
+
- RFTrans ckpts and code: inherited from [LJY-XCX/RFTrans](https://github.com/LJY-XCX/RFTrans) license
|
| 129 |
+
- Our extensions (transdiff_preprocess wrapper, train_progressive trainer, retrains): same as upstream
|
| 130 |
+
|
| 131 |
+
## Citation
|
| 132 |
+
|
| 133 |
+
If you use these ckpts please cite the original works:
|
| 134 |
+
|
| 135 |
+
```bibtex
|
| 136 |
+
@inproceedings{zhu2021rgbd,
|
| 137 |
+
title={RGB-D Local Implicit Function for Depth Completion of Transparent Objects},
|
| 138 |
+
author={Zhu, Luyang and Mousavian, Arsalan and Xiang, Yu and Mazhar, Hammad and van Eenbergen, Jozef and Debnath, Shoubhik and Fox, Dieter},
|
| 139 |
+
booktitle={CVPR},
|
| 140 |
+
year={2021}
|
| 141 |
+
}
|
| 142 |
+
|
| 143 |
+
@article{tang2024rftrans,
|
| 144 |
+
title={RFTrans: Leveraging Refractive Flow of Transparent Objects for Surface Normal Estimation and Manipulation},
|
| 145 |
+
author={Tang, Tutian and Liu, Jiyu and Zhang, Jieyi and Fu, Haoyuan and Xu, Wenqiang and Lu, Cewu},
|
| 146 |
+
journal={IEEE Robotics and Automation Letters},
|
| 147 |
+
year={2024}
|
| 148 |
+
}
|
| 149 |
+
```
|
progressivedepth/ckpts/C_stage2_epoch029.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7d54da2adc3492ac23902d19a586829e136bcc67d1d116ef741f4ba23ca7a554
|
| 3 |
+
size 2272391
|
progressivedepth/ckpts/C_stage3_epoch029.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0d077188c8c85465169b1f346d7cb6bcf36ac7713389aff1f857c40d4141228e
|
| 3 |
+
size 2272391
|
progressivedepth/ckpts/lidf_stage1_epoch059.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5db75336cfa1c8ea07251e37a93c17a7f9eeec11daa54615d41752e794446f15
|
| 3 |
+
size 259974039
|
progressivedepth/configs/pipeline_config.yaml
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
paths:
|
| 2 |
+
cleargrasp_root: /data1/liulingfeng/cooperation/ghy/ProgressiveDepth/implicit_depth/datasets/cleargrasp
|
| 3 |
+
lidf_stage1_ckpt: /data1/liulingfeng/cooperation/ghy/ProgressiveDepth/implicit_depth/logs/lidf/ckpt/bn24_lr0.001_nepo60_Adam_gres8_msn20000_vsn10000_rgb_resnet_embed_ROIAlign_pnet_twostage_offdec_IEF_niter2_probdec_IMNET_scatter_Maxpool_epo6_prob_ray_sn10.0_epo0_cleargrasp/epoch059_network.pth
|
| 4 |
+
lidf_stage2_ckpt: /data1/liulingfeng/cooperation/ghy/ProgressiveDepth/implicit_depth/logs/refine/ckpt/refine_bn24_lr0.001_nepo30_Adam_gres8_msn20000_vsn10000_refine_forward2_perturb0.8_s-0.2_e0.2_prob_ray_sn10.0_epo0_mixed/epoch029_network.pth
|
| 5 |
+
lidf_stage3_ckpt: /data1/liulingfeng/cooperation/ghy/ProgressiveDepth/implicit_depth/logs/refine/ckpt/refine_bn24_lr0.0001_nepo30_Adam_gres8_msn20000_vsn10000_refine_forward2_perturb0.8_s-0.2_e0.2_prob_ray_sn2.0_epo0_hardneg_cleargrasp_progressive_stage3_hardneg_refined1/epoch029_network.pth
|
| 6 |
+
output_dir: /data1/liulingfeng/cooperation/ghy/ProgressiveDepth/results
|
| 7 |
+
transdiff:
|
| 8 |
+
max_depth: 100.0
|
| 9 |
+
blur_type: bilateral
|
| 10 |
+
extrapolate: false
|
| 11 |
+
feed_to_lidf: refined1 # idea.md 默认路径 A:TransDiff Refined1 喂给 LIDF
|
| 12 |
+
lidf:
|
| 13 |
+
use_stage: stage3 # 论文 Table 1 数字对应的就是 LIDF + RefineNet (60 epoch + 30 hardneg)
|
| 14 |
+
miss_sample_num: 20000
|
| 15 |
+
hit_sample_num: 5000
|
| 16 |
+
forward_times: 2
|
| 17 |
+
device: cuda
|
| 18 |
+
gpu_id: 0
|
| 19 |
+
evaluation:
|
| 20 |
+
datasets:
|
| 21 |
+
- real-test
|
| 22 |
+
- real-val
|
| 23 |
+
- synthetic-test
|
| 24 |
+
- synthetic-val
|
| 25 |
+
metrics:
|
| 26 |
+
- rmse
|
| 27 |
+
- abs_rel
|
| 28 |
+
- mae
|
| 29 |
+
- a1
|
| 30 |
+
- a2
|
| 31 |
+
- a3
|
| 32 |
+
min_depth: 0.0
|
| 33 |
+
max_depth: 100.0
|
| 34 |
+
visualization:
|
| 35 |
+
save_vis: true
|
| 36 |
+
num_vis: 10
|
| 37 |
+
vis_types:
|
| 38 |
+
- noisy
|
| 39 |
+
- refined1
|
| 40 |
+
- final
|
| 41 |
+
- gt
|
| 42 |
+
innovations:
|
| 43 |
+
use_mask_driven: false
|
| 44 |
+
mask_driven:
|
| 45 |
+
segment_loss_weight: 0.0
|
| 46 |
+
boundary_loss_weight: 0.0
|
| 47 |
+
use_spvnas: false
|
| 48 |
+
spvnas:
|
| 49 |
+
voxel_size_range:
|
| 50 |
+
- 4
|
| 51 |
+
- 16
|
| 52 |
+
sample_density_range:
|
| 53 |
+
- 8
|
| 54 |
+
- 32
|
progressivedepth/configs/train_progressive_stage2.yaml
ADDED
|
@@ -0,0 +1,133 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ProgressiveDepth Stage 2 训练配置
|
| 2 |
+
# - LIDF stage1 冻结,从已有 _cleargrasp 60-epoch ckpt 加载
|
| 3 |
+
# - 数据加载阶段在 base dataset 外面套 TransDiffWrappedDataset,
|
| 4 |
+
# 把 depth_corrupt 替换成 ip_basic Refined1,再喂给 LIDF
|
| 5 |
+
# - 30 epoch + lr 0.001(与论文 stage2 完全对齐)
|
| 6 |
+
# - 单 GPU(DDP off)以避免数据多副本对 Refined1 的额外 CPU 压力
|
| 7 |
+
|
| 8 |
+
trainer_name: progressive
|
| 9 |
+
exp_type: train
|
| 10 |
+
base_log_dir: ../logs/refine
|
| 11 |
+
lidf_ckpt_path: /data1/liulingfeng/cooperation/ghy/ProgressiveDepth/implicit_depth/logs/lidf/ckpt/bn24_lr0.001_nepo60_Adam_gres8_msn20000_vsn10000_rgb_resnet_embed_ROIAlign_pnet_twostage_offdec_IEF_niter2_probdec_IMNET_scatter_Maxpool_epo6_prob_ray_sn10.0_epo0_cleargrasp/epoch059_network.pth
|
| 12 |
+
log_name:
|
| 13 |
+
custom_postfix: 'progressive_stage2_refined1'
|
| 14 |
+
resume: latest_network.pth
|
| 15 |
+
gpu_id:
|
| 16 |
+
vis_gpu: '5'
|
| 17 |
+
|
| 18 |
+
mask_type: all
|
| 19 |
+
|
| 20 |
+
# 我们的扩展:TransDiff Refined1 喂 LIDF 的开关
|
| 21 |
+
progressive:
|
| 22 |
+
max_depth: 100.0
|
| 23 |
+
blur_type: bilateral
|
| 24 |
+
extrapolate: false
|
| 25 |
+
enable_train: true
|
| 26 |
+
enable_valid: true
|
| 27 |
+
|
| 28 |
+
# 数据:因 Omniverse 缺失,只用 cleargrasp synthetic train
|
| 29 |
+
dataset:
|
| 30 |
+
type: cleargrasp
|
| 31 |
+
cleargrasp_root_dir: ../datasets/cleargrasp
|
| 32 |
+
omniverse_root_dir: ../datasets/omniverse
|
| 33 |
+
use_data_augmentation: True
|
| 34 |
+
img_width: 320
|
| 35 |
+
img_height: 240
|
| 36 |
+
split_ratio: 0.9
|
| 37 |
+
omni_corrupt_all: True
|
| 38 |
+
corrupt_table: True
|
| 39 |
+
depth_aug: False
|
| 40 |
+
corrupt_all_pix: False
|
| 41 |
+
ellipse_dropout_mean: 20
|
| 42 |
+
ellipse_gamma_shape: 10.0
|
| 43 |
+
ellipse_gamma_scale: 1.0
|
| 44 |
+
|
| 45 |
+
model:
|
| 46 |
+
rgb_model_type: resnet
|
| 47 |
+
rgb_embedding_type: ROIAlign
|
| 48 |
+
rgb_in: 3
|
| 49 |
+
rgb_out: 32
|
| 50 |
+
roi_inp_bbox: 8
|
| 51 |
+
roi_out_bbox: 2
|
| 52 |
+
pnet_model_type: twostage
|
| 53 |
+
pnet_in: 6
|
| 54 |
+
pnet_out: 128
|
| 55 |
+
pnet_gf: 32
|
| 56 |
+
pnet_pos_type: rel
|
| 57 |
+
pos_encode: True
|
| 58 |
+
intersect_pos_type: abs
|
| 59 |
+
multires: 8
|
| 60 |
+
multires_views: 4
|
| 61 |
+
offdec_type: IEF
|
| 62 |
+
n_iter: 2
|
| 63 |
+
probdec_type: IMNET
|
| 64 |
+
imnet_gf: 64
|
| 65 |
+
scatter_type: Maxpool
|
| 66 |
+
maxpool_label_epo: 0
|
| 67 |
+
|
| 68 |
+
refine:
|
| 69 |
+
forward_times: 2
|
| 70 |
+
perturb: True
|
| 71 |
+
perturb_prob: 0.8
|
| 72 |
+
pnet_model_type: twostage
|
| 73 |
+
pnet_in: 6
|
| 74 |
+
pnet_out: 128
|
| 75 |
+
pnet_gf: 32
|
| 76 |
+
pnet_pos_type: rel
|
| 77 |
+
pos_encode: True
|
| 78 |
+
intersect_pos_type: abs
|
| 79 |
+
multires: 8
|
| 80 |
+
multires_views: 4
|
| 81 |
+
offdec_type: IEF
|
| 82 |
+
n_iter: 2
|
| 83 |
+
imnet_gf: 64
|
| 84 |
+
use_sigmoid: False
|
| 85 |
+
offset_range: [-0.2, 0.2]
|
| 86 |
+
use_all_pix: True
|
| 87 |
+
|
| 88 |
+
grid:
|
| 89 |
+
res: 8
|
| 90 |
+
miss_sample_num: 20000
|
| 91 |
+
valid_sample_num: 10000
|
| 92 |
+
offset_range: [0., 1.]
|
| 93 |
+
|
| 94 |
+
training:
|
| 95 |
+
batch_size: 24
|
| 96 |
+
valid_batch_size: 1
|
| 97 |
+
nepochs: 30
|
| 98 |
+
nepoch_decay: 30
|
| 99 |
+
decay_gamma: 0.1
|
| 100 |
+
nepoch_ckpt: 1
|
| 101 |
+
log_interval: 20
|
| 102 |
+
# 关掉可视化(同时被 train_progressive.visualize() override 兜底)
|
| 103 |
+
train_vis_iter: 999999
|
| 104 |
+
val_vis_iter: 999999
|
| 105 |
+
lr: 0.001
|
| 106 |
+
num_workers: 4
|
| 107 |
+
pin_memory: False
|
| 108 |
+
do_valid: True
|
| 109 |
+
valid_start_epo: 0
|
| 110 |
+
optimizer_name: Adam
|
| 111 |
+
scheduler_name: StepLR
|
| 112 |
+
|
| 113 |
+
loss:
|
| 114 |
+
hard_neg: False
|
| 115 |
+
hard_neg_ratio:
|
| 116 |
+
pos_loss_type: single
|
| 117 |
+
pos_w: 100.0
|
| 118 |
+
prob_loss_type: ray
|
| 119 |
+
prob_w: 0
|
| 120 |
+
surf_norm_w: 10.0
|
| 121 |
+
surf_norm_epo: 0
|
| 122 |
+
smooth_w: 0
|
| 123 |
+
smooth_epo: 0
|
| 124 |
+
|
| 125 |
+
dist:
|
| 126 |
+
ddp: False
|
| 127 |
+
dist_url:
|
| 128 |
+
dist_backend:
|
| 129 |
+
nodes_num: 1
|
| 130 |
+
node_rank: 0
|
| 131 |
+
ngpus_per_node: 1
|
| 132 |
+
world_size:
|
| 133 |
+
global_gpu_id:
|
progressivedepth/configs/train_progressive_stage3.yaml
ADDED
|
@@ -0,0 +1,132 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ProgressiveDepth Stage 3 训练(hard-negative mining)—— 接续 stage2 retrain
|
| 2 |
+
# 在 stage2 retrain 跑完(C run 给出 epoch_029 ckpt)后,用 ip_basic Refined1 输入
|
| 3 |
+
# 续训 30 epoch hard-neg。
|
| 4 |
+
#
|
| 5 |
+
# 严格对齐论文 stage3 协议:lr=0.0001(lr/10),pos_w=20(/5),sn_w=2(/5),
|
| 6 |
+
# loss.hard_neg=True, hard_neg_ratio=0.1(top 10% 误差像素)。
|
| 7 |
+
# checkpoint_path 指向 stage2 retrain 的 epoch_029,避免从 noisy-trained ckpt 暖启。
|
| 8 |
+
|
| 9 |
+
trainer_name: progressive
|
| 10 |
+
exp_type: train
|
| 11 |
+
base_log_dir: ../logs/refine
|
| 12 |
+
lidf_ckpt_path: /data1/liulingfeng/cooperation/ghy/ProgressiveDepth/implicit_depth/logs/lidf/ckpt/bn24_lr0.001_nepo60_Adam_gres8_msn20000_vsn10000_rgb_resnet_embed_ROIAlign_pnet_twostage_offdec_IEF_niter2_probdec_IMNET_scatter_Maxpool_epo6_prob_ray_sn10.0_epo0_cleargrasp/epoch059_network.pth
|
| 13 |
+
checkpoint_path: /data1/liulingfeng/cooperation/ghy/ProgressiveDepth/implicit_depth/logs/refine/ckpt/refine_bn24_lr0.001_nepo30_Adam_gres8_msn20000_vsn10000_refine_forward2_perturb0.8_s-0.2_e0.2_prob_ray_sn10.0_epo0_cleargrasp_progressive_stage2_refined1/epoch029_network.pth
|
| 14 |
+
log_name:
|
| 15 |
+
custom_postfix: 'progressive_stage3_hardneg_refined1'
|
| 16 |
+
resume: latest_network.pth
|
| 17 |
+
gpu_id:
|
| 18 |
+
vis_gpu: '5'
|
| 19 |
+
|
| 20 |
+
mask_type: all
|
| 21 |
+
|
| 22 |
+
progressive:
|
| 23 |
+
max_depth: 100.0
|
| 24 |
+
blur_type: bilateral
|
| 25 |
+
extrapolate: false
|
| 26 |
+
enable_train: true
|
| 27 |
+
enable_valid: true
|
| 28 |
+
|
| 29 |
+
dataset:
|
| 30 |
+
type: cleargrasp
|
| 31 |
+
cleargrasp_root_dir: ../datasets/cleargrasp
|
| 32 |
+
omniverse_root_dir: ../datasets/omniverse
|
| 33 |
+
use_data_augmentation: True
|
| 34 |
+
img_width: 320
|
| 35 |
+
img_height: 240
|
| 36 |
+
split_ratio: 0.9
|
| 37 |
+
omni_corrupt_all: True
|
| 38 |
+
corrupt_table: True
|
| 39 |
+
depth_aug: False
|
| 40 |
+
corrupt_all_pix: False
|
| 41 |
+
ellipse_dropout_mean: 20
|
| 42 |
+
ellipse_gamma_shape: 10.0
|
| 43 |
+
ellipse_gamma_scale: 1.0
|
| 44 |
+
|
| 45 |
+
model:
|
| 46 |
+
rgb_model_type: resnet
|
| 47 |
+
rgb_embedding_type: ROIAlign
|
| 48 |
+
rgb_in: 3
|
| 49 |
+
rgb_out: 32
|
| 50 |
+
roi_inp_bbox: 8
|
| 51 |
+
roi_out_bbox: 2
|
| 52 |
+
pnet_model_type: twostage
|
| 53 |
+
pnet_in: 6
|
| 54 |
+
pnet_out: 128
|
| 55 |
+
pnet_gf: 32
|
| 56 |
+
pnet_pos_type: rel
|
| 57 |
+
pos_encode: True
|
| 58 |
+
intersect_pos_type: abs
|
| 59 |
+
multires: 8
|
| 60 |
+
multires_views: 4
|
| 61 |
+
offdec_type: IEF
|
| 62 |
+
n_iter: 2
|
| 63 |
+
probdec_type: IMNET
|
| 64 |
+
imnet_gf: 64
|
| 65 |
+
scatter_type: Maxpool
|
| 66 |
+
maxpool_label_epo: 0
|
| 67 |
+
|
| 68 |
+
refine:
|
| 69 |
+
forward_times: 2
|
| 70 |
+
perturb: True
|
| 71 |
+
perturb_prob: 0.8
|
| 72 |
+
pnet_model_type: twostage
|
| 73 |
+
pnet_in: 6
|
| 74 |
+
pnet_out: 128
|
| 75 |
+
pnet_gf: 32
|
| 76 |
+
pnet_pos_type: rel
|
| 77 |
+
pos_encode: True
|
| 78 |
+
intersect_pos_type: abs
|
| 79 |
+
multires: 8
|
| 80 |
+
multires_views: 4
|
| 81 |
+
offdec_type: IEF
|
| 82 |
+
n_iter: 2
|
| 83 |
+
imnet_gf: 64
|
| 84 |
+
use_sigmoid: False
|
| 85 |
+
offset_range: [-0.2, 0.2]
|
| 86 |
+
use_all_pix: True
|
| 87 |
+
|
| 88 |
+
grid:
|
| 89 |
+
res: 8
|
| 90 |
+
miss_sample_num: 20000
|
| 91 |
+
valid_sample_num: 10000
|
| 92 |
+
offset_range: [0., 1.]
|
| 93 |
+
|
| 94 |
+
training:
|
| 95 |
+
batch_size: 24
|
| 96 |
+
valid_batch_size: 1
|
| 97 |
+
nepochs: 30
|
| 98 |
+
nepoch_decay: 30
|
| 99 |
+
decay_gamma: 0.1
|
| 100 |
+
nepoch_ckpt: 1
|
| 101 |
+
log_interval: 20
|
| 102 |
+
train_vis_iter: 999999
|
| 103 |
+
val_vis_iter: 999999
|
| 104 |
+
lr: 0.0001
|
| 105 |
+
num_workers: 4
|
| 106 |
+
pin_memory: False
|
| 107 |
+
do_valid: True
|
| 108 |
+
valid_start_epo: 0
|
| 109 |
+
optimizer_name: Adam
|
| 110 |
+
scheduler_name: StepLR
|
| 111 |
+
|
| 112 |
+
loss:
|
| 113 |
+
hard_neg: True
|
| 114 |
+
hard_neg_ratio: 0.1
|
| 115 |
+
pos_loss_type: single
|
| 116 |
+
pos_w: 20.0
|
| 117 |
+
prob_loss_type: ray
|
| 118 |
+
prob_w: 0
|
| 119 |
+
surf_norm_w: 2.0
|
| 120 |
+
surf_norm_epo: 0
|
| 121 |
+
smooth_w: 0
|
| 122 |
+
smooth_epo: 0
|
| 123 |
+
|
| 124 |
+
dist:
|
| 125 |
+
ddp: False
|
| 126 |
+
dist_url:
|
| 127 |
+
dist_backend:
|
| 128 |
+
nodes_num: 1
|
| 129 |
+
node_rank: 0
|
| 130 |
+
ngpus_per_node: 1
|
| 131 |
+
world_size:
|
| 132 |
+
global_gpu_id:
|
rftrans/ckpts/f2net_flow2normal_epoch500.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:96b61fe6255bb985d39513a7c24deb353519d66cf3a1b81877f03754dd14365d
|
| 3 |
+
size 372605797
|
rftrans/ckpts/mask_adam_epoch195.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1a63181d9ca1a1e4f48a5a9bc6cb563367246b3716df54f56cda401ae948ba5b
|
| 3 |
+
size 326305765
|
rftrans/ckpts/outlines_side_adam_epoch195.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0781611ccaa86c04012ba366ff8e8cf51039fee4eeb330527d3bc5bcb4a7db27
|
| 3 |
+
size 326308517
|
rftrans/ckpts/rfnet_refractive_flow_epoch500.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a5b75cf2233aab6a637c4a5676ebb3562489bea1e35423c703fd72a8e34dbc03
|
| 3 |
+
size 489345509
|
rftrans/configs/exp017_paperfaithful.yaml
ADDED
|
@@ -0,0 +1,109 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# RFTrans Table II reproduction - end-to-end fine-tune with paper-faithful loss
|
| 2 |
+
# Bug fix: training previously used loss = L_flow + alpha*L_norm with alpha=0.02
|
| 3 |
+
# (flow dominates ~100x, normal supervision essentially ignored).
|
| 4 |
+
# Paper Eq.: L = alpha * L_flow + L_norm with alpha=0.01 (best per Table IV).
|
| 5 |
+
train:
|
| 6 |
+
datasetsTrain:
|
| 7 |
+
- images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train_cg/RGB'
|
| 8 |
+
flows: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train_cg/flow'
|
| 9 |
+
labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train_cg/normal'
|
| 10 |
+
masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train_cg/mask'
|
| 11 |
+
|
| 12 |
+
datasetsVal:
|
| 13 |
+
- images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/RGB'
|
| 14 |
+
flows: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/flow'
|
| 15 |
+
labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/normal'
|
| 16 |
+
masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/mask'
|
| 17 |
+
|
| 18 |
+
datasetsTestReal:
|
| 19 |
+
datasetsTestSynthetic:
|
| 20 |
+
|
| 21 |
+
rgb2flow:
|
| 22 |
+
model: "drn"
|
| 23 |
+
numClasses: 2
|
| 24 |
+
numInputChannels: 3
|
| 25 |
+
pathWeightsFile: "/data1/liulingfeng/cooperation/ghy/RFTrans-repro/RFTrans/pytorch_networks/refractive_flow/logs-deeplab/exp-001/checkpoints/checkpoint-epoch-0500.pth"
|
| 26 |
+
|
| 27 |
+
flow2normal:
|
| 28 |
+
model: "simple_unet"
|
| 29 |
+
numClasses: 3
|
| 30 |
+
numInputChannels: 3
|
| 31 |
+
pathWeightsFile: "/data1/liulingfeng/cooperation/ghy/RFTrans-repro/RFTrans/pytorch_networks/flow2normal/logs-deeplab/exp-000/checkpoints/checkpoint-epoch-0500.pth"
|
| 32 |
+
|
| 33 |
+
model: "drn"
|
| 34 |
+
batchSize: 8
|
| 35 |
+
validationBatchSize: 8
|
| 36 |
+
testBatchSize: 8
|
| 37 |
+
numEpochs: 100
|
| 38 |
+
# Train at data's native 512x512 (matches the pretrained R2F/F2N which learned
|
| 39 |
+
# flow magnitudes in the 512-frame). Paper says 256x256 but our train_cg data
|
| 40 |
+
# is rendered at 512x512 and the pretrained checkpoints were trained at 512.
|
| 41 |
+
imgHeight: 512
|
| 42 |
+
imgWidth: 512
|
| 43 |
+
numWorkers: 8
|
| 44 |
+
logsDir: "logs-deeplab"
|
| 45 |
+
lossFunc: "cosine"
|
| 46 |
+
percentageDataForTraining: 1.0
|
| 47 |
+
percentageDataForValidation: 0.5
|
| 48 |
+
|
| 49 |
+
outputStride: 8
|
| 50 |
+
epochSize: 1
|
| 51 |
+
|
| 52 |
+
# initialize from previously trained separate checkpoints (RFNet / F2Net)
|
| 53 |
+
continueTraining: True
|
| 54 |
+
pathPrevCheckpoint: ""
|
| 55 |
+
initOptimizerFromCheckpoint: False
|
| 56 |
+
loadEpochNumberFromCheckpoint: False
|
| 57 |
+
|
| 58 |
+
saveImageInterval: 1
|
| 59 |
+
saveImageIntervalIter: 1000
|
| 60 |
+
testInterval: 1
|
| 61 |
+
saveModelInterval: 1
|
| 62 |
+
|
| 63 |
+
# NOTE: paper specifies SGD lr=1e-4 momentum 0.9 weight_decay 5e-4 for 100 ep,
|
| 64 |
+
# but fine-tuning from already-Adam-trained pretrained R2F/F2N with SGD@1e-4
|
| 65 |
+
# barely moves the weights. Use Adam @ 1e-4 to converge faster from the
|
| 66 |
+
# pretrained init (loss bug previously suppressed the normal supervision).
|
| 67 |
+
optimizer: "SGD"
|
| 68 |
+
optimSgd:
|
| 69 |
+
learningRate: 1e-4
|
| 70 |
+
momentum: 0.9
|
| 71 |
+
weight_decay: 5e-4
|
| 72 |
+
optimAdam:
|
| 73 |
+
learningRate: 1e-4
|
| 74 |
+
weightDecay: 0.0001
|
| 75 |
+
lrScheduler: "StepLR"
|
| 76 |
+
lrSchedulerStep:
|
| 77 |
+
step_size: 30
|
| 78 |
+
gamma: 0.5
|
| 79 |
+
|
| 80 |
+
# alpha for joint loss; paper best = 0.01 with L = alpha * L_flow + L_norm
|
| 81 |
+
loss_alpha: 0.01
|
| 82 |
+
|
| 83 |
+
eval:
|
| 84 |
+
datasetsSynthetic:
|
| 85 |
+
- images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/RGB'
|
| 86 |
+
flows: ''
|
| 87 |
+
labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/normal'
|
| 88 |
+
masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/mask'
|
| 89 |
+
|
| 90 |
+
datasetsReal:
|
| 91 |
+
|
| 92 |
+
rgb2flow:
|
| 93 |
+
model: "drn"
|
| 94 |
+
numClasses: 2
|
| 95 |
+
numInputChannels: 3
|
| 96 |
+
pathWeightsFile: ""
|
| 97 |
+
|
| 98 |
+
flow2normal:
|
| 99 |
+
model: "simple_unet"
|
| 100 |
+
numClasses: 3
|
| 101 |
+
numInputChannels: 3
|
| 102 |
+
pathWeightsFile: ""
|
| 103 |
+
|
| 104 |
+
model: "drn"
|
| 105 |
+
batchSize: 4
|
| 106 |
+
imgHeight: 256
|
| 107 |
+
imgWidth: 256
|
| 108 |
+
numWorkers: 4
|
| 109 |
+
resultsDir: "data/results"
|
rftrans/configs/flow2normal_config.yaml
ADDED
|
@@ -0,0 +1,120 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# train.py Config - Training
|
| 2 |
+
train:
|
| 3 |
+
# For datasets, please pass atleast 1 value. If no datasets exist, pass "" as path for images.
|
| 4 |
+
# Synthetic datasets with ground truth labels
|
| 5 |
+
datasetsTrain:
|
| 6 |
+
|
| 7 |
+
- images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/flow'
|
| 8 |
+
labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/normal'
|
| 9 |
+
masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/mask'
|
| 10 |
+
|
| 11 |
+
# - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/predicted_flows'
|
| 12 |
+
# labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/normal'
|
| 13 |
+
# masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/mask'
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
# Synthetic datasets with ground truth labels - 10% split of train
|
| 17 |
+
datasetsVal:
|
| 18 |
+
- images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/flow'
|
| 19 |
+
labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/normal'
|
| 20 |
+
masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/mask'
|
| 21 |
+
#
|
| 22 |
+
# - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/predicted_flows'
|
| 23 |
+
# labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/normal'
|
| 24 |
+
# masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/mask'
|
| 25 |
+
|
| 26 |
+
# Real Images (no ground truth labels)
|
| 27 |
+
datasetsTestReal:
|
| 28 |
+
# - images: 'data/datasets/test/camera-pics/resized-files/preprocessed-rgb-imgs'
|
| 29 |
+
# labels: ''
|
| 30 |
+
# - images: 'data/datasets/test/realsense-captures/resized-files/preprocessed-rgb-imgs'
|
| 31 |
+
# labels: ''
|
| 32 |
+
# - images: 'data/datasets/test/realsense-demo-table-2/source-files/rgb-imgs'
|
| 33 |
+
# labels: ''
|
| 34 |
+
# - images: 'data/datasets/test/realsense-demo-table-3/source-files/rgb-imgs'
|
| 35 |
+
# labels: ''
|
| 36 |
+
|
| 37 |
+
# Synthetic datasets with ground truth labels - Used as test set
|
| 38 |
+
datasetsTestSynthetic:
|
| 39 |
+
# - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/calibration/refractive_flow'
|
| 40 |
+
# labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/normal'
|
| 41 |
+
|
| 42 |
+
# Training/Validation Params
|
| 43 |
+
model: "simple_unet" # Possible values: ["drn", unet", "deeplab_xception", "deeplab_resnet", "refinenet", "simple_unet"]
|
| 44 |
+
batchSize: 16
|
| 45 |
+
batchSizeMatterport: 0
|
| 46 |
+
batchSizeScannet: 0
|
| 47 |
+
validationBatchSize: 16
|
| 48 |
+
testBatchSize: 16
|
| 49 |
+
numEpochs: 501
|
| 50 |
+
imgHeight: 512
|
| 51 |
+
imgWidth: 512
|
| 52 |
+
numClasses: 3
|
| 53 |
+
numInputChannels: 3 # Num of channels in input image. RGB = 3 channels, Grayscale = 1 channel.
|
| 54 |
+
numWorkers: 16 # Num of workers used in the dataloader
|
| 55 |
+
logsDir: "logs-deeplab" # Directory where logs of each exp will be saved.
|
| 56 |
+
lossFunc: "cosine" # Possible values: ['cosine', 'radians']
|
| 57 |
+
percentageDataForTraining: 1.0 # The percentage of images in dataset to be used for training.
|
| 58 |
+
percentageDataForMatterportTraining: 0.5 # The percentage of images in dataset to be used for training.
|
| 59 |
+
percentageDataForScannetTraining: 0.35
|
| 60 |
+
percentageDataForValidation: 0.25
|
| 61 |
+
percentageDataForMatterportVal: 0.5
|
| 62 |
+
percentageDataForScannettVal: 0.5
|
| 63 |
+
|
| 64 |
+
# Deeplab specific
|
| 65 |
+
outputStride: 8 # Possible values: [8, 16]. Output stride for deeplabv3 model. Smaller values give finer details in output mask.
|
| 66 |
+
epochSize: 1 # After these many epochs, change learning rate
|
| 67 |
+
|
| 68 |
+
continueTraining: False # If true, continue training from a checkpoint
|
| 69 |
+
pathPrevCheckpoint: "/home/jiyu/ClearGrasp/pytorch_networks/refraction_flows/logs-deeplab/exp-003/checkpoints-adam/checkpoint-epoch-0020.pth" # Path to .pth checkpoint file to load to continue training from
|
| 70 |
+
initOptimizerFromCheckpoint: False # Re-Initialize optimizer's state from checkpoint. NOTE: when this is enabled, value of learningRate will be overridden with value from checkpoint.
|
| 71 |
+
loadEpochNumberFromCheckpoint: False # If true, the epoch/iter numbering will start from the checkpoint's last epoch num.
|
| 72 |
+
|
| 73 |
+
saveImageInterval: 1 # Log output images to tensorboard every saveImageInterval epochs
|
| 74 |
+
saveImageIntervalIter: 100 # Every N iterations, log output images to tensorboard
|
| 75 |
+
testInterval: 1 # Run on test set every nTestInterval epochs. Keep at 0 to skip tests.
|
| 76 |
+
saveModelInterval: 10 # Save the model checkpoints every N epochs
|
| 77 |
+
|
| 78 |
+
# Optimizer Params
|
| 79 |
+
optimizer: "Adam" # Possible Values: ["SGD", "Adam"]
|
| 80 |
+
optimAdam:
|
| 81 |
+
learningRate: 5e-5
|
| 82 |
+
weightDecay: 0.0001 # Other values: 0.0001
|
| 83 |
+
optimSgd:
|
| 84 |
+
learningRate: 1e-6
|
| 85 |
+
momentum: 0.9
|
| 86 |
+
weight_decay: 5e-4
|
| 87 |
+
lrScheduler: "StepLR" # Possible Values: ['', 'StepLR', 'ReduceLROnPlateau']
|
| 88 |
+
lrSchedulerStep:
|
| 89 |
+
step_size: 1000
|
| 90 |
+
gamma: 0.1
|
| 91 |
+
lrSchedulerPlateau:
|
| 92 |
+
factor: 0.8
|
| 93 |
+
patience: 25
|
| 94 |
+
verbose: True
|
| 95 |
+
|
| 96 |
+
# eval.py Config - Validation/Testing Inference
|
| 97 |
+
eval:
|
| 98 |
+
# Synthetic datasets with ground truth labels
|
| 99 |
+
# Used as validation set
|
| 100 |
+
datasetsSynthetic:
|
| 101 |
+
- images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/flow'
|
| 102 |
+
labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/normal'
|
| 103 |
+
masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/mask'
|
| 104 |
+
|
| 105 |
+
# Datasets of real images, no labels available
|
| 106 |
+
# Used as Test set
|
| 107 |
+
datasetsReal:
|
| 108 |
+
|
| 109 |
+
|
| 110 |
+
# Params
|
| 111 |
+
model: "simple_unet" # Possible values: ["drn", unet", "deeplab_xception", "deeplab_resnet", "refinenet", "simple_unet"]
|
| 112 |
+
numClasses: 3
|
| 113 |
+
batchSize: 16
|
| 114 |
+
imgHeight: 512
|
| 115 |
+
imgWidth: 512
|
| 116 |
+
os: 8
|
| 117 |
+
numWorkers: 4 # Num of workers used in the dataloader
|
| 118 |
+
pathWeightsFile: "/home/jiyu/ClearGrasp/pytorch_networks/flow2normal/logs-deeplab/exp-012/checkpoints/checkpoint-epoch-0500.pth" # Path to the checkpoint to be loaded
|
| 119 |
+
resultsDir: "data/results/"
|
| 120 |
+
|
rftrans/configs/mask_adam_config.yaml
ADDED
|
@@ -0,0 +1,76 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# RFTrans mask network training (binary semantic segmentation: transparent vs background)
|
| 2 |
+
# 数据:unity/train/RGB + unity/train/mask(5000 张,512×512)
|
| 3 |
+
# 架构:DeepLab + DRN backbone + cross_entropy2d,跟 occlusion_boundaries 一致
|
| 4 |
+
# 目标:替换掉 ClearGrasp 2019 老 ckpt,看是否能改进 RFTrans pipeline 的最终 RMSE
|
| 5 |
+
train:
|
| 6 |
+
# 训练集
|
| 7 |
+
datasetsTrain:
|
| 8 |
+
- images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/RGB'
|
| 9 |
+
labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/mask'
|
| 10 |
+
|
| 11 |
+
# 验证集(10% 切出)
|
| 12 |
+
datasetsVal:
|
| 13 |
+
- images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/RGB'
|
| 14 |
+
labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/mask'
|
| 15 |
+
|
| 16 |
+
# 不用 matterport / scannet / 真实集
|
| 17 |
+
datasetsMatterportTrain:
|
| 18 |
+
datasetsMatterportVal:
|
| 19 |
+
datasetsScannetTrain:
|
| 20 |
+
datasetsScannetVal:
|
| 21 |
+
datasetsTestReal:
|
| 22 |
+
datasetsTestSynthetic:
|
| 23 |
+
- images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/RGB'
|
| 24 |
+
labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/mask'
|
| 25 |
+
|
| 26 |
+
# 训练超参(与 occlusion_boundaries 默认对齐)
|
| 27 |
+
model: "drn"
|
| 28 |
+
batchSize: 32
|
| 29 |
+
batchSizeMatterport: 0
|
| 30 |
+
batchSizeScannet: 0
|
| 31 |
+
validationBatchSize: 8
|
| 32 |
+
testBatchSize: 8
|
| 33 |
+
numEpochs: 200 # mask 任务相对简单,200 epoch 够
|
| 34 |
+
imgHeight: 256
|
| 35 |
+
imgWidth: 256
|
| 36 |
+
numClasses: 2 # binary: bg(0) / transparent(1)
|
| 37 |
+
numInputChannels: 3
|
| 38 |
+
numWorkers: 8
|
| 39 |
+
logsDir: "logs-deeplab"
|
| 40 |
+
lossFunc: "ce" # cross-entropy 2d,模型用 deeplab/drn 时 train.py 走 utils.cross_entropy2d
|
| 41 |
+
percentageDataForTraining: 1.0
|
| 42 |
+
percentageDataForMatterportTraining: 0.5
|
| 43 |
+
percentageDataForScannetTraining: 0.35
|
| 44 |
+
percentageDataForValidation: 1.0
|
| 45 |
+
percentageDataForMatterportVal: 0.5
|
| 46 |
+
percentageDataForScannettVal: 0.5
|
| 47 |
+
|
| 48 |
+
outputStride: 8
|
| 49 |
+
epochSize: 1
|
| 50 |
+
|
| 51 |
+
continueTraining: False
|
| 52 |
+
pathPrevCheckpoint: ""
|
| 53 |
+
initOptimizerFromCheckpoint: False
|
| 54 |
+
loadEpochNumberFromCheckpoint: False
|
| 55 |
+
|
| 56 |
+
saveImageInterval: 5
|
| 57 |
+
saveImageIntervalIter: 200
|
| 58 |
+
testInterval: 1
|
| 59 |
+
saveModelInterval: 5
|
| 60 |
+
|
| 61 |
+
optimizer: "Adam"
|
| 62 |
+
optimAdam:
|
| 63 |
+
learningRate: 0.0001
|
| 64 |
+
weightDecay: 0
|
| 65 |
+
optimSgd:
|
| 66 |
+
learningRate: 1.0e-6
|
| 67 |
+
momentum: 0.9
|
| 68 |
+
weight_decay: 5.0e-4
|
| 69 |
+
lrScheduler: "StepLR"
|
| 70 |
+
lrSchedulerStep:
|
| 71 |
+
step_size: 20
|
| 72 |
+
gamma: 0.95
|
| 73 |
+
lrSchedulerPlateau:
|
| 74 |
+
factor: 0.8
|
| 75 |
+
patience: 25
|
| 76 |
+
verbose: True
|
rftrans/configs/outlines_side_adam_config.yaml
ADDED
|
@@ -0,0 +1,130 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# train.py Config - Training
|
| 2 |
+
train:
|
| 3 |
+
# For datasets, please pass atleast 1 value. If no datasets exist, pass "" as path for images.
|
| 4 |
+
# Synthetic datasets with ground truth labels
|
| 5 |
+
datasetsTrain:
|
| 6 |
+
- images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/RGB'
|
| 7 |
+
labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/boundary'
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
# Synthetic datasets with ground truth labels - 10% split of train
|
| 11 |
+
datasetsVal:
|
| 12 |
+
- images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/RGB'
|
| 13 |
+
labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/boundary'
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
datasetsMatterportTrain:
|
| 17 |
+
# - images: 'data/datasets/matterport3d/train/matterport_rgb/v1/scans'
|
| 18 |
+
# labels: 'data/datasets/matterport3d/train/matterport_render_normal'
|
| 19 |
+
datasetsMatterportVal:
|
| 20 |
+
# - images: 'data/datasets/matterport3d/val/matterport_rgb/v1/scans'
|
| 21 |
+
# labels: 'data/datasets/matterport3d/val/matterport_render_normal'
|
| 22 |
+
datasetsScannetTrain:
|
| 23 |
+
# - images: 'data/datasets/scannet/scannet-rgb/scans/train'
|
| 24 |
+
# labels: 'data/datasets/scannet/scannet_render_normal/train'
|
| 25 |
+
datasetsScannetVal:
|
| 26 |
+
# - images: 'data/datasets/scannet/scannet-rgb/scans/val'
|
| 27 |
+
# labels: 'data/datasets/scannet/scannet_render_normal/val'
|
| 28 |
+
|
| 29 |
+
# Real Images (no ground truth labels)
|
| 30 |
+
datasetsTestReal:
|
| 31 |
+
# - images: 'data/datasets/test/camera-pics/resized-files/preprocessed-rgb-imgs'
|
| 32 |
+
# labels: ''
|
| 33 |
+
# - images: 'data/datasets/test/realsense-captures/resized-files/preprocessed-rgb-imgs'
|
| 34 |
+
# labels: ''
|
| 35 |
+
# - images: 'data/datasets/test/realsense-demo-table-2/source-files/rgb-imgs'
|
| 36 |
+
# labels: ''
|
| 37 |
+
# - images: 'data/datasets/test/realsense-demo-table-3/source-files/rgb-imgs'
|
| 38 |
+
# labels: ''
|
| 39 |
+
|
| 40 |
+
# Synthetic datasets with ground truth labels - Used as test set
|
| 41 |
+
datasetsTestSynthetic:
|
| 42 |
+
- images: 'data/data/datasets/test-synthetic/scoop-val/source-files/rgb-imgs'
|
| 43 |
+
labels: 'data/data/datasets/test-synthetic/scoop-val/source-files/outlines'
|
| 44 |
+
|
| 45 |
+
# Training/Validation Params
|
| 46 |
+
model: "drn" # Possible values: ['deeplab_xception', 'deeplab_resnet', 'drn']
|
| 47 |
+
batchSize: 64
|
| 48 |
+
batchSizeMatterport: 0
|
| 49 |
+
batchSizeScannet: 0
|
| 50 |
+
validationBatchSize: 8
|
| 51 |
+
testBatchSize: 8
|
| 52 |
+
numEpochs: 200
|
| 53 |
+
imgHeight: 256
|
| 54 |
+
imgWidth: 256
|
| 55 |
+
numClasses: 3
|
| 56 |
+
numInputChannels: 3 # Num of channels in input image. RGB = 3 channels, Grayscale = 1 channel.
|
| 57 |
+
numWorkers: 8 # Num of workers used in the dataloader
|
| 58 |
+
logsDir: "logs-deeplab-side" # Directory where logs of each exp will be saved.
|
| 59 |
+
lossFunc: "cosine" # Possible values: ['cosine', 'radians']
|
| 60 |
+
percentageDataForTraining: 1.0 # The percentage of images in dataset to be used for training.
|
| 61 |
+
percentageDataForMatterportTraining: 0.5 # The percentage of images in dataset to be used for training.
|
| 62 |
+
percentageDataForScannetTraining: 0.35
|
| 63 |
+
percentageDataForValidation: 1.0
|
| 64 |
+
percentageDataForMatterportVal: 0.5
|
| 65 |
+
percentageDataForScannettVal: 0.5
|
| 66 |
+
|
| 67 |
+
# Deeplab specific
|
| 68 |
+
outputStride: 8 # Possible values: [8, 16]. Output stride for deeplabv3 model. Smaller values give finer details in output mask.
|
| 69 |
+
epochSize: 1 # After these many epochs, change learning rate
|
| 70 |
+
|
| 71 |
+
continueTraining: True # If true, continue training from a checkpoint
|
| 72 |
+
pathPrevCheckpoint: "/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/cleargrasp_orig/cleargrasp-checkpoints/outlines/checkpoint_outlines.pth" # Path to .pth checkpoint file to load to continue training from
|
| 73 |
+
initOptimizerFromCheckpoint: False # Re-Initialize optimizer's state from checkpoint. NOTE: when this is enabled, value of learningRate will be overridden with value from checkpoint.
|
| 74 |
+
loadEpochNumberFromCheckpoint: False # If true, the epoch/iter numbering will start from the checkpoint's last epoch num.
|
| 75 |
+
|
| 76 |
+
saveImageInterval: 1 # Log output images to tensorboard every saveImageInterval epochs
|
| 77 |
+
saveImageIntervalIter: 100 # Every N iterations, log output images to tensorboard
|
| 78 |
+
testInterval: 1 # Run on test set every nTestInterval epochs. Keep at 0 to skip tests.
|
| 79 |
+
saveModelInterval: 5 # Save the model checkpoints every N epochs
|
| 80 |
+
|
| 81 |
+
# Optimizer Params
|
| 82 |
+
optimAdam:
|
| 83 |
+
learningRate: 0.0001
|
| 84 |
+
weightDecay: 0 # Other values: 0.0001
|
| 85 |
+
optimSgd:
|
| 86 |
+
learningRate: 1e-7
|
| 87 |
+
momentum: 0.9
|
| 88 |
+
weight_decay: 5e-4
|
| 89 |
+
lrScheduler: "StepLR" # Possible Values: ['', 'StepLR', 'ReduceLROnPlateau']
|
| 90 |
+
lrSchedulerStep:
|
| 91 |
+
step_size: 7
|
| 92 |
+
gamma: 0.1
|
| 93 |
+
lrSchedulerPlateau:
|
| 94 |
+
factor: 0.8
|
| 95 |
+
patience: 25
|
| 96 |
+
verbose: True
|
| 97 |
+
|
| 98 |
+
# eval.py Config - Validation/Testing Inference
|
| 99 |
+
eval:
|
| 100 |
+
# Synthetic datasets with ground truth labels
|
| 101 |
+
# Used as validation set
|
| 102 |
+
datasetsSynthetic:
|
| 103 |
+
- images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train_rgmask/RGB'
|
| 104 |
+
labels: ''
|
| 105 |
+
|
| 106 |
+
# Datasets of real images, no labels available
|
| 107 |
+
# Used as Test set
|
| 108 |
+
datasetsReal:
|
| 109 |
+
# - images: "datasets-transparent/studio_pics_sorted/selected_test/d415"
|
| 110 |
+
# labels: "datasets-transparent/studio_pics_sorted/selected_test/d415"
|
| 111 |
+
# - images: "datasets-transparent/studio_pics_sorted/selected_test/d435"
|
| 112 |
+
# labels: "datasets-transparent/studio_pics_sorted/selected_test/d435"
|
| 113 |
+
# - images: "datasets-transparent/studio_pics_sorted/selected_val/d435"
|
| 114 |
+
# labels: "datasets-transparent/studio_pics_sorted/selected_val/d435"
|
| 115 |
+
|
| 116 |
+
datasetsMatterport:
|
| 117 |
+
# - images: 'data/datasets/matterport3d/train/matterport_rgb/v1/scans'
|
| 118 |
+
# labels: 'data/datasets/matterport3d/train/matterport_render_normal'
|
| 119 |
+
|
| 120 |
+
# Params
|
| 121 |
+
model: "drn" # Possible values: ['deeplab_xception', 'deeplab_resnet', 'drn']
|
| 122 |
+
numClasses: 3
|
| 123 |
+
batchSize: 32
|
| 124 |
+
imgHeight: 256
|
| 125 |
+
imgWidth: 256
|
| 126 |
+
os: 8
|
| 127 |
+
numWorkers: 4 # Num of workers used in the dataloader
|
| 128 |
+
pathWeightsFile: "/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/cleargrasp_orig/cleargrasp-checkpoints/outlines/checkpoint_outlines.pth" # Path to the checkpoint to be loaded
|
| 129 |
+
resultsDir: "data/results"
|
| 130 |
+
|
rftrans/configs/refractive_flow_config.yaml
ADDED
|
@@ -0,0 +1,100 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# train.py Config - Training
|
| 2 |
+
train:
|
| 3 |
+
# For datasets, please pass atleast 1 value. If no datasets exist, pass "" as path for images.
|
| 4 |
+
# Synthetic datasets with ground truth labels
|
| 5 |
+
datasetsTrain:
|
| 6 |
+
- images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/RGB'
|
| 7 |
+
labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/flow'
|
| 8 |
+
masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/mask'
|
| 9 |
+
|
| 10 |
+
# Synthetic datasets with ground truth labels - 10% split of train
|
| 11 |
+
datasetsVal:
|
| 12 |
+
- images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/RGB'
|
| 13 |
+
labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/flow'
|
| 14 |
+
masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/mask'
|
| 15 |
+
|
| 16 |
+
# Real Images (no ground truth labels)
|
| 17 |
+
datasetsTestReal:
|
| 18 |
+
|
| 19 |
+
# Synthetic datasets with ground truth labels - Used as test set
|
| 20 |
+
datasetsTestSynthetic:
|
| 21 |
+
|
| 22 |
+
# Training/Validation Params
|
| 23 |
+
model: "drn" # Possible values: ['deeplab_xception', 'deeplab_resnet', 'drn']
|
| 24 |
+
batchSize: 24
|
| 25 |
+
batchSizeMatterport: 0
|
| 26 |
+
batchSizeScannet: 0
|
| 27 |
+
validationBatchSize: 64
|
| 28 |
+
testBatchSize: 16
|
| 29 |
+
numEpochs: 501
|
| 30 |
+
imgHeight: 512
|
| 31 |
+
imgWidth: 512
|
| 32 |
+
numClasses: 2
|
| 33 |
+
numInputChannels: 3 # Num of channels in input image. RGB = 3 channels, Grayscale = 1 channel.
|
| 34 |
+
numWorkers: 4 # Num of workers used in the dataloader
|
| 35 |
+
logsDir: "logs-deeplab" # Directory where logs of each exp will be saved.
|
| 36 |
+
#lossFunc: "cosine" # Possible values: ['cosine', 'radians']
|
| 37 |
+
percentageDataForTraining: 1.0 # The percentage of images in dataset to be used for training.
|
| 38 |
+
percentageDataForMatterportTraining: 0.5 # The percentage of images in dataset to be used for training.
|
| 39 |
+
percentageDataForScannetTraining: 0.35
|
| 40 |
+
percentageDataForValidation: 1.0
|
| 41 |
+
percentageDataForMatterportVal: 0.5
|
| 42 |
+
percentageDataForScannettVal: 0.5
|
| 43 |
+
|
| 44 |
+
# Deeplab specific
|
| 45 |
+
outputStride: 8 # Possible values: [8, 16]. Output stride for deeplabv3 model. Smaller values give finer details in output mask.
|
| 46 |
+
epochSize: 1 # After these many epochs, change learning rate
|
| 47 |
+
|
| 48 |
+
continueTraining: False # If true, continue training from a checkpoint
|
| 49 |
+
pathPrevCheckpoint: "" # Path to .pth checkpoint file to load to continue training from
|
| 50 |
+
initOptimizerFromCheckpoint: False # Re-Initialize optimizer's state from checkpoint. NOTE: when this is enabled, value of learningRate will be overridden with value from checkpoint.
|
| 51 |
+
loadEpochNumberFromCheckpoint: True # If true, the epoch/iter numbering will start from the checkpoint's last epoch num.
|
| 52 |
+
|
| 53 |
+
saveImageInterval: 1 # Log output images to tensorboard every saveImageInterval epochs
|
| 54 |
+
saveImageIntervalIter: 1000 # Every N iterations, log output images to tensorboard
|
| 55 |
+
testInterval: 10 # Run on test set every nTestInterval epochs. Keep at 0 to skip tests.
|
| 56 |
+
saveModelInterval: 10 # Save the model checkpoints every N epochs
|
| 57 |
+
|
| 58 |
+
# Optimizer Params
|
| 59 |
+
optimizer: "Adam" # Possible Values: ["SGD", "Adam"]
|
| 60 |
+
optimAdam:
|
| 61 |
+
learningRate: 5e-5 # Origin value: 0.0001
|
| 62 |
+
weightDecay: 0 # Other values: 0.0001
|
| 63 |
+
optimSgd:
|
| 64 |
+
learningRate: 1e-6
|
| 65 |
+
momentum: 0.9
|
| 66 |
+
weight_decay: 5e-4
|
| 67 |
+
lrScheduler: "StepLR" # Possible Values: ['', 'StepLR', 'ReduceLROnPlateau']
|
| 68 |
+
lrSchedulerStep:
|
| 69 |
+
step_size: 1000
|
| 70 |
+
gamma: 0.1
|
| 71 |
+
lrSchedulerPlateau:
|
| 72 |
+
factor: 0.8
|
| 73 |
+
patience: 25
|
| 74 |
+
verbose: True
|
| 75 |
+
|
| 76 |
+
# eval.py Config - Validation/Testing Inference
|
| 77 |
+
eval:
|
| 78 |
+
# Synthetic datasets with ground truth labels
|
| 79 |
+
# Used as validation set
|
| 80 |
+
datasetsSynthetic:
|
| 81 |
+
- images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/RGB'
|
| 82 |
+
labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/flow'
|
| 83 |
+
masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/mask'
|
| 84 |
+
|
| 85 |
+
# Datasets of real images, no labels available
|
| 86 |
+
# Used as Test set
|
| 87 |
+
datasetsReal:
|
| 88 |
+
|
| 89 |
+
|
| 90 |
+
# Params
|
| 91 |
+
model: "drn" # Possible values: ['deeplab_xception', 'deeplab_resnet', 'drn']
|
| 92 |
+
numClasses: 2
|
| 93 |
+
batchSize: 64
|
| 94 |
+
imgHeight: 512
|
| 95 |
+
imgWidth: 512
|
| 96 |
+
os: 8
|
| 97 |
+
numWorkers: 4 # Num of workers used in the dataloader
|
| 98 |
+
pathWeightsFile: "/home/jiyu/ClearGrasp/pytorch_networks/refractive_flow/logs-deeplab/exp-033/checkpoints/checkpoint-epoch-0500.pth" # Path to the checkpoint to be loaded
|
| 99 |
+
resultsDir: "data/results"
|
| 100 |
+
|