agnoie commited on
Commit
736c1b5
·
verified ·
1 Parent(s): 9068a69

Initial upload: ProgressiveDepth + RFTrans final ckpts and configs

Browse files
README.md CHANGED
@@ -1,3 +1,149 @@
1
- # GridDepth - placeholder
 
 
 
 
 
 
 
 
 
 
 
 
 
2
 
3
- Upload test from progressive depth bot.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ language:
4
+ - en
5
+ tags:
6
+ - depth-completion
7
+ - transparent-objects
8
+ - robotics
9
+ - cleargrasp
10
+ - lidf
11
+ - rftrans
12
+ - transdiff
13
+ library_name: pytorch
14
+ ---
15
 
16
+ # GridDepth: Pretrained Checkpoints for Transparent-Object Depth Completion
17
+
18
+ This repo hosts the **pretrained checkpoints** that go with the
19
+ [atom525/ProgressiveDepth](https://github.com/atom525/ProgressiveDepth) codebase
20
+ (idea.md series-joint pipeline: TransDiff Refined1 → LIDF) **plus** our local
21
+ **RFTrans reproduction** baselines.
22
+
23
+ > **Recipe**: see [atom525/ProgressiveDepth README.md](https://github.com/atom525/ProgressiveDepth)
24
+ > and [docs/PIPELINE.md](https://github.com/atom525/ProgressiveDepth/blob/main/docs/PIPELINE.md).
25
+
26
+ ---
27
+
28
+ ## File layout
29
+
30
+ ```
31
+ GridDepth/
32
+ ├── progressivedepth/ # idea.md 主线(Module A=ip_basic + Module B=LIDF)
33
+ │ ├── ckpts/
34
+ │ │ ├── lidf_stage1_epoch059.pth # 248 MB — LIDF Stage 1 (frozen baseline, CG-only Adam 60 ep)
35
+ │ │ ├── C_stage2_epoch029.pth # 2.2 MB — Stage 2 RefineNet, retrained on Refined1 input (idea.md C run)
36
+ │ │ └── C_stage3_epoch029.pth # 2.2 MB — Stage 3 RefineNet hard-neg, retrained on Refined1 input
37
+ │ └── configs/
38
+ │ ├── train_progressive_stage2.yaml
39
+ │ ├── train_progressive_stage3.yaml
40
+ │ └── pipeline_config.yaml # inference / evaluate config
41
+
42
+ └── rftrans/ # RFTrans 复现产物
43
+ ├── ckpts/
44
+ │ ├── rfnet_refractive_flow_epoch500.pth # 467 MB — RFNet (DRN backbone), Adam 500 ep on unity/train
45
+ │ ├── f2net_flow2normal_epoch500.pth # 356 MB — F2Net (simple_unet), Adam 500 ep on unity/train
46
+ │ ├── mask_adam_epoch195.pth # 312 MB — mask network (DRN), Adam 200 ep on unity/train, mIoU 0.847
47
+ │ └── outlines_side_adam_epoch195.pth # 312 MB — boundary network (DRN side-output), Adam 200 ep on unity/train
48
+ └── configs/
49
+ ├── refractive_flow_config.yaml # RFNet train config (Adam, 500 ep)
50
+ ├── flow2normal_config.yaml # F2Net train config (Adam, 500 ep)
51
+ ├── mask_adam_config.yaml # mask train config (Adam, 200 ep)
52
+ ├── outlines_side_adam_config.yaml # boundary train config (Adam, 200 ep)
53
+ └── exp017_paperfaithful.yaml # rgb2normal e2e config (paper-faithful: SGD 100 ep, lr=1e-4 mom=0.9 wd=5e-4)
54
+ ```
55
+
56
+ ---
57
+
58
+ ## ProgressiveDepth (idea.md series-joint pipeline)
59
+
60
+ Pipeline:
61
+ ```
62
+ RGB + Noisy Depth
63
+
64
+ ▼ Module A: TransDiff Data Preprocessing (ip_basic 多尺度形态学填充)
65
+ Refined Depth1
66
+
67
+ ▼ Module B: LIDF (Stage 1 frozen + Stage 2 / 3 retrained on Refined1)
68
+ Final Depth
69
+ ```
70
+
71
+ ### Final results (paper protocol: 256×144 + per-image avg + corrupt mask)
72
+
73
+ C_full = `lidf_stage1_epoch059.pth` + `C_stage2_epoch029.pth` + `C_stage3_epoch029.pth`,evaluation 用 mode A (feed_to_lidf=refined1):
74
+
75
+ | Dataset | C_full RMSE↓ | C_full δ1.05↑ | B baseline RMSE | B baseline δ1.05 | LIDF paper Table 1 |
76
+ |---|---:|---:|---:|---:|---:|
77
+ | **real-test (Real-novel)** ⭐ | **0.0403** | **45.28** | 0.0443 | 40.18 | 0.0250 / 76.21 |
78
+ | real-val (Real-known) | 0.0351 | 77.22 | 0.0358 | 77.18 | 0.0280 / 82.37 |
79
+ | synthetic-test (Syn-novel) | 0.0328 | 62.82 | 0.0305 | 66.12 | 0.0280 / 68.62 |
80
+ | synthetic-val (Syn-known) | 0.0129 | 93.72 | 0.0111 | 96.07 | 0.0120 / 94.79 |
81
+
82
+ **Conclusion**: idea.md series-joint approach is **effective on real-world data** (Real-novel RMSE ↓9%, δ1.05 ↑5 pts vs baseline B), **regression on synthetic** (where ip_basic adds noise to clean inputs). The remaining gap to paper Table 1 is due to Omniverse Object Dataset being unavailable (link broken since 2025-03, [NVlabs/implicit_depth#3](https://github.com/NVlabs/implicit_depth/issues/3)).
83
+
84
+ ---
85
+
86
+ ## RFTrans reproduction
87
+
88
+ Pipeline (per RFTrans paper §III-C):
89
+ ```
90
+ RGB ──> RFNet ──> refractive flow + mask + boundary
91
+
92
+ └──> F2Net ──> surface normal
93
+
94
+ └──> depth2depth global opt ──> Refined Depth
95
+ ```
96
+
97
+ ### Caveats
98
+
99
+ 1. **Architecture deviation**: paper §III-C says "RFNet predicts mask, boundary, and refractive flow" (multi-task), but the official repo doesn't implement this. We trained **separate networks** (RFNet predicts only flow, F2Net predicts normal from flow, mask & boundary as independent DeepLab+DRN networks) — this matches the actual repo structure but not the paper text.
100
+ 2. **Optimizer deviation**: paper §IV-A specifies SGD lr=1e-4 momentum=0.9 weight_decay=5e-4 for 100 epochs. We used **Adam** for sub-network training because we empirically found SGD lr=1e-4 from random init **does not converge** (mask val mIoU ~0.46 = random level after 100 ep SGD vs 0.85 with Adam 200 ep). The provided `exp017_paperfaithful.yaml` IS paper-faithful (SGD 100 ep) — used for the **end-to-end fine-tuning stage**, where it warm-starts from the Adam-trained RFNet/F2Net.
101
+ 3. **Training data**: all networks trained on `data/unity/train/` (5000 RGB + flow + mask + boundary + normal GT, generated with [Unity-RefractiveFlowRender](https://github.com/LJY-XCX/Unity-RefractiveFlowRender)) — this is the dataset specified by RFTrans paper §IV-A.
102
+
103
+ ### How to use these RFTrans ckpts
104
+
105
+ In your `RFTrans/eval_depth_completion/config_*.yaml`:
106
+ ```yaml
107
+ rgb2flow:
108
+ pathWeightsFile: <path_to>/rfnet_refractive_flow_epoch500.pth
109
+ flow2normal:
110
+ pathWeightsFile: <path_to>/f2net_flow2normal_epoch500.pth
111
+ masks:
112
+ pathWeightsFile: <path_to>/mask_adam_epoch195.pth # OR cleargrasp_orig/.../checkpoint_mask.pth
113
+ outlines:
114
+ pathWeightsFile: <path_to>/outlines_side_adam_epoch195.pth # OR cleargrasp_orig/.../checkpoint_outlines.pth
115
+ ```
116
+
117
+ ---
118
+
119
+ ## Environment / dependencies
120
+
121
+ - python 3.8, pytorch 2.0.0+cu118
122
+ - LIDF: see [implicit_depth/requirements.txt](https://github.com/atom525/ProgressiveDepth/blob/main/implicit_depth/requirements.txt)
123
+ - RFTrans: needs `depth2depth` C++ binary and `libhdf5.so` from conda env
124
+
125
+ ## License
126
+
127
+ - LIDF Stage 1 ckpt and code: NVIDIA Source Code License (Non-Commercial), inherited from [NVlabs/implicit_depth](https://github.com/NVlabs/implicit_depth)
128
+ - RFTrans ckpts and code: inherited from [LJY-XCX/RFTrans](https://github.com/LJY-XCX/RFTrans) license
129
+ - Our extensions (transdiff_preprocess wrapper, train_progressive trainer, retrains): same as upstream
130
+
131
+ ## Citation
132
+
133
+ If you use these ckpts please cite the original works:
134
+
135
+ ```bibtex
136
+ @inproceedings{zhu2021rgbd,
137
+ title={RGB-D Local Implicit Function for Depth Completion of Transparent Objects},
138
+ author={Zhu, Luyang and Mousavian, Arsalan and Xiang, Yu and Mazhar, Hammad and van Eenbergen, Jozef and Debnath, Shoubhik and Fox, Dieter},
139
+ booktitle={CVPR},
140
+ year={2021}
141
+ }
142
+
143
+ @article{tang2024rftrans,
144
+ title={RFTrans: Leveraging Refractive Flow of Transparent Objects for Surface Normal Estimation and Manipulation},
145
+ author={Tang, Tutian and Liu, Jiyu and Zhang, Jieyi and Fu, Haoyuan and Xu, Wenqiang and Lu, Cewu},
146
+ journal={IEEE Robotics and Automation Letters},
147
+ year={2024}
148
+ }
149
+ ```
progressivedepth/ckpts/C_stage2_epoch029.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7d54da2adc3492ac23902d19a586829e136bcc67d1d116ef741f4ba23ca7a554
3
+ size 2272391
progressivedepth/ckpts/C_stage3_epoch029.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0d077188c8c85465169b1f346d7cb6bcf36ac7713389aff1f857c40d4141228e
3
+ size 2272391
progressivedepth/ckpts/lidf_stage1_epoch059.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5db75336cfa1c8ea07251e37a93c17a7f9eeec11daa54615d41752e794446f15
3
+ size 259974039
progressivedepth/configs/pipeline_config.yaml ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ paths:
2
+ cleargrasp_root: /data1/liulingfeng/cooperation/ghy/ProgressiveDepth/implicit_depth/datasets/cleargrasp
3
+ lidf_stage1_ckpt: /data1/liulingfeng/cooperation/ghy/ProgressiveDepth/implicit_depth/logs/lidf/ckpt/bn24_lr0.001_nepo60_Adam_gres8_msn20000_vsn10000_rgb_resnet_embed_ROIAlign_pnet_twostage_offdec_IEF_niter2_probdec_IMNET_scatter_Maxpool_epo6_prob_ray_sn10.0_epo0_cleargrasp/epoch059_network.pth
4
+ lidf_stage2_ckpt: /data1/liulingfeng/cooperation/ghy/ProgressiveDepth/implicit_depth/logs/refine/ckpt/refine_bn24_lr0.001_nepo30_Adam_gres8_msn20000_vsn10000_refine_forward2_perturb0.8_s-0.2_e0.2_prob_ray_sn10.0_epo0_mixed/epoch029_network.pth
5
+ lidf_stage3_ckpt: /data1/liulingfeng/cooperation/ghy/ProgressiveDepth/implicit_depth/logs/refine/ckpt/refine_bn24_lr0.0001_nepo30_Adam_gres8_msn20000_vsn10000_refine_forward2_perturb0.8_s-0.2_e0.2_prob_ray_sn2.0_epo0_hardneg_cleargrasp_progressive_stage3_hardneg_refined1/epoch029_network.pth
6
+ output_dir: /data1/liulingfeng/cooperation/ghy/ProgressiveDepth/results
7
+ transdiff:
8
+ max_depth: 100.0
9
+ blur_type: bilateral
10
+ extrapolate: false
11
+ feed_to_lidf: refined1 # idea.md 默认路径 A:TransDiff Refined1 喂给 LIDF
12
+ lidf:
13
+ use_stage: stage3 # 论文 Table 1 数字对应的就是 LIDF + RefineNet (60 epoch + 30 hardneg)
14
+ miss_sample_num: 20000
15
+ hit_sample_num: 5000
16
+ forward_times: 2
17
+ device: cuda
18
+ gpu_id: 0
19
+ evaluation:
20
+ datasets:
21
+ - real-test
22
+ - real-val
23
+ - synthetic-test
24
+ - synthetic-val
25
+ metrics:
26
+ - rmse
27
+ - abs_rel
28
+ - mae
29
+ - a1
30
+ - a2
31
+ - a3
32
+ min_depth: 0.0
33
+ max_depth: 100.0
34
+ visualization:
35
+ save_vis: true
36
+ num_vis: 10
37
+ vis_types:
38
+ - noisy
39
+ - refined1
40
+ - final
41
+ - gt
42
+ innovations:
43
+ use_mask_driven: false
44
+ mask_driven:
45
+ segment_loss_weight: 0.0
46
+ boundary_loss_weight: 0.0
47
+ use_spvnas: false
48
+ spvnas:
49
+ voxel_size_range:
50
+ - 4
51
+ - 16
52
+ sample_density_range:
53
+ - 8
54
+ - 32
progressivedepth/configs/train_progressive_stage2.yaml ADDED
@@ -0,0 +1,133 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ProgressiveDepth Stage 2 训练配置
2
+ # - LIDF stage1 冻结,从已有 _cleargrasp 60-epoch ckpt 加载
3
+ # - 数据加载阶段在 base dataset 外面套 TransDiffWrappedDataset,
4
+ # 把 depth_corrupt 替换成 ip_basic Refined1,再喂给 LIDF
5
+ # - 30 epoch + lr 0.001(与论文 stage2 完全对齐)
6
+ # - 单 GPU(DDP off)以避免数据多副本对 Refined1 的额外 CPU 压力
7
+
8
+ trainer_name: progressive
9
+ exp_type: train
10
+ base_log_dir: ../logs/refine
11
+ lidf_ckpt_path: /data1/liulingfeng/cooperation/ghy/ProgressiveDepth/implicit_depth/logs/lidf/ckpt/bn24_lr0.001_nepo60_Adam_gres8_msn20000_vsn10000_rgb_resnet_embed_ROIAlign_pnet_twostage_offdec_IEF_niter2_probdec_IMNET_scatter_Maxpool_epo6_prob_ray_sn10.0_epo0_cleargrasp/epoch059_network.pth
12
+ log_name:
13
+ custom_postfix: 'progressive_stage2_refined1'
14
+ resume: latest_network.pth
15
+ gpu_id:
16
+ vis_gpu: '5'
17
+
18
+ mask_type: all
19
+
20
+ # 我们的扩展:TransDiff Refined1 喂 LIDF 的开关
21
+ progressive:
22
+ max_depth: 100.0
23
+ blur_type: bilateral
24
+ extrapolate: false
25
+ enable_train: true
26
+ enable_valid: true
27
+
28
+ # 数据:因 Omniverse 缺失,只用 cleargrasp synthetic train
29
+ dataset:
30
+ type: cleargrasp
31
+ cleargrasp_root_dir: ../datasets/cleargrasp
32
+ omniverse_root_dir: ../datasets/omniverse
33
+ use_data_augmentation: True
34
+ img_width: 320
35
+ img_height: 240
36
+ split_ratio: 0.9
37
+ omni_corrupt_all: True
38
+ corrupt_table: True
39
+ depth_aug: False
40
+ corrupt_all_pix: False
41
+ ellipse_dropout_mean: 20
42
+ ellipse_gamma_shape: 10.0
43
+ ellipse_gamma_scale: 1.0
44
+
45
+ model:
46
+ rgb_model_type: resnet
47
+ rgb_embedding_type: ROIAlign
48
+ rgb_in: 3
49
+ rgb_out: 32
50
+ roi_inp_bbox: 8
51
+ roi_out_bbox: 2
52
+ pnet_model_type: twostage
53
+ pnet_in: 6
54
+ pnet_out: 128
55
+ pnet_gf: 32
56
+ pnet_pos_type: rel
57
+ pos_encode: True
58
+ intersect_pos_type: abs
59
+ multires: 8
60
+ multires_views: 4
61
+ offdec_type: IEF
62
+ n_iter: 2
63
+ probdec_type: IMNET
64
+ imnet_gf: 64
65
+ scatter_type: Maxpool
66
+ maxpool_label_epo: 0
67
+
68
+ refine:
69
+ forward_times: 2
70
+ perturb: True
71
+ perturb_prob: 0.8
72
+ pnet_model_type: twostage
73
+ pnet_in: 6
74
+ pnet_out: 128
75
+ pnet_gf: 32
76
+ pnet_pos_type: rel
77
+ pos_encode: True
78
+ intersect_pos_type: abs
79
+ multires: 8
80
+ multires_views: 4
81
+ offdec_type: IEF
82
+ n_iter: 2
83
+ imnet_gf: 64
84
+ use_sigmoid: False
85
+ offset_range: [-0.2, 0.2]
86
+ use_all_pix: True
87
+
88
+ grid:
89
+ res: 8
90
+ miss_sample_num: 20000
91
+ valid_sample_num: 10000
92
+ offset_range: [0., 1.]
93
+
94
+ training:
95
+ batch_size: 24
96
+ valid_batch_size: 1
97
+ nepochs: 30
98
+ nepoch_decay: 30
99
+ decay_gamma: 0.1
100
+ nepoch_ckpt: 1
101
+ log_interval: 20
102
+ # 关掉可视化(同时被 train_progressive.visualize() override 兜底)
103
+ train_vis_iter: 999999
104
+ val_vis_iter: 999999
105
+ lr: 0.001
106
+ num_workers: 4
107
+ pin_memory: False
108
+ do_valid: True
109
+ valid_start_epo: 0
110
+ optimizer_name: Adam
111
+ scheduler_name: StepLR
112
+
113
+ loss:
114
+ hard_neg: False
115
+ hard_neg_ratio:
116
+ pos_loss_type: single
117
+ pos_w: 100.0
118
+ prob_loss_type: ray
119
+ prob_w: 0
120
+ surf_norm_w: 10.0
121
+ surf_norm_epo: 0
122
+ smooth_w: 0
123
+ smooth_epo: 0
124
+
125
+ dist:
126
+ ddp: False
127
+ dist_url:
128
+ dist_backend:
129
+ nodes_num: 1
130
+ node_rank: 0
131
+ ngpus_per_node: 1
132
+ world_size:
133
+ global_gpu_id:
progressivedepth/configs/train_progressive_stage3.yaml ADDED
@@ -0,0 +1,132 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ProgressiveDepth Stage 3 训练(hard-negative mining)—— 接续 stage2 retrain
2
+ # 在 stage2 retrain 跑完(C run 给出 epoch_029 ckpt)后,用 ip_basic Refined1 输入
3
+ # 续训 30 epoch hard-neg。
4
+ #
5
+ # 严格对齐论文 stage3 协议:lr=0.0001(lr/10),pos_w=20(/5),sn_w=2(/5),
6
+ # loss.hard_neg=True, hard_neg_ratio=0.1(top 10% 误差像素)。
7
+ # checkpoint_path 指向 stage2 retrain 的 epoch_029,避免从 noisy-trained ckpt 暖启。
8
+
9
+ trainer_name: progressive
10
+ exp_type: train
11
+ base_log_dir: ../logs/refine
12
+ lidf_ckpt_path: /data1/liulingfeng/cooperation/ghy/ProgressiveDepth/implicit_depth/logs/lidf/ckpt/bn24_lr0.001_nepo60_Adam_gres8_msn20000_vsn10000_rgb_resnet_embed_ROIAlign_pnet_twostage_offdec_IEF_niter2_probdec_IMNET_scatter_Maxpool_epo6_prob_ray_sn10.0_epo0_cleargrasp/epoch059_network.pth
13
+ checkpoint_path: /data1/liulingfeng/cooperation/ghy/ProgressiveDepth/implicit_depth/logs/refine/ckpt/refine_bn24_lr0.001_nepo30_Adam_gres8_msn20000_vsn10000_refine_forward2_perturb0.8_s-0.2_e0.2_prob_ray_sn10.0_epo0_cleargrasp_progressive_stage2_refined1/epoch029_network.pth
14
+ log_name:
15
+ custom_postfix: 'progressive_stage3_hardneg_refined1'
16
+ resume: latest_network.pth
17
+ gpu_id:
18
+ vis_gpu: '5'
19
+
20
+ mask_type: all
21
+
22
+ progressive:
23
+ max_depth: 100.0
24
+ blur_type: bilateral
25
+ extrapolate: false
26
+ enable_train: true
27
+ enable_valid: true
28
+
29
+ dataset:
30
+ type: cleargrasp
31
+ cleargrasp_root_dir: ../datasets/cleargrasp
32
+ omniverse_root_dir: ../datasets/omniverse
33
+ use_data_augmentation: True
34
+ img_width: 320
35
+ img_height: 240
36
+ split_ratio: 0.9
37
+ omni_corrupt_all: True
38
+ corrupt_table: True
39
+ depth_aug: False
40
+ corrupt_all_pix: False
41
+ ellipse_dropout_mean: 20
42
+ ellipse_gamma_shape: 10.0
43
+ ellipse_gamma_scale: 1.0
44
+
45
+ model:
46
+ rgb_model_type: resnet
47
+ rgb_embedding_type: ROIAlign
48
+ rgb_in: 3
49
+ rgb_out: 32
50
+ roi_inp_bbox: 8
51
+ roi_out_bbox: 2
52
+ pnet_model_type: twostage
53
+ pnet_in: 6
54
+ pnet_out: 128
55
+ pnet_gf: 32
56
+ pnet_pos_type: rel
57
+ pos_encode: True
58
+ intersect_pos_type: abs
59
+ multires: 8
60
+ multires_views: 4
61
+ offdec_type: IEF
62
+ n_iter: 2
63
+ probdec_type: IMNET
64
+ imnet_gf: 64
65
+ scatter_type: Maxpool
66
+ maxpool_label_epo: 0
67
+
68
+ refine:
69
+ forward_times: 2
70
+ perturb: True
71
+ perturb_prob: 0.8
72
+ pnet_model_type: twostage
73
+ pnet_in: 6
74
+ pnet_out: 128
75
+ pnet_gf: 32
76
+ pnet_pos_type: rel
77
+ pos_encode: True
78
+ intersect_pos_type: abs
79
+ multires: 8
80
+ multires_views: 4
81
+ offdec_type: IEF
82
+ n_iter: 2
83
+ imnet_gf: 64
84
+ use_sigmoid: False
85
+ offset_range: [-0.2, 0.2]
86
+ use_all_pix: True
87
+
88
+ grid:
89
+ res: 8
90
+ miss_sample_num: 20000
91
+ valid_sample_num: 10000
92
+ offset_range: [0., 1.]
93
+
94
+ training:
95
+ batch_size: 24
96
+ valid_batch_size: 1
97
+ nepochs: 30
98
+ nepoch_decay: 30
99
+ decay_gamma: 0.1
100
+ nepoch_ckpt: 1
101
+ log_interval: 20
102
+ train_vis_iter: 999999
103
+ val_vis_iter: 999999
104
+ lr: 0.0001
105
+ num_workers: 4
106
+ pin_memory: False
107
+ do_valid: True
108
+ valid_start_epo: 0
109
+ optimizer_name: Adam
110
+ scheduler_name: StepLR
111
+
112
+ loss:
113
+ hard_neg: True
114
+ hard_neg_ratio: 0.1
115
+ pos_loss_type: single
116
+ pos_w: 20.0
117
+ prob_loss_type: ray
118
+ prob_w: 0
119
+ surf_norm_w: 2.0
120
+ surf_norm_epo: 0
121
+ smooth_w: 0
122
+ smooth_epo: 0
123
+
124
+ dist:
125
+ ddp: False
126
+ dist_url:
127
+ dist_backend:
128
+ nodes_num: 1
129
+ node_rank: 0
130
+ ngpus_per_node: 1
131
+ world_size:
132
+ global_gpu_id:
rftrans/ckpts/f2net_flow2normal_epoch500.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:96b61fe6255bb985d39513a7c24deb353519d66cf3a1b81877f03754dd14365d
3
+ size 372605797
rftrans/ckpts/mask_adam_epoch195.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1a63181d9ca1a1e4f48a5a9bc6cb563367246b3716df54f56cda401ae948ba5b
3
+ size 326305765
rftrans/ckpts/outlines_side_adam_epoch195.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0781611ccaa86c04012ba366ff8e8cf51039fee4eeb330527d3bc5bcb4a7db27
3
+ size 326308517
rftrans/ckpts/rfnet_refractive_flow_epoch500.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a5b75cf2233aab6a637c4a5676ebb3562489bea1e35423c703fd72a8e34dbc03
3
+ size 489345509
rftrans/configs/exp017_paperfaithful.yaml ADDED
@@ -0,0 +1,109 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # RFTrans Table II reproduction - end-to-end fine-tune with paper-faithful loss
2
+ # Bug fix: training previously used loss = L_flow + alpha*L_norm with alpha=0.02
3
+ # (flow dominates ~100x, normal supervision essentially ignored).
4
+ # Paper Eq.: L = alpha * L_flow + L_norm with alpha=0.01 (best per Table IV).
5
+ train:
6
+ datasetsTrain:
7
+ - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train_cg/RGB'
8
+ flows: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train_cg/flow'
9
+ labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train_cg/normal'
10
+ masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train_cg/mask'
11
+
12
+ datasetsVal:
13
+ - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/RGB'
14
+ flows: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/flow'
15
+ labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/normal'
16
+ masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/mask'
17
+
18
+ datasetsTestReal:
19
+ datasetsTestSynthetic:
20
+
21
+ rgb2flow:
22
+ model: "drn"
23
+ numClasses: 2
24
+ numInputChannels: 3
25
+ pathWeightsFile: "/data1/liulingfeng/cooperation/ghy/RFTrans-repro/RFTrans/pytorch_networks/refractive_flow/logs-deeplab/exp-001/checkpoints/checkpoint-epoch-0500.pth"
26
+
27
+ flow2normal:
28
+ model: "simple_unet"
29
+ numClasses: 3
30
+ numInputChannels: 3
31
+ pathWeightsFile: "/data1/liulingfeng/cooperation/ghy/RFTrans-repro/RFTrans/pytorch_networks/flow2normal/logs-deeplab/exp-000/checkpoints/checkpoint-epoch-0500.pth"
32
+
33
+ model: "drn"
34
+ batchSize: 8
35
+ validationBatchSize: 8
36
+ testBatchSize: 8
37
+ numEpochs: 100
38
+ # Train at data's native 512x512 (matches the pretrained R2F/F2N which learned
39
+ # flow magnitudes in the 512-frame). Paper says 256x256 but our train_cg data
40
+ # is rendered at 512x512 and the pretrained checkpoints were trained at 512.
41
+ imgHeight: 512
42
+ imgWidth: 512
43
+ numWorkers: 8
44
+ logsDir: "logs-deeplab"
45
+ lossFunc: "cosine"
46
+ percentageDataForTraining: 1.0
47
+ percentageDataForValidation: 0.5
48
+
49
+ outputStride: 8
50
+ epochSize: 1
51
+
52
+ # initialize from previously trained separate checkpoints (RFNet / F2Net)
53
+ continueTraining: True
54
+ pathPrevCheckpoint: ""
55
+ initOptimizerFromCheckpoint: False
56
+ loadEpochNumberFromCheckpoint: False
57
+
58
+ saveImageInterval: 1
59
+ saveImageIntervalIter: 1000
60
+ testInterval: 1
61
+ saveModelInterval: 1
62
+
63
+ # NOTE: paper specifies SGD lr=1e-4 momentum 0.9 weight_decay 5e-4 for 100 ep,
64
+ # but fine-tuning from already-Adam-trained pretrained R2F/F2N with SGD@1e-4
65
+ # barely moves the weights. Use Adam @ 1e-4 to converge faster from the
66
+ # pretrained init (loss bug previously suppressed the normal supervision).
67
+ optimizer: "SGD"
68
+ optimSgd:
69
+ learningRate: 1e-4
70
+ momentum: 0.9
71
+ weight_decay: 5e-4
72
+ optimAdam:
73
+ learningRate: 1e-4
74
+ weightDecay: 0.0001
75
+ lrScheduler: "StepLR"
76
+ lrSchedulerStep:
77
+ step_size: 30
78
+ gamma: 0.5
79
+
80
+ # alpha for joint loss; paper best = 0.01 with L = alpha * L_flow + L_norm
81
+ loss_alpha: 0.01
82
+
83
+ eval:
84
+ datasetsSynthetic:
85
+ - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/RGB'
86
+ flows: ''
87
+ labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/normal'
88
+ masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/mask'
89
+
90
+ datasetsReal:
91
+
92
+ rgb2flow:
93
+ model: "drn"
94
+ numClasses: 2
95
+ numInputChannels: 3
96
+ pathWeightsFile: ""
97
+
98
+ flow2normal:
99
+ model: "simple_unet"
100
+ numClasses: 3
101
+ numInputChannels: 3
102
+ pathWeightsFile: ""
103
+
104
+ model: "drn"
105
+ batchSize: 4
106
+ imgHeight: 256
107
+ imgWidth: 256
108
+ numWorkers: 4
109
+ resultsDir: "data/results"
rftrans/configs/flow2normal_config.yaml ADDED
@@ -0,0 +1,120 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # train.py Config - Training
2
+ train:
3
+ # For datasets, please pass atleast 1 value. If no datasets exist, pass "" as path for images.
4
+ # Synthetic datasets with ground truth labels
5
+ datasetsTrain:
6
+
7
+ - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/flow'
8
+ labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/normal'
9
+ masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/mask'
10
+
11
+ # - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/predicted_flows'
12
+ # labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/normal'
13
+ # masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/mask'
14
+
15
+
16
+ # Synthetic datasets with ground truth labels - 10% split of train
17
+ datasetsVal:
18
+ - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/flow'
19
+ labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/normal'
20
+ masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/mask'
21
+ #
22
+ # - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/predicted_flows'
23
+ # labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/normal'
24
+ # masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/mask'
25
+
26
+ # Real Images (no ground truth labels)
27
+ datasetsTestReal:
28
+ # - images: 'data/datasets/test/camera-pics/resized-files/preprocessed-rgb-imgs'
29
+ # labels: ''
30
+ # - images: 'data/datasets/test/realsense-captures/resized-files/preprocessed-rgb-imgs'
31
+ # labels: ''
32
+ # - images: 'data/datasets/test/realsense-demo-table-2/source-files/rgb-imgs'
33
+ # labels: ''
34
+ # - images: 'data/datasets/test/realsense-demo-table-3/source-files/rgb-imgs'
35
+ # labels: ''
36
+
37
+ # Synthetic datasets with ground truth labels - Used as test set
38
+ datasetsTestSynthetic:
39
+ # - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/calibration/refractive_flow'
40
+ # labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/normal'
41
+
42
+ # Training/Validation Params
43
+ model: "simple_unet" # Possible values: ["drn", unet", "deeplab_xception", "deeplab_resnet", "refinenet", "simple_unet"]
44
+ batchSize: 16
45
+ batchSizeMatterport: 0
46
+ batchSizeScannet: 0
47
+ validationBatchSize: 16
48
+ testBatchSize: 16
49
+ numEpochs: 501
50
+ imgHeight: 512
51
+ imgWidth: 512
52
+ numClasses: 3
53
+ numInputChannels: 3 # Num of channels in input image. RGB = 3 channels, Grayscale = 1 channel.
54
+ numWorkers: 16 # Num of workers used in the dataloader
55
+ logsDir: "logs-deeplab" # Directory where logs of each exp will be saved.
56
+ lossFunc: "cosine" # Possible values: ['cosine', 'radians']
57
+ percentageDataForTraining: 1.0 # The percentage of images in dataset to be used for training.
58
+ percentageDataForMatterportTraining: 0.5 # The percentage of images in dataset to be used for training.
59
+ percentageDataForScannetTraining: 0.35
60
+ percentageDataForValidation: 0.25
61
+ percentageDataForMatterportVal: 0.5
62
+ percentageDataForScannettVal: 0.5
63
+
64
+ # Deeplab specific
65
+ outputStride: 8 # Possible values: [8, 16]. Output stride for deeplabv3 model. Smaller values give finer details in output mask.
66
+ epochSize: 1 # After these many epochs, change learning rate
67
+
68
+ continueTraining: False # If true, continue training from a checkpoint
69
+ pathPrevCheckpoint: "/home/jiyu/ClearGrasp/pytorch_networks/refraction_flows/logs-deeplab/exp-003/checkpoints-adam/checkpoint-epoch-0020.pth" # Path to .pth checkpoint file to load to continue training from
70
+ initOptimizerFromCheckpoint: False # Re-Initialize optimizer's state from checkpoint. NOTE: when this is enabled, value of learningRate will be overridden with value from checkpoint.
71
+ loadEpochNumberFromCheckpoint: False # If true, the epoch/iter numbering will start from the checkpoint's last epoch num.
72
+
73
+ saveImageInterval: 1 # Log output images to tensorboard every saveImageInterval epochs
74
+ saveImageIntervalIter: 100 # Every N iterations, log output images to tensorboard
75
+ testInterval: 1 # Run on test set every nTestInterval epochs. Keep at 0 to skip tests.
76
+ saveModelInterval: 10 # Save the model checkpoints every N epochs
77
+
78
+ # Optimizer Params
79
+ optimizer: "Adam" # Possible Values: ["SGD", "Adam"]
80
+ optimAdam:
81
+ learningRate: 5e-5
82
+ weightDecay: 0.0001 # Other values: 0.0001
83
+ optimSgd:
84
+ learningRate: 1e-6
85
+ momentum: 0.9
86
+ weight_decay: 5e-4
87
+ lrScheduler: "StepLR" # Possible Values: ['', 'StepLR', 'ReduceLROnPlateau']
88
+ lrSchedulerStep:
89
+ step_size: 1000
90
+ gamma: 0.1
91
+ lrSchedulerPlateau:
92
+ factor: 0.8
93
+ patience: 25
94
+ verbose: True
95
+
96
+ # eval.py Config - Validation/Testing Inference
97
+ eval:
98
+ # Synthetic datasets with ground truth labels
99
+ # Used as validation set
100
+ datasetsSynthetic:
101
+ - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/flow'
102
+ labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/normal'
103
+ masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/mask'
104
+
105
+ # Datasets of real images, no labels available
106
+ # Used as Test set
107
+ datasetsReal:
108
+
109
+
110
+ # Params
111
+ model: "simple_unet" # Possible values: ["drn", unet", "deeplab_xception", "deeplab_resnet", "refinenet", "simple_unet"]
112
+ numClasses: 3
113
+ batchSize: 16
114
+ imgHeight: 512
115
+ imgWidth: 512
116
+ os: 8
117
+ numWorkers: 4 # Num of workers used in the dataloader
118
+ pathWeightsFile: "/home/jiyu/ClearGrasp/pytorch_networks/flow2normal/logs-deeplab/exp-012/checkpoints/checkpoint-epoch-0500.pth" # Path to the checkpoint to be loaded
119
+ resultsDir: "data/results/"
120
+
rftrans/configs/mask_adam_config.yaml ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # RFTrans mask network training (binary semantic segmentation: transparent vs background)
2
+ # 数据:unity/train/RGB + unity/train/mask(5000 张,512×512)
3
+ # 架构:DeepLab + DRN backbone + cross_entropy2d,跟 occlusion_boundaries 一致
4
+ # 目标:替换掉 ClearGrasp 2019 老 ckpt,看是否能改进 RFTrans pipeline 的最终 RMSE
5
+ train:
6
+ # 训练集
7
+ datasetsTrain:
8
+ - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/RGB'
9
+ labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/mask'
10
+
11
+ # 验证集(10% 切出)
12
+ datasetsVal:
13
+ - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/RGB'
14
+ labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/mask'
15
+
16
+ # 不用 matterport / scannet / 真实集
17
+ datasetsMatterportTrain:
18
+ datasetsMatterportVal:
19
+ datasetsScannetTrain:
20
+ datasetsScannetVal:
21
+ datasetsTestReal:
22
+ datasetsTestSynthetic:
23
+ - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/RGB'
24
+ labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/mask'
25
+
26
+ # 训练超参(与 occlusion_boundaries 默认对齐)
27
+ model: "drn"
28
+ batchSize: 32
29
+ batchSizeMatterport: 0
30
+ batchSizeScannet: 0
31
+ validationBatchSize: 8
32
+ testBatchSize: 8
33
+ numEpochs: 200 # mask 任务相对简单,200 epoch 够
34
+ imgHeight: 256
35
+ imgWidth: 256
36
+ numClasses: 2 # binary: bg(0) / transparent(1)
37
+ numInputChannels: 3
38
+ numWorkers: 8
39
+ logsDir: "logs-deeplab"
40
+ lossFunc: "ce" # cross-entropy 2d,模型用 deeplab/drn 时 train.py 走 utils.cross_entropy2d
41
+ percentageDataForTraining: 1.0
42
+ percentageDataForMatterportTraining: 0.5
43
+ percentageDataForScannetTraining: 0.35
44
+ percentageDataForValidation: 1.0
45
+ percentageDataForMatterportVal: 0.5
46
+ percentageDataForScannettVal: 0.5
47
+
48
+ outputStride: 8
49
+ epochSize: 1
50
+
51
+ continueTraining: False
52
+ pathPrevCheckpoint: ""
53
+ initOptimizerFromCheckpoint: False
54
+ loadEpochNumberFromCheckpoint: False
55
+
56
+ saveImageInterval: 5
57
+ saveImageIntervalIter: 200
58
+ testInterval: 1
59
+ saveModelInterval: 5
60
+
61
+ optimizer: "Adam"
62
+ optimAdam:
63
+ learningRate: 0.0001
64
+ weightDecay: 0
65
+ optimSgd:
66
+ learningRate: 1.0e-6
67
+ momentum: 0.9
68
+ weight_decay: 5.0e-4
69
+ lrScheduler: "StepLR"
70
+ lrSchedulerStep:
71
+ step_size: 20
72
+ gamma: 0.95
73
+ lrSchedulerPlateau:
74
+ factor: 0.8
75
+ patience: 25
76
+ verbose: True
rftrans/configs/outlines_side_adam_config.yaml ADDED
@@ -0,0 +1,130 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # train.py Config - Training
2
+ train:
3
+ # For datasets, please pass atleast 1 value. If no datasets exist, pass "" as path for images.
4
+ # Synthetic datasets with ground truth labels
5
+ datasetsTrain:
6
+ - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/RGB'
7
+ labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/boundary'
8
+
9
+
10
+ # Synthetic datasets with ground truth labels - 10% split of train
11
+ datasetsVal:
12
+ - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/RGB'
13
+ labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/boundary'
14
+
15
+
16
+ datasetsMatterportTrain:
17
+ # - images: 'data/datasets/matterport3d/train/matterport_rgb/v1/scans'
18
+ # labels: 'data/datasets/matterport3d/train/matterport_render_normal'
19
+ datasetsMatterportVal:
20
+ # - images: 'data/datasets/matterport3d/val/matterport_rgb/v1/scans'
21
+ # labels: 'data/datasets/matterport3d/val/matterport_render_normal'
22
+ datasetsScannetTrain:
23
+ # - images: 'data/datasets/scannet/scannet-rgb/scans/train'
24
+ # labels: 'data/datasets/scannet/scannet_render_normal/train'
25
+ datasetsScannetVal:
26
+ # - images: 'data/datasets/scannet/scannet-rgb/scans/val'
27
+ # labels: 'data/datasets/scannet/scannet_render_normal/val'
28
+
29
+ # Real Images (no ground truth labels)
30
+ datasetsTestReal:
31
+ # - images: 'data/datasets/test/camera-pics/resized-files/preprocessed-rgb-imgs'
32
+ # labels: ''
33
+ # - images: 'data/datasets/test/realsense-captures/resized-files/preprocessed-rgb-imgs'
34
+ # labels: ''
35
+ # - images: 'data/datasets/test/realsense-demo-table-2/source-files/rgb-imgs'
36
+ # labels: ''
37
+ # - images: 'data/datasets/test/realsense-demo-table-3/source-files/rgb-imgs'
38
+ # labels: ''
39
+
40
+ # Synthetic datasets with ground truth labels - Used as test set
41
+ datasetsTestSynthetic:
42
+ - images: 'data/data/datasets/test-synthetic/scoop-val/source-files/rgb-imgs'
43
+ labels: 'data/data/datasets/test-synthetic/scoop-val/source-files/outlines'
44
+
45
+ # Training/Validation Params
46
+ model: "drn" # Possible values: ['deeplab_xception', 'deeplab_resnet', 'drn']
47
+ batchSize: 64
48
+ batchSizeMatterport: 0
49
+ batchSizeScannet: 0
50
+ validationBatchSize: 8
51
+ testBatchSize: 8
52
+ numEpochs: 200
53
+ imgHeight: 256
54
+ imgWidth: 256
55
+ numClasses: 3
56
+ numInputChannels: 3 # Num of channels in input image. RGB = 3 channels, Grayscale = 1 channel.
57
+ numWorkers: 8 # Num of workers used in the dataloader
58
+ logsDir: "logs-deeplab-side" # Directory where logs of each exp will be saved.
59
+ lossFunc: "cosine" # Possible values: ['cosine', 'radians']
60
+ percentageDataForTraining: 1.0 # The percentage of images in dataset to be used for training.
61
+ percentageDataForMatterportTraining: 0.5 # The percentage of images in dataset to be used for training.
62
+ percentageDataForScannetTraining: 0.35
63
+ percentageDataForValidation: 1.0
64
+ percentageDataForMatterportVal: 0.5
65
+ percentageDataForScannettVal: 0.5
66
+
67
+ # Deeplab specific
68
+ outputStride: 8 # Possible values: [8, 16]. Output stride for deeplabv3 model. Smaller values give finer details in output mask.
69
+ epochSize: 1 # After these many epochs, change learning rate
70
+
71
+ continueTraining: True # If true, continue training from a checkpoint
72
+ pathPrevCheckpoint: "/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/cleargrasp_orig/cleargrasp-checkpoints/outlines/checkpoint_outlines.pth" # Path to .pth checkpoint file to load to continue training from
73
+ initOptimizerFromCheckpoint: False # Re-Initialize optimizer's state from checkpoint. NOTE: when this is enabled, value of learningRate will be overridden with value from checkpoint.
74
+ loadEpochNumberFromCheckpoint: False # If true, the epoch/iter numbering will start from the checkpoint's last epoch num.
75
+
76
+ saveImageInterval: 1 # Log output images to tensorboard every saveImageInterval epochs
77
+ saveImageIntervalIter: 100 # Every N iterations, log output images to tensorboard
78
+ testInterval: 1 # Run on test set every nTestInterval epochs. Keep at 0 to skip tests.
79
+ saveModelInterval: 5 # Save the model checkpoints every N epochs
80
+
81
+ # Optimizer Params
82
+ optimAdam:
83
+ learningRate: 0.0001
84
+ weightDecay: 0 # Other values: 0.0001
85
+ optimSgd:
86
+ learningRate: 1e-7
87
+ momentum: 0.9
88
+ weight_decay: 5e-4
89
+ lrScheduler: "StepLR" # Possible Values: ['', 'StepLR', 'ReduceLROnPlateau']
90
+ lrSchedulerStep:
91
+ step_size: 7
92
+ gamma: 0.1
93
+ lrSchedulerPlateau:
94
+ factor: 0.8
95
+ patience: 25
96
+ verbose: True
97
+
98
+ # eval.py Config - Validation/Testing Inference
99
+ eval:
100
+ # Synthetic datasets with ground truth labels
101
+ # Used as validation set
102
+ datasetsSynthetic:
103
+ - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train_rgmask/RGB'
104
+ labels: ''
105
+
106
+ # Datasets of real images, no labels available
107
+ # Used as Test set
108
+ datasetsReal:
109
+ # - images: "datasets-transparent/studio_pics_sorted/selected_test/d415"
110
+ # labels: "datasets-transparent/studio_pics_sorted/selected_test/d415"
111
+ # - images: "datasets-transparent/studio_pics_sorted/selected_test/d435"
112
+ # labels: "datasets-transparent/studio_pics_sorted/selected_test/d435"
113
+ # - images: "datasets-transparent/studio_pics_sorted/selected_val/d435"
114
+ # labels: "datasets-transparent/studio_pics_sorted/selected_val/d435"
115
+
116
+ datasetsMatterport:
117
+ # - images: 'data/datasets/matterport3d/train/matterport_rgb/v1/scans'
118
+ # labels: 'data/datasets/matterport3d/train/matterport_render_normal'
119
+
120
+ # Params
121
+ model: "drn" # Possible values: ['deeplab_xception', 'deeplab_resnet', 'drn']
122
+ numClasses: 3
123
+ batchSize: 32
124
+ imgHeight: 256
125
+ imgWidth: 256
126
+ os: 8
127
+ numWorkers: 4 # Num of workers used in the dataloader
128
+ pathWeightsFile: "/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/cleargrasp_orig/cleargrasp-checkpoints/outlines/checkpoint_outlines.pth" # Path to the checkpoint to be loaded
129
+ resultsDir: "data/results"
130
+
rftrans/configs/refractive_flow_config.yaml ADDED
@@ -0,0 +1,100 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # train.py Config - Training
2
+ train:
3
+ # For datasets, please pass atleast 1 value. If no datasets exist, pass "" as path for images.
4
+ # Synthetic datasets with ground truth labels
5
+ datasetsTrain:
6
+ - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/RGB'
7
+ labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/flow'
8
+ masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/train/mask'
9
+
10
+ # Synthetic datasets with ground truth labels - 10% split of train
11
+ datasetsVal:
12
+ - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/RGB'
13
+ labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/flow'
14
+ masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/mask'
15
+
16
+ # Real Images (no ground truth labels)
17
+ datasetsTestReal:
18
+
19
+ # Synthetic datasets with ground truth labels - Used as test set
20
+ datasetsTestSynthetic:
21
+
22
+ # Training/Validation Params
23
+ model: "drn" # Possible values: ['deeplab_xception', 'deeplab_resnet', 'drn']
24
+ batchSize: 24
25
+ batchSizeMatterport: 0
26
+ batchSizeScannet: 0
27
+ validationBatchSize: 64
28
+ testBatchSize: 16
29
+ numEpochs: 501
30
+ imgHeight: 512
31
+ imgWidth: 512
32
+ numClasses: 2
33
+ numInputChannels: 3 # Num of channels in input image. RGB = 3 channels, Grayscale = 1 channel.
34
+ numWorkers: 4 # Num of workers used in the dataloader
35
+ logsDir: "logs-deeplab" # Directory where logs of each exp will be saved.
36
+ #lossFunc: "cosine" # Possible values: ['cosine', 'radians']
37
+ percentageDataForTraining: 1.0 # The percentage of images in dataset to be used for training.
38
+ percentageDataForMatterportTraining: 0.5 # The percentage of images in dataset to be used for training.
39
+ percentageDataForScannetTraining: 0.35
40
+ percentageDataForValidation: 1.0
41
+ percentageDataForMatterportVal: 0.5
42
+ percentageDataForScannettVal: 0.5
43
+
44
+ # Deeplab specific
45
+ outputStride: 8 # Possible values: [8, 16]. Output stride for deeplabv3 model. Smaller values give finer details in output mask.
46
+ epochSize: 1 # After these many epochs, change learning rate
47
+
48
+ continueTraining: False # If true, continue training from a checkpoint
49
+ pathPrevCheckpoint: "" # Path to .pth checkpoint file to load to continue training from
50
+ initOptimizerFromCheckpoint: False # Re-Initialize optimizer's state from checkpoint. NOTE: when this is enabled, value of learningRate will be overridden with value from checkpoint.
51
+ loadEpochNumberFromCheckpoint: True # If true, the epoch/iter numbering will start from the checkpoint's last epoch num.
52
+
53
+ saveImageInterval: 1 # Log output images to tensorboard every saveImageInterval epochs
54
+ saveImageIntervalIter: 1000 # Every N iterations, log output images to tensorboard
55
+ testInterval: 10 # Run on test set every nTestInterval epochs. Keep at 0 to skip tests.
56
+ saveModelInterval: 10 # Save the model checkpoints every N epochs
57
+
58
+ # Optimizer Params
59
+ optimizer: "Adam" # Possible Values: ["SGD", "Adam"]
60
+ optimAdam:
61
+ learningRate: 5e-5 # Origin value: 0.0001
62
+ weightDecay: 0 # Other values: 0.0001
63
+ optimSgd:
64
+ learningRate: 1e-6
65
+ momentum: 0.9
66
+ weight_decay: 5e-4
67
+ lrScheduler: "StepLR" # Possible Values: ['', 'StepLR', 'ReduceLROnPlateau']
68
+ lrSchedulerStep:
69
+ step_size: 1000
70
+ gamma: 0.1
71
+ lrSchedulerPlateau:
72
+ factor: 0.8
73
+ patience: 25
74
+ verbose: True
75
+
76
+ # eval.py Config - Validation/Testing Inference
77
+ eval:
78
+ # Synthetic datasets with ground truth labels
79
+ # Used as validation set
80
+ datasetsSynthetic:
81
+ - images: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/RGB'
82
+ labels: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/flow'
83
+ masks: '/data1/liulingfeng/cooperation/ghy/RFTrans-repro/data/unity/valid/mask'
84
+
85
+ # Datasets of real images, no labels available
86
+ # Used as Test set
87
+ datasetsReal:
88
+
89
+
90
+ # Params
91
+ model: "drn" # Possible values: ['deeplab_xception', 'deeplab_resnet', 'drn']
92
+ numClasses: 2
93
+ batchSize: 64
94
+ imgHeight: 512
95
+ imgWidth: 512
96
+ os: 8
97
+ numWorkers: 4 # Num of workers used in the dataloader
98
+ pathWeightsFile: "/home/jiyu/ClearGrasp/pytorch_networks/refractive_flow/logs-deeplab/exp-033/checkpoints/checkpoint-epoch-0500.pth" # Path to the checkpoint to be loaded
99
+ resultsDir: "data/results"
100
+