_________________________________
|:::::::::::::;;::::::::::::::::::|
|:::::::::::'~||~~~``:::::::::::::|
|::::::::' .': o`:::::::::::|
|:::::::' oo | |o o ::::::::::|
|::::::: 8 .'.' 8 o :::::::::|
|::::::: 8 | | 8 :::::::::| ishu@ishu-Dell-G15-5530
|::::::: _._| |_,...8 :::::::::| -----------------------
|::::::'~--. .--. `. `::::::::| Star system: Linux
|:::::' =8 ~ \ o ::::::::| Flagship: Dell G15 5530
|::::' 8._ 88. \ o::::::::| Reactor core: 6.17.0-1012-oem
|:::' __. ,.ooo~~. \ o`::::::| Mission Duration: 2 days, 8 ho
|::: . -. 88`78o/: \ `:::::| Imperial Assets: 2178 (dpkg),
|::' /. o o \ :: \88`::::| "All I am surrounded by is Command Interface: bash 5.2.21
|:; o|| 8 8 |d. `8 `:::| Fear and Dead Men" Deflector Shields: 1920x1080
|:. - ^ ^ -' `-`::| Targeting System: Cinnamon 6.6
|::. .:::| Holonet Uplink: gnome-terminal
|:::::..... ::' ``::| CPU: 13th Gen Intel i5-13450HX
|::::::::-'`- 88 `| GPU: NVIDIA GeForce RTX 3050 6
|:::::-'. - :: | GPU: Intel Raptor Lake-S UHD G
|:-~. . . : | Memory: 6406MiB / 31779MiB
| .. . ..: o:8 88o |
|. . ::: 8:P d888. . . |
|. . :88 88 888' . . |
| o8 d88P . 88 ' d88P .. |
| 88P 888 d8P ' 888 |
| 8 d88P.'d:8 .- dP~ o8 |
| 888 888 d~ o888 |
|_________________________________|
Darth-Ishu@Executor:~$ conda activate ghost_test
(ghost_test) Darth-Ishu@Executor:~$ ghost help
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣀⣤⣤⡤⣤⢤⣤⣄⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣠⡾⠛⠉⠀⠀⠀⠀⠀⠀⠉⠻⣦⡄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣾⠏⠀⠀⠀⠀⣀⡀⠀⠀⠀⠀⠀⠈⢻⡆⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣼⠯⠀⠀⠀⢰⣿⣿⣿⣆⡀⣴⣾⣿⣦⡈⢿⡄⠀ G H O S T
⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⣏⠀⠀⠀⠀⢸⣿⣿⣿⡿⠇⣿⣿⣿⣿⠆⠘⣷⡀ Generalizable Hyperspectral
⠀⠀⠀⠀⠀⠀⠀⠀⠀⣿⠇⠀⠀⠀⠀⠈⠿⠿⡿⠃⠀⢿⣿⣿⠏⠀⠀⢹⣗ Observation & Segmentation
⠀⠀⠀⠀⠀⠀⠀⠀⢸⡏⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠀⠀⠀⠀⠀⣿⡀ Toolkit
Unrecognized command: 'help'
Available commands: train | train_spt | predict | visualize | demo | version | flower
Use 'ghost --version' or 'ghost version' to check installed version
usage: ghost <command> [<args>]
GHOST: Generalizable Hyperspectral Observation & Segmentation Toolkit
positional arguments:
command Subcommand: train | train_spt | predict | visualize | demo | version | flower
options:
-h, --help show this help message and exit
-v, --version show program's version number and exit
(ghost_test) Darth-Ishu@Executor:~$ ghost demo
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣀⣤⣤⡤⣤⢤⣤⣄⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣠⡾⠛⠉⠀⠀⠀⠀⠀⠀⠉⠻⣦⡄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣾⠏⠀⠀⠀⠀⣀⡀⠀⠀⠀⠀⠀⠈⢻⡆⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣼⠯⠀⠀⠀⢰⣿⣿⣿⣆⡀⣴⣾⣿⣦⡈⢿⡄⠀ G H O S T
⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⣏⠀⠀⠀⠀⢸⣿⣿⣿⡿⠇⣿⣿⣿⣿⠆⠘⣷⡀ Generalizable Hyperspectral
⠀⠀⠀⠀⠀⠀⠀⠀⠀⣿⠇⠀⠀⠀⠀⠈⠿⠿⡿⠃⠀⢿⣿⣿⠏⠀⠀⢹⣗ Observation & Segmentation
⠀⠀⠀⠀⠀⠀⠀⠀⢸⡏⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠀⠀⠀⠀⠀⣿⡀ Toolkit
Bundled Indian Pines dataset:
Data : /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_corrected.mat
GT : /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_gt.mat
Run this to start training:
ghost train_spt \
--data /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_corrected.mat \
--gt /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_gt.mat \
--loss dice \
--base_filters 32 --num_filters 8 \
--ensembles 5 --leaf_ensembles 3 \
--epochs 400 --patience 50 --min_epochs 40 \
--out-dir runs/indian_pines
(ghost_test) Darth-Ishu@Executor:~$ ghost train_spt \
--data /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_corrected.mat \
--gt /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_gt.mat \
--loss dice \
--base_filters 32 --num_filters 8 \
--ensembles 5 --leaf_ensembles 3 \
--epochs 400 --patience 50 --min_epochs 40 \
--out-dir runs/indian_pines
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣀⣤⣤⡤⣤⢤⣤⣄⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣠⡾⠛⠉⠀⠀⠀⠀⠀⠀⠉⠻⣦⡄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣾⠏⠀⠀⠀⠀⣀⡀⠀⠀⠀⠀⠀⠈⢻⡆⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣼⠯⠀⠀⠀⢰⣿⣿⣿⣆⡀⣴⣾⣿⣦⡈⢿⡄⠀ G H O S T
⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⣏⠀⠀⠀⠀⢸⣿⣿⣿⡿⠇⣿⣿⣿⣿⠆⠘⣷⡀ Generalizable Hyperspectral
⠀⠀⠀⠀⠀⠀⠀⠀⠀⣿⠇⠀⠀⠀⠀⠈⠿⠿⡿⠃⠀⢿⣿⣿⠏⠀⠀⢹⣗ Observation & Segmentation
⠀⠀⠀⠀⠀⠀⠀⠀⢸⡏⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠀⠀⠀⠀⠀⣿⡀ Toolkit
Ensemble routing selected — skipping SSM pretraining (set --ssm_epochs manually to override)
train: 2045 pixels across 16 classes
val: 1018 pixels across 16 classes
test: 7186 pixels across 16 classes
════════════════════════════════════════════════════════════
GHOST SPT Training
Device : cuda (NVIDIA GeForce RTX 3050 6GB Laptop GPU, 6 GB)
Loss : dice
Routing : forest
Train pixels : 2,045 (16 classes)
Val pixels : 1,018
Test pixels : 7,186
Output : runs/indian_pines
════════════════════════════════════════════════════════════
=== SSM Encoder ===
⚠ SSM Encoder is experimental — results may vary
SSM will be saved to: runs/indian_pines/ssm_pretrained.pt
=== Pre-training SSM Encoder ===
d_model=64 d_state=16 epochs=1 batch_size=512
Train pixels: 2045 | Val pixels: 1018
SSM pre-trained | Best Val Acc: 0.0000 | Saved → runs/indian_pines/ssm_pretrained.pt
Pre-computing fingerprint map ...
Fingerprint map shape: (145, 145, 64)
=== Building Spectral Partition Tree ===
Building Spectral Partition Tree for 16 classes
Pixel counts: {1: 46, 2: 1428, 3: 830, 4: 237, 5: 483, 6: 730, 7: 28, 8: 478, 9: 20, 10: 972, 11: 2455, 12: 593, 13: 205, 14: 1265, 15: 386, 16: 93}
Node (depth 0): classes [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]
├── LEFT:
Node (depth 1): classes [7, 2, 3, 10, 11, 13]
├── LEFT:
Node (depth 2): classes [3, 7, 2, 13]
├── LEFT:
Node (depth 3): classes [7, 2]
└── RIGHT:
Node (depth 3): classes [13, 3]
└── RIGHT:
Node (depth 2): classes [11, 10]
└── RIGHT:
Node (depth 1): classes [8, 1, 4, 5, 6, 9, 12, 14, 15, 16]
├── LEFT:
Node (depth 2): classes [6, 1, 5, 14, 15, 16]
├── LEFT:
Node (depth 3): classes [6, 5, 15, 16]
└── RIGHT:
Node (depth 3): classes [1, 14]
└── RIGHT:
Node (depth 2): classes [9, 8, 4, 12]
├── LEFT:
Node (depth 3): classes [8]
└── RIGHT:
Node (depth 3): classes [4, 9, 12]
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣀⣤⣤⡤⣤⢤⣤⣄⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣠⡾⠛⠉⠀⠀⠀⠀⠀⠀⠉⠻⣦⡄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣾⠏⠀⠀⠀⠀⣀⡀⠀⠀⠀⠀⠀⠈⢻⡆⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣼⠯⠀⠀⠀⢰⣿⣿⣿⣆⡀⣴⣾⣿⣦⡈⢿⡄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⣏⠀⠀⠀⠀⢸⣿⣿⣿⡿⠇⣿⣿⣿⣿⠆⠘⣷⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⣿⠇⠀⠀⠀⠀⠈⠿⠿⡿⠃⠀⢿⣿⣿⠏⠀⠀⢹⣗⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⢸⡏⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠀⠀⠀⠀⠀⣿⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀ Training initiated...
=== Training SPT Ensembles (loss=dice) ===
════════════════════════════════════════════════════════════
Node 'root'
Classes : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]
Local classes : 16 Epochs: 400 Ensembles: 5 Loss: dice
Train pixels : 2045 Val pixels: 1018
════════════════════════════════════════════════════════════
── Ensemble 1/5 seed=42 node='root'
Epoch 20/400 | █░░░░░░░░░░░░░░░░░░░░░░░ 5% | Loss 1.8599 | OA 0.0098 | mIoU 0.0093 | κ 0.0028
Epoch 40/400 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.7689 | OA 0.1100 | mIoU 0.0601 | κ 0.0844
Epoch 60/400 | ███░░░░░░░░░░░░░░░░░░░░░ 15% | Loss 1.6448 | OA 0.2652 | mIoU 0.0882 | κ 0.1994
Epoch 80/400 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 1.5505 | OA 0.3831 | mIoU 0.1089 | κ 0.2656
Epoch 100/400 | ██████░░░░░░░░░░░░░░░░░░ 25% | Loss 1.3803 | OA 0.4479 | mIoU 0.1295 | κ 0.3360
Epoch 120/400 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 1.2616 | OA 0.5884 | mIoU 0.2201 | κ 0.5164
Epoch 140/400 | ████████░░░░░░░░░░░░░░░░ 35% | Loss 1.1684 | OA 0.6277 | mIoU 0.2572 | κ 0.5663
Epoch 160/400 | █████████░░░░░░░░░░░░░░░ 40% | Loss 1.1124 | OA 0.7230 | mIoU 0.3322 | κ 0.6803
Epoch 180/400 | ██████████░░░░░░░░░░░░░░ 45% | Loss 1.0255 | OA 0.7760 | mIoU 0.3815 | κ 0.7413
Epoch 200/400 | ████████████░░░░░░░░░░░░ 50% | Loss 0.9272 | OA 0.7338 | mIoU 0.3525 | κ 0.6928
Epoch 220/400 | █████████████░░░░░░░░░░░ 55% | Loss 0.9028 | OA 0.8389 | mIoU 0.4608 | κ 0.8151
Epoch 240/400 | ██████████████░░░░░░░░░░ 60% | Loss 0.8665 | OA 0.7908 | mIoU 0.4067 | κ 0.7596
Epoch 260/400 | ███████████████░░░░░░░░░ 65% | Loss 0.7707 | OA 0.8654 | mIoU 0.4881 | κ 0.8454
Epoch 280/400 | ████████████████░░░░░░░░ 70% | Loss 0.7999 | OA 0.8762 | mIoU 0.5289 | κ 0.8582
Epoch 300/400 | ██████████████████░░░░░░ 75% | Loss 0.8166 | OA 0.8978 | mIoU 0.5504 | κ 0.8830
Epoch 320/400 | ███████████████████░░░░░ 80% | Loss 0.8132 | OA 0.8959 | mIoU 0.5468 | κ 0.8807
Epoch 340/400 | ████████████████████░░░░ 85% | Loss 0.7841 | OA 0.8841 | mIoU 0.5315 | κ 0.8671
Epoch 360/400 | █████████████████████░░░ 90% | Loss 0.8073 | OA 0.8870 | mIoU 0.5399 | κ 0.8706
⚡ Early stop: no improvement for 50 epochs at epoch 360
✓ Ensemble 1/5 done Best @ epoch 300 mIoU 0.5504 OA 0.8978 AA 0.6176 κ 0.8830
2m 54s elapsed | GPU 71°C | 3447 / 6144 MB | 100% util
── Ensemble 2/5 seed=43 node='root'
Epoch 20/400 | █░░░░░░░░░░░░░░░░░░░░░░░ 5% | Loss 1.8549 | OA 0.0216 | mIoU 0.0129 | κ 0.0139
Epoch 40/400 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.7744 | OA 0.2073 | mIoU 0.0591 | κ 0.1250
Epoch 60/400 | ███░░░░░░░░░░░░░░░░░░░░░ 15% | Loss 1.5492 | OA 0.3733 | mIoU 0.0918 | κ 0.2303
Epoch 80/400 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 1.3864 | OA 0.5069 | mIoU 0.1489 | κ 0.4124
Epoch 100/400 | ██████░░░░░░░░░░░░░░░░░░ 25% | Loss 1.2873 | OA 0.5147 | mIoU 0.1546 | κ 0.4265
Epoch 120/400 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 1.1665 | OA 0.5806 | mIoU 0.1908 | κ 0.5107
Epoch 140/400 | ████████░░░░░░░░░░░░░░░░ 35% | Loss 1.0866 | OA 0.6051 | mIoU 0.2185 | κ 0.5415
Epoch 160/400 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.9869 | OA 0.7073 | mIoU 0.2729 | κ 0.6612
Epoch 180/400 | ██████████░░░░░░░░░░░░░░ 45% | Loss 0.9113 | OA 0.7220 | mIoU 0.2967 | κ 0.6788
Epoch 200/400 | ████████████░░░░░░░░░░░░ 50% | Loss 0.9396 | OA 0.7554 | mIoU 0.3116 | κ 0.7178
Epoch 220/400 | █████████████░░░░░░░░░░░ 55% | Loss 0.8157 | OA 0.7741 | mIoU 0.3454 | κ 0.7392
Epoch 240/400 | ██████████████░░░░░░░░░░ 60% | Loss 0.8435 | OA 0.7819 | mIoU 0.3648 | κ 0.7493
Epoch 260/400 | ███████████████░░░░░░░░░ 65% | Loss 0.7999 | OA 0.8261 | mIoU 0.4054 | κ 0.8002
Epoch 280/400 | ████████████████░░░░░░░░ 70% | Loss 0.7536 | OA 0.7967 | mIoU 0.3788 | κ 0.7664
Epoch 300/400 | ██████████████████░░░░░░ 75% | Loss 0.7395 | OA 0.8183 | mIoU 0.4035 | κ 0.7911
Epoch 320/400 | ███████████████████░░░░░ 80% | Loss 0.7504 | OA 0.8251 | mIoU 0.4213 | κ 0.7992
Epoch 340/400 | ████████████████████░░░░ 85% | Loss 0.8073 | OA 0.8360 | mIoU 0.4313 | κ 0.8115
Epoch 360/400 | █████████████████████░░░ 90% | Loss 0.7397 | OA 0.8379 | mIoU 0.4385 | κ 0.8138
Epoch 380/400 | ██████████████████████░░ 95% | Loss 0.7283 | OA 0.8428 | mIoU 0.4351 | κ 0.8195
Epoch 400/400 | ████████████████████████ 100% | Loss 0.7413 | OA 0.8242 | mIoU 0.4178 | κ 0.7977
✓ Ensemble 2/5 done Best @ epoch 360 mIoU 0.4385 OA 0.8379 AA 0.5116 κ 0.8138
6m 9s elapsed | GPU 76°C | 3447 / 6144 MB | 100% util
── Ensemble 3/5 seed=44 node='root'
Epoch 20/400 | █░░░░░░░░░░░░░░░░░░░░░░░ 5% | Loss 1.7897 | OA 0.0589 | mIoU 0.0040 | κ 0.0012
Epoch 40/400 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.7001 | OA 0.3094 | mIoU 0.0657 | κ 0.1711
Epoch 60/400 | ███░░░░░░░░░░░░░░░░░░░░░ 15% | Loss 1.4537 | OA 0.3556 | mIoU 0.0807 | κ 0.1879
Epoch 80/400 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 1.2966 | OA 0.4528 | mIoU 0.1117 | κ 0.3265
Epoch 100/400 | ██████░░░░░░░░░░░░░░░░░░ 25% | Loss 1.2077 | OA 0.5000 | mIoU 0.1394 | κ 0.4124
Epoch 120/400 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 1.1276 | OA 0.4273 | mIoU 0.1181 | κ 0.3450
Epoch 140/400 | ████████░░░░░░░░░░░░░░░░ 35% | Loss 1.0681 | OA 0.6022 | mIoU 0.2025 | κ 0.5306
Epoch 160/400 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.9939 | OA 0.5933 | mIoU 0.2219 | κ 0.5213
Epoch 180/400 | ██████████░░░░░░░░░░░░░░ 45% | Loss 0.9966 | OA 0.7063 | mIoU 0.2943 | κ 0.6566
Epoch 200/400 | ████████████░░░░░░░░░░░░ 50% | Loss 0.8787 | OA 0.7633 | mIoU 0.3708 | κ 0.7259
Epoch 220/400 | █████████████░░░░░░░░░░░ 55% | Loss 0.9065 | OA 0.7515 | mIoU 0.3657 | κ 0.7108
Epoch 240/400 | ██████████████░░░░░░░░░░ 60% | Loss 0.7831 | OA 0.8124 | mIoU 0.3999 | κ 0.7837
Epoch 260/400 | ███████████████░░░░░░░░░ 65% | Loss 0.7967 | OA 0.8163 | mIoU 0.4389 | κ 0.7878
Epoch 280/400 | ████████████████░░░░░░░░ 70% | Loss 0.7737 | OA 0.8448 | mIoU 0.4637 | κ 0.8216
Epoch 300/400 | ██████████████████░░░░░░ 75% | Loss 0.7388 | OA 0.8497 | mIoU 0.4681 | κ 0.8271
Epoch 320/400 | ███████████████████░░░░░ 80% | Loss 0.7679 | OA 0.8399 | mIoU 0.4573 | κ 0.8155
Epoch 340/400 | ████████████████████░░░░ 85% | Loss 0.7449 | OA 0.8428 | mIoU 0.4580 | κ 0.8190
Epoch 360/400 | █████████████████████░░░ 90% | Loss 0.8102 | OA 0.8418 | mIoU 0.4572 | κ 0.8179
⚡ Early stop: no improvement for 50 epochs at epoch 360
✓ Ensemble 3/5 done Best @ epoch 300 mIoU 0.4681 OA 0.8497 AA 0.5350 κ 0.8271
9m 6s elapsed | GPU 79°C | 3447 / 6144 MB | 100% util
── Ensemble 4/5 seed=45 node='root'
Epoch 20/400 | █░░░░░░░░░░░░░░░░░░░░░░░ 5% | Loss 1.8160 | OA 0.1660 | mIoU 0.0194 | κ 0.0141
Epoch 40/400 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.7151 | OA 0.2996 | mIoU 0.0536 | κ 0.1273
Epoch 60/400 | ███░░░░░░░░░░░░░░░░░░░░░ 15% | Loss 1.5736 | OA 0.3507 | mIoU 0.0776 | κ 0.1881
Epoch 80/400 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 1.3492 | OA 0.3831 | mIoU 0.0999 | κ 0.2228
Epoch 100/400 | ██████░░░░░░░░░░░░░░░░░░ 25% | Loss 1.2651 | OA 0.4705 | mIoU 0.1796 | κ 0.3478
Epoch 120/400 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 1.1363 | OA 0.5619 | mIoU 0.2081 | κ 0.4900
Epoch 140/400 | ████████░░░░░░░░░░░░░░░░ 35% | Loss 1.0675 | OA 0.6316 | mIoU 0.2841 | κ 0.5682
Epoch 160/400 | █████████░░░░░░░░░░░░░░░ 40% | Loss 1.0657 | OA 0.6699 | mIoU 0.3085 | κ 0.6189
Epoch 180/400 | ██████████░░░░░░░░░░░░░░ 45% | Loss 0.9443 | OA 0.6316 | mIoU 0.2867 | κ 0.5759
Epoch 200/400 | ████████████░░░░░░░░░░░░ 50% | Loss 0.8603 | OA 0.7289 | mIoU 0.3479 | κ 0.6837
Epoch 220/400 | █████████████░░░░░░░░░░░ 55% | Loss 0.8411 | OA 0.7495 | mIoU 0.3526 | κ 0.7122
Epoch 240/400 | ██████████████░░░░░░░░░░ 60% | Loss 0.7780 | OA 0.8291 | mIoU 0.4342 | κ 0.8031
Epoch 260/400 | ███████████████░░░░░░░░░ 65% | Loss 0.7888 | OA 0.8281 | mIoU 0.4377 | κ 0.8034
Epoch 280/400 | ████████████████░░░░░░░░ 70% | Loss 0.6737 | OA 0.8939 | mIoU 0.5115 | κ 0.8778
Epoch 300/400 | ██████████████████░░░░░░ 75% | Loss 0.6619 | OA 0.8792 | mIoU 0.5075 | κ 0.8617
Epoch 320/400 | ███████████████████░░░░░ 80% | Loss 0.6660 | OA 0.8585 | mIoU 0.4922 | κ 0.8378
Epoch 340/400 | ████████████████████░░░░ 85% | Loss 0.5801 | OA 0.8988 | mIoU 0.5328 | κ 0.8841
Epoch 360/400 | █████████████████████░░░ 90% | Loss 0.5413 | OA 0.8998 | mIoU 0.5296 | κ 0.8852
Epoch 380/400 | ██████████████████████░░ 95% | Loss 0.5567 | OA 0.9077 | mIoU 0.5377 | κ 0.8939
Epoch 400/400 | ████████████████████████ 100% | Loss 0.5373 | OA 0.9047 | mIoU 0.5310 | κ 0.8905
✓ Ensemble 4/5 done Best @ epoch 380 mIoU 0.5377 OA 0.9077 AA 0.5889 κ 0.8939
12m 22s elapsed | GPU 81°C | 3447 / 6144 MB | 100% util
── Ensemble 5/5 seed=46 node='root'
Epoch 20/400 | █░░░░░░░░░░░░░░░░░░░░░░░ 5% | Loss 1.8160 | OA 0.0147 | mIoU 0.0082 | κ 0.0004
Epoch 40/400 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.6716 | OA 0.2800 | mIoU 0.0728 | κ 0.1434
Epoch 60/400 | ███░░░░░░░░░░░░░░░░░░░░░ 15% | Loss 1.5168 | OA 0.3576 | mIoU 0.0910 | κ 0.2121
Epoch 80/400 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 1.3345 | OA 0.3595 | mIoU 0.0938 | κ 0.2564
Epoch 100/400 | ██████░░░░░░░░░░░░░░░░░░ 25% | Loss 1.2019 | OA 0.4941 | mIoU 0.1494 | κ 0.4012
Epoch 120/400 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 1.1171 | OA 0.5363 | mIoU 0.1868 | κ 0.4407
Epoch 140/400 | ████████░░░░░░░░░░░░░░░░ 35% | Loss 1.1258 | OA 0.5914 | mIoU 0.2106 | κ 0.5111
Epoch 160/400 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.9936 | OA 0.5835 | mIoU 0.2242 | κ 0.4984
Epoch 180/400 | ██████████░░░░░░░░░░░░░░ 45% | Loss 0.9368 | OA 0.6631 | mIoU 0.2854 | κ 0.6010
Epoch 200/400 | ████████████░░░░░░░░░░░░ 50% | Loss 0.9357 | OA 0.7033 | mIoU 0.3175 | κ 0.6509
Epoch 220/400 | █████████████░░░░░░░░░░░ 55% | Loss 0.9930 | OA 0.7161 | mIoU 0.3337 | κ 0.6659
Epoch 240/400 | ██████████████░░░░░░░░░░ 60% | Loss 0.8949 | OA 0.7240 | mIoU 0.3363 | κ 0.6770
Epoch 260/400 | ███████████████░░░░░░░░░ 65% | Loss 0.9040 | OA 0.7112 | mIoU 0.3283 | κ 0.6620
Epoch 280/400 | ████████████████░░░░░░░░ 70% | Loss 0.9122 | OA 0.7181 | mIoU 0.3331 | κ 0.6699
Epoch 300/400 | ██████████████████░░░░░░ 75% | Loss 0.9048 | OA 0.7073 | mIoU 0.3268 | κ 0.6565
⚡ Early stop: no improvement for 50 epochs at epoch 300
✓ Ensemble 5/5 done Best @ epoch 240 mIoU 0.3363 OA 0.7240 AA 0.4194 κ 0.6770
14m 50s elapsed | GPU 82°C | 3447 / 6144 MB | 100% util
Node 'root' — 5 ensembles trained in 14m 50s
mIoU 0.5504 0.4385 0.4681 0.5377 0.3363 mean 0.4662 ± 0.0772
Best: Ensemble 1 (0.5504) Worst: Ensemble 5 (0.3363) Early stops: 3/5
Training SSSR router for node 'root' ...
Router: train_acc=0.5765 | val_acc=0.5796 | left=1182 right=863
Router trained.
════════════════════════════════════════════════════════════
Node 'root_L'
Classes : [7, 2, 3, 10, 11, 13]
Local classes : 6 Epochs: 200 Ensembles: 5 Loss: dice
Train pixels : 1182 Val pixels: 589
════════════════════════════════════════════════════════════
── Ensemble 1/5 seed=42 node='root_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.2960 | OA 0.2462 | mIoU 0.0585 | κ 0.0132
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 1.0854 | OA 0.5025 | mIoU 0.1766 | κ 0.2151
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.7780 | OA 0.5705 | mIoU 0.1999 | κ 0.3154
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.5256 | OA 0.7487 | mIoU 0.3870 | κ 0.6275
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.3791 | OA 0.8353 | mIoU 0.5563 | κ 0.7634
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.3741 | OA 0.8149 | mIoU 0.5272 | κ 0.7338
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.2856 | OA 0.8438 | mIoU 0.5943 | κ 0.7757
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.2618 | OA 0.9660 | mIoU 0.7624 | κ 0.9528
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.2122 | OA 0.9626 | mIoU 0.7508 | κ 0.9481
Epoch 200/200 | ████████████████████████ 100% | Loss 0.1973 | OA 0.9915 | mIoU 0.7980 | κ 0.9883
✓ Ensemble 1/5 done Best @ epoch 200 mIoU 0.7980 OA 0.9915 AA 0.8313 κ 0.9883
16m 29s elapsed | GPU 81°C | 3449 / 6144 MB | 100% util
── Ensemble 2/5 seed=43 node='root_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.4046 | OA 0.0034 | mIoU 0.0006 | κ -0.0012
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 1.2950 | OA 0.4041 | mIoU 0.1107 | κ 0.0834
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 1.0134 | OA 0.5263 | mIoU 0.1908 | κ 0.3016
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.7543 | OA 0.7453 | mIoU 0.3581 | κ 0.6410
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.6444 | OA 0.8744 | mIoU 0.5604 | κ 0.8248
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.5043 | OA 0.9168 | mIoU 0.5989 | κ 0.8852
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.4260 | OA 0.9134 | mIoU 0.6209 | κ 0.8799
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.3923 | OA 0.9491 | mIoU 0.6625 | κ 0.9297
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.3318 | OA 0.9576 | mIoU 0.6759 | κ 0.9414
Epoch 200/200 | ████████████████████████ 100% | Loss 0.3819 | OA 0.9593 | mIoU 0.6849 | κ 0.9437
✓ Ensemble 2/5 done Best @ epoch 200 mIoU 0.6849 OA 0.9593 AA 0.8383 κ 0.9437
18m 7s elapsed | GPU 83°C | 3451 / 6144 MB | 86% util
── Ensemble 3/5 seed=44 node='root_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.3436 | OA 0.0628 | mIoU 0.0368 | κ 0.0047
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 1.2167 | OA 0.4737 | mIoU 0.1834 | κ 0.2197
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.9377 | OA 0.4873 | mIoU 0.1476 | κ 0.1632
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.7717 | OA 0.5348 | mIoU 0.1853 | κ 0.2442
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.5961 | OA 0.7844 | mIoU 0.5008 | κ 0.7122
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.4797 | OA 0.8625 | mIoU 0.5812 | κ 0.8134
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.3872 | OA 0.9338 | mIoU 0.7074 | κ 0.9075
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.3289 | OA 0.9796 | mIoU 0.7828 | κ 0.9717
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.3413 | OA 0.9830 | mIoU 0.7956 | κ 0.9765
Epoch 200/200 | ████████████████████████ 100% | Loss 0.2486 | OA 0.9932 | mIoU 0.8257 | κ 0.9906
✓ Ensemble 3/5 done Best @ epoch 200 mIoU 0.8257 OA 0.9932 AA 0.8310 κ 0.9906
19m 46s elapsed | GPU 82°C | 3451 / 6144 MB | 100% util
── Ensemble 4/5 seed=45 node='root_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.2776 | OA 0.2411 | mIoU 0.0453 | κ 0.0056
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 1.0087 | OA 0.4465 | mIoU 0.1084 | κ 0.0784
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.7376 | OA 0.6520 | mIoU 0.2570 | κ 0.4755
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.5426 | OA 0.6876 | mIoU 0.2879 | κ 0.5501
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.4759 | OA 0.7538 | mIoU 0.3600 | κ 0.6463
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.3663 | OA 0.8693 | mIoU 0.5174 | κ 0.8119
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.3121 | OA 0.8896 | mIoU 0.5470 | κ 0.8438
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.3099 | OA 0.9457 | mIoU 0.6421 | κ 0.9235
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.2486 | OA 0.9389 | mIoU 0.6325 | κ 0.9140
Epoch 200/200 | ████████████████████████ 100% | Loss 0.2344 | OA 0.9694 | mIoU 0.7159 | κ 0.9573
✓ Ensemble 4/5 done Best @ epoch 200 mIoU 0.7159 OA 0.9694 AA 0.7352 κ 0.9573
21m 24s elapsed | GPU 83°C | 3451 / 6144 MB | 100% util
── Ensemble 5/5 seed=46 node='root_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.3402 | OA 0.0102 | mIoU 0.0144 | κ 0.0068
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 1.0883 | OA 0.4465 | mIoU 0.1160 | κ 0.0965
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.8300 | OA 0.5433 | mIoU 0.2586 | κ 0.3976
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.6647 | OA 0.7776 | mIoU 0.5016 | κ 0.7011
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.5247 | OA 0.8234 | mIoU 0.5576 | κ 0.7630
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.4627 | OA 0.9643 | mIoU 0.7457 | κ 0.9507
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.4134 | OA 0.9559 | mIoU 0.7373 | κ 0.9389
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.2641 | OA 0.9508 | mIoU 0.7291 | κ 0.9321
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.2421 | OA 0.9677 | mIoU 0.7814 | κ 0.9553
Epoch 200/200 | ████████████████████████ 100% | Loss 0.2173 | OA 0.9779 | mIoU 0.7756 | κ 0.9695
✓ Ensemble 5/5 done Best @ epoch 180 mIoU 0.7814 OA 0.9677 AA 0.8130 κ 0.9553
23m 3s elapsed | GPU 83°C | 3451 / 6144 MB | 100% util
Node 'root_L' — 5 ensembles trained in 8m 12s
mIoU 0.7980 0.6849 0.8257 0.7159 0.7814 mean 0.7612 ± 0.0525
Best: Ensemble 3 (0.8257) Worst: Ensemble 2 (0.6849) Early stops: 0/5
Training SSSR router for node 'root_L' ...
Router: train_acc=0.5601 | val_acc=0.5501 | left=497 right=685
Router trained.
════════════════════════════════════════════════════════════
Node 'root_L_L'
Classes : [3, 7, 2, 13]
Local classes : 4 Epochs: 200 Ensembles: 5 Loss: dice
Train pixels : 497 Val pixels: 247
════════════════════════════════════════════════════════════
── Ensemble 1/5 seed=42 node='root_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.0981 | OA 0.5668 | mIoU 0.1434 | κ 0.0071
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.7913 | OA 0.6073 | mIoU 0.2220 | κ 0.2346
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.4630 | OA 0.5061 | mIoU 0.2059 | κ 0.2277
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.3415 | OA 0.4170 | mIoU 0.2710 | κ 0.1076
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.1938 | OA 0.4939 | mIoU 0.3551 | κ 0.2495
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.1350 | OA 0.8826 | mIoU 0.6204 | κ 0.8066
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.1155 | OA 0.8178 | mIoU 0.6072 | κ 0.7112
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0883 | OA 0.9393 | mIoU 0.7444 | κ 0.8958
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0827 | OA 0.9798 | mIoU 0.8245 | κ 0.9639
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0641 | OA 0.9960 | mIoU 0.9982 | κ 0.9927
⚡ Early stop: mIoU ≥ 0.99 at epoch 200
✓ Ensemble 1/5 done Best @ epoch 200 mIoU 0.9982 OA 0.9960 AA 0.9982 κ 0.9927
24m 41s elapsed | GPU 83°C | 3451 / 6144 MB | 100% util
── Ensemble 2/5 seed=43 node='root_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.1453 | OA 0.0202 | mIoU 0.0352 | κ 0.0127
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.8788 | OA 0.6437 | mIoU 0.3717 | κ 0.3255
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.3980 | OA 0.7692 | mIoU 0.3659 | κ 0.6037
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.3289 | OA 0.8745 | mIoU 0.4888 | κ 0.7779
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.2870 | OA 0.9514 | mIoU 0.6691 | κ 0.9112
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.2250 | OA 0.9838 | mIoU 0.8608 | κ 0.9705
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.1439 | OA 0.9352 | mIoU 0.7733 | κ 0.8885
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0944 | OA 0.9919 | mIoU 0.9816 | κ 0.9854
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0505 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 180
✓ Ensemble 2/5 done Best @ epoch 180 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
26m 10s elapsed | GPU 83°C | 3451 / 6144 MB | 100% util
── Ensemble 3/5 seed=44 node='root_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.9789 | OA 0.5830 | mIoU 0.1509 | κ 0.0235
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.7573 | OA 0.6964 | mIoU 0.2875 | κ 0.4421
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.4620 | OA 0.4049 | mIoU 0.1336 | κ 0.0789
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.2841 | OA 0.5911 | mIoU 0.2836 | κ 0.3326
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.1401 | OA 0.8583 | mIoU 0.5436 | κ 0.7584
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0860 | OA 0.7895 | mIoU 0.5912 | κ 0.6568
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0485 | OA 0.9352 | mIoU 0.8826 | κ 0.8869
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0210 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 160
✓ Ensemble 3/5 done Best @ epoch 160 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
27m 29s elapsed | GPU 82°C | 3451 / 6144 MB | 100% util
── Ensemble 4/5 seed=45 node='root_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.0371 | OA 0.3684 | mIoU 0.1075 | κ 0.0164
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.7636 | OA 0.6883 | mIoU 0.2777 | κ 0.4048
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.4346 | OA 0.8381 | mIoU 0.4187 | κ 0.6975
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.1897 | OA 0.5668 | mIoU 0.2858 | κ 0.3590
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.1370 | OA 0.7571 | mIoU 0.5664 | κ 0.6384
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.1012 | OA 0.9393 | mIoU 0.6569 | κ 0.8943
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0821 | OA 0.9879 | mIoU 0.8566 | κ 0.9780
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0916 | OA 0.9838 | mIoU 0.8610 | κ 0.9707
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0754 | OA 0.9919 | mIoU 0.8703 | κ 0.9853
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0577 | OA 0.9919 | mIoU 0.8316 | κ 0.9853
✓ Ensemble 4/5 done Best @ epoch 180 mIoU 0.8703 OA 0.9919 AA 0.8732 κ 0.9853
29m 7s elapsed | GPU 83°C | 3451 / 6144 MB | 100% util
── Ensemble 5/5 seed=46 node='root_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.0807 | OA 0.3644 | mIoU 0.1328 | κ 0.1051
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.7912 | OA 0.6478 | mIoU 0.2812 | κ 0.3924
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.4722 | OA 0.8340 | mIoU 0.3988 | κ 0.6972
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.2954 | OA 0.8745 | mIoU 0.5936 | κ 0.7880
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.2146 | OA 0.7085 | mIoU 0.5140 | κ 0.5756
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.1746 | OA 0.9555 | mIoU 0.7342 | κ 0.9220
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.1662 | OA 0.9595 | mIoU 0.7889 | κ 0.9291
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0514 | OA 0.9960 | mIoU 0.9851 | κ 0.9927
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0506 | OA 0.9879 | mIoU 0.8874 | κ 0.9781
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0370 | OA 0.9960 | mIoU 0.9851 | κ 0.9927
✓ Ensemble 5/5 done Best @ epoch 160 mIoU 0.9851 OA 0.9960 AA 0.9970 κ 0.9927
30m 46s elapsed | GPU 84°C | 3451 / 6144 MB | 100% util
Node 'root_L_L' — 5 ensembles trained in 7m 43s
mIoU 0.9982 1.0000 1.0000 0.8703 0.9851 mean 0.9707 ± 0.0505
Best: Ensemble 2 (1.0000) Worst: Ensemble 4 (0.8703) Early stops: 3/5
Training SSSR router for node 'root_L_L' ...
Router: train_acc=0.4386 | val_acc=0.4372 | left=290 right=207
Router trained.
════════════════════════════════════════════════════════════
Node 'root_L_L_L'
Classes : [7, 2]
Local classes : 2 Epochs: 200 Ensembles: 3 Loss: dice
Train pixels : 290 Val pixels: 144
════════════════════════════════════════════════════════════
── Ensemble 1/3 seed=42 node='root_L_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.6891 | OA 0.9653 | mIoU 0.4826 | κ -0.0112
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.3404 | OA 0.9861 | mIoU 0.4931 | κ 0.0000
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0900 | OA 0.9861 | mIoU 0.7430 | κ 0.6604
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0324 | OA 0.7292 | mIoU 0.3871 | κ 0.0683
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0145 | OA 0.8542 | mIoU 0.4695 | κ 0.1380
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0093 | OA 0.9931 | mIoU 0.8298 | κ 0.7966
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0079 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 140
✓ Ensemble 1/3 done Best @ epoch 140 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
31m 56s elapsed | GPU 84°C | 3451 / 6144 MB | 100% util
── Ensemble 2/3 seed=43 node='root_L_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.8272 | OA 0.0139 | mIoU 0.0070 | κ 0.0001
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.5268 | OA 0.9653 | mIoU 0.6824 | κ 0.4340
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.1922 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 60
✓ Ensemble 2/3 done Best @ epoch 60 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
32m 25s elapsed | GPU 84°C | 3451 / 6144 MB | 100% util
── Ensemble 3/3 seed=44 node='root_L_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.6955 | OA 0.0486 | mIoU 0.0248 | κ 0.0010
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.3466 | OA 0.9653 | mIoU 0.4826 | κ -0.0141
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.1048 | OA 0.9722 | mIoU 0.6859 | κ 0.4903
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0692 | OA 0.9236 | mIoU 0.5382 | κ 0.2486
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0352 | OA 0.9722 | mIoU 0.6526 | κ 0.4894
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0151 | OA 0.9931 | mIoU 0.8298 | κ 0.7966
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0110 | OA 0.9931 | mIoU 0.8298 | κ 0.7966
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0070 | OA 0.9931 | mIoU 0.8298 | κ 0.7966
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0070 | OA 0.9931 | mIoU 0.8298 | κ 0.7966
⚡ Early stop: no improvement for 50 epochs at epoch 180
✓ Ensemble 3/3 done Best @ epoch 120 mIoU 0.8298 OA 0.9931 AA 0.9965 κ 0.7966
33m 54s elapsed | GPU 84°C | 3451 / 6144 MB | 89% util
Node 'root_L_L_L' — 3 ensembles trained in 3m 7s
mIoU 1.0000 1.0000 0.8298 mean 0.9433 ± 0.0802
Best: Ensemble 1 (1.0000) Worst: Ensemble 3 (0.8298) Early stops: 3/3
════════════════════════════════════════════════════════════
Node 'root_L_L_R'
Classes : [13, 3]
Local classes : 2 Epochs: 200 Ensembles: 3 Loss: dice
Train pixels : 207 Val pixels: 103
════════════════════════════════════════════════════════════
── Ensemble 1/3 seed=42 node='root_L_L_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.6538 | OA 0.7476 | mIoU 0.3775 | κ -0.0376
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.3004 | OA 0.8932 | mIoU 0.7426 | κ 0.6564
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0141 | OA 0.9709 | mIoU 0.9167 | κ 0.9119
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0050 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 80
✓ Ensemble 1/3 done Best @ epoch 80 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
34m 34s elapsed | GPU 85°C | 3451 / 6144 MB | 100% util
── Ensemble 2/3 seed=43 node='root_L_L_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.7553 | OA 0.1942 | mIoU 0.0971 | κ 0.0000
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.5283 | OA 0.9029 | mIoU 0.7740 | κ 0.7197
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0594 | OA 0.9903 | mIoU 0.9940 | κ 0.9697
⚡ Early stop: mIoU ≥ 0.99 at epoch 60
✓ Ensemble 2/3 done Best @ epoch 60 mIoU 0.9940 OA 0.9903 AA 0.9940 κ 0.9697
35m 3s elapsed | GPU 86°C | 3451 / 6144 MB | 100% util
── Ensemble 3/3 seed=44 node='root_L_L_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.5922 | OA 0.3981 | mIoU 0.2482 | κ 0.1006
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.2463 | OA 0.9320 | mIoU 0.8129 | κ 0.7841
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0232 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 60
✓ Ensemble 3/3 done Best @ epoch 60 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
35m 33s elapsed | GPU 85°C | 3451 / 6144 MB | 100% util
Node 'root_L_L_R' — 3 ensembles trained in 1m 38s
mIoU 1.0000 0.9940 1.0000 mean 0.9980 ± 0.0028
Best: Ensemble 1 (1.0000) Worst: Ensemble 2 (0.9940) Early stops: 3/3
════════════════════════════════════════════════════════════
Node 'root_L_R'
Classes : [11, 10]
Local classes : 2 Epochs: 200 Ensembles: 3 Loss: dice
Train pixels : 685 Val pixels: 342
════════════════════════════════════════════════════════════
── Ensemble 1/3 seed=42 node='root_L_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.6873 | OA 0.7456 | mIoU 0.4405 | κ 0.1971
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.3295 | OA 0.7749 | mIoU 0.4991 | κ 0.2988
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.1116 | OA 0.7778 | mIoU 0.4958 | κ 0.2960
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0575 | OA 0.9620 | mIoU 0.9078 | κ 0.9025
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0405 | OA 0.9883 | mIoU 0.9713 | κ 0.9709
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0337 | OA 0.9971 | mIoU 0.9928 | κ 0.9928
⚡ Early stop: mIoU ≥ 0.99 at epoch 120
✓ Ensemble 1/3 done Best @ epoch 120 mIoU 0.9928 OA 0.9971 AA 0.9948 κ 0.9928
36m 32s elapsed | GPU 86°C | 3451 / 6144 MB | 100% util
── Ensemble 2/3 seed=43 node='root_L_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.6768 | OA 0.7193 | mIoU 0.3691 | κ 0.0262
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.4573 | OA 0.7807 | mIoU 0.5010 | κ 0.3062
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.1581 | OA 0.8860 | mIoU 0.7352 | κ 0.6836
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0838 | OA 0.9708 | mIoU 0.9307 | κ 0.9261
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0485 | OA 0.9883 | mIoU 0.9753 | κ 0.9712
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0459 | OA 0.9942 | mIoU 0.9856 | κ 0.9855
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0380 | OA 0.9795 | mIoU 0.9520 | κ 0.9488
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0359 | OA 0.9971 | mIoU 0.9928 | κ 0.9928
⚡ Early stop: mIoU ≥ 0.99 at epoch 160
✓ Ensemble 2/3 done Best @ epoch 160 mIoU 0.9928 OA 0.9971 AA 0.9948 κ 0.9928
37m 51s elapsed | GPU 86°C | 3451 / 6144 MB | 100% util
── Ensemble 3/3 seed=44 node='root_L_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.5992 | OA 0.7544 | mIoU 0.4416 | κ 0.1928
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.3487 | OA 0.7807 | mIoU 0.5022 | κ 0.3080
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.1439 | OA 0.7924 | mIoU 0.5217 | κ 0.3441
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0789 | OA 0.7895 | mIoU 0.5153 | κ 0.3322
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0370 | OA 0.9766 | mIoU 0.9430 | κ 0.9410
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0279 | OA 0.9327 | mIoU 0.8385 | κ 0.8217
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0237 | OA 0.9854 | mIoU 0.9642 | κ 0.9635
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0171 | OA 0.9708 | mIoU 0.9288 | κ 0.9257
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0229 | OA 0.9912 | mIoU 0.9785 | κ 0.9782
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0147 | OA 0.9942 | mIoU 0.9856 | κ 0.9855
✓ Ensemble 3/3 done Best @ epoch 200 mIoU 0.9856 OA 0.9942 AA 0.9897 κ 0.9855
39m 30s elapsed | GPU 86°C | 3451 / 6144 MB | 100% util
Node 'root_L_R' — 3 ensembles trained in 3m 57s
mIoU 0.9928 0.9928 0.9856 mean 0.9904 ± 0.0034
Best: Ensemble 1 (0.9928) Worst: Ensemble 3 (0.9856) Early stops: 2/3
════════════════════════════════════════════════════════════
Node 'root_R'
Classes : [8, 1, 4, 5, 6, 9, 12, 14, 15, 16]
Local classes : 10 Epochs: 250 Ensembles: 5 Loss: dice
Train pixels : 863 Val pixels: 429
════════════════════════════════════════════════════════════
── Ensemble 1/5 seed=42 node='root_R'
Epoch 20/250 | █░░░░░░░░░░░░░░░░░░░░░░░ 8% | Loss 1.4678 | OA 0.0443 | mIoU 0.0217 | κ 0.0312
Epoch 40/250 | ███░░░░░░░░░░░░░░░░░░░░░ 16% | Loss 1.2878 | OA 0.3357 | mIoU 0.1252 | κ 0.2509
Epoch 60/250 | █████░░░░░░░░░░░░░░░░░░░ 24% | Loss 1.0830 | OA 0.3916 | mIoU 0.1424 | κ 0.2902
Epoch 80/250 | ███████░░░░░░░░░░░░░░░░░ 32% | Loss 0.9847 | OA 0.5548 | mIoU 0.2625 | κ 0.4687
Epoch 100/250 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.8405 | OA 0.7786 | mIoU 0.4161 | κ 0.7320
Epoch 120/250 | ███████████░░░░░░░░░░░░░ 48% | Loss 0.7154 | OA 0.8065 | mIoU 0.4351 | κ 0.7658
Epoch 140/250 | █████████████░░░░░░░░░░░ 56% | Loss 0.6298 | OA 0.7389 | mIoU 0.3782 | κ 0.6805
Epoch 160/250 | ███████████████░░░░░░░░░ 64% | Loss 0.5012 | OA 0.7413 | mIoU 0.3964 | κ 0.6827
Epoch 180/250 | █████████████████░░░░░░░ 72% | Loss 0.4267 | OA 0.8741 | mIoU 0.5044 | κ 0.8472
Epoch 200/250 | ███████████████████░░░░░ 80% | Loss 0.3552 | OA 0.9510 | mIoU 0.6457 | κ 0.9408
Epoch 220/250 | █████████████████████░░░ 88% | Loss 0.3504 | OA 0.9161 | mIoU 0.5884 | κ 0.8983
Epoch 240/250 | ███████████████████████░ 96% | Loss 0.3021 | OA 0.9464 | mIoU 0.6482 | κ 0.9352
Epoch 250/250 | ████████████████████████ 100% | Loss 0.3122 | OA 0.9464 | mIoU 0.6463 | κ 0.9352
✓ Ensemble 1/5 done Best @ epoch 240 mIoU 0.6482 OA 0.9464 AA 0.6808 κ 0.9352
41m 34s elapsed | GPU 85°C | 3453 / 6144 MB | 100% util
── Ensemble 2/5 seed=43 node='root_R'
Epoch 20/250 | █░░░░░░░░░░░░░░░░░░░░░░░ 8% | Loss 1.5350 | OA 0.1632 | mIoU 0.0259 | κ 0.0455
Epoch 40/250 | ███░░░░░░░░░░░░░░░░░░░░░ 16% | Loss 1.4042 | OA 0.3916 | mIoU 0.1274 | κ 0.2765
Epoch 60/250 | █████░░░░░░░░░░░░░░░░░░░ 24% | Loss 1.2340 | OA 0.5268 | mIoU 0.2182 | κ 0.4418
Epoch 80/250 | ███████░░░░░░░░░░░░░░░░░ 32% | Loss 1.0179 | OA 0.6247 | mIoU 0.2800 | κ 0.5500
Epoch 100/250 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.9441 | OA 0.6387 | mIoU 0.3023 | κ 0.5673
Epoch 120/250 | ███████████░░░░░░░░░░░░░ 48% | Loss 0.7664 | OA 0.7016 | mIoU 0.3538 | κ 0.6340
Epoch 140/250 | █████████████░░░░░░░░░░░ 56% | Loss 0.6687 | OA 0.8089 | mIoU 0.4497 | κ 0.7683
Epoch 160/250 | ███████████████░░░░░░░░░ 64% | Loss 0.5775 | OA 0.8322 | mIoU 0.4626 | κ 0.7952
Epoch 180/250 | █████████████████░░░░░░░ 72% | Loss 0.4215 | OA 0.9161 | mIoU 0.5882 | κ 0.8980
Epoch 200/250 | ███████████████████░░░░░ 80% | Loss 0.4077 | OA 0.8998 | mIoU 0.5730 | κ 0.8780
Epoch 220/250 | █████████████████████░░░ 88% | Loss 0.3124 | OA 0.9464 | mIoU 0.6736 | κ 0.9351
Epoch 240/250 | ███████████████████████░ 96% | Loss 0.4044 | OA 0.9441 | mIoU 0.6616 | κ 0.9322
Epoch 250/250 | ████████████████████████ 100% | Loss 0.4491 | OA 0.9697 | mIoU 0.7158 | κ 0.9634
✓ Ensemble 2/5 done Best @ epoch 250 mIoU 0.7158 OA 0.9697 AA 0.7375 κ 0.9634
43m 53s elapsed | GPU 83°C | 3453 / 6144 MB | 100% util
── Ensemble 3/5 seed=44 node='root_R'
Epoch 20/250 | █░░░░░░░░░░░░░░░░░░░░░░░ 8% | Loss 1.5364 | OA 0.0676 | mIoU 0.0261 | κ 0.0287
Epoch 40/250 | ███░░░░░░░░░░░░░░░░░░░░░ 16% | Loss 1.4231 | OA 0.2471 | mIoU 0.0917 | κ 0.1615
Epoch 60/250 | █████░░░░░░░░░░░░░░░░░░░ 24% | Loss 1.1630 | OA 0.4219 | mIoU 0.2091 | κ 0.3308
Epoch 80/250 | ███████░░░░░░░░░░░░░░░░░ 32% | Loss 1.0003 | OA 0.5012 | mIoU 0.2442 | κ 0.4040
Epoch 100/250 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.8068 | OA 0.6993 | mIoU 0.4028 | κ 0.6437
Epoch 120/250 | ███████████░░░░░░░░░░░░░ 48% | Loss 0.7139 | OA 0.7646 | mIoU 0.4614 | κ 0.7205
Epoch 140/250 | █████████████░░░░░░░░░░░ 56% | Loss 0.6253 | OA 0.5548 | mIoU 0.2619 | κ 0.4660
Epoch 160/250 | ███████████████░░░░░░░░░ 64% | Loss 0.4840 | OA 0.7599 | mIoU 0.4063 | κ 0.7103
Epoch 180/250 | █████████████████░░░░░░░ 72% | Loss 0.5208 | OA 0.8182 | mIoU 0.4920 | κ 0.7801
Epoch 200/250 | ███████████████████░░░░░ 80% | Loss 0.3466 | OA 0.7646 | mIoU 0.4417 | κ 0.7158
Epoch 220/250 | █████████████████████░░░ 88% | Loss 0.4067 | OA 0.9254 | mIoU 0.6578 | κ 0.9102
Epoch 240/250 | ███████████████████████░ 96% | Loss 0.2559 | OA 0.9510 | mIoU 0.7010 | κ 0.9410
Epoch 250/250 | ████████████████████████ 100% | Loss 0.2345 | OA 0.9790 | mIoU 0.7584 | κ 0.9746
✓ Ensemble 3/5 done Best @ epoch 250 mIoU 0.7584 OA 0.9790 AA 0.7734 κ 0.9746
45m 56s elapsed | GPU 84°C | 3453 / 6144 MB | 85% util
── Ensemble 4/5 seed=45 node='root_R'
Epoch 20/250 | █░░░░░░░░░░░░░░░░░░░░░░░ 8% | Loss 1.5189 | OA 0.1282 | mIoU 0.0264 | κ 0.0200
Epoch 40/250 | ███░░░░░░░░░░░░░░░░░░░░░ 16% | Loss 1.3051 | OA 0.3963 | mIoU 0.1483 | κ 0.2785
Epoch 60/250 | █████░░░░░░░░░░░░░░░░░░░ 24% | Loss 1.0766 | OA 0.4965 | mIoU 0.1966 | κ 0.3900
Epoch 80/250 | ███████░░░░░░░░░░░░░░░░░ 32% | Loss 0.8293 | OA 0.6270 | mIoU 0.2373 | κ 0.5368
Epoch 100/250 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.7762 | OA 0.5571 | mIoU 0.1973 | κ 0.4526
Epoch 120/250 | ███████████░░░░░░░░░░░░░ 48% | Loss 0.5907 | OA 0.9068 | mIoU 0.5874 | κ 0.8865
Epoch 140/250 | █████████████░░░░░░░░░░░ 56% | Loss 0.5426 | OA 0.9347 | mIoU 0.6200 | κ 0.9210
Epoch 160/250 | ███████████████░░░░░░░░░ 64% | Loss 0.5059 | OA 0.9510 | mIoU 0.6491 | κ 0.9408
Epoch 180/250 | █████████████████░░░░░░░ 72% | Loss 0.4647 | OA 0.9324 | mIoU 0.6092 | κ 0.9182
Epoch 200/250 | ███████████████████░░░░░ 80% | Loss 0.3799 | OA 0.9487 | mIoU 0.6450 | κ 0.9379
Epoch 220/250 | █████████████████████░░░ 88% | Loss 0.4408 | OA 0.9441 | mIoU 0.6325 | κ 0.9323
⚡ Early stop: no improvement for 50 epochs at epoch 220
✓ Ensemble 4/5 done Best @ epoch 160 mIoU 0.6491 OA 0.9510 AA 0.6880 κ 0.9408
47m 44s elapsed | GPU 85°C | 3453 / 6144 MB | 85% util
── Ensemble 5/5 seed=46 node='root_R'
Epoch 20/250 | █░░░░░░░░░░░░░░░░░░░░░░░ 8% | Loss 1.4536 | OA 0.3800 | mIoU 0.0962 | κ 0.2438
Epoch 40/250 | ███░░░░░░░░░░░░░░░░░░░░░ 16% | Loss 1.2562 | OA 0.4709 | mIoU 0.1750 | κ 0.3446
Epoch 60/250 | █████░░░░░░░░░░░░░░░░░░░ 24% | Loss 1.0947 | OA 0.4779 | mIoU 0.1891 | κ 0.3252
Epoch 80/250 | ███████░░░░░░░░░░░░░░░░░ 32% | Loss 0.9811 | OA 0.4732 | mIoU 0.1585 | κ 0.3116
Epoch 100/250 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.8368 | OA 0.4312 | mIoU 0.1408 | κ 0.2393
Epoch 120/250 | ███████████░░░░░░░░░░░░░ 48% | Loss 0.7301 | OA 0.6620 | mIoU 0.3041 | κ 0.5733
Epoch 140/250 | █████████████░░░░░░░░░░░ 56% | Loss 0.6496 | OA 0.7925 | mIoU 0.4268 | κ 0.7476
Epoch 160/250 | ███████████████░░░░░░░░░ 64% | Loss 0.5221 | OA 0.5711 | mIoU 0.2995 | κ 0.4995
Epoch 180/250 | █████████████████░░░░░░░ 72% | Loss 0.4670 | OA 0.8578 | mIoU 0.5343 | κ 0.8281
Epoch 200/250 | ███████████████████░░░░░ 80% | Loss 0.4725 | OA 0.9277 | mIoU 0.6201 | κ 0.9125
Epoch 220/250 | █████████████████████░░░ 88% | Loss 0.5522 | OA 0.9184 | mIoU 0.6059 | κ 0.9012
Epoch 240/250 | ███████████████████████░ 96% | Loss 0.4726 | OA 0.9301 | mIoU 0.6197 | κ 0.9153
Epoch 250/250 | ████████████████████████ 100% | Loss 0.4327 | OA 0.9254 | mIoU 0.6136 | κ 0.9097
⚡ Early stop: no improvement for 50 epochs at epoch 250
✓ Ensemble 5/5 done Best @ epoch 200 mIoU 0.6201 OA 0.9277 AA 0.6572 κ 0.9125
49m 47s elapsed | GPU 84°C | 3453 / 6144 MB | 100% util
Node 'root_R' — 5 ensembles trained in 10m 17s
mIoU 0.6482 0.7158 0.7584 0.6491 0.6201 mean 0.6783 ± 0.0509
Best: Ensemble 3 (0.7584) Worst: Ensemble 5 (0.6201) Early stops: 2/5
Training SSSR router for node 'root_R' ...
Router: train_acc=0.4820 | val_acc=0.4429 | left=599 right=264
Router trained.
════════════════════════════════════════════════════════════
Node 'root_R_L'
Classes : [6, 1, 5, 14, 15, 16]
Local classes : 6 Epochs: 200 Ensembles: 5 Loss: dice
Train pixels : 599 Val pixels: 298
════════════════════════════════════════════════════════════
── Ensemble 1/5 seed=42 node='root_R_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.1816 | OA 0.3624 | mIoU 0.1111 | κ 0.1855
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.9489 | OA 0.4597 | mIoU 0.1577 | κ 0.2760
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.7582 | OA 0.4899 | mIoU 0.1701 | κ 0.2753
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.6203 | OA 0.6913 | mIoU 0.3543 | κ 0.5761
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.3525 | OA 0.8960 | mIoU 0.6720 | κ 0.8547
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.2489 | OA 0.9799 | mIoU 0.8462 | κ 0.9720
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.1918 | OA 0.9732 | mIoU 0.8568 | κ 0.9626
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.1671 | OA 0.9799 | mIoU 0.8166 | κ 0.9720
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.1387 | OA 0.9832 | mIoU 0.8356 | κ 0.9766
Epoch 200/200 | ████████████████████████ 100% | Loss 0.1205 | OA 0.9765 | mIoU 0.7967 | κ 0.9672
⚡ Early stop: no improvement for 50 epochs at epoch 200
✓ Ensemble 1/5 done Best @ epoch 140 mIoU 0.8568 OA 0.9732 AA 0.9349 κ 0.9626
51m 26s elapsed | GPU 84°C | 3453 / 6144 MB | 100% util
── Ensemble 2/5 seed=43 node='root_R_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.3772 | OA 0.2450 | mIoU 0.0412 | κ 0.0031
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 1.2901 | OA 0.3456 | mIoU 0.1235 | κ 0.1833
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.9867 | OA 0.6376 | mIoU 0.2902 | κ 0.4856
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.6859 | OA 0.6577 | mIoU 0.2437 | κ 0.4830
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.5764 | OA 0.8221 | mIoU 0.4628 | κ 0.7413
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.4183 | OA 0.9195 | mIoU 0.6373 | κ 0.8861
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.3020 | OA 0.9497 | mIoU 0.6683 | κ 0.9296
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.2654 | OA 0.9564 | mIoU 0.7291 | κ 0.9391
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.1863 | OA 0.9161 | mIoU 0.6758 | κ 0.8795
Epoch 200/200 | ████████████████████████ 100% | Loss 0.1847 | OA 0.9497 | mIoU 0.8320 | κ 0.9288
✓ Ensemble 2/5 done Best @ epoch 200 mIoU 0.8320 OA 0.9497 AA 0.8726 κ 0.9288
53m 5s elapsed | GPU 84°C | 3453 / 6144 MB | 86% util
── Ensemble 3/5 seed=44 node='root_R_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.2131 | OA 0.2315 | mIoU 0.0759 | κ 0.0821
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 1.1141 | OA 0.4497 | mIoU 0.1564 | κ 0.2353
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.7923 | OA 0.4899 | mIoU 0.2013 | κ 0.2612
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.5943 | OA 0.8154 | mIoU 0.4542 | κ 0.7367
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.3425 | OA 0.9430 | mIoU 0.6496 | κ 0.9195
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.2498 | OA 0.9765 | mIoU 0.7544 | κ 0.9672
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.2245 | OA 0.9027 | mIoU 0.6252 | κ 0.8663
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.1642 | OA 0.9094 | mIoU 0.6452 | κ 0.8771
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.1559 | OA 0.9799 | mIoU 0.8071 | κ 0.9720
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0795 | OA 0.9899 | mIoU 0.8518 | κ 0.9860
✓ Ensemble 3/5 done Best @ epoch 200 mIoU 0.8518 OA 0.9899 AA 0.8750 κ 0.9860
54m 43s elapsed | GPU 83°C | 3453 / 6144 MB | 86% util
── Ensemble 4/5 seed=45 node='root_R_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.2342 | OA 0.0839 | mIoU 0.0329 | κ 0.0402
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 1.0768 | OA 0.4295 | mIoU 0.1772 | κ 0.2742
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.7868 | OA 0.5470 | mIoU 0.2834 | κ 0.4192
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.5704 | OA 0.6409 | mIoU 0.4189 | κ 0.5473
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.4673 | OA 0.8691 | mIoU 0.5452 | κ 0.8186
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.3122 | OA 0.9597 | mIoU 0.7794 | κ 0.9439
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.2368 | OA 0.9430 | mIoU 0.7497 | κ 0.9201
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.1805 | OA 0.9832 | mIoU 0.8354 | κ 0.9766
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.1369 | OA 0.9597 | mIoU 0.7506 | κ 0.9437
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0727 | OA 0.9664 | mIoU 0.8212 | κ 0.9532
✓ Ensemble 4/5 done Best @ epoch 160 mIoU 0.8354 OA 0.9832 AA 0.8681 κ 0.9766
56m 22s elapsed | GPU 85°C | 3453 / 6144 MB | 86% util
── Ensemble 5/5 seed=46 node='root_R_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.1967 | OA 0.1208 | mIoU 0.0413 | κ 0.0398
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.9847 | OA 0.4094 | mIoU 0.2228 | κ 0.2433
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.7648 | OA 0.5302 | mIoU 0.2909 | κ 0.3143
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.5769 | OA 0.7584 | mIoU 0.4942 | κ 0.6460
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.3937 | OA 0.7517 | mIoU 0.4293 | κ 0.6339
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.3567 | OA 0.8893 | mIoU 0.6712 | κ 0.8451
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.3242 | OA 0.9664 | mIoU 0.7877 | κ 0.9533
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.1405 | OA 0.7584 | mIoU 0.5402 | κ 0.6361
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0849 | OA 0.8826 | mIoU 0.7588 | κ 0.8305
Epoch 200/200 | ████████████████████████ 100% | Loss 0.1116 | OA 0.9866 | mIoU 0.8933 | κ 0.9813
✓ Ensemble 5/5 done Best @ epoch 200 mIoU 0.8933 OA 0.9866 AA 0.9079 κ 0.9813
58m 1s elapsed | GPU 84°C | 3453 / 6144 MB | 100% util
Node 'root_R_L' — 5 ensembles trained in 8m 13s
mIoU 0.8568 0.8320 0.8518 0.8354 0.8933 mean 0.8539 ± 0.0219
Best: Ensemble 5 (0.8933) Worst: Ensemble 2 (0.8320) Early stops: 1/5
Training SSSR router for node 'root_R_L' ...
Router: train_acc=0.6277 | val_acc=0.5940 | left=337 right=262
Router trained.
════════════════════════════════════════════════════════════
Node 'root_R_L_L'
Classes : [6, 5, 15, 16]
Local classes : 4 Epochs: 200 Ensembles: 5 Loss: dice
Train pixels : 337 Val pixels: 168
════════════════════════════════════════════════════════════
── Ensemble 1/5 seed=42 node='root_R_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.0554 | OA 0.2976 | mIoU 0.0861 | κ 0.0209
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.8071 | OA 0.4464 | mIoU 0.2550 | κ 0.2776
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.5565 | OA 0.5774 | mIoU 0.3633 | κ 0.4225
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.3329 | OA 0.8214 | mIoU 0.5602 | κ 0.7215
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.1979 | OA 0.9345 | mIoU 0.7710 | κ 0.9019
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.1187 | OA 0.9702 | mIoU 0.9043 | κ 0.9564
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0544 | OA 0.9762 | mIoU 0.9412 | κ 0.9650
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0298 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 160
✓ Ensemble 1/5 done Best @ epoch 160 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
59m 20s elapsed | GPU 84°C | 3453 / 6144 MB | 85% util
── Ensemble 2/5 seed=43 node='root_R_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.0680 | OA 0.0119 | mIoU 0.0128 | κ 0.0076
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.9053 | OA 0.3036 | mIoU 0.2180 | κ 0.1955
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.4890 | OA 0.7143 | mIoU 0.4883 | κ 0.5971
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.2514 | OA 0.9405 | mIoU 0.7286 | κ 0.9119
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.1553 | OA 0.9048 | mIoU 0.6640 | κ 0.8583
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.1283 | OA 0.9821 | mIoU 0.9133 | κ 0.9735
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0905 | OA 0.9762 | mIoU 0.9256 | κ 0.9649
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0354 | OA 0.9940 | mIoU 0.9883 | κ 0.9912
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0149 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 180
✓ Ensemble 2/5 done Best @ epoch 180 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 0m 49s elapsed | GPU 84°C | 3453 / 6144 MB | 100% util
── Ensemble 3/5 seed=44 node='root_R_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.0188 | OA 0.2976 | mIoU 0.0863 | κ 0.0152
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.8580 | OA 0.4464 | mIoU 0.1964 | κ 0.1879
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.3927 | OA 0.7798 | mIoU 0.4828 | κ 0.6493
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.2094 | OA 0.8214 | mIoU 0.5662 | κ 0.7356
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.1361 | OA 0.8631 | mIoU 0.6053 | κ 0.7954
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0974 | OA 0.9881 | mIoU 0.9344 | κ 0.9823
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0437 | OA 0.9881 | mIoU 0.9588 | κ 0.9824
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0145 | OA 0.9762 | mIoU 0.9394 | κ 0.9649
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0387 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 180
✓ Ensemble 3/5 done Best @ epoch 180 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 2m 17s elapsed | GPU 84°C | 3453 / 6144 MB | 100% util
── Ensemble 4/5 seed=45 node='root_R_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.0662 | OA 0.5655 | mIoU 0.2457 | κ 0.3060
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.8699 | OA 0.5952 | mIoU 0.2785 | κ 0.3680
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.6479 | OA 0.5119 | mIoU 0.2682 | κ 0.3294
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.3972 | OA 0.5595 | mIoU 0.3028 | κ 0.3804
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.1491 | OA 0.8571 | mIoU 0.7366 | κ 0.7938
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0587 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 120
✓ Ensemble 4/5 done Best @ epoch 120 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 3m 16s elapsed | GPU 84°C | 3453 / 6144 MB | 100% util
── Ensemble 5/5 seed=46 node='root_R_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.0522 | OA 0.0833 | mIoU 0.0647 | κ 0.0519
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.8068 | OA 0.4048 | mIoU 0.2652 | κ 0.2599
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.5643 | OA 0.7083 | mIoU 0.4945 | κ 0.5540
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.2595 | OA 0.7619 | mIoU 0.5308 | κ 0.6253
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.1312 | OA 0.8333 | mIoU 0.6515 | κ 0.7642
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.1081 | OA 0.9464 | mIoU 0.8968 | κ 0.9221
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0880 | OA 0.9226 | mIoU 0.8918 | κ 0.8881
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0259 | OA 0.9762 | mIoU 0.9625 | κ 0.9650
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0283 | OA 0.9881 | mIoU 0.9867 | κ 0.9825
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0191 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 200
✓ Ensemble 5/5 done Best @ epoch 200 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 4m 55s elapsed | GPU 84°C | 3453 / 6144 MB | 100% util
Node 'root_R_L_L' — 5 ensembles trained in 6m 53s
mIoU 1.0000 1.0000 1.0000 1.0000 1.0000 mean 1.0000 ± 0.0000
Best: Ensemble 1 (1.0000) Worst: Ensemble 1 (1.0000) Early stops: 5/5
════════════════════════════════════════════════════════════
Node 'root_R_L_R'
Classes : [1, 14]
Local classes : 2 Epochs: 200 Ensembles: 3 Loss: dice
Train pixels : 262 Val pixels: 130
════════════════════════════════════════════════════════════
── Ensemble 1/3 seed=42 node='root_R_L_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.5594 | OA 0.4846 | mIoU 0.4921 | κ 0.0539
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.3241 | OA 0.9692 | mIoU 0.9841 | κ 0.6561
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0778 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 60
✓ Ensemble 1/3 done Best @ epoch 60 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 5m 25s elapsed | GPU 85°C | 3453 / 6144 MB | 86% util
── Ensemble 2/3 seed=43 node='root_R_L_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.5474 | OA 0.9692 | mIoU 0.4846 | κ 0.0000
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.3980 | OA 0.9769 | mIoU 0.7460 | κ 0.6585
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0802 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 60
✓ Ensemble 2/3 done Best @ epoch 60 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 5m 54s elapsed | GPU 85°C | 3453 / 6144 MB | 97% util
── Ensemble 3/3 seed=44 node='root_R_L_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.5314 | OA 0.9692 | mIoU 0.4884 | κ 0.1950
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.3775 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 40
✓ Ensemble 3/3 done Best @ epoch 40 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 6m 14s elapsed | GPU 84°C | 3453 / 6144 MB | 100% util
Node 'root_R_L_R' — 3 ensembles trained in 1m 19s
mIoU 1.0000 1.0000 1.0000 mean 1.0000 ± 0.0000
Best: Ensemble 1 (1.0000) Worst: Ensemble 1 (1.0000) Early stops: 3/3
════════════════════════════════════════════════════════════
Node 'root_R_R'
Classes : [9, 8, 4, 12]
Local classes : 4 Epochs: 200 Ensembles: 5 Loss: dice
Train pixels : 264 Val pixels: 131
════════════════════════════════════════════════════════════
── Ensemble 1/5 seed=42 node='root_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.9698 | OA 0.0916 | mIoU 0.0631 | κ 0.0379
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.7184 | OA 0.7710 | mIoU 0.4889 | κ 0.6430
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.5738 | OA 0.7863 | mIoU 0.4667 | κ 0.6627
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.3574 | OA 0.5267 | mIoU 0.2143 | κ 0.2168
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.2143 | OA 0.5573 | mIoU 0.2366 | κ 0.2657
⚡ Early stop: no improvement for 50 epochs at epoch 100
✓ Ensemble 1/5 done Best @ epoch 40 mIoU 0.4889 OA 0.7710 AA 0.6583 κ 0.6430
1h 7m 3s elapsed | GPU 84°C | 3453 / 6144 MB | 100% util
── Ensemble 2/5 seed=43 node='root_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.0021 | OA 0.0458 | mIoU 0.0353 | κ 0.0195
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.8153 | OA 0.3740 | mIoU 0.2580 | κ 0.2548
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.7034 | OA 0.7634 | mIoU 0.4699 | κ 0.6413
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.5582 | OA 0.8397 | mIoU 0.5295 | κ 0.7457
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.4637 | OA 0.7405 | mIoU 0.4454 | κ 0.5834
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.2418 | OA 0.7176 | mIoU 0.4632 | κ 0.5864
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.1265 | OA 0.7099 | mIoU 0.5020 | κ 0.5959
⚡ Early stop: no improvement for 50 epochs at epoch 140
✓ Ensemble 2/5 done Best @ epoch 80 mIoU 0.5295 OA 0.8397 AA 0.6122 κ 0.7457
1h 8m 12s elapsed | GPU 84°C | 3453 / 6144 MB | 100% util
── Ensemble 3/5 seed=44 node='root_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.9900 | OA 0.1985 | mIoU 0.0609 | κ 0.0228
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.8179 | OA 0.4198 | mIoU 0.2233 | κ 0.2505
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.5533 | OA 0.6336 | mIoU 0.3508 | κ 0.4727
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.3473 | OA 0.5420 | mIoU 0.2187 | κ 0.2480
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.1384 | OA 0.8092 | mIoU 0.4943 | κ 0.7115
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.1022 | OA 0.8626 | mIoU 0.5738 | κ 0.7944
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0365 | OA 0.9695 | mIoU 0.9417 | κ 0.9525
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0208 | OA 0.9084 | mIoU 0.7720 | κ 0.8602
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0550 | OA 0.9695 | mIoU 0.9417 | κ 0.9525
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0114 | OA 0.9542 | mIoU 0.8401 | κ 0.9292
⚡ Early stop: no improvement for 50 epochs at epoch 200
✓ Ensemble 3/5 done Best @ epoch 140 mIoU 0.9417 OA 0.9695 AA 0.9787 κ 0.9525
1h 9m 50s elapsed | GPU 84°C | 3453 / 6144 MB | 100% util
── Ensemble 4/5 seed=45 node='root_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.9732 | OA 0.1298 | mIoU 0.0921 | κ 0.0650
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.7240 | OA 0.6183 | mIoU 0.3290 | κ 0.4084
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.3308 | OA 0.7328 | mIoU 0.3772 | κ 0.5500
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.1710 | OA 0.8550 | mIoU 0.5476 | κ 0.7748
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.1300 | OA 0.8550 | mIoU 0.5638 | κ 0.7802
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0772 | OA 0.9542 | mIoU 0.6928 | κ 0.9289
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0482 | OA 0.9084 | mIoU 0.7083 | κ 0.8602
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0304 | OA 0.9771 | mIoU 0.9651 | κ 0.9644
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0171 | OA 0.9695 | mIoU 0.8217 | κ 0.9521
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0075 | OA 0.9466 | mIoU 0.8007 | κ 0.9179
✓ Ensemble 4/5 done Best @ epoch 160 mIoU 0.9651 OA 0.9771 AA 0.9851 κ 0.9644
1h 11m 29s elapsed | GPU 84°C | 3453 / 6144 MB | 100% util
── Ensemble 5/5 seed=46 node='root_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.0211 | OA 0.0153 | mIoU 0.0106 | κ -0.0025
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.7027 | OA 0.5191 | mIoU 0.3360 | κ 0.3813
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.5026 | OA 0.7939 | mIoU 0.5141 | κ 0.6975
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.2638 | OA 0.7634 | mIoU 0.4644 | κ 0.6462
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.1141 | OA 0.9313 | mIoU 0.7987 | κ 0.8953
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0828 | OA 0.8015 | mIoU 0.5247 | κ 0.6975
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0740 | OA 0.8626 | mIoU 0.7053 | κ 0.7844
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0206 | OA 0.6870 | mIoU 0.5197 | κ 0.5398
⚡ Early stop: no improvement for 50 epochs at epoch 160
✓ Ensemble 5/5 done Best @ epoch 100 mIoU 0.7987 OA 0.9313 AA 0.8357 κ 0.8953
1h 12m 47s elapsed | GPU 84°C | 3453 / 6144 MB | 100% util
Node 'root_R_R' — 5 ensembles trained in 6m 33s
mIoU 0.4889 0.5295 0.9417 0.9651 0.7987 mean 0.7448 ± 0.2010
Best: Ensemble 4 (0.9651) Worst: Ensemble 1 (0.4889) Early stops: 4/5
Training SSSR router for node 'root_R_R' ...
Router: train_acc=0.6402 | val_acc=0.6641 | left=95 right=169
Router trained.
════════════════════════════════════════════════════════════
Node 'root_R_R_R'
Classes : [4, 9, 12]
Local classes : 3 Epochs: 200 Ensembles: 5 Loss: dice
Train pixels : 169 Val pixels: 84
════════════════════════════════════════════════════════════
── Ensemble 1/5 seed=42 node='root_R_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.8333 | OA 0.1190 | mIoU 0.1111 | κ 0.0668
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.4654 | OA 0.8929 | mIoU 0.5393 | κ 0.7329
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.1308 | OA 0.9524 | mIoU 0.6268 | κ 0.8870
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0967 | OA 0.9762 | mIoU 0.6667 | κ 0.9448
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0441 | OA 0.9881 | mIoU 0.9800 | κ 0.9721
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0149 | OA 0.9762 | mIoU 0.9601 | κ 0.9434
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0058 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 140
✓ Ensemble 1/5 done Best @ epoch 140 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 13m 57s elapsed | GPU 84°C | 3453 / 6144 MB | 86% util
── Ensemble 2/5 seed=43 node='root_R_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.8307 | OA 0.6905 | mIoU 0.2447 | κ 0.1100
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.6346 | OA 0.8810 | mIoU 0.5204 | κ 0.6994
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.4110 | OA 0.7857 | mIoU 0.3602 | κ 0.3729
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0793 | OA 0.9881 | mIoU 0.8333 | κ 0.9724
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0404 | OA 0.9405 | mIoU 0.9015 | κ 0.8532
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0196 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 120
✓ Ensemble 2/5 done Best @ epoch 120 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 14m 56s elapsed | GPU 85°C | 3453 / 6144 MB | 99% util
── Ensemble 3/5 seed=44 node='root_R_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.7524 | OA 0.7738 | mIoU 0.3391 | κ 0.3109
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.5859 | OA 0.8095 | mIoU 0.3927 | κ 0.4455
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.1416 | OA 0.8929 | mIoU 0.5211 | κ 0.7175
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0566 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 80
✓ Ensemble 3/5 done Best @ epoch 80 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 15m 35s elapsed | GPU 84°C | 3453 / 6144 MB | 86% util
── Ensemble 4/5 seed=45 node='root_R_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.8961 | OA 0.0000 | mIoU 0.0000 | κ 0.0000
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.6942 | OA 0.8690 | mIoU 0.5611 | κ 0.7106
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.3777 | OA 0.9048 | mIoU 0.5399 | κ 0.7522
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.1146 | OA 0.9881 | mIoU 0.8333 | κ 0.9724
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0642 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 100
✓ Ensemble 4/5 done Best @ epoch 100 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 16m 24s elapsed | GPU 84°C | 3453 / 6144 MB | 100% util
── Ensemble 5/5 seed=46 node='root_R_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.8997 | OA 0.7024 | mIoU 0.2562 | κ 0.1132
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.6154 | OA 0.8690 | mIoU 0.4879 | κ 0.6528
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.3101 | OA 0.9524 | mIoU 0.6165 | κ 0.8824
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0842 | OA 0.9762 | mIoU 0.8776 | κ 0.9468
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0261 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 100
✓ Ensemble 5/5 done Best @ epoch 100 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 17m 13s elapsed | GPU 84°C | 3453 / 6144 MB | 86% util
Node 'root_R_R_R' — 5 ensembles trained in 4m 25s
mIoU 1.0000 1.0000 1.0000 1.0000 1.0000 mean 1.0000 ± 0.0000
Best: Ensemble 1 (1.0000) Worst: Ensemble 1 (1.0000) Early stops: 5/5
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣀⣀⣀⣀⣀⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣀⣤⡶⠟⠛⠛⠛⠉⠉⠛⠛⠻⢶⣦⣄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣠⣴⠟⠋⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠙⢿⣤⠀⠀⠀⠀⠀⣠⣤⡀⢀⣀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣾⠟⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠹⣷⡀⠀⠀⢸⡏⠀⠹⠋⢹⣧
⠀⠀⠀⠀⠀⠀⠀⠀⠀⢠⡿⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠹⣿⡀⠀⠀⠻⢶⣤⣴⠾⠃
⠀⠀⠀⠀⠀⠀⠀⠀⢠⣿⠃⠀⠀⢀⣀⡀⠀⠀⠀⠀⠀⠀⣀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢻⣧⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⣾⡇⠀⠀⠀⢿⣿⡏⠀⡀⠀⡀⠀⣾⣿⣷⠀⠀⠀⠀⠀⠀⠀⠀⠀⠸⣿⡀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⢀⣿⠁⠐⡀⢠⠀⠀⡄⠀⠙⠛⠁⠀⠈⠉⠁⠀⠐⠄⠀⠀⠀⠀⠀⠀⠀⢿⣇⠀⠀⠀⠀⠀⠀⠀
⠀ Training complete!
=== Running Cascade Inference (routing=forest) ===
──────────────────────────────────────────
Test Results [forest]
──────────────────────────────────────────
OA 0.9736 ███████████████████░
mIoU 0.8092 ████████████████░░░░
Dice 0.8510 █████████████████░░░
Precision 0.8801 █████████████████░░░
Recall 0.8362 ████████████████░░░░
AA 0.8362 ████████████████░░░░
kappa 0.9698 ███████████████████░
──────────────────────────────────────────
Saved → runs/indian_pines/test_results.csv
Per-class IoU:
Class 1 0.8182 ████████████████
Class 2 0.9650 ███████████████████
Class 3 0.9520 ███████████████████
Class 4 0.8656 █████████████████
Class 5 0.9256 ██████████████████
Class 6 0.9883 ███████████████████
Class 7 0.0000
Class 8 0.8899 █████████████████
Class 9 0.1765 ███
Class 10 0.9536 ███████████████████
Class 11 0.9800 ███████████████████
Class 12 0.9258 ██████████████████
Class 13 0.9660 ███████████████████
Class 14 0.9547 ███████████████████
Class 15 0.9747 ███████████████████
Class 16 0.6119 ████████████
Saved → runs/indian_pines/class_report_forest.csv
Saved → runs/indian_pines/spt_models.pkl
What's next?
# Evaluate all routing modes:
ghost predict \
--data /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_corrected.mat \
--gt /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_gt.mat \
--model runs/indian_pines/spt_models.pkl \
--routing all --out-dir runs/indian_pines
# Visualize predictions:
ghost visualize \
--data /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_corrected.mat \
--gt /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_gt.mat \
--model runs/indian_pines/spt_models.pkl \
--out-dir runs/indian_pines
(ghost_test) Darth-Ishu@Executor:~$ ghost demo
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣀⣤⣤⡤⣤⢤⣤⣄⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣠⡾⠛⠉⠀⠀⠀⠀⠀⠀⠉⠻⣦⡄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣾⠏⠀⠀⠀⠀⣀⡀⠀⠀⠀⠀⠀⠈⢻⡆⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣼⠯⠀⠀⠀⢰⣿⣿⣿⣆⡀⣴⣾⣿⣦⡈⢿⡄⠀ G H O S T
⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⣏⠀⠀⠀⠀⢸⣿⣿⣿⡿⠇⣿⣿⣿⣿⠆⠘⣷⡀ Generalizable Hyperspectral
⠀⠀⠀⠀⠀⠀⠀⠀⠀⣿⠇⠀⠀⠀⠀⠈⠿⠿⡿⠃⠀⢿⣿⣿⠏⠀⠀⢹⣗ Observation & Segmentation
⠀⠀⠀⠀⠀⠀⠀⠀⢸⡏⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠀⠀⠀⠀⠀⣿⡀ Toolkit
Bundled Indian Pines dataset:
Data : /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_corrected.mat
GT : /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_gt.mat
Run this to start training:
ghost train_spt \
--data /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_corrected.mat \
--gt /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_gt.mat \
--loss dice \
--base_filters 32 --num_filters 8 \
--ensembles 5 --leaf_ensembles 3 \
--epochs 400 --patience 50 --min_epochs 40 \
--out-dir runs/indian_pines
(ghost_test) Darth-Ishu@Executor:~$ ghost train_spt \
--data /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_corrected.mat \
--gt /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_gt.mat \
--loss dice \
--base_filters 64 --num_filters 16 \
--ensembles 5 --leaf_ensembles 3 \
--epochs 400 --patience 100 --min_epochs 40 \
--out-dir runs/indian_pines
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣀⣤⣤⡤⣤⢤⣤⣄⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣠⡾⠛⠉⠀⠀⠀⠀⠀⠀⠉⠻⣦⡄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣾⠏⠀⠀⠀⠀⣀⡀⠀⠀⠀⠀⠀⠈⢻⡆⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣼⠯⠀⠀⠀⢰⣿⣿⣿⣆⡀⣴⣾⣿⣦⡈⢿⡄⠀ G H O S T
⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⣏⠀⠀⠀⠀⢸⣿⣿⣿⡿⠇⣿⣿⣿⣿⠆⠘⣷⡀ Generalizable Hyperspectral
⠀⠀⠀⠀⠀⠀⠀⠀⠀⣿⠇⠀⠀⠀⠀⠈⠿⠿⡿⠃⠀⢿⣿⣿⠏⠀⠀⢹⣗ Observation & Segmentation
⠀⠀⠀⠀⠀⠀⠀⠀⢸⡏⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠀⠀⠀⠀⠀⣿⡀ Toolkit
Ensemble routing selected — skipping SSM pretraining (set --ssm_epochs manually to override)
train: 2045 pixels across 16 classes
val: 1018 pixels across 16 classes
test: 7186 pixels across 16 classes
════════════════════════════════════════════════════════════
GHOST SPT Training
Device : cuda (NVIDIA GeForce RTX 3050 6GB Laptop GPU, 6 GB)
Loss : dice
Routing : forest
Train pixels : 2,045 (16 classes)
Val pixels : 1,018
Test pixels : 7,186
Output : runs/indian_pines
════════════════════════════════════════════════════════════
=== SSM Encoder ===
⚠ SSM Encoder is experimental — results may vary
SSM will be saved to: runs/indian_pines/ssm_pretrained.pt
=== Pre-training SSM Encoder ===
d_model=64 d_state=16 epochs=1 batch_size=512
Train pixels: 2045 | Val pixels: 1018
SSM pre-trained | Best Val Acc: 0.0000 | Saved → runs/indian_pines/ssm_pretrained.pt
Pre-computing fingerprint map ...
Fingerprint map shape: (145, 145, 64)
=== Building Spectral Partition Tree ===
Building Spectral Partition Tree for 16 classes
Pixel counts: {1: 46, 2: 1428, 3: 830, 4: 237, 5: 483, 6: 730, 7: 28, 8: 478, 9: 20, 10: 972, 11: 2455, 12: 593, 13: 205, 14: 1265, 15: 386, 16: 93}
Node (depth 0): classes [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]
├── LEFT:
Node (depth 1): classes [7, 2, 3, 10, 11, 13]
├── LEFT:
Node (depth 2): classes [3, 7, 2, 13]
├── LEFT:
Node (depth 3): classes [7, 2]
└── RIGHT:
Node (depth 3): classes [13, 3]
└── RIGHT:
Node (depth 2): classes [11, 10]
└── RIGHT:
Node (depth 1): classes [8, 1, 4, 5, 6, 9, 12, 14, 15, 16]
├── LEFT:
Node (depth 2): classes [6, 1, 5, 14, 15, 16]
├── LEFT:
Node (depth 3): classes [6, 5, 15, 16]
└── RIGHT:
Node (depth 3): classes [1, 14]
└── RIGHT:
Node (depth 2): classes [9, 8, 4, 12]
├── LEFT:
Node (depth 3): classes [8]
└── RIGHT:
Node (depth 3): classes [4, 9, 12]
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣀⣤⣤⡤⣤⢤⣤⣄⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣠⡾⠛⠉⠀⠀⠀⠀⠀⠀⠉⠻⣦⡄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣾⠏⠀⠀⠀⠀⣀⡀⠀⠀⠀⠀⠀⠈⢻⡆⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣼⠯⠀⠀⠀⢰⣿⣿⣿⣆⡀⣴⣾⣿⣦⡈⢿⡄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⣏⠀⠀⠀⠀⢸⣿⣿⣿⡿⠇⣿⣿⣿⣿⠆⠘⣷⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⣿⠇⠀⠀⠀⠀⠈⠿⠿⡿⠃⠀⢿⣿⣿⠏⠀⠀⢹⣗⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⢸⡏⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠀⠀⠀⠀⠀⣿⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀ Training initiated...
=== Training SPT Ensembles (loss=dice) ===
════════════════════════════════════════════════════════════
Node 'root'
Classes : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16]
Local classes : 16 Epochs: 400 Ensembles: 5 Loss: dice
Train pixels : 2045 Val pixels: 1018
════════════════════════════════════════════════════════════
── Ensemble 1/5 seed=42 node='root'
Epoch 20/400 | █░░░░░░░░░░░░░░░░░░░░░░░ 5% | Loss 1.7303 | OA 0.1935 | mIoU 0.0592 | κ 0.1268
Epoch 40/400 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.4330 | OA 0.3811 | mIoU 0.0921 | κ 0.2262
Epoch 60/400 | ███░░░░░░░░░░░░░░░░░░░░░ 15% | Loss 1.0880 | OA 0.5501 | mIoU 0.1755 | κ 0.4680
Epoch 80/400 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.9168 | OA 0.7544 | mIoU 0.3896 | κ 0.7158
Epoch 100/400 | ██████░░░░░░░░░░░░░░░░░░ 25% | Loss 0.7392 | OA 0.7750 | mIoU 0.4144 | κ 0.7436
Epoch 120/400 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.6452 | OA 0.8743 | mIoU 0.5071 | κ 0.8565
Epoch 140/400 | ████████░░░░░░░░░░░░░░░░ 35% | Loss 0.5234 | OA 0.8173 | mIoU 0.4669 | κ 0.7926
Epoch 160/400 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.4843 | OA 0.9470 | mIoU 0.6566 | κ 0.9396
Epoch 180/400 | ██████████░░░░░░░░░░░░░░ 45% | Loss 0.3822 | OA 0.9666 | mIoU 0.7025 | κ 0.9619
Epoch 200/400 | ████████████░░░░░░░░░░░░ 50% | Loss 0.3098 | OA 0.9234 | mIoU 0.6292 | κ 0.9128
Epoch 220/400 | █████████████░░░░░░░░░░░ 55% | Loss 0.3242 | OA 0.9204 | mIoU 0.6306 | κ 0.9097
Epoch 240/400 | ██████████████░░░░░░░░░░ 60% | Loss 0.2550 | OA 0.9695 | mIoU 0.7063 | κ 0.9653
Epoch 260/400 | ███████████████░░░░░░░░░ 65% | Loss 0.2489 | OA 0.9676 | mIoU 0.7040 | κ 0.9629
Epoch 280/400 | ████████████████░░░░░░░░ 70% | Loss 0.2335 | OA 0.9745 | mIoU 0.7157 | κ 0.9708
Epoch 300/400 | ██████████████████░░░░░░ 75% | Loss 0.2379 | OA 0.9725 | mIoU 0.7061 | κ 0.9686
Epoch 320/400 | ███████████████████░░░░░ 80% | Loss 0.2614 | OA 0.9754 | mIoU 0.7101 | κ 0.9720
Epoch 340/400 | ████████████████████░░░░ 85% | Loss 0.2765 | OA 0.9745 | mIoU 0.7169 | κ 0.9709
Epoch 360/400 | █████████████████████░░░ 90% | Loss 0.2233 | OA 0.9754 | mIoU 0.7160 | κ 0.9720
Epoch 380/400 | ██████████████████████░░ 95% | Loss 0.2426 | OA 0.9764 | mIoU 0.7190 | κ 0.9731
Epoch 400/400 | ████████████████████████ 100% | Loss 0.2260 | OA 0.9774 | mIoU 0.7217 | κ 0.9742
✓ Ensemble 1/5 done Best @ epoch 400 mIoU 0.7217 OA 0.9774 AA 0.7504 κ 0.9742
6m 17s elapsed | GPU 76°C | 4307 / 6144 MB | 100% util
── Ensemble 2/5 seed=43 node='root'
Epoch 20/400 | █░░░░░░░░░░░░░░░░░░░░░░░ 5% | Loss 1.6586 | OA 0.3222 | mIoU 0.0600 | κ 0.1406
Epoch 40/400 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.3553 | OA 0.3870 | mIoU 0.0940 | κ 0.2266
Epoch 60/400 | ███░░░░░░░░░░░░░░░░░░░░░ 15% | Loss 1.0973 | OA 0.4116 | mIoU 0.1193 | κ 0.2580
Epoch 80/400 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.9196 | OA 0.5943 | mIoU 0.2410 | κ 0.5091
Epoch 100/400 | ██████░░░░░░░░░░░░░░░░░░ 25% | Loss 0.7952 | OA 0.7083 | mIoU 0.3691 | κ 0.6541
Epoch 120/400 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.6135 | OA 0.7800 | mIoU 0.4703 | κ 0.7426
Epoch 140/400 | ████████░░░░░░░░░░░░░░░░ 35% | Loss 0.4849 | OA 0.8851 | mIoU 0.5478 | κ 0.8664
Epoch 160/400 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.4759 | OA 0.8468 | mIoU 0.5646 | κ 0.8211
Epoch 180/400 | ██████████░░░░░░░░░░░░░░ 45% | Loss 0.3431 | OA 0.8998 | mIoU 0.6187 | κ 0.8837
Epoch 200/400 | ████████████░░░░░░░░░░░░ 50% | Loss 0.3778 | OA 0.7446 | mIoU 0.3979 | κ 0.7082
Epoch 220/400 | █████████████░░░░░░░░░░░ 55% | Loss 0.2724 | OA 0.8644 | mIoU 0.5853 | κ 0.8432
Epoch 240/400 | ██████████████░░░░░░░░░░ 60% | Loss 0.2720 | OA 0.9381 | mIoU 0.6530 | κ 0.9293
Epoch 260/400 | ███████████████░░░░░░░░░ 65% | Loss 0.2535 | OA 0.9273 | mIoU 0.6519 | κ 0.9165
Epoch 280/400 | ████████████████░░░░░░░░ 70% | Loss 0.1893 | OA 0.8694 | mIoU 0.6147 | κ 0.8527
Epoch 300/400 | ██████████████████░░░░░░ 75% | Loss 0.1722 | OA 0.9676 | mIoU 0.7180 | κ 0.9629
Epoch 320/400 | ███████████████████░░░░░ 80% | Loss 0.1854 | OA 0.9705 | mIoU 0.7206 | κ 0.9663
Epoch 340/400 | ████████████████████░░░░ 85% | Loss 0.2434 | OA 0.9705 | mIoU 0.7171 | κ 0.9663
Epoch 360/400 | █████████████████████░░░ 90% | Loss 0.1861 | OA 0.9764 | mIoU 0.7258 | κ 0.9730
Epoch 380/400 | ██████████████████████░░ 95% | Loss 0.1685 | OA 0.9764 | mIoU 0.7284 | κ 0.9730
Epoch 400/400 | ████████████████████████ 100% | Loss 0.1750 | OA 0.9764 | mIoU 0.7270 | κ 0.9730
✓ Ensemble 2/5 done Best @ epoch 380 mIoU 0.7284 OA 0.9764 AA 0.7445 κ 0.9730
12m 38s elapsed | GPU 80°C | 4307 / 6144 MB | 100% util
── Ensemble 3/5 seed=44 node='root'
Epoch 20/400 | █░░░░░░░░░░░░░░░░░░░░░░░ 5% | Loss 1.6605 | OA 0.3360 | mIoU 0.0641 | κ 0.1604
Epoch 40/400 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.3698 | OA 0.3870 | mIoU 0.0824 | κ 0.2331
Epoch 60/400 | ███░░░░░░░░░░░░░░░░░░░░░ 15% | Loss 1.0826 | OA 0.4528 | mIoU 0.1455 | κ 0.3342
Epoch 80/400 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.8970 | OA 0.5874 | mIoU 0.2334 | κ 0.5062
Epoch 100/400 | ██████░░░░░░░░░░░░░░░░░░ 25% | Loss 0.7563 | OA 0.6719 | mIoU 0.2964 | κ 0.6145
Epoch 120/400 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.5708 | OA 0.7544 | mIoU 0.3671 | κ 0.7162
Epoch 140/400 | ████████░░░░░░░░░░░░░░░░ 35% | Loss 0.4706 | OA 0.9126 | mIoU 0.5749 | κ 0.9000
Epoch 160/400 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.3993 | OA 0.9509 | mIoU 0.6856 | κ 0.9439
Epoch 180/400 | ██████████░░░░░░░░░░░░░░ 45% | Loss 0.3814 | OA 0.9607 | mIoU 0.6996 | κ 0.9552
Epoch 200/400 | ████████████░░░░░░░░░░░░ 50% | Loss 0.3137 | OA 0.9607 | mIoU 0.6940 | κ 0.9552
Epoch 220/400 | █████████████░░░░░░░░░░░ 55% | Loss 0.2995 | OA 0.9676 | mIoU 0.7387 | κ 0.9630
Epoch 240/400 | ██████████████░░░░░░░░░░ 60% | Loss 0.2504 | OA 0.9745 | mIoU 0.7762 | κ 0.9708
Epoch 260/400 | ███████████████░░░░░░░░░ 65% | Loss 0.2128 | OA 0.9725 | mIoU 0.7385 | κ 0.9686
Epoch 280/400 | ████████████████░░░░░░░░ 70% | Loss 0.2277 | OA 0.9754 | mIoU 0.7571 | κ 0.9720
Epoch 300/400 | ██████████████████░░░░░░ 75% | Loss 0.2323 | OA 0.9735 | mIoU 0.7389 | κ 0.9697
Epoch 320/400 | ███████████████████░░░░░ 80% | Loss 0.2599 | OA 0.9774 | mIoU 0.7622 | κ 0.9742
Epoch 340/400 | ████████████████████░░░░ 85% | Loss 0.2190 | OA 0.9754 | mIoU 0.7467 | κ 0.9720
⚡ Early stop: no improvement for 100 epochs at epoch 340
✓ Ensemble 3/5 done Best @ epoch 240 mIoU 0.7762 OA 0.9745 AA 0.8075 κ 0.9708
18m 3s elapsed | GPU 81°C | 4307 / 6144 MB | 100% util
── Ensemble 4/5 seed=45 node='root'
Epoch 20/400 | █░░░░░░░░░░░░░░░░░░░░░░░ 5% | Loss 1.6358 | OA 0.3242 | mIoU 0.0748 | κ 0.1774
Epoch 40/400 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.2928 | OA 0.3674 | mIoU 0.0781 | κ 0.1938
Epoch 60/400 | ███░░░░░░░░░░░░░░░░░░░░░ 15% | Loss 1.0039 | OA 0.4813 | mIoU 0.1408 | κ 0.3642
Epoch 80/400 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.7634 | OA 0.6974 | mIoU 0.3043 | κ 0.6508
Epoch 100/400 | ██████░░░░░░░░░░░░░░░░░░ 25% | Loss 0.6649 | OA 0.7043 | mIoU 0.3488 | κ 0.6553
Epoch 120/400 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.5566 | OA 0.8310 | mIoU 0.4723 | κ 0.8051
Epoch 140/400 | ████████░░░░░░░░░░░░░░░░ 35% | Loss 0.4840 | OA 0.8743 | mIoU 0.5417 | κ 0.8551
Epoch 160/400 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.4235 | OA 0.9293 | mIoU 0.6299 | κ 0.9188
Epoch 180/400 | ██████████░░░░░░░░░░░░░░ 45% | Loss 0.3754 | OA 0.8752 | mIoU 0.5617 | κ 0.8575
Epoch 200/400 | ████████████░░░░░░░░░░░░ 50% | Loss 0.3494 | OA 0.8360 | mIoU 0.5262 | κ 0.8090
Epoch 220/400 | █████████████░░░░░░░░░░░ 55% | Loss 0.3033 | OA 0.6857 | mIoU 0.4058 | κ 0.6493
Epoch 240/400 | ██████████████░░░░░░░░░░ 60% | Loss 0.2445 | OA 0.9273 | mIoU 0.6709 | κ 0.9166
Epoch 260/400 | ███████████████░░░░░░░░░ 65% | Loss 0.2473 | OA 0.9695 | mIoU 0.7378 | κ 0.9652
Epoch 280/400 | ████████████████░░░░░░░░ 70% | Loss 0.2109 | OA 0.9450 | mIoU 0.7216 | κ 0.9371
Epoch 300/400 | ██████████████████░░░░░░ 75% | Loss 0.2174 | OA 0.9754 | mIoU 0.7582 | κ 0.9719
Epoch 320/400 | ███████████████████░░░░░ 80% | Loss 0.2093 | OA 0.9784 | mIoU 0.7768 | κ 0.9753
Epoch 340/400 | ████████████████████░░░░ 85% | Loss 0.2020 | OA 0.9774 | mIoU 0.7719 | κ 0.9742
Epoch 360/400 | █████████████████████░░░ 90% | Loss 0.2337 | OA 0.9784 | mIoU 0.7736 | κ 0.9753
Epoch 380/400 | ██████████████████████░░ 95% | Loss 0.2277 | OA 0.9794 | mIoU 0.7774 | κ 0.9765
Epoch 400/400 | ████████████████████████ 100% | Loss 0.2314 | OA 0.9774 | mIoU 0.7719 | κ 0.9742
✓ Ensemble 4/5 done Best @ epoch 380 mIoU 0.7774 OA 0.9794 AA 0.7967 κ 0.9765
24m 25s elapsed | GPU 83°C | 4307 / 6144 MB | 100% util
── Ensemble 5/5 seed=46 node='root'
Epoch 20/400 | █░░░░░░░░░░░░░░░░░░░░░░░ 5% | Loss 1.6816 | OA 0.3055 | mIoU 0.0613 | κ 0.1094
Epoch 40/400 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.3498 | OA 0.3733 | mIoU 0.0892 | κ 0.2019
Epoch 60/400 | ███░░░░░░░░░░░░░░░░░░░░░ 15% | Loss 1.0831 | OA 0.4980 | mIoU 0.1778 | κ 0.3821
Epoch 80/400 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.8779 | OA 0.5639 | mIoU 0.2493 | κ 0.4924
Epoch 100/400 | ██████░░░░░░░░░░░░░░░░░░ 25% | Loss 0.7049 | OA 0.8173 | mIoU 0.4416 | κ 0.7896
Epoch 120/400 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.5657 | OA 0.8232 | mIoU 0.4642 | κ 0.7968
Epoch 140/400 | ████████░░░░░░░░░░░░░░░░ 35% | Loss 0.5469 | OA 0.9303 | mIoU 0.6042 | κ 0.9201
Epoch 160/400 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.4205 | OA 0.9352 | mIoU 0.6387 | κ 0.9258
Epoch 180/400 | ██████████░░░░░░░░░░░░░░ 45% | Loss 0.3810 | OA 0.9342 | mIoU 0.6683 | κ 0.9248
Epoch 200/400 | ████████████░░░░░░░░░░░░ 50% | Loss 0.3018 | OA 0.9676 | mIoU 0.7037 | κ 0.9630
Epoch 220/400 | █████████████░░░░░░░░░░░ 55% | Loss 0.2453 | OA 0.9077 | mIoU 0.6587 | κ 0.8952
Epoch 240/400 | ██████████████░░░░░░░░░░ 60% | Loss 0.2506 | OA 0.9833 | mIoU 0.7804 | κ 0.9809
Epoch 260/400 | ███████████████░░░░░░░░░ 65% | Loss 0.1932 | OA 0.9804 | mIoU 0.7804 | κ 0.9775
Epoch 280/400 | ████████████████░░░░░░░░ 70% | Loss 0.1919 | OA 0.9833 | mIoU 0.7874 | κ 0.9809
Epoch 300/400 | ██████████████████░░░░░░ 75% | Loss 0.1774 | OA 0.9843 | mIoU 0.7919 | κ 0.9820
Epoch 320/400 | ███████████████████░░░░░ 80% | Loss 0.1683 | OA 0.9853 | mIoU 0.7834 | κ 0.9832
Epoch 340/400 | ████████████████████░░░░ 85% | Loss 0.2059 | OA 0.9853 | mIoU 0.7849 | κ 0.9832
Epoch 360/400 | █████████████████████░░░ 90% | Loss 0.1734 | OA 0.9853 | mIoU 0.7886 | κ 0.9832
Epoch 380/400 | ██████████████████████░░ 95% | Loss 0.1691 | OA 0.9853 | mIoU 0.7844 | κ 0.9832
Epoch 400/400 | ████████████████████████ 100% | Loss 0.1760 | OA 0.9843 | mIoU 0.7775 | κ 0.9821
⚡ Early stop: no improvement for 100 epochs at epoch 400
✓ Ensemble 5/5 done Best @ epoch 300 mIoU 0.7919 OA 0.9843 AA 0.8054 κ 0.9820
30m 46s elapsed | GPU 82°C | 4307 / 6144 MB | 100% util
Node 'root' — 5 ensembles trained in 30m 46s
mIoU 0.7217 0.7284 0.7762 0.7774 0.7919 mean 0.7591 ± 0.0284
Best: Ensemble 5 (0.7919) Worst: Ensemble 1 (0.7217) Early stops: 2/5
Training SSSR router for node 'root' ...
Router: train_acc=0.5775 | val_acc=0.5776 | left=1182 right=863
Router trained.
════════════════════════════════════════════════════════════
Node 'root_L'
Classes : [7, 2, 3, 10, 11, 13]
Local classes : 6 Epochs: 200 Ensembles: 5 Loss: dice
Train pixels : 1182 Val pixels: 589
════════════════════════════════════════════════════════════
── Ensemble 1/5 seed=42 node='root_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.2088 | OA 0.4329 | mIoU 0.0921 | κ 0.0488
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.5994 | OA 0.4363 | mIoU 0.0878 | κ 0.0500
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.2948 | OA 0.8370 | mIoU 0.5049 | κ 0.7632
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.2371 | OA 0.9542 | mIoU 0.6891 | κ 0.9354
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.1541 | OA 0.9423 | mIoU 0.7406 | κ 0.9187
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.1183 | OA 0.9032 | mIoU 0.7711 | κ 0.8670
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0952 | OA 0.9728 | mIoU 0.8613 | κ 0.9621
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0917 | OA 0.9949 | mIoU 0.9913 | κ 0.9929
⚡ Early stop: mIoU ≥ 0.99 at epoch 160
✓ Ensemble 1/5 done Best @ epoch 160 mIoU 0.9913 OA 0.9949 AA 0.9959 κ 0.9929
33m 20s elapsed | GPU 80°C | 4307 / 6144 MB | 95% util
── Ensemble 2/5 seed=43 node='root_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.0817 | OA 0.4397 | mIoU 0.1007 | κ 0.0605
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.6813 | OA 0.4601 | mIoU 0.1097 | κ 0.0929
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.3565 | OA 0.5552 | mIoU 0.2010 | κ 0.2814
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.2007 | OA 0.7827 | mIoU 0.4588 | κ 0.6772
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.1572 | OA 0.9406 | mIoU 0.7664 | κ 0.9163
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0913 | OA 0.9728 | mIoU 0.7944 | κ 0.9623
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0525 | OA 0.9236 | mIoU 0.8894 | κ 0.8945
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0398 | OA 0.9966 | mIoU 0.9414 | κ 0.9953
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0215 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 180
✓ Ensemble 2/5 done Best @ epoch 180 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
36m 12s elapsed | GPU 81°C | 4309 / 6144 MB | 80% util
── Ensemble 3/5 seed=44 node='root_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.1883 | OA 0.4295 | mIoU 0.0963 | κ 0.0583
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.6265 | OA 0.5195 | mIoU 0.1752 | κ 0.2613
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.3035 | OA 0.6876 | mIoU 0.3549 | κ 0.5545
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.1979 | OA 0.9049 | mIoU 0.6845 | κ 0.8669
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.1438 | OA 0.9083 | mIoU 0.6797 | κ 0.8718
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.1151 | OA 0.9593 | mIoU 0.7637 | κ 0.9429
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0959 | OA 0.9660 | mIoU 0.7754 | κ 0.9527
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0818 | OA 0.9626 | mIoU 0.7767 | κ 0.9480
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0640 | OA 0.9864 | mIoU 0.8771 | κ 0.9812
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0392 | OA 0.9796 | mIoU 0.9194 | κ 0.9716
✓ Ensemble 3/5 done Best @ epoch 200 mIoU 0.9194 OA 0.9796 AA 0.9821 κ 0.9716
39m 23s elapsed | GPU 84°C | 4309 / 6144 MB | 99% util
── Ensemble 4/5 seed=45 node='root_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.2623 | OA 0.4380 | mIoU 0.1096 | κ 0.0865
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.8198 | OA 0.4618 | mIoU 0.1225 | κ 0.1066
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.3631 | OA 0.6316 | mIoU 0.2891 | κ 0.4638
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.2397 | OA 0.9168 | mIoU 0.7185 | κ 0.8823
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.1813 | OA 0.9100 | mIoU 0.9049 | κ 0.8722
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.1455 | OA 0.9253 | mIoU 0.8140 | κ 0.8957
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.1295 | OA 0.9440 | mIoU 0.8553 | κ 0.9214
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.1108 | OA 0.9728 | mIoU 0.9032 | κ 0.9623
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0828 | OA 0.9440 | mIoU 0.8457 | κ 0.9220
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0691 | OA 0.9253 | mIoU 0.8338 | κ 0.8974
⚡ Early stop: no improvement for 100 epochs at epoch 200
✓ Ensemble 4/5 done Best @ epoch 100 mIoU 0.9049 OA 0.9100 AA 0.9345 κ 0.8722
42m 35s elapsed | GPU 86°C | 4309 / 6144 MB | 100% util
── Ensemble 5/5 seed=46 node='root_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.1599 | OA 0.4312 | mIoU 0.0910 | κ 0.0420
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.7112 | OA 0.5178 | mIoU 0.1579 | κ 0.2112
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.3511 | OA 0.6757 | mIoU 0.3833 | κ 0.5031
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.1898 | OA 0.9015 | mIoU 0.6962 | κ 0.8598
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.1368 | OA 0.9610 | mIoU 0.7569 | κ 0.9455
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.1076 | OA 0.9491 | mIoU 0.7565 | κ 0.9286
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.1096 | OA 0.9915 | mIoU 0.8127 | κ 0.9882
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0860 | OA 0.9694 | mIoU 0.8691 | κ 0.9573
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0553 | OA 0.9830 | mIoU 0.9693 | κ 0.9764
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0271 | OA 0.9949 | mIoU 0.9916 | κ 0.9929
⚡ Early stop: mIoU ≥ 0.99 at epoch 200
✓ Ensemble 5/5 done Best @ epoch 200 mIoU 0.9916 OA 0.9949 AA 0.9959 κ 0.9929
45m 48s elapsed | GPU 86°C | 4309 / 6144 MB | 100% util
Node 'root_L' — 5 ensembles trained in 15m 0s
mIoU 0.9913 1.0000 0.9194 0.9049 0.9916 mean 0.9614 ± 0.0406
Best: Ensemble 2 (1.0000) Worst: Ensemble 4 (0.9049) Early stops: 4/5
Training SSSR router for node 'root_L' ...
Router: train_acc=0.5525 | val_acc=0.5484 | left=497 right=685
Router trained.
════════════════════════════════════════════════════════════
Node 'root_L_L'
Classes : [3, 7, 2, 13]
Local classes : 4 Epochs: 200 Ensembles: 5 Loss: dice
Train pixels : 497 Val pixels: 247
════════════════════════════════════════════════════════════
── Ensemble 1/5 seed=42 node='root_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.9074 | OA 0.5830 | mIoU 0.1583 | κ 0.0553
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.3751 | OA 0.6680 | mIoU 0.3021 | κ 0.4402
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.1421 | OA 0.4939 | mIoU 0.2350 | κ 0.2817
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0438 | OA 0.5547 | mIoU 0.3962 | κ 0.4185
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0198 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 100
✓ Ensemble 1/5 done Best @ epoch 100 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
47m 25s elapsed | GPU 87°C | 4309 / 6144 MB | 100% util
── Ensemble 2/5 seed=43 node='root_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.8642 | OA 0.5951 | mIoU 0.1653 | κ 0.0718
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.3078 | OA 0.5223 | mIoU 0.3449 | κ 0.2759
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0816 | OA 0.5263 | mIoU 0.2772 | κ 0.3078
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0175 | OA 0.8947 | mIoU 0.7928 | κ 0.8171
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0156 | OA 0.9757 | mIoU 0.9726 | κ 0.9563
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0041 | OA 0.9960 | mIoU 0.9953 | κ 0.9927
⚡ Early stop: mIoU ≥ 0.99 at epoch 120
✓ Ensemble 2/5 done Best @ epoch 120 mIoU 0.9953 OA 0.9960 AA 0.9982 κ 0.9927
49m 21s elapsed | GPU 87°C | 4309 / 6144 MB | 100% util
── Ensemble 3/5 seed=44 node='root_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.9623 | OA 0.3117 | mIoU 0.0916 | κ 0.0214
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.3417 | OA 0.4008 | mIoU 0.2164 | κ 0.1020
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.1053 | OA 0.5061 | mIoU 0.3473 | κ 0.3065
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0467 | OA 0.6032 | mIoU 0.3983 | κ 0.4484
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0267 | OA 0.9676 | mIoU 0.8378 | κ 0.9428
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0134 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 120
✓ Ensemble 3/5 done Best @ epoch 120 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
51m 17s elapsed | GPU 85°C | 4309 / 6144 MB | 100% util
── Ensemble 4/5 seed=45 node='root_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.9918 | OA 0.4291 | mIoU 0.1680 | κ 0.1170
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.3434 | OA 0.3320 | mIoU 0.0833 | κ -0.0025
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.1578 | OA 0.5628 | mIoU 0.3326 | κ 0.4119
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0366 | OA 0.6478 | mIoU 0.5114 | κ 0.5257
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0192 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 100
✓ Ensemble 4/5 done Best @ epoch 100 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
52m 53s elapsed | GPU 85°C | 4309 / 6144 MB | 100% util
── Ensemble 5/5 seed=46 node='root_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.0045 | OA 0.5830 | mIoU 0.1980 | κ 0.1316
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.4343 | OA 0.5425 | mIoU 0.2205 | κ 0.2636
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.1606 | OA 0.7328 | mIoU 0.4177 | κ 0.5716
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0997 | OA 0.8988 | mIoU 0.6326 | κ 0.8280
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0494 | OA 0.8462 | mIoU 0.6572 | κ 0.7541
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0167 | OA 0.9798 | mIoU 0.8771 | κ 0.9635
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0169 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 140
✓ Ensemble 5/5 done Best @ epoch 140 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
55m 39s elapsed | GPU 87°C | 4309 / 6144 MB | 100% util
Node 'root_L_L' — 5 ensembles trained in 9m 51s
mIoU 1.0000 0.9953 1.0000 1.0000 1.0000 mean 0.9991 ± 0.0019
Best: Ensemble 1 (1.0000) Worst: Ensemble 2 (0.9953) Early stops: 5/5
Training SSSR router for node 'root_L_L' ...
Router: train_acc=0.4386 | val_acc=0.4332 | left=290 right=207
Router trained.
════════════════════════════════════════════════════════════
Node 'root_L_L_L'
Classes : [7, 2]
Local classes : 2 Epochs: 200 Ensembles: 3 Loss: dice
Train pixels : 290 Val pixels: 144
════════════════════════════════════════════════════════════
── Ensemble 1/3 seed=42 node='root_L_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.4918 | OA 0.9722 | mIoU 0.4861 | κ -0.0105
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.1166 | OA 0.9861 | mIoU 0.4931 | κ 0.0000
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0486 | OA 0.7431 | mIoU 0.3960 | κ 0.0734
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0058 | OA 0.4722 | mIoU 0.2456 | κ 0.0239
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0040 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 100
✓ Ensemble 1/3 done Best @ epoch 100 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
57m 16s elapsed | GPU 85°C | 4309 / 6144 MB | 100% util
── Ensemble 2/3 seed=43 node='root_L_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.4644 | OA 0.9861 | mIoU 0.4931 | κ 0.0000
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.0953 | OA 0.9931 | mIoU 0.7465 | κ 0.6636
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0111 | OA 0.9375 | mIoU 0.5683 | κ 0.2918
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0044 | OA 0.9861 | mIoU 0.7430 | κ 0.6604
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0041 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 100
✓ Ensemble 2/3 done Best @ epoch 100 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
58m 53s elapsed | GPU 86°C | 4309 / 6144 MB | 86% util
── Ensemble 3/3 seed=44 node='root_L_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.3889 | OA 0.9792 | mIoU 0.4896 | κ -0.0047
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.0968 | OA 0.7153 | mIoU 0.3800 | κ 0.0646
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0118 | OA 0.4306 | mIoU 0.2232 | κ 0.0199
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0123 | OA 0.9861 | mIoU 0.7430 | κ 0.6604
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0034 | OA 0.9931 | mIoU 0.8298 | κ 0.7966
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0023 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 120
✓ Ensemble 3/3 done Best @ epoch 120 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 0m 48s elapsed | GPU 85°C | 4309 / 6144 MB | 81% util
Node 'root_L_L_L' — 3 ensembles trained in 5m 9s
mIoU 1.0000 1.0000 1.0000 mean 1.0000 ± 0.0000
Best: Ensemble 1 (1.0000) Worst: Ensemble 1 (1.0000) Early stops: 3/3
════════════════════════════════════════════════════════════
Node 'root_L_L_R'
Classes : [13, 3]
Local classes : 2 Epochs: 200 Ensembles: 3 Loss: dice
Train pixels : 207 Val pixels: 103
════════════════════════════════════════════════════════════
── Ensemble 1/3 seed=42 node='root_L_L_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.4822 | OA 0.8058 | mIoU 0.4486 | κ 0.1166
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.0238 | OA 0.9806 | mIoU 0.9631 | κ 0.9383
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0016 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 60
✓ Ensemble 1/3 done Best @ epoch 60 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 1m 46s elapsed | GPU 85°C | 4309 / 6144 MB | 86% util
── Ensemble 2/3 seed=43 node='root_L_L_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.4163 | OA 0.8058 | mIoU 0.4029 | κ 0.0000
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.0116 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 40
✓ Ensemble 2/3 done Best @ epoch 40 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 2m 25s elapsed | GPU 85°C | 4309 / 6144 MB | 94% util
── Ensemble 3/3 seed=44 node='root_L_L_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.3157 | OA 0.7961 | mIoU 0.4415 | κ 0.0969
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.0134 | OA 0.9903 | mIoU 0.9690 | κ 0.9684
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0022 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 60
✓ Ensemble 3/3 done Best @ epoch 60 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 3m 22s elapsed | GPU 84°C | 4309 / 6144 MB | 100% util
Node 'root_L_L_R' — 3 ensembles trained in 2m 33s
mIoU 1.0000 1.0000 1.0000 mean 1.0000 ± 0.0000
Best: Ensemble 1 (1.0000) Worst: Ensemble 1 (1.0000) Early stops: 3/3
════════════════════════════════════════════════════════════
Node 'root_L_R'
Classes : [11, 10]
Local classes : 2 Epochs: 200 Ensembles: 3 Loss: dice
Train pixels : 685 Val pixels: 342
════════════════════════════════════════════════════════════
── Ensemble 1/3 seed=42 node='root_L_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.4761 | OA 0.7427 | mIoU 0.4319 | κ 0.1736
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.0829 | OA 0.8450 | mIoU 0.6610 | κ 0.5710
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0197 | OA 0.9152 | mIoU 0.8038 | κ 0.7742
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0146 | OA 0.9971 | mIoU 0.9928 | κ 0.9928
⚡ Early stop: mIoU ≥ 0.99 at epoch 80
✓ Ensemble 1/3 done Best @ epoch 80 mIoU 0.9928 OA 0.9971 AA 0.9948 κ 0.9928
1h 4m 40s elapsed | GPU 86°C | 4309 / 6144 MB | 100% util
── Ensemble 2/3 seed=43 node='root_L_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.4371 | OA 0.7690 | mIoU 0.4912 | κ 0.2881
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.0960 | OA 0.7778 | mIoU 0.5042 | κ 0.3062
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0156 | OA 0.8801 | mIoU 0.7204 | κ 0.6662
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0088 | OA 0.9854 | mIoU 0.9642 | κ 0.9635
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0111 | OA 0.9883 | mIoU 0.9713 | κ 0.9709
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0062 | OA 0.9971 | mIoU 0.9928 | κ 0.9928
⚡ Early stop: mIoU ≥ 0.99 at epoch 120
✓ Ensemble 2/3 done Best @ epoch 120 mIoU 0.9928 OA 0.9971 AA 0.9948 κ 0.9928
1h 6m 35s elapsed | GPU 84°C | 4309 / 6144 MB | 86% util
── Ensemble 3/3 seed=44 node='root_L_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.3524 | OA 0.7515 | mIoU 0.4519 | κ 0.2166
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.0524 | OA 0.8480 | mIoU 0.6555 | κ 0.5648
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0169 | OA 0.8713 | mIoU 0.6991 | κ 0.6358
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0096 | OA 0.9825 | mIoU 0.9571 | κ 0.9560
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0067 | OA 0.9766 | mIoU 0.9430 | κ 0.9410
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0047 | OA 0.9883 | mIoU 0.9713 | κ 0.9709
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0039 | OA 0.9825 | mIoU 0.9571 | κ 0.9560
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0039 | OA 0.9854 | mIoU 0.9642 | κ 0.9635
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0034 | OA 0.9854 | mIoU 0.9642 | κ 0.9635
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0030 | OA 0.9854 | mIoU 0.9642 | κ 0.9635
✓ Ensemble 3/3 done Best @ epoch 120 mIoU 0.9713 OA 0.9883 AA 0.9794 κ 0.9709
1h 9m 46s elapsed | GPU 85°C | 4309 / 6144 MB | 100% util
Node 'root_L_R' — 3 ensembles trained in 6m 23s
mIoU 0.9928 0.9928 0.9713 mean 0.9857 ± 0.0101
Best: Ensemble 1 (0.9928) Worst: Ensemble 3 (0.9713) Early stops: 2/3
════════════════════════════════════════════════════════════
Node 'root_R'
Classes : [8, 1, 4, 5, 6, 9, 12, 14, 15, 16]
Local classes : 10 Epochs: 250 Ensembles: 5 Loss: dice
Train pixels : 863 Val pixels: 429
════════════════════════════════════════════════════════════
── Ensemble 1/5 seed=42 node='root_R'
Epoch 20/250 | █░░░░░░░░░░░░░░░░░░░░░░░ 8% | Loss 1.3911 | OA 0.2238 | mIoU 0.0873 | κ 0.1533
Epoch 40/250 | ███░░░░░░░░░░░░░░░░░░░░░ 16% | Loss 1.0459 | OA 0.4476 | mIoU 0.1448 | κ 0.3281
Epoch 60/250 | █████░░░░░░░░░░░░░░░░░░░ 24% | Loss 0.6154 | OA 0.4825 | mIoU 0.1680 | κ 0.3301
Epoch 80/250 | ███████░░░░░░░░░░░░░░░░░ 32% | Loss 0.3934 | OA 0.8671 | mIoU 0.5292 | κ 0.8376
Epoch 100/250 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.2626 | OA 0.9021 | mIoU 0.6142 | κ 0.8813
Epoch 120/250 | ███████████░░░░░░░░░░░░░ 48% | Loss 0.1760 | OA 0.9814 | mIoU 0.7703 | κ 0.9775
Epoch 140/250 | █████████████░░░░░░░░░░░ 56% | Loss 0.1174 | OA 0.9883 | mIoU 0.9046 | κ 0.9859
Epoch 160/250 | ███████████████░░░░░░░░░ 64% | Loss 0.1080 | OA 0.9907 | mIoU 0.9172 | κ 0.9887
Epoch 180/250 | █████████████████░░░░░░░ 72% | Loss 0.0723 | OA 0.9930 | mIoU 0.9422 | κ 0.9916
Epoch 200/250 | ███████████████████░░░░░ 80% | Loss 0.0539 | OA 0.9930 | mIoU 0.9436 | κ 0.9916
Epoch 220/250 | █████████████████████░░░ 88% | Loss 0.0634 | OA 0.9953 | mIoU 0.9473 | κ 0.9944
Epoch 240/250 | ███████████████████████░ 96% | Loss 0.0532 | OA 0.9953 | mIoU 0.9473 | κ 0.9944
Epoch 250/250 | ████████████████████████ 100% | Loss 0.0580 | OA 0.9953 | mIoU 0.9473 | κ 0.9944
✓ Ensemble 1/5 done Best @ epoch 220 mIoU 0.9473 OA 0.9953 AA 0.9500 κ 0.9944
1h 13m 46s elapsed | GPU 83°C | 4311 / 6144 MB | 86% util
── Ensemble 2/5 seed=43 node='root_R'
Epoch 20/250 | █░░░░░░░░░░░░░░░░░░░░░░░ 8% | Loss 1.2719 | OA 0.3147 | mIoU 0.0789 | κ 0.1779
Epoch 40/250 | ███░░░░░░░░░░░░░░░░░░░░░ 16% | Loss 0.9931 | OA 0.5408 | mIoU 0.1894 | κ 0.4322
Epoch 60/250 | █████░░░░░░░░░░░░░░░░░░░ 24% | Loss 0.6674 | OA 0.5641 | mIoU 0.2211 | κ 0.4649
Epoch 80/250 | ███████░░░░░░░░░░░░░░░░░ 32% | Loss 0.3712 | OA 0.4149 | mIoU 0.1526 | κ 0.3052
Epoch 100/250 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.3339 | OA 0.8881 | mIoU 0.5927 | κ 0.8646
Epoch 120/250 | ███████████░░░░░░░░░░░░░ 48% | Loss 0.2103 | OA 0.9487 | mIoU 0.6878 | κ 0.9379
Epoch 140/250 | █████████████░░░░░░░░░░░ 56% | Loss 0.1152 | OA 0.9417 | mIoU 0.7618 | κ 0.9299
Epoch 160/250 | ███████████████░░░░░░░░░ 64% | Loss 0.0727 | OA 0.9277 | mIoU 0.7386 | κ 0.9125
Epoch 180/250 | █████████████████░░░░░░░ 72% | Loss 0.0402 | OA 0.8228 | mIoU 0.6912 | κ 0.7849
Epoch 200/250 | ███████████████████░░░░░ 80% | Loss 0.0709 | OA 0.9930 | mIoU 0.9444 | κ 0.9916
Epoch 220/250 | █████████████████████░░░ 88% | Loss 0.0301 | OA 0.9953 | mIoU 0.9461 | κ 0.9944
Epoch 240/250 | ███████████████████████░ 96% | Loss 0.0349 | OA 0.9907 | mIoU 0.8910 | κ 0.9888
Epoch 250/250 | ████████████████████████ 100% | Loss 0.0589 | OA 0.9930 | mIoU 0.9423 | κ 0.9916
✓ Ensemble 2/5 done Best @ epoch 220 mIoU 0.9461 OA 0.9953 AA 0.9500 κ 0.9944
1h 17m 45s elapsed | GPU 84°C | 4313 / 6144 MB | 100% util
── Ensemble 3/5 seed=44 node='root_R'
Epoch 20/250 | █░░░░░░░░░░░░░░░░░░░░░░░ 8% | Loss 1.2991 | OA 0.1841 | mIoU 0.0643 | κ 0.1164
Epoch 40/250 | ███░░░░░░░░░░░░░░░░░░░░░ 16% | Loss 1.0455 | OA 0.5455 | mIoU 0.2096 | κ 0.4462
Epoch 60/250 | █████░░░░░░░░░░░░░░░░░░░ 24% | Loss 0.6822 | OA 0.6061 | mIoU 0.2521 | κ 0.5135
Epoch 80/250 | ███████░░░░░░░░░░░░░░░░░ 32% | Loss 0.3556 | OA 0.7599 | mIoU 0.4151 | κ 0.7047
Epoch 100/250 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.2849 | OA 0.8718 | mIoU 0.5739 | κ 0.8440
Epoch 120/250 | ███████████░░░░░░░░░░░░░ 48% | Loss 0.1793 | OA 0.9534 | mIoU 0.7010 | κ 0.9436
Epoch 140/250 | █████████████░░░░░░░░░░░ 56% | Loss 0.1416 | OA 0.9674 | mIoU 0.7463 | κ 0.9605
Epoch 160/250 | ███████████████░░░░░░░░░ 64% | Loss 0.0859 | OA 0.9534 | mIoU 0.7533 | κ 0.9438
Epoch 180/250 | █████████████████░░░░░░░ 72% | Loss 0.0801 | OA 0.9883 | mIoU 0.8528 | κ 0.9859
Epoch 200/250 | ███████████████████░░░░░ 80% | Loss 0.0396 | OA 0.9953 | mIoU 0.9830 | κ 0.9944
Epoch 220/250 | █████████████████████░░░ 88% | Loss 0.0569 | OA 0.9953 | mIoU 0.9216 | κ 0.9944
Epoch 240/250 | ███████████████████████░ 96% | Loss 0.0333 | OA 0.9883 | mIoU 0.8515 | κ 0.9859
Epoch 250/250 | ████████████████████████ 100% | Loss 0.0304 | OA 0.9907 | mIoU 0.8786 | κ 0.9888
✓ Ensemble 3/5 done Best @ epoch 200 mIoU 0.9830 OA 0.9953 AA 0.9863 κ 0.9944
1h 21m 44s elapsed | GPU 85°C | 4313 / 6144 MB | 100% util
── Ensemble 4/5 seed=45 node='root_R'
Epoch 20/250 | █░░░░░░░░░░░░░░░░░░░░░░░ 8% | Loss 1.2642 | OA 0.3613 | mIoU 0.0918 | κ 0.2283
Epoch 40/250 | ███░░░░░░░░░░░░░░░░░░░░░ 16% | Loss 0.9576 | OA 0.5758 | mIoU 0.1987 | κ 0.4678
Epoch 60/250 | █████░░░░░░░░░░░░░░░░░░░ 24% | Loss 0.5513 | OA 0.6970 | mIoU 0.3356 | κ 0.6213
Epoch 80/250 | ███████░░░░░░░░░░░░░░░░░ 32% | Loss 0.3312 | OA 0.6410 | mIoU 0.2966 | κ 0.5533
Epoch 100/250 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.2660 | OA 0.8345 | mIoU 0.5355 | κ 0.7980
Epoch 120/250 | ███████████░░░░░░░░░░░░░ 48% | Loss 0.1563 | OA 0.7622 | mIoU 0.4589 | κ 0.7099
Epoch 140/250 | █████████████░░░░░░░░░░░ 56% | Loss 0.1191 | OA 0.9044 | mIoU 0.6884 | κ 0.8858
Epoch 160/250 | ███████████████░░░░░░░░░ 64% | Loss 0.0781 | OA 0.8765 | mIoU 0.6146 | κ 0.8527
Epoch 180/250 | █████████████████░░░░░░░ 72% | Loss 0.0623 | OA 0.8765 | mIoU 0.7510 | κ 0.8540
Epoch 200/250 | ███████████████████░░░░░ 80% | Loss 0.0393 | OA 0.9627 | mIoU 0.8924 | κ 0.9551
Epoch 220/250 | █████████████████████░░░ 88% | Loss 0.0397 | OA 0.9604 | mIoU 0.7904 | κ 0.9520
Epoch 240/250 | ███████████████████████░ 96% | Loss 0.0394 | OA 0.9860 | mIoU 0.9242 | κ 0.9831
Epoch 250/250 | ████████████████████████ 100% | Loss 0.0343 | OA 0.9883 | mIoU 0.9275 | κ 0.9859
✓ Ensemble 4/5 done Best @ epoch 250 mIoU 0.9275 OA 0.9883 AA 0.9346 κ 0.9859
1h 25m 44s elapsed | GPU 83°C | 4313 / 6144 MB | 93% util
── Ensemble 5/5 seed=46 node='root_R'
Epoch 20/250 | █░░░░░░░░░░░░░░░░░░░░░░░ 8% | Loss 1.2965 | OA 0.3357 | mIoU 0.0837 | κ 0.1991
Epoch 40/250 | ███░░░░░░░░░░░░░░░░░░░░░ 16% | Loss 0.9258 | OA 0.4918 | mIoU 0.1686 | κ 0.3730
Epoch 60/250 | █████░░░░░░░░░░░░░░░░░░░ 24% | Loss 0.6451 | OA 0.5897 | mIoU 0.2694 | κ 0.5011
Epoch 80/250 | ███████░░░░░░░░░░░░░░░░░ 32% | Loss 0.4006 | OA 0.6154 | mIoU 0.2863 | κ 0.5290
Epoch 100/250 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.2669 | OA 0.8415 | mIoU 0.5251 | κ 0.8070
Epoch 120/250 | ███████████░░░░░░░░░░░░░ 48% | Loss 0.1822 | OA 0.9301 | mIoU 0.6157 | κ 0.9153
Epoch 140/250 | █████████████░░░░░░░░░░░ 56% | Loss 0.2069 | OA 0.8368 | mIoU 0.5458 | κ 0.8034
Epoch 160/250 | ███████████████░░░░░░░░░ 64% | Loss 0.1068 | OA 0.9487 | mIoU 0.7886 | κ 0.9381
Epoch 180/250 | █████████████████░░░░░░░ 72% | Loss 0.0534 | OA 0.9417 | mIoU 0.8462 | κ 0.9296
Epoch 200/250 | ███████████████████░░░░░ 80% | Loss 0.0265 | OA 0.9674 | mIoU 0.9010 | κ 0.9606
Epoch 220/250 | █████████████████████░░░ 88% | Loss 0.0213 | OA 0.9044 | mIoU 0.7166 | κ 0.8850
Epoch 240/250 | ███████████████████████░ 96% | Loss 0.0240 | OA 0.9977 | mIoU 0.9729 | κ 0.9972
Epoch 250/250 | ████████████████████████ 100% | Loss 0.0216 | OA 0.9953 | mIoU 0.9667 | κ 0.9944
✓ Ensemble 5/5 done Best @ epoch 240 mIoU 0.9729 OA 0.9977 AA 0.9750 κ 0.9972
1h 29m 43s elapsed | GPU 85°C | 4313 / 6144 MB | 100% util
Node 'root_R' — 5 ensembles trained in 19m 57s
mIoU 0.9473 0.9461 0.9830 0.9275 0.9729 mean 0.9554 ± 0.0200
Best: Ensemble 3 (0.9830) Worst: Ensemble 4 (0.9275) Early stops: 0/5
Training SSSR router for node 'root_R' ...
Router: train_acc=0.4878 | val_acc=0.4382 | left=599 right=264
Router trained.
════════════════════════════════════════════════════════════
Node 'root_R_L'
Classes : [6, 1, 5, 14, 15, 16]
Local classes : 6 Epochs: 200 Ensembles: 5 Loss: dice
Train pixels : 599 Val pixels: 298
════════════════════════════════════════════════════════════
── Ensemble 1/5 seed=42 node='root_R_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.0742 | OA 0.4631 | mIoU 0.1852 | κ 0.2980
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.7235 | OA 0.5336 | mIoU 0.2207 | κ 0.2821
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.3385 | OA 0.8154 | mIoU 0.5309 | κ 0.7300
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.1465 | OA 0.9631 | mIoU 0.7401 | κ 0.9485
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0536 | OA 0.9698 | mIoU 0.8366 | κ 0.9577
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0451 | OA 0.9866 | mIoU 0.9009 | κ 0.9812
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0244 | OA 0.9799 | mIoU 0.8870 | κ 0.9719
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0621 | OA 0.9933 | mIoU 0.9122 | κ 0.9906
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0126 | OA 0.9832 | mIoU 0.8918 | κ 0.9766
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0094 | OA 0.9899 | mIoU 0.9038 | κ 0.9860
✓ Ensemble 1/5 done Best @ epoch 160 mIoU 0.9122 OA 0.9933 AA 0.9167 κ 0.9906
1h 32m 55s elapsed | GPU 85°C | 4313 / 6144 MB | 100% util
── Ensemble 2/5 seed=43 node='root_R_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.0121 | OA 0.4094 | mIoU 0.1565 | κ 0.2527
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.6173 | OA 0.6779 | mIoU 0.4098 | κ 0.5729
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.2646 | OA 0.8658 | mIoU 0.6847 | κ 0.8138
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.1123 | OA 0.8993 | mIoU 0.7221 | κ 0.8605
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.1133 | OA 0.9933 | mIoU 0.9628 | κ 0.9907
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0175 | OA 0.9933 | mIoU 0.9846 | κ 0.9906
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0137 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 140
✓ Ensemble 2/5 done Best @ epoch 140 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 35m 10s elapsed | GPU 85°C | 4313 / 6144 MB | 100% util
── Ensemble 3/5 seed=44 node='root_R_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.9515 | OA 0.3691 | mIoU 0.1261 | κ 0.2174
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.5218 | OA 0.6745 | mIoU 0.3524 | κ 0.5432
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.1856 | OA 0.8826 | mIoU 0.6391 | κ 0.8321
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0892 | OA 0.9698 | mIoU 0.8493 | κ 0.9581
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0429 | OA 0.9732 | mIoU 0.8580 | κ 0.9628
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0156 | OA 0.9899 | mIoU 0.9479 | κ 0.9860
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0179 | OA 0.9966 | mIoU 0.9799 | κ 0.9953
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0116 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 160
✓ Ensemble 3/5 done Best @ epoch 160 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 37m 43s elapsed | GPU 86°C | 4313 / 6144 MB | 100% util
── Ensemble 4/5 seed=45 node='root_R_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.9950 | OA 0.3926 | mIoU 0.1397 | κ 0.2311
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.6121 | OA 0.5134 | mIoU 0.1535 | κ 0.2061
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.2175 | OA 0.6141 | mIoU 0.3308 | κ 0.3850
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0811 | OA 0.8154 | mIoU 0.6315 | κ 0.7310
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0472 | OA 0.8893 | mIoU 0.7796 | κ 0.8420
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0156 | OA 0.9765 | mIoU 0.8810 | κ 0.9671
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0149 | OA 0.9933 | mIoU 0.9928 | κ 0.9906
⚡ Early stop: mIoU ≥ 0.99 at epoch 140
✓ Ensemble 4/5 done Best @ epoch 140 mIoU 0.9928 OA 0.9933 AA 0.9954 κ 0.9906
1h 39m 57s elapsed | GPU 85°C | 4313 / 6144 MB | 86% util
── Ensemble 5/5 seed=46 node='root_R_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 1.0579 | OA 0.2919 | mIoU 0.1111 | κ 0.1569
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.6443 | OA 0.5168 | mIoU 0.1907 | κ 0.2974
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.2832 | OA 0.8356 | mIoU 0.4607 | κ 0.7643
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.1471 | OA 0.8221 | mIoU 0.5987 | κ 0.7411
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0681 | OA 0.9799 | mIoU 0.8658 | κ 0.9719
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0334 | OA 0.9933 | mIoU 0.9083 | κ 0.9906
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0505 | OA 0.9933 | mIoU 0.9101 | κ 0.9906
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0147 | OA 0.9899 | mIoU 0.9045 | κ 0.9859
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0163 | OA 0.9933 | mIoU 0.9483 | κ 0.9906
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0072 | OA 0.9933 | mIoU 0.9483 | κ 0.9906
✓ Ensemble 5/5 done Best @ epoch 180 mIoU 0.9483 OA 0.9933 AA 0.9549 κ 0.9906
1h 43m 9s elapsed | GPU 85°C | 4313 / 6144 MB | 100% util
Node 'root_R_L' — 5 ensembles trained in 13m 25s
mIoU 0.9122 1.0000 1.0000 0.9928 0.9483 mean 0.9707 ± 0.0350
Best: Ensemble 2 (1.0000) Worst: Ensemble 1 (0.9122) Early stops: 3/5
Training SSSR router for node 'root_R_L' ...
Router: train_acc=0.6260 | val_acc=0.5940 | left=337 right=262
Router trained.
════════════════════════════════════════════════════════════
Node 'root_R_L_L'
Classes : [6, 5, 15, 16]
Local classes : 4 Epochs: 200 Ensembles: 5 Loss: dice
Train pixels : 337 Val pixels: 168
════════════════════════════════════════════════════════════
── Ensemble 1/5 seed=42 node='root_R_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.8291 | OA 0.3452 | mIoU 0.1460 | κ 0.1112
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.3408 | OA 0.6310 | mIoU 0.3502 | κ 0.4549
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.1052 | OA 0.7917 | mIoU 0.5422 | κ 0.6693
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0191 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 80
✓ Ensemble 1/5 done Best @ epoch 80 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 44m 26s elapsed | GPU 85°C | 4313 / 6144 MB | 85% util
── Ensemble 2/5 seed=43 node='root_R_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.7936 | OA 0.3690 | mIoU 0.2122 | κ 0.2077
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.3953 | OA 0.6964 | mIoU 0.4509 | κ 0.5464
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0520 | OA 0.8095 | mIoU 0.6682 | κ 0.7046
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0114 | OA 0.9464 | mIoU 0.8823 | κ 0.9202
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0303 | OA 0.9821 | mIoU 0.9656 | κ 0.9735
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0035 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 120
✓ Ensemble 2/5 done Best @ epoch 120 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 46m 21s elapsed | GPU 85°C | 4313 / 6144 MB | 85% util
── Ensemble 3/5 seed=44 node='root_R_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.7761 | OA 0.5060 | mIoU 0.3138 | κ 0.3462
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.3698 | OA 0.6845 | mIoU 0.3993 | κ 0.4865
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0569 | OA 0.8750 | mIoU 0.6602 | κ 0.8116
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0104 | OA 0.9940 | mIoU 0.9902 | κ 0.9912
⚡ Early stop: mIoU ≥ 0.99 at epoch 80
✓ Ensemble 3/5 done Best @ epoch 80 mIoU 0.9902 OA 0.9940 AA 0.9966 κ 0.9912
1h 47m 38s elapsed | GPU 86°C | 4313 / 6144 MB | 100% util
── Ensemble 4/5 seed=45 node='root_R_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.7874 | OA 0.2024 | mIoU 0.1512 | κ 0.1330
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.2633 | OA 0.6607 | mIoU 0.3566 | κ 0.4441
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0682 | OA 0.8571 | mIoU 0.7433 | κ 0.7844
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0115 | OA 0.9940 | mIoU 0.9915 | κ 0.9912
⚡ Early stop: mIoU ≥ 0.99 at epoch 80
✓ Ensemble 4/5 done Best @ epoch 80 mIoU 0.9915 OA 0.9940 AA 0.9966 κ 0.9912
1h 48m 55s elapsed | GPU 84°C | 4313 / 6144 MB | 86% util
── Ensemble 5/5 seed=46 node='root_R_L_L'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.9192 | OA 0.1667 | mIoU 0.1599 | κ 0.1221
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.3214 | OA 0.7798 | mIoU 0.5973 | κ 0.6769
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0867 | OA 0.7857 | mIoU 0.5829 | κ 0.6845
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0326 | OA 0.9762 | mIoU 0.9174 | κ 0.9652
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0094 | OA 0.9762 | mIoU 0.9639 | κ 0.9649
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0069 | OA 0.9940 | mIoU 0.9722 | κ 0.9912
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0075 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 140
✓ Ensemble 5/5 done Best @ epoch 140 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 51m 16s elapsed | GPU 85°C | 4313 / 6144 MB | 100% util
Node 'root_R_L_L' — 5 ensembles trained in 8m 6s
mIoU 1.0000 1.0000 0.9902 0.9915 1.0000 mean 0.9963 ± 0.0045
Best: Ensemble 1 (1.0000) Worst: Ensemble 3 (0.9902) Early stops: 5/5
════════════════════════════════════════════════════════════
Node 'root_R_L_R'
Classes : [1, 14]
Local classes : 2 Epochs: 200 Ensembles: 3 Loss: dice
Train pixels : 262 Val pixels: 130
════════════════════════════════════════════════════════════
── Ensemble 1/3 seed=42 node='root_R_L_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.3861 | OA 0.9615 | mIoU 0.8802 | κ 0.6022
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.0572 | OA 0.9923 | mIoU 0.8711 | κ 0.8533
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0054 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 60
✓ Ensemble 1/3 done Best @ epoch 60 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 52m 14s elapsed | GPU 85°C | 4313 / 6144 MB | 85% util
── Ensemble 2/3 seed=43 node='root_R_L_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.3135 | OA 0.8000 | mIoU 0.7968 | κ 0.2103
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.0254 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 40
✓ Ensemble 2/3 done Best @ epoch 40 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 52m 52s elapsed | GPU 86°C | 4313 / 6144 MB | 99% util
── Ensemble 3/3 seed=44 node='root_R_L_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.2713 | OA 0.8308 | mIoU 0.8127 | κ 0.2426
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.0078 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 40
✓ Ensemble 3/3 done Best @ epoch 40 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
1h 53m 31s elapsed | GPU 86°C | 4313 / 6144 MB | 100% util
Node 'root_R_L_R' — 3 ensembles trained in 2m 14s
mIoU 1.0000 1.0000 1.0000 mean 1.0000 ± 0.0000
Best: Ensemble 1 (1.0000) Worst: Ensemble 1 (1.0000) Early stops: 3/3
════════════════════════════════════════════════════════════
Node 'root_R_R'
Classes : [9, 8, 4, 12]
Local classes : 4 Epochs: 200 Ensembles: 5 Loss: dice
Train pixels : 264 Val pixels: 131
════════════════════════════════════════════════════════════
── Ensemble 1/5 seed=42 node='root_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.8057 | OA 0.0840 | mIoU 0.0664 | κ 0.0445
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.5738 | OA 0.8244 | mIoU 0.5484 | κ 0.7281
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.1610 | OA 0.7634 | mIoU 0.6809 | κ 0.6367
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0803 | OA 0.9847 | mIoU 0.9694 | κ 0.9762
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0225 | OA 0.8931 | mIoU 0.6886 | κ 0.8389
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0125 | OA 0.9695 | mIoU 0.8754 | κ 0.9527
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0094 | OA 0.9924 | mIoU 0.9843 | κ 0.9880
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0096 | OA 0.9924 | mIoU 0.9843 | κ 0.9880
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0034 | OA 0.9924 | mIoU 0.9843 | κ 0.9880
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0028 | OA 0.9924 | mIoU 0.9843 | κ 0.9880
✓ Ensemble 1/5 done Best @ epoch 140 mIoU 0.9843 OA 0.9924 AA 0.9947 κ 0.9880
1h 56m 43s elapsed | GPU 87°C | 4313 / 6144 MB | 100% util
── Ensemble 2/5 seed=43 node='root_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.6217 | OA 0.5573 | mIoU 0.2114 | κ 0.2173
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.3045 | OA 0.6947 | mIoU 0.3576 | κ 0.4950
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0847 | OA 0.8321 | mIoU 0.5346 | κ 0.7340
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0150 | OA 0.9008 | mIoU 0.8500 | κ 0.8492
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0133 | OA 0.9389 | mIoU 0.7878 | κ 0.9064
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0015 | OA 0.9771 | mIoU 0.9552 | κ 0.9643
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0010 | OA 0.9313 | mIoU 0.8043 | κ 0.8945
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0047 | OA 0.9542 | mIoU 0.8401 | κ 0.9292
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0008 | OA 0.9389 | mIoU 0.8158 | κ 0.9060
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0014 | OA 0.9466 | mIoU 0.8277 | κ 0.9176
✓ Ensemble 2/5 done Best @ epoch 120 mIoU 0.9552 OA 0.9771 AA 0.9840 κ 0.9643
1h 59m 56s elapsed | GPU 86°C | 4313 / 6144 MB | 100% util
── Ensemble 3/5 seed=44 node='root_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.7709 | OA 0.1145 | mIoU 0.0717 | κ 0.0581
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.4699 | OA 0.6794 | mIoU 0.3499 | κ 0.4740
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.1224 | OA 0.6336 | mIoU 0.3309 | κ 0.4189
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0340 | OA 0.8244 | mIoU 0.5962 | κ 0.7439
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0105 | OA 0.8168 | mIoU 0.5736 | κ 0.7297
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0043 | OA 0.9237 | mIoU 0.7235 | κ 0.8843
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0028 | OA 0.9313 | mIoU 0.7360 | κ 0.8942
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0024 | OA 0.9389 | mIoU 0.7500 | κ 0.9062
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0033 | OA 0.9466 | mIoU 0.7642 | κ 0.9182
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0017 | OA 0.9618 | mIoU 0.8034 | κ 0.9411
✓ Ensemble 3/5 done Best @ epoch 200 mIoU 0.8034 OA 0.9618 AA 0.9734 κ 0.9411
2h 3m 7s elapsed | GPU 86°C | 4313 / 6144 MB | 100% util
── Ensemble 4/5 seed=45 node='root_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.7088 | OA 0.4962 | mIoU 0.1817 | κ 0.1737
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.3893 | OA 0.6107 | mIoU 0.2824 | κ 0.3332
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.1205 | OA 0.7405 | mIoU 0.5002 | κ 0.6304
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0388 | OA 0.6565 | mIoU 0.4894 | κ 0.5478
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0210 | OA 0.4580 | mIoU 0.3664 | κ 0.3579
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0036 | OA 0.7786 | mIoU 0.6013 | κ 0.6896
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0022 | OA 0.8168 | mIoU 0.6220 | κ 0.7363
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0026 | OA 0.8931 | mIoU 0.6940 | κ 0.8409
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0022 | OA 0.9160 | mIoU 0.7265 | κ 0.8738
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0020 | OA 0.9160 | mIoU 0.7269 | κ 0.8731
✓ Ensemble 4/5 done Best @ epoch 200 mIoU 0.7269 OA 0.9160 AA 0.9415 κ 0.8731
2h 6m 19s elapsed | GPU 87°C | 4313 / 6144 MB | 100% util
── Ensemble 5/5 seed=46 node='root_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.7320 | OA 0.4885 | mIoU 0.1834 | κ 0.1677
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.4213 | OA 0.7099 | mIoU 0.3709 | κ 0.5204
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.1146 | OA 0.7557 | mIoU 0.4529 | κ 0.6109
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0752 | OA 0.9008 | mIoU 0.7473 | κ 0.8478
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0125 | OA 0.8779 | mIoU 0.7382 | κ 0.8185
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0081 | OA 0.9924 | mIoU 0.9843 | κ 0.9880
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0103 | OA 0.9924 | mIoU 0.9843 | κ 0.9880
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0030 | OA 0.9847 | mIoU 0.9789 | κ 0.9762
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0025 | OA 0.9847 | mIoU 0.9694 | κ 0.9762
Epoch 200/200 | ████████████████████████ 100% | Loss 0.0024 | OA 0.9771 | mIoU 0.9736 | κ 0.9645
✓ Ensemble 5/5 done Best @ epoch 120 mIoU 0.9843 OA 0.9924 AA 0.9947 κ 0.9880
2h 9m 31s elapsed | GPU 85°C | 4313 / 6144 MB | 100% util
Node 'root_R_R' — 5 ensembles trained in 16m 0s
mIoU 0.9843 0.9552 0.8034 0.7269 0.9843 mean 0.8908 ± 0.1059
Best: Ensemble 1 (0.9843) Worst: Ensemble 4 (0.7269) Early stops: 0/5
Training SSSR router for node 'root_R_R' ...
Router: train_acc=0.6553 | val_acc=0.7023 | left=95 right=169
Router trained.
════════════════════════════════════════════════════════════
Node 'root_R_R_R'
Classes : [4, 9, 12]
Local classes : 3 Epochs: 200 Ensembles: 5 Loss: dice
Train pixels : 169 Val pixels: 84
════════════════════════════════════════════════════════════
── Ensemble 1/5 seed=42 node='root_R_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.5992 | OA 0.7143 | mIoU 0.2543 | κ 0.0815
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.0717 | OA 0.9048 | mIoU 0.5399 | κ 0.7522
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0037 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 60
✓ Ensemble 1/5 done Best @ epoch 60 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
2h 10m 29s elapsed | GPU 85°C | 4313 / 6144 MB | 88% util
── Ensemble 2/5 seed=43 node='root_R_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.6425 | OA 0.6190 | mIoU 0.3088 | κ 0.2636
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.1176 | OA 0.8333 | mIoU 0.4326 | κ 0.5397
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0057 | OA 0.9762 | mIoU 0.8043 | κ 0.9455
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0027 | OA 0.9881 | mIoU 0.8744 | κ 0.9726
Epoch 100/200 | ████████████░░░░░░░░░░░░ 50% | Loss 0.0030 | OA 0.9881 | mIoU 0.8744 | κ 0.9726
Epoch 120/200 | ██████████████░░░░░░░░░░ 60% | Loss 0.0009 | OA 0.9881 | mIoU 0.8744 | κ 0.9726
Epoch 140/200 | ████████████████░░░░░░░░ 70% | Loss 0.0011 | OA 0.9881 | mIoU 0.8744 | κ 0.9726
Epoch 160/200 | ███████████████████░░░░░ 80% | Loss 0.0023 | OA 0.9881 | mIoU 0.8744 | κ 0.9726
Epoch 180/200 | █████████████████████░░░ 90% | Loss 0.0005 | OA 0.9881 | mIoU 0.8744 | κ 0.9726
⚡ Early stop: no improvement for 100 epochs at epoch 180
✓ Ensemble 2/5 done Best @ epoch 80 mIoU 0.8744 OA 0.9881 AA 0.9855 κ 0.9726
2h 13m 22s elapsed | GPU 86°C | 4313 / 6144 MB | 100% util
── Ensemble 3/5 seed=44 node='root_R_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.5050 | OA 0.7500 | mIoU 0.3069 | κ 0.2340
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.0600 | OA 0.8214 | mIoU 0.4107 | κ 0.4878
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0037 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 60
✓ Ensemble 3/5 done Best @ epoch 60 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
2h 14m 19s elapsed | GPU 86°C | 4313 / 6144 MB | 100% util
── Ensemble 4/5 seed=45 node='root_R_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.6307 | OA 0.7262 | mIoU 0.2688 | κ 0.1105
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.0925 | OA 0.7381 | mIoU 0.2863 | κ 0.1630
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0044 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 60
✓ Ensemble 4/5 done Best @ epoch 60 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
2h 15m 17s elapsed | GPU 85°C | 4313 / 6144 MB | 100% util
── Ensemble 5/5 seed=46 node='root_R_R_R'
Epoch 20/200 | ██░░░░░░░░░░░░░░░░░░░░░░ 10% | Loss 0.7007 | OA 0.7500 | mIoU 0.3069 | κ 0.2340
Epoch 40/200 | ████░░░░░░░░░░░░░░░░░░░░ 20% | Loss 0.2312 | OA 0.7738 | mIoU 0.3391 | κ 0.3109
Epoch 60/200 | ███████░░░░░░░░░░░░░░░░░ 30% | Loss 0.0126 | OA 0.9762 | mIoU 0.9601 | κ 0.9434
Epoch 80/200 | █████████░░░░░░░░░░░░░░░ 40% | Loss 0.0029 | OA 1.0000 | mIoU 1.0000 | κ 1.0000
⚡ Early stop: mIoU ≥ 0.99 at epoch 80
✓ Ensemble 5/5 done Best @ epoch 80 mIoU 1.0000 OA 1.0000 AA 1.0000 κ 1.0000
2h 16m 33s elapsed | GPU 87°C | 4313 / 6144 MB | 100% util
Node 'root_R_R_R' — 5 ensembles trained in 7m 2s
mIoU 1.0000 0.8744 1.0000 1.0000 1.0000 mean 0.9749 ± 0.0502
Best: Ensemble 1 (1.0000) Worst: Ensemble 2 (0.8744) Early stops: 5/5
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣀⣀⣀⣀⣀⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣀⣤⡶⠟⠛⠛⠛⠉⠉⠛⠛⠻⢶⣦⣄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣠⣴⠟⠋⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠙⢿⣤⠀⠀⠀⠀⠀⣠⣤⡀⢀⣀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣾⠟⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠹⣷⡀⠀⠀⢸⡏⠀⠹⠋⢹⣧
⠀⠀⠀⠀⠀⠀⠀⠀⠀⢠⡿⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠹⣿⡀⠀⠀⠻⢶⣤⣴⠾⠃
⠀⠀⠀⠀⠀⠀⠀⠀⢠⣿⠃⠀⠀⢀⣀⡀⠀⠀⠀⠀⠀⠀⣀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢻⣧⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⣾⡇⠀⠀⠀⢿⣿⡏⠀⡀⠀⡀⠀⣾⣿⣷⠀⠀⠀⠀⠀⠀⠀⠀⠀⠸⣿⡀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⢀⣿⠁⠐⡀⢠⠀⠀⡄⠀⠙⠛⠁⠀⠈⠉⠁⠀⠐⠄⠀⠀⠀⠀⠀⠀⠀⢿⣇⠀⠀⠀⠀⠀⠀⠀
⠀ Training complete!
=== Running Cascade Inference (routing=forest) ===
──────────────────────────────────────────
Test Results [forest]
──────────────────────────────────────────
OA 0.9783 ███████████████████░
mIoU 0.8959 █████████████████░░░
Dice 0.9397 ██████████████████░░
Precision 0.9533 ███████████████████░
Recall 0.9341 ██████████████████░░
AA 0.9341 ██████████████████░░
kappa 0.9752 ███████████████████░
──────────────────────────────────────────
Saved → runs/indian_pines/test_results.csv
Per-class IoU:
Class 1 0.8889 █████████████████
Class 2 0.9441 ██████████████████
Class 3 0.9764 ███████████████████
Class 4 0.8610 █████████████████
Class 5 0.9600 ███████████████████
Class 6 0.9941 ███████████████████
Class 7 0.6190 ████████████
Class 8 0.9018 ██████████████████
Class 9 0.5500 ███████████
Class 10 0.9634 ███████████████████
Class 11 0.9695 ███████████████████
Class 12 0.9541 ███████████████████
Class 13 0.9726 ███████████████████
Class 14 0.9704 ███████████████████
Class 15 0.9890 ███████████████████
Class 16 0.8209 ████████████████
Saved → runs/indian_pines/class_report_forest.csv
Saved → runs/indian_pines/spt_models.pkl
What's next?
# Evaluate all routing modes:
ghost predict \
--data /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_corrected.mat \
--gt /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_gt.mat \
--model runs/indian_pines/spt_models.pkl \
--routing all --out-dir runs/indian_pines
# Visualize predictions:
ghost visualize \
--data /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_corrected.mat \
--gt /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_gt.mat \
--model runs/indian_pines/spt_models.pkl \
--out-dir runs/indian_pines
(ghost_test) Darth-Ishu@Executor:~$ ghost visualize \
--data /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_corrected.mat \
--gt /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_gt.mat \
--model runs/indian_pines/spt_models.pkl \
--out-dir runs/indian_pines
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣀⣤⣤⡤⣤⢤⣤⣄⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣠⡾⠛⠉⠀⠀⠀⠀⠀⠀⠉⠻⣦⡄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣾⠏⠀⠀⠀⠀⣀⡀⠀⠀⠀⠀⠀⠈⢻⡆⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣼⠯⠀⠀⠀⢰⣿⣿⣿⣆⡀⣴⣾⣿⣦⡈⢿⡄⠀ G H O S T
⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⣏⠀⠀⠀⠀⢸⣿⣿⣿⡿⠇⣿⣿⣿⣿⠆⠘⣷⡀ Generalizable Hyperspectral
⠀⠀⠀⠀⠀⠀⠀⠀⠀⣿⠇⠀⠀⠀⠀⠈⠿⠿⡿⠃⠀⢿⣿⣿⠏⠀⠀⢹⣗ Observation & Segmentation
⠀⠀⠀⠀⠀⠀⠀⠀⢸⡏⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠀⠀⠀⠀⠀⣿⡀ Toolkit
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣠⠤⠒⠒⠒⠒⠢⢄⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡠⠾⠁⠀⠀⠀⠀⠀⠀⠀⠀⠈⢦⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠀⢠⠊⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢳⠀⠀⠀⠀⠀⡤⢄⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⡰⠃⠀⠀⠀⠀⠀⠀⠀⣼⡆⠀⢀⢀⠀⠸⡟⡇⠀⠀⡰⢾⢁⢸⣄⠀
⠀⠀⠀⠀⠀⠀⠀⣰⠁⠀⠀⠀⠀⠀⠀⠀⠀⠙⠈⠀⠈⠉⠀⠀⠀⢸⠀⠀⡝⡆⠚⠠⣤⠇
⠀⠀⠀⠀⠀⠀⢠⠃⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⡀⠀⠙⢸⠧⣤⠃⠀
⠀⠀⠀⠀⠀⢠⠏⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣇⡀⣤⣼⠧⡀⠀⠀
⠀⠀⠀⠀⢠⡏⠀⠀⠀⠀⡀⠀⠀⣠⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠐⡇⠀⣀⣧⢸⠀⠀⠀
⠀⠀⠀⢠⡞⠀⠀⠀⠀⠀⢟⠀⢀⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⠒⠉⠁⢸⡄⠀⠀⠀
⠀⠀⢀⡞⠀⠀⠀⠀⠀⠀⢸⠀⡜⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⠀⠀⠀⠈⠀⠀⠀⠀
⠀⣠⠏⠀⠀⠀⠀⠀⠀⠀⠈⠋⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣸⠀⠀⠀⠀⠀⠀⠀⠀
⣰⠃⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡏⠀⠀⠀⠀⠀⠀⠀⠀
⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣸⠁⠀⠀⠀⠀⠀⠀⠀⠀
⠙⠶⠦⠤⠶⠖⠒⢤⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡴⠃⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠈⣷⠀⠀⠀⠀⢀⣤⠴⠶⣤⣀⠀⠀⣠⠞⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀⠀⠀⠀⠀⠀⠀⠀⠙⠲⠤⠖⠚⠉⠀⠀⠀⠀⠈⠉⠉⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
⠀ Visualizing...
════════════════════════════════════════════════════════════
GHOST Visualize
Device : cuda (NVIDIA GeForce RTX 3050 6GB Laptop GPU, 6 GB)
Model : runs/indian_pines/spt_models.pkl
Data : /home/ishu/miniconda3/envs/ghost_test/lib/python3.12/site-packages/ghost/data/indian_pines/Indian_pines_corrected.mat
Routing : forest
Dataset : auto
════════════════════════════════════════════════════════════
test: 7186 pixels across 16 classes
Running inference (routing=forest) ...
Saved → runs/indian_pines/segmentation_forest.png
Total visualize time: 31.6s
(ghost_test) Darth-Ishu@Executor:~$