Euler View
1 pending

Run Dossier

Training run for SD 2.1. Mari

21 metrics · 4,736 train · 42 val

Chart Overlays

Compare this run against its peers

0/48
Add peer runs here to layer their train and val traces onto the current metric charts.
crashed Apr 13, 23:02 Duration: 10h 56m Python 3.11.7 train_joint.py --config /cluster/home/drothenpiele/models/stable_diffusion_mari/mari/.euler-launches/euler_launch_e82c77ef-53f8-4f6a-aaf1-be3062aa3cd8/joint_vkitti.yaml
lr 3e-5batch 16epochs 200precision fp16model stable-diffusion-2-1 seed 42wd 0grad accum 4warmup 0.05000ema nograd clip 1
ID 63209365
mari/runs/2026-04-13_23-02-17_d461
Error
BrokenPipeError: [Errno 32] Broken pipe
Metrics 21/21
depth.train 4
diag 3
depth_abs_rel
depth_delta1
depth_mae
loss 1
depth
rgb.train 11
diag 2
image_psnr
image_ssim
loss 7
dehaze
preserve
total
visibility
visibility_aux
visibility_rank
visibility_tv
stat 2
visibility_mean
visibility_std
sys.train 6
gpu_mem_total_gb
gpu_mem_used_gb
gpu_mem_util_pct
gpu_util_pct
lr
visibility_aux_scale
21 selected
X-Axis
Y-Scale
Series
Group
Namespace

Depth Abs Rel

↓ kind=diag
V
2026-04-13_23-02-17_d461 val (smoothed) —
2026-04-13_23-02-17_d461 val (raw) —

Depth Delta1

↑ kind=diag
V
2026-04-13_23-02-17_d461 val (smoothed) —
2026-04-13_23-02-17_d461 val (raw) —

Depth Mae

kind=diag
V
2026-04-13_23-02-17_d461 val (smoothed) —
2026-04-13_23-02-17_d461 val (raw) —

Depth

↓ kind=loss
T
2026-04-13_23-02-17_d461 train (smoothed) —
2026-04-13_23-02-17_d461 train (raw) —

Image Psnr

↑ dB kind=diag
V
2026-04-13_23-02-17_d461 val (smoothed) —
2026-04-13_23-02-17_d461 val (raw) —

Image Ssim

↑ kind=diag
V
2026-04-13_23-02-17_d461 val (smoothed) —
2026-04-13_23-02-17_d461 val (raw) —

Dehaze

↓ kind=loss
T
2026-04-13_23-02-17_d461 train (smoothed) —
2026-04-13_23-02-17_d461 train (raw) —

Preserve

↓ kind=loss
T
2026-04-13_23-02-17_d461 train (smoothed) —
2026-04-13_23-02-17_d461 train (raw) —

Total

↓ kind=loss
T
2026-04-13_23-02-17_d461 train (smoothed) —
2026-04-13_23-02-17_d461 train (raw) —

Visibility

↓ kind=loss
T
2026-04-13_23-02-17_d461 train (smoothed) —
2026-04-13_23-02-17_d461 train (raw) —

Visibility Aux

↓ kind=loss
T
2026-04-13_23-02-17_d461 train (smoothed) —
2026-04-13_23-02-17_d461 train (raw) —

Visibility Rank

↓ kind=loss
T
2026-04-13_23-02-17_d461 train (smoothed) —
2026-04-13_23-02-17_d461 train (raw) —

Visibility Tv

↓ kind=loss
T
2026-04-13_23-02-17_d461 train (smoothed) —
2026-04-13_23-02-17_d461 train (raw) —

Visibility Mean

kind=stat
T V
2026-04-13_23-02-17_d461 train (smoothed) —
2026-04-13_23-02-17_d461 val (smoothed) —
2026-04-13_23-02-17_d461 train (raw) —
2026-04-13_23-02-17_d461 val (raw) —

Visibility Std

kind=stat
T V
2026-04-13_23-02-17_d461 train (smoothed) —
2026-04-13_23-02-17_d461 val (smoothed) —
2026-04-13_23-02-17_d461 train (raw) —
2026-04-13_23-02-17_d461 val (raw) —

Gpu Mem Total Gb

T
2026-04-13_23-02-17_d461 train (smoothed) —
2026-04-13_23-02-17_d461 train (raw) —

Gpu Mem Used Gb

T
2026-04-13_23-02-17_d461 train (smoothed) —
2026-04-13_23-02-17_d461 train (raw) —

Gpu Mem Util Pct

T
2026-04-13_23-02-17_d461 train (smoothed) —
2026-04-13_23-02-17_d461 train (raw) —

Gpu Util Pct

T
2026-04-13_23-02-17_d461 train (smoothed) —
2026-04-13_23-02-17_d461 train (raw) —

Lr

T
2026-04-13_23-02-17_d461 train (smoothed) —
2026-04-13_23-02-17_d461 train (raw) —

Visibility Aux Scale

T
2026-04-13_23-02-17_d461 train (smoothed) —
2026-04-13_23-02-17_d461 train (raw) —

No output snapshots found for this run.

Outputs are generated during training and saved to outputs/epoch_N_step_M/ directories.

epoch 1 / step 612 Checkpoint #2759
4/13/2026, 11:17:54 PM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-04-13-23-02-17-d461/checkpoint-1
epoch 3 / step 1224 Checkpoint #2761
4/14/2026, 9:53:46 AM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-04-13-23-02-17-d461/checkpoint-3
epoch 5 / step 1836 Checkpoint #2762
4/14/2026, 9:53:46 AM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-04-13-23-02-17-d461/checkpoint-5
epoch 7 / step 2448 Checkpoint #2763
4/14/2026, 9:53:46 AM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-04-13-23-02-17-d461/checkpoint-7
epoch 9 / step 3060 Checkpoint #2764
4/14/2026, 9:53:46 AM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-04-13-23-02-17-d461/checkpoint-9
epoch 11 / step 3672 Checkpoint #2765
4/14/2026, 9:53:46 AM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-04-13-23-02-17-d461/checkpoint-11

Run-Owned Exports

Publish direct filesystem paths here.

Published

No run-owned exports are published yet.

Raw Artifacts

Run-owned exports are typically direct paths, so there are no captured artifacts to publish from here.

Euler View - ML Experiment Monitor