Euler View
No jobs

Run Dossier

Training run for SD 2.1. Mari

42 metrics · 40,611 train · 109 val

Chart Overlays

Compare this run against its peers

0/105
Add peer runs here to layer their train and val traces onto the current metric charts.
crashed Jul 18, 23:04 Duration: 13h 22m Python 3.11.7 train_joint.py --config /cluster/home/drothenpiele/models/stable_diffusion_mari/mari/.euler-launches/euler_launch_0cea3305-0c98-4c7c-8b6a-36cfdb7782bc/joint_vkitti.yaml
lr 4e-5batch 16epochs 200precision fp16model stable-diffusion-2-1 seed 42wd 0grad accum 1warmup 0.05000ema nograd clip 1
ID 7669321
mari/runs/2026-07-18_23-04-18_940d
Error
BrokenPipeError: [Errno 32] Broken pipe
Metrics 42/42
depth.train 21
diag 15
depth_abs_rel
depth_delta1
depth_mae
valset_muses_absrel
valset_muses_delta1
valset_muses_delta2
valset_muses_delta3
valset_muses_mae
valset_muses_num_samples
valset_muses_projected_pixels
valset_muses_rmse
valset_muses_rmse_log
valset_muses_silog
valset_muses_sqrel
valset_muses_valid_pixels
loss 6
depth
depth_head
depth_tv
edge_smoothness
multiscale_smoothness
smoothness_aux
rgb.train 11
diag 2
image_psnr
image_ssim
loss 7
dehaze
preserve
total
visibility
visibility_aux
visibility_rank
visibility_tv
stat 2
visibility_mean
visibility_std
sys.train 6
gpu_mem_total_gb
gpu_mem_used_gb
gpu_mem_util_pct
gpu_util_pct
lr
visibility_aux_scale
Legacy 4
data/camera 3
enabled/any/source
fields/emitted
rays/emitted
train/camera/valid/fraction
42 selected
X-Axis
Y-Scale
Series
Group
Namespace

Depth Abs Rel

↓ kind=diag
V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Depth Delta1

↑ kind=diag
V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Depth Mae

kind=diag
V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Valset Muses Absrel

kind=diag
V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Valset Muses Delta1

kind=diag
V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Valset Muses Delta2

kind=diag
V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Valset Muses Delta3

kind=diag
V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Valset Muses Mae

kind=diag
V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Valset Muses Num Samples

kind=diag
V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Valset Muses Projected Pixels

kind=diag
V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Valset Muses Rmse

kind=diag
V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Valset Muses Rmse Log

kind=diag
V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Valset Muses Silog

kind=diag
V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Valset Muses Sqrel

kind=diag
V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Valset Muses Valid Pixels

kind=diag
V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Depth

↓ kind=loss
T
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d train (raw) —

Depth Head

kind=loss
T
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d train (raw) —

Depth Tv

↓ kind=loss
T
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d train (raw) —

Edge Smoothness

↓ kind=loss
T
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d train (raw) —

Multiscale Smoothness

↓ kind=loss
T
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d train (raw) —

Smoothness Aux

↓ kind=loss
T
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d train (raw) —

Image Psnr

↑ dB kind=diag
V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Image Ssim

↑ kind=diag
V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Dehaze

↓ kind=loss
T
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d train (raw) —

Preserve

↓ kind=loss
T
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d train (raw) —

Total

↓ kind=loss
T
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d train (raw) —

Visibility

↓ kind=loss
T
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d train (raw) —

Visibility Aux

↓ kind=loss
T
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d train (raw) —

Visibility Rank

↓ kind=loss
T
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d train (raw) —

Visibility Tv

↓ kind=loss
T
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d train (raw) —

Visibility Mean

kind=stat
T
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d train (raw) —

Visibility Std

kind=stat
T
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d train (raw) —

Gpu Mem Total Gb

T V
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d train (raw) —
2026-07-18_23-04-18_940d val (raw) —

Gpu Mem Used Gb

T V
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d train (raw) —
2026-07-18_23-04-18_940d val (raw) —

Gpu Mem Util Pct

T V
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d train (raw) —
2026-07-18_23-04-18_940d val (raw) —

Gpu Util Pct

T V
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d train (raw) —
2026-07-18_23-04-18_940d val (raw) —

Lr

T
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d train (raw) —

Visibility Aux Scale

T
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d train (raw) —

Data/Camera Enabled Any Source

V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Data/Camera Fields Emitted

V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Data/Camera Rays Emitted

V
2026-07-18_23-04-18_940d val (smoothed) —
2026-07-18_23-04-18_940d val (raw) —

Train/Camera Valid Fraction

T
2026-07-18_23-04-18_940d train (smoothed) —
2026-07-18_23-04-18_940d train (raw) —

No output snapshots found for this run.

Outputs are generated during training and saved to outputs/epoch_N_step_M/ directories.

epoch 1 / step 3238 Checkpoint #3932
7/19/2026, 9:33:32 AM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-07-18-23-04-18-940d/checkpoint-1
epoch 3 / step 6476 Checkpoint #3933
7/19/2026, 9:33:32 AM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-07-18-23-04-18-940d/checkpoint-3
epoch 5 / step 9714 Checkpoint #3934
7/19/2026, 9:33:32 AM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-07-18-23-04-18-940d/checkpoint-5
epoch 7 / step 12952 Checkpoint #3935
7/19/2026, 9:33:32 AM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-07-18-23-04-18-940d/checkpoint-7
epoch 9 / step 16190 Checkpoint #3936
7/19/2026, 9:33:32 AM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-07-18-23-04-18-940d/checkpoint-9
epoch 11 / step 19428 Checkpoint #3937
7/19/2026, 9:33:32 AM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-07-18-23-04-18-940d/checkpoint-11
Producer launch exports are available. Manage launch-owned exports here for quick reference.
Open Launch

Inherited Launch Exports

These exports are published by the run's producer launch.

Published

The producer launch does not publish any exports yet.

Run-Owned Exports

Publish direct filesystem paths here.

Published

No run-owned exports are published yet.

Raw Artifacts

Run-owned exports are typically direct paths, so there are no captured artifacts to publish from here.

Euler View - ML Experiment Monitor