Euler View
No jobs

Run Dossier

Training run for SD 2.1. Mari

86 metrics · 72,476 train · 533 val

Chart Overlays

Compare this run against its peers

0/63
Add peer runs here to layer their train and val traces onto the current metric charts.
crashed Jun 30, 12:44 Duration: 21h 59m Python 3.11.7 train_joint.py --config /cluster/home/drothenpiele/models/stable_diffusion_mari/mari/.euler-launches/euler_launch_7681f163-59af-465e-89cd-5e97b7433bbc/joint_vkitti.yaml --resume /cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-06-30-01-17-37-a2c0/checkpoint-5
lr 4e-5batch 16epochs 200precision fp16model stable-diffusion-2-1 seed 42wd 0grad accum 1warmup 0.05000ema nograd clip 1
ID 5244736
mari/runs/2026-06-30_12-44-58_d837
Error
BrokenPipeError: [Errno 32] Broken pipe
Metrics 86/86
camera.train 9
camera_cx_abs_error_px
camera_cy_abs_error_px
camera_focal_rel_error
camera_fx_rel_error
camera_fy_rel_error
camera_intrinsics_rel_error
camera_principal_point_saturation_fraction
camera_principal_rel_error
camera_valid_fraction
depth.train 29
diag 13
depth_abs_rel
depth_delta1
depth_edge_smoothness
depth_head_abs_rel
depth_head_delta1
depth_head_mae
depth_hf_energy
depth_mae
depth_road_hsmooth
depth_tv
vae_decode_abs_rel
vae_decode_delta1
vae_decode_mae
loss 14
camera_ray
depth
depth_head
depth_tv
edge_smoothness
eg_ssi
geometry
geometry_consistency
geometry_log_depth
lambda_mse
multiscale_smoothness
point_aux
smoothness_aux
uncertainty
stat 2
geometry_depth_mean
geometry_uncertainty_mean
geometry.train 26
geometry_abs_rel
geometry_delta0_5
geometry_delta1
geometry_delta2
geometry_delta3
geometry_edge_f1
geometry_edge_grad_mae
geometry_edge_precision
geometry_edge_recall
geometry_enabled
geometry_log_depth_error_visibility_0p00_0p33
geometry_log_depth_error_visibility_0p33_0p66
geometry_log_depth_error_visibility_0p66_1p00
geometry_log10
geometry_mae
geometry_ray_angle_deg
geometry_ray_angle_median_deg
geometry_ray_rho_auc_15
geometry_rmse
geometry_rmse_log
geometry_silog
geometry_sq_rel
geometry_texture_edge_false_positive_rate
geometry_uncertainty_ause
geometry_uncertainty_nause
geometry_uncertainty_spearman
rgb.train 11
diag 2
image_psnr
image_ssim
loss 7
dehaze
preserve
total
visibility
visibility_aux
visibility_rank
visibility_tv
stat 2
visibility_mean
visibility_std
sys.train 6
gpu_mem_total_gb
gpu_mem_used_gb
gpu_mem_util_pct
gpu_util_pct
lr
visibility_aux_scale
Legacy 5
data/camera 3
enabled/any/source
fields/emitted
rays/emitted
train/camera/valid/fraction
val/source/rds/camera/valid/fraction
86 selected
X-Axis
Y-Scale
Series
Group
Namespace

Camera Cx Abs Error Px

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Camera Cy Abs Error Px

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Camera Focal Rel Error

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Camera Fx Rel Error

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Camera Fy Rel Error

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Camera Intrinsics Rel Error

↓ kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Camera Principal Point Saturation Fraction

↓ kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Camera Principal Rel Error

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Camera Valid Fraction

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Depth Abs Rel

↓ kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Depth Delta1

↑ kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Depth Edge Smoothness

↓ kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Depth Head Abs Rel

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Depth Head Delta1

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Depth Head Mae

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Depth Hf Energy

↓ kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Depth Mae

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Depth Road Hsmooth

↓ kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Depth Tv

↓ kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Vae Decode Abs Rel

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Vae Decode Delta1

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Vae Decode Mae

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Camera Ray

kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Depth

↓ kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Depth Head

kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Depth Tv

↓ kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Edge Smoothness

↓ kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Eg Ssi

kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Geometry

kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Geometry Consistency

kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Geometry Log Depth

kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Lambda Mse

kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Multiscale Smoothness

↓ kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Point Aux

kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Smoothness Aux

↓ kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Uncertainty

kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Geometry Depth Mean

kind=stat
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Geometry Uncertainty Mean

kind=stat
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Geometry Abs Rel

↓ kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Delta0 5

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Delta1

↑ kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Delta2

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Delta3

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Edge F1

↑ kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Edge Grad Mae

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Edge Precision

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Edge Recall

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Enabled

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Log Depth Error Visibility 0p00 0p33

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Log Depth Error Visibility 0p33 0p66

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Log Depth Error Visibility 0p66 1p00

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Log10

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Mae

↓ kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Ray Angle Deg

↓ deg kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Ray Angle Median Deg

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Ray Rho Auc 15

↑ kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Rmse

↓ kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Rmse Log

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Silog

↓ kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Sq Rel

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Texture Edge False Positive Rate

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Uncertainty Ause

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Uncertainty Nause

↓ kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Geometry Uncertainty Spearman

kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Image Psnr

↑ dB kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Image Ssim

↑ kind=diag
V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Dehaze

↓ kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Preserve

↓ kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Total

↓ kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Visibility

↓ kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Visibility Aux

↓ kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Visibility Rank

↓ kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Visibility Tv

↓ kind=loss
T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Visibility Mean

kind=stat
T V
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —
2026-06-30_12-44-58_d837 val (raw) —

Visibility Std

kind=stat
T V
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —
2026-06-30_12-44-58_d837 val (raw) —

Gpu Mem Total Gb

T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Gpu Mem Used Gb

T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Gpu Mem Util Pct

T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Gpu Util Pct

T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Lr

T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Visibility Aux Scale

T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Data/Camera Enabled Any Source

V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Data/Camera Fields Emitted

V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Data/Camera Rays Emitted

V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

Train/Camera Valid Fraction

T
2026-06-30_12-44-58_d837 train (smoothed) —
2026-06-30_12-44-58_d837 train (raw) —

Val/Source/Rds/Camera Valid Fraction

V
2026-06-30_12-44-58_d837 val (smoothed) —
2026-06-30_12-44-58_d837 val (raw) —

No output snapshots found for this run.

Outputs are generated during training and saved to outputs/epoch_N_step_M/ directories.

epoch 7 / step 9768 Checkpoint #3730
6/30/2026, 1:13:04 PM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-06-30-12-44-58-d837/checkpoint-7
epoch 9 / step 12210 Checkpoint #3736
6/30/2026, 3:54:52 PM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-06-30-12-44-58-d837/checkpoint-9
epoch 11 / step 14652 Checkpoint #3741
6/30/2026, 6:17:18 PM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-06-30-12-44-58-d837/checkpoint-11
epoch 13 / step 17094 Checkpoint #3748
6/30/2026, 8:35:46 PM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-06-30-12-44-58-d837/checkpoint-13
epoch 15 / step 19536 Checkpoint #3753
6/30/2026, 11:30:50 PM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-06-30-12-44-58-d837/checkpoint-15
epoch 17 / step 21978 Checkpoint #3764
7/1/2026, 8:40:59 AM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-06-30-12-44-58-d837/checkpoint-17
epoch 19 / step 24420 Checkpoint #3765
7/1/2026, 8:40:59 AM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-06-30-12-44-58-d837/checkpoint-19
epoch 21 / step 26862 Checkpoint #3766
7/1/2026, 8:40:59 AM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-06-30-12-44-58-d837/checkpoint-21
epoch 23 / step 29304 Checkpoint #3767
7/1/2026, 8:40:59 AM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-06-30-12-44-58-d837/checkpoint-23
epoch 25 / step 31746 Checkpoint #3768
7/1/2026, 8:40:59 AM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-06-30-12-44-58-d837/checkpoint-25
Producer launch exports are available. Manage launch-owned exports here for quick reference.
Open Launch

Inherited Launch Exports

These exports are published by the run's producer launch.

Published

The producer launch does not publish any exports yet.

Run-Owned Exports

Publish direct filesystem paths here.

Published

No run-owned exports are published yet.

Raw Artifacts

Run-owned exports are typically direct paths, so there are no captured artifacts to publish from here.

Euler View - ML Experiment Monitor