Euler View
No jobs

Run Dossier

Training run for SD 2.1. Mari

86 metrics · 15,732 train · 109 val

Chart Overlays

Compare this run against its peers

0/63
Add peer runs here to layer their train and val traces onto the current metric charts.
crashed Jun 29, 09:35 Duration: 4h 34m Python 3.11.7 train_joint.py --config /cluster/home/drothenpiele/models/stable_diffusion_mari/mari/.euler-launches/euler_launch_a4c5aab1-77f0-4998-9a82-d853bed69ec0/joint_vkitti.yaml --resume /cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-06-28-16-36-15-2d4a/checkpoint-11
lr 4e-5batch 16epochs 200precision fp16model stable-diffusion-2-1 seed 42wd 0grad accum 1warmup 0.05000ema nograd clip 1
ID 5109406
mari/runs/2026-06-29_09-35-32_9a35
Error
BrokenPipeError: [Errno 32] Broken pipe
Metrics 86/86
camera.train 9
camera_cx_abs_error_px
camera_cy_abs_error_px
camera_focal_rel_error
camera_fx_rel_error
camera_fy_rel_error
camera_intrinsics_rel_error
camera_principal_point_saturation_fraction
camera_principal_rel_error
camera_valid_fraction
depth.train 29
diag 13
depth_abs_rel
depth_delta1
depth_edge_smoothness
depth_head_abs_rel
depth_head_delta1
depth_head_mae
depth_hf_energy
depth_mae
depth_road_hsmooth
depth_tv
vae_decode_abs_rel
vae_decode_delta1
vae_decode_mae
loss 14
camera_ray
depth
depth_head
depth_tv
edge_smoothness
eg_ssi
geometry
geometry_consistency
geometry_log_depth
lambda_mse
multiscale_smoothness
point_aux
smoothness_aux
uncertainty
stat 2
geometry_depth_mean
geometry_uncertainty_mean
geometry.train 26
geometry_abs_rel
geometry_delta0_5
geometry_delta1
geometry_delta2
geometry_delta3
geometry_edge_f1
geometry_edge_grad_mae
geometry_edge_precision
geometry_edge_recall
geometry_enabled
geometry_log_depth_error_visibility_0p00_0p33
geometry_log_depth_error_visibility_0p33_0p66
geometry_log_depth_error_visibility_0p66_1p00
geometry_log10
geometry_mae
geometry_ray_angle_deg
geometry_ray_angle_median_deg
geometry_ray_rho_auc_15
geometry_rmse
geometry_rmse_log
geometry_silog
geometry_sq_rel
geometry_texture_edge_false_positive_rate
geometry_uncertainty_ause
geometry_uncertainty_nause
geometry_uncertainty_spearman
rgb.train 11
diag 2
image_psnr
image_ssim
loss 7
dehaze
preserve
total
visibility
visibility_aux
visibility_rank
visibility_tv
stat 2
visibility_mean
visibility_std
sys.train 6
gpu_mem_total_gb
gpu_mem_used_gb
gpu_mem_util_pct
gpu_util_pct
lr
visibility_aux_scale
Legacy 5
data/camera 3
enabled/any/source
fields/emitted
rays/emitted
train/camera/valid/fraction
val/source/rds/camera/valid/fraction
86 selected
X-Axis
Y-Scale
Series
Group
Namespace

Camera Cx Abs Error Px

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Camera Cy Abs Error Px

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Camera Focal Rel Error

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Camera Fx Rel Error

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Camera Fy Rel Error

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Camera Intrinsics Rel Error

↓ kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Camera Principal Point Saturation Fraction

↓ kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Camera Principal Rel Error

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Camera Valid Fraction

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Depth Abs Rel

↓ kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Depth Delta1

↑ kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Depth Edge Smoothness

↓ kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Depth Head Abs Rel

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Depth Head Delta1

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Depth Head Mae

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Depth Hf Energy

↓ kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Depth Mae

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Depth Road Hsmooth

↓ kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Depth Tv

↓ kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Vae Decode Abs Rel

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Vae Decode Delta1

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Vae Decode Mae

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Camera Ray

kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Depth

↓ kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Depth Head

kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Depth Tv

↓ kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Edge Smoothness

↓ kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Eg Ssi

kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Geometry

kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Geometry Consistency

kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Geometry Log Depth

kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Lambda Mse

kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Multiscale Smoothness

↓ kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Point Aux

kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Smoothness Aux

↓ kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Uncertainty

kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Geometry Depth Mean

kind=stat
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Geometry Uncertainty Mean

kind=stat
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Geometry Abs Rel

↓ kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Delta0 5

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Delta1

↑ kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Delta2

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Delta3

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Edge F1

↑ kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Edge Grad Mae

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Edge Precision

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Edge Recall

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Enabled

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Log Depth Error Visibility 0p00 0p33

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Log Depth Error Visibility 0p33 0p66

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Log Depth Error Visibility 0p66 1p00

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Log10

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Mae

↓ kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Ray Angle Deg

↓ deg kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Ray Angle Median Deg

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Ray Rho Auc 15

↑ kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Rmse

↓ kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Rmse Log

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Silog

↓ kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Sq Rel

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Texture Edge False Positive Rate

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Uncertainty Ause

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Uncertainty Nause

↓ kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Geometry Uncertainty Spearman

kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Image Psnr

↑ dB kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Image Ssim

↑ kind=diag
V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Dehaze

↓ kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Preserve

↓ kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Total

↓ kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Visibility

↓ kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Visibility Aux

↓ kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Visibility Rank

↓ kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Visibility Tv

↓ kind=loss
T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Visibility Mean

kind=stat
T V
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —
2026-06-29_09-35-32_9a35 val (raw) —

Visibility Std

kind=stat
T V
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —
2026-06-29_09-35-32_9a35 val (raw) —

Gpu Mem Total Gb

T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Gpu Mem Used Gb

T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Gpu Mem Util Pct

T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Gpu Util Pct

T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Lr

T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Visibility Aux Scale

T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Data/Camera Enabled Any Source

V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Data/Camera Fields Emitted

V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Data/Camera Rays Emitted

V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

Train/Camera Valid Fraction

T
2026-06-29_09-35-32_9a35 train (smoothed) —
2026-06-29_09-35-32_9a35 train (raw) —

Val/Source/Rds/Camera Valid Fraction

V
2026-06-29_09-35-32_9a35 val (smoothed) —
2026-06-29_09-35-32_9a35 val (raw) —

No output snapshots found for this run.

Outputs are generated during training and saved to outputs/epoch_N_step_M/ directories.

epoch 13 / step 17094 Checkpoint #3704
6/29/2026, 9:54:07 AM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-06-29-09-35-32-9a35/checkpoint-13
epoch 15 / step 19536 Checkpoint #3712
6/29/2026, 11:57:10 AM
/cluster/scratch/drothenpiele/euler_train/mari/checkpoints/joint-dehaze-metric-2026-06-29-09-35-32-9a35/checkpoint-15
Producer launch exports are available. Manage launch-owned exports here for quick reference.
Open Launch

Inherited Launch Exports

These exports are published by the run's producer launch.

Published

The producer launch does not publish any exports yet.

Run-Owned Exports

Publish direct filesystem paths here.

Published

No run-owned exports are published yet.

Raw Artifacts

Run-owned exports are typically direct paths, so there are no captured artifacts to publish from here.

Euler View - ML Experiment Monitor