Chart Overlays
Compare this run against its peers
Add peer runs here to layer their train and val traces onto the current metric charts.
interrupted Apr 11, 22:50 Duration: 11h 58m Python 3.11.7
src/train.py --config configs/model/inverted_joint_metric.yaml --config configs/model/backbone_dinov2_vitl14.yaml --config configs/data/real_drive_sim_euler_loading.yaml --config configs/training/phase_b_joint_structure.yaml --init-from /cluster/scratch/drothenpiele/euler_train/unifuse/checkpoints/geometry-prompt/checkpoint-epoch-2.ptlr 2e-4batch 4epochs 12precision automodel weather_metric_system wd 1e-4
ID 63055282 Job metric_diffusion_pty Part gpupr.24h Node eu-a65-05 CPU 8 GPU 1
unifuse/runs/2026-04-11_22-50-53_32e9 Error
Signal: SIGINT
Metrics 71/71
depth.train 49
diag 26
consolidation 1
delta_vs_prior
final 5
depth_abs_rel
depth_mae
depth_rmse
log_radius_l1
xyz_l1
pred 5
depth_abs_rel
depth_mae
depth_rmse
log_radius_l1
xyz_l1
prior 5
depth_abs_rel
depth_mae
depth_rmse
log_radius_l1
xyz_l1
structure 10
depth_l1
edge_l1
eval_ensemble_size
first_nonfinite_step_index
first_nonfinite_timestep
had_nonfinite
mode_is_sampled
mode_is_teacher_forced
sampled_debug_enabled
used_fp32_sampler
loss 18
consolidation 3
low_frequency
structure_align
structure_align_edge
final 4
confidence
depth
log_radius
xyz
pred 1
log_radius
prior 4
aux_metric_probe
aux_metric_probe_confidence
confidence
log_radius
structure 5
confidence
depth
depth_denoise_mse
edge
rgb_denoise_mse
no stage 1
total
stat 5
final 1
confidence
prior 1
confidence
structure 3
confidence
pred_finite_fraction
pred_has_nonfinite
rgb.train 6
diag 2
image_mae
image_psnr
loss 3
clear
confidence
rgb_denoise_mse
stat 1
visibility
sys.train 16
data_mean_depth
data_median_depth
data_valid_fraction
flag_refiner_enabled
flag_structure_enabled
gpu_mem_total_gb
gpu_mem_used_gb
gpu_mem_util_pct
gpu_util_pct
lr
lr.default
lr.geometry_encoder_adapter
lr.joint_structure_expert_heads
lr.joint_structure_expert_model
lr.prompt_encoder
ray_source_is_predicted
71 selected
X-Axis
Y-Scale
Series
Group
Namespace
Delta Vs Prior
↓ kind=diagstage=consolidation T V
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
2026-04-11_22-50-53_32e9 val (raw) —
Depth Abs Rel
↓ kind=diagstage=final V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Depth Mae
↓ meters kind=diagstage=final T V
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
2026-04-11_22-50-53_32e9 val (raw) —
Depth Rmse
↓ meters kind=diagstage=final V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Log Radius L1
↓ kind=diagstage=final T V
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
2026-04-11_22-50-53_32e9 val (raw) —
Xyz L1
↓ meters kind=diagstage=final T V
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
2026-04-11_22-50-53_32e9 val (raw) —
Depth Abs Rel
↓ kind=diagstage=pred V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Depth Mae
↓ meters kind=diagstage=pred T V
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
2026-04-11_22-50-53_32e9 val (raw) —
Depth Rmse
↓ meters kind=diagstage=pred V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Log Radius L1
↓ kind=diagstage=pred T V
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
2026-04-11_22-50-53_32e9 val (raw) —
Xyz L1
↓ meters kind=diagstage=pred T V
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
2026-04-11_22-50-53_32e9 val (raw) —
Depth Abs Rel
↓ kind=diagstage=prior V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Depth Mae
↓ meters kind=diagstage=prior T V
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
2026-04-11_22-50-53_32e9 val (raw) —
Depth Rmse
↓ meters kind=diagstage=prior V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Log Radius L1
↓ kind=diagstage=prior T V
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
2026-04-11_22-50-53_32e9 val (raw) —
Xyz L1
↓ meters kind=diagstage=prior T V
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
2026-04-11_22-50-53_32e9 val (raw) —
Depth L1
↓ kind=diagstage=structure T V
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
2026-04-11_22-50-53_32e9 val (raw) —
Edge L1
↓ kind=diagstage=structure T V
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
2026-04-11_22-50-53_32e9 val (raw) —
Eval Ensemble Size
kind=diagstage=structure V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
First Nonfinite Step Index
kind=diagstage=structure V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
First Nonfinite Timestep
kind=diagstage=structure V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Had Nonfinite
kind=diagstage=structure V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Mode Is Sampled
kind=diagstage=structure V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Mode Is Teacher Forced
kind=diagstage=structure V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Sampled Debug Enabled
kind=diagstage=structure V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Used Fp32 Sampler
kind=diagstage=structure V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Low Frequency
↓ kind=lossstage=consolidation T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Structure Align
↓ kind=lossstage=consolidation T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Structure Align Edge
↓ kind=lossstage=consolidation T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Confidence
↑ kind=lossstage=final T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Depth
↓ meters kind=lossstage=final T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Log Radius
↓ kind=lossstage=final T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Xyz
↓ meters kind=lossstage=final T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Log Radius
↓ kind=lossstage=pred T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Aux Metric Probe
kind=lossstage=prior T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Aux Metric Probe Confidence
kind=lossstage=prior T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Confidence
↑ kind=lossstage=prior T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Log Radius
↓ kind=lossstage=prior T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Confidence
↑ kind=lossstage=structure T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Depth
↓ meters kind=lossstage=structure T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Depth Denoise Mse
kind=lossstage=structure T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Edge
↓ kind=lossstage=structure T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Rgb Denoise Mse
kind=lossstage=structure T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Total
↓ kind=loss T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Confidence
↑ kind=statstage=final V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Confidence
↑ kind=statstage=prior V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Confidence
↑ kind=statstage=structure T V
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
2026-04-11_22-50-53_32e9 val (raw) —
Pred Finite Fraction
kind=statstage=structure V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Pred Has Nonfinite
kind=statstage=structure V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Image Mae
↓ kind=diagstage=structure V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Image Psnr
↑ dB kind=diagstage=structure V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Clear
kind=lossstage=structure T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Confidence
↑ kind=lossstage=structure T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Rgb Denoise Mse
kind=lossstage=structure T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Visibility
↑ kind=statstage=weather V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Data Mean Depth
V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Data Median Depth
V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Data Valid Fraction
V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Flag Refiner Enabled
V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Flag Structure Enabled
V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
Gpu Mem Total Gb
T V
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
2026-04-11_22-50-53_32e9 val (raw) —
Gpu Mem Used Gb
T V
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
2026-04-11_22-50-53_32e9 val (raw) —
Gpu Mem Util Pct
T V
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
2026-04-11_22-50-53_32e9 val (raw) —
Gpu Util Pct
T V
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
2026-04-11_22-50-53_32e9 val (raw) —
Lr
T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Lr.Default
T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Lr.Geometry Encoder Adapter
T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Lr.Joint Structure Expert Heads
T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Lr.Joint Structure Expert Model
T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Lr.Prompt Encoder
T
2026-04-11_22-50-53_32e9 train (smoothed) —
2026-04-11_22-50-53_32e9 train (raw) —
Ray Source Is Predicted
V
2026-04-11_22-50-53_32e9 val (smoothed) —
2026-04-11_22-50-53_32e9 val (raw) —
No output snapshots found for this run.
Outputs are generated during training and saved to outputs/epoch_N_step_M/ directories.
epoch 1 / step 800 Checkpoint #2689
4/11/2026, 10:34:39 PM
/cluster/scratch/drothenpiele/euler_train/unifuse/checkpoints/joint-structure-expert-2026-04-11-22-50-53-32e9/checkpoint-epoch-1.ptepoch 2 / step 1600 Checkpoint #2693
4/11/2026, 11:58:16 PM
/cluster/scratch/drothenpiele/euler_train/unifuse/checkpoints/joint-structure-expert-2026-04-11-22-50-53-32e9/checkpoint-epoch-2.ptepoch 3 / step 2400 Checkpoint #2703
4/12/2026, 8:42:37 AM
/cluster/scratch/drothenpiele/euler_train/unifuse/checkpoints/joint-structure-expert-2026-04-11-22-50-53-32e9/checkpoint-epoch-3.ptepoch 4 / step 3200 Checkpoint #2705
4/12/2026, 8:42:37 AM
/cluster/scratch/drothenpiele/euler_train/unifuse/checkpoints/joint-structure-expert-2026-04-11-22-50-53-32e9/checkpoint-epoch-4.ptepoch 5 / step 4000 Checkpoint #2707
4/12/2026, 8:42:37 AM
/cluster/scratch/drothenpiele/euler_train/unifuse/checkpoints/joint-structure-expert-2026-04-11-22-50-53-32e9/checkpoint-epoch-5.ptepoch 6 / step 4800 Checkpoint #2708
4/12/2026, 8:42:37 AM
/cluster/scratch/drothenpiele/euler_train/unifuse/checkpoints/joint-structure-expert-2026-04-11-22-50-53-32e9/checkpoint-epoch-6.ptepoch 7 / step 5600 Checkpoint #2709
4/12/2026, 8:42:37 AM
/cluster/scratch/drothenpiele/euler_train/unifuse/checkpoints/joint-structure-expert-2026-04-11-22-50-53-32e9/checkpoint-epoch-7.ptepoch 8 / step 6400 Checkpoint #2710
4/12/2026, 8:42:37 AM
/cluster/scratch/drothenpiele/euler_train/unifuse/checkpoints/joint-structure-expert-2026-04-11-22-50-53-32e9/checkpoint-epoch-8.ptepoch 9 / step 7200 Checkpoint #2711
4/12/2026, 8:42:37 AM
/cluster/scratch/drothenpiele/euler_train/unifuse/checkpoints/joint-structure-expert-2026-04-11-22-50-53-32e9/checkpoint-epoch-9.ptepoch 10 / step 8000 Checkpoint #2712
4/12/2026, 8:42:37 AM
/cluster/scratch/drothenpiele/euler_train/unifuse/checkpoints/joint-structure-expert-2026-04-11-22-50-53-32e9/checkpoint-epoch-10.ptepoch 11 / step 8800 Checkpoint #2713
4/12/2026, 8:42:37 AM
/cluster/scratch/drothenpiele/euler_train/unifuse/checkpoints/joint-structure-expert-2026-04-11-22-50-53-32e9/checkpoint-epoch-11.ptRun-Owned Exports
Publish direct filesystem paths here.
Published
No run-owned exports are published yet.
Raw Artifacts
Run-owned exports are typically direct paths, so there are no captured artifacts to publish from here.