No jobs
Back to SD 2.1. Mari

Train: Optimised single_pass — soft-inverse τ30 + L2 + sky0.1 area-pooled + 10% zero-SNR (C3) f02817a5

Supersedes config 96 (C2) for a cold-start soft-inverse-depth experiment on model “SD 2.1. Mari”. Primary changes: 1. Uses model.depth_mode=metric_soft_inverse with one global tau=30 m. The forward mapping is y=(d−tau)/(d+tau), so near depth is low and far depth approaches +1. 2. Uses a single L2 diffusion-prediction loss from epoch 0; removes the C2 L2/Charbonnier ramp. 3. Samples the zero-terminal-SNR endpoint for 10% of examples and samples the remaining 90% uniformly from first through penultimate timestep. 4. Supervises finite sky targets at 700 m with weight 0.1 and area-pools depth supervision weights. 5. Sets the model decode/evaluation/export cap to 700 m, the shared dataset validity minimum to 2.5 m, and the general validity maximum to 700 m. Soft inverse intentionally has no normalization_min_depth_m; 2.5 m is only a dataset validity cutoff. 6. Enables explicit mask-aware depth resizing. VKITTI2 remains a necessary source-specific exception at valid_max_depth_m=650 m because its 16-bit-centimetre sky sentinel is 655.35 m. Raising that validity threshold to 700 m without a semantic sky mask would classify the sentinel as ordinary valid depth and defeat sky supervision. The finite sky target and model cap remain 700 m. RDS and MUSES use the shared 2.5–700 m validity interval. Euler’s RDS metadata reports a raw 0–1000 m range, so RDS samples above 700 m intentionally enter the far/sky fallback and receive the finite 700 m target. This successor does not resume a metric-log checkpoint; the inherited launch script starts train_joint.py without --resume, avoiding representation reinterpretation.

training failed

Output

pending f02817a5
Launch Output
Waiting for SLURM job to start...

Failure Metadata

failed
interactive.monitor.poll INT_EXIT_CODE_NONZERO

Interactive command exited with code 255

Context JSON
{
  "reason": null,
  "exitCode": 255,
  "remotePid": 2432808,
  "computePid": null,
  "allocationJobId": null
}

Datasets

14
Dataset clear_root_rds
Type rgb
Split full
Path /cluster/work/igp_psr/drothenpiele/data/rds/rgb.zip/
Dataset depth_root_rds
Type depth
Split full
Path /cluster/work/igp_psr/drothenpiele/data/rds/depth.zip/
Dataset intrinsics_root_rds
Type intrinsics
Split full
Path /cluster/work/igp_psr/drothenpiele/data/rds/calibration.zip/
Dataset train_hazy_root_rds
Type rgb
Split full
Path /cluster/work/igp_psr/drothenpiele/data/rds/gpu/balanced_full/foggy_rgb.zip/
Dataset val_hazy_root_rds
Type rgb
Split val
Path /cluster/work/igp_psr/drothenpiele/data/rds/gloomy_fixed/foggy_rgb.zip/
Dataset clear_root_vkitti2
Dataset VKITTI2
Type rgb
Split full
Path /cluster/work/igp_psr/drothenpiele/data/vkitti_2.0.3_rgb.zip
Dataset depth_root_vkitti2
Dataset VKITTI2
Type depth
Split full
Path /cluster/work/igp_psr/drothenpiele/data/vkitti_2.0.3_depth.zip
Dataset intrinsics_root_vkitti2
Dataset VKITTI2
Type intrinsics
Split full
Path /cluster/work/igp_psr/drothenpiele/data/vkitti_2.0.3_textgt.zip/
Dataset train_hazy_vkitti
Type rgb
Split train
Path /cluster/work/igp_psr/drothenpiele/data/vkitti_next/gpu_2/foggy_rgb.zip/
Dataset val_hazy_vkitti
Type rgb
Split val
Path /cluster/work/igp_psr/drothenpiele/data/vkitti_next/gpu_2/foggy_rgb.zip/
Split muses_rgb
Dataset muses
Type rgb
Split fog_day
Path /cluster/work/igp_psr/drothenpiele/data/muses/frame_camera_trainvaltest.zip/
Split muses_sparse_depth
Dataset muses
Type sparse_depth
Split full
Path /cluster/work/igp_psr/drothenpiele/data/muses/lidar_trainvaltest.zip/
Split muses_intrinsics
Dataset muses
Type intrinsics
Split full
Path /cluster/work/igp_psr/drothenpiele/data/muses/frame_camera_trainvaltest.zip/
Split muses_extrinsics
Dataset muses
Type camera_extrinsics
Split full
Path /cluster/work/igp_psr/drothenpiele/data/muses/frame_camera_trainvaltest.zip/

Execution Artifacts

2
run.sh

Published Exports

Launch-Owned Exports

These semantic handles are what downstream pipelines should resolve against. Auto-published exports come from config-template metadata captured on this launch.

Published

This launch does not publish any exports yet.

Raw Artifacts

Parameters

275

Typed Parameters

Dataset clear_root_rds
real-drive-sim / full (4)
Dataset depth_root_rds
real-drive-sim / full (5)
Dataset intrinsics_root_rds
real-drive-sim / full (6)
Dataset train_hazy_root_rds
real-drive-sim / full (244)
Dataset val_hazy_root_rds
real-drive-sim / val (196)
Dataset clear_root_vkitti2
VKITTI2 / full (1)
Dataset depth_root_vkitti2
VKITTI2 / full (2)
Dataset intrinsics_root_vkitti2
VKITTI2 / full (12)
Dataset train_hazy_vkitti
VKITTI2 / train (216)
Dataset val_hazy_vkitti
VKITTI2 / val (238)
Split muses_rgb
muses / fog_day (77)
Split muses_sparse_depth
muses / full (76)
Split muses_intrinsics
muses / full (82)
Split muses_extrinsics
muses / full (81)

Simple Parameters

clip
true
crop
center
seed
42
tracker
wandb
use_ema
false
gpu_type
rtx_4090:1
job_name
sd_train_concat
norm_max
1
norm_min
-1
run_name
joint-opt-l2-softinv-tau30-sky01-noise10
tmp_size
10G
adam_8bit
true
data_kind
real_drive_sim
ema_decay
0.9999
ema_dtype
float32
log_every
10
adam_beta1
0.9
adam_beta2
0.999
batch_size
16
clear_root
4
depth_mode
metric_soft_inverse
depth_root
5
image_size
[384, 768]
joint_mode
single_pass
log_images
true
num_epochs
200
output_dir
/cluster/scratch/drothenpiele/SD21/exp_1
time_limit
2-00:00:00
mem_per_cpu
8G
num_workers
4
adam_epsilon
1e-8
adam_foreach
false
aspect_ratio
[1, 2]
camera_model
pinhole
conditioning
concat
lambda_depth
1
lambda_point
0.0
min_lr_ratio
0.01
project_name
joint-dehazing
use_geometry
false
warmup_ratio
0.05
weight_decay
0
cpus_per_task
8
lambda_camera
0.0
lambda_dehaze
1
lambda_eg_ssi
0.0
learning_rate
0.00004
max_grad_norm
1
val_hazy_root
104
camera_enabled
true
freeze_encoder
false
log_visibility
true
num_log_images
4
use_depth_head
true
val_batch_size
10
camera_fallback
invalid
depth_head_type
base_residual
enable_xformers
true
euler_train_dir
/cluster/work/igp_psr/drothenpiele/data/out/train/sd21-joint
lambda_depth_tv
0.0002
mixed_precision
fp16
prediction_type
v_prediction
source_kind_rds
real_drive_sim
train_hazy_root
103
visibility_cmap
viridis
camera_embedding
sine
camera_emit_rays
false
camera_mlp_ratio
4
camera_num_heads
4
camera_token_dim
256
depth_noise_type
annealed_multires
lambda_log_depth
0.0
lr_schedule_type
cosine_with_warmup
min_max_quantile
0.02
no_decay_enabled
false
pretrained_model
sd2-community/stable-diffusion-2-1
camera_num_layers
2
camera_num_tokens
4
conv_in_init_mode
joint_marigold
geometry_lambda_z
0.15
geometry_timestep
0
joint_weight_psnr
1
joint_weight_ssim
10
lambda_depth_head
0.2
lambda_lambda_mse
0.0
lambda_visibility
0.05
sampling_strategy
source_balanced
source_weight_rds
1
valid_max_depth_m
700
valid_min_depth_m
2.5
depth_ensemble_tol
0.001
lambda_uncertainty
0.0
model_depth_output
vae_decode
val_every_n_epochs
1
camera_feature_pool
avg
depth_ensemble_size
1
geometry_lambda_phi
1
num_inference_steps
25
rds_val_max_samples
128
sampling_epoch_size
all
save_every_n_epochs
1
source_kind_vkitti2
vkitti2
use_visibility_head
true
warmup_start_factor
0.001
camera_metadata_root
null
conv_in_target_scale
0.7071
external_camera_mode
none
geometry_loss_robust
charbonnier
lambda_vae_depth_rec
0.0
lambda_visibility_tv
0.005
sky_depth_override_m
700
camera_branch_enabled
true
camera_embedding_size
latent
depth_gradient_robust
charbonnier
depth_multires_levels
4
encoder_learning_rate
0.000007
geometry_head_enabled
false
geometry_lambda_theta
1
lambda_depth_gradient
0.05
lambda_geometry_total
0.0
source_weight_vkitti2
1
use_cross_task_fusion
true
vae_decoder_trainable
false
visibility_rank_pairs
128
weight_decay_backbone
0
weight_decay_no_decay
0
zero_grad_set_to_none
true
camera_feature_sources
auto
camera_metadata_format
euler_loading
depth_ensemble_max_res
1024
depth_smoothness_space
latent_x0
freeze_cross_attention
true
gradient_checkpointing
true
inference_depth_output
vae_decode
lambda_geo_consistency
0.0
lambda_visibility_rank
0
use_task_skip_adapters
false
visibility_rank_margin
0.05
camera_embedding_stride
8
camera_prediction_space
residual_intrinsics
depth_ensemble_max_iter
2
depth_multires_strength
0.9
depth_smoothness_scales
4
geometric_pairs_enabled
false
joint_noise_correlation
0
joint_weight_delta1_pct
0.5
keep_last_n_checkpoints
5
mask_aware_depth_resize
true
validation_depth_output
vae_decode
visibility_target_gamma
4
visibility_warmup_steps
1000
vkitti2_val_max_samples
0
cross_task_fusion_blocks
[0,1,2,3]
cross_task_fusion_detach
false
cross_task_fusion_kernel
adaptive
depth_ensemble_reduction
median
geometry_camera_encoding
sine
geometry_head_num_blocks
2
joint_weight_abs_rel_pct
0.5
soft_inverse_depth_tau_m
30
camera_intrinsics_loading
hierarchical
depth_diagnostics_enabled
true
depth_head_residual_scale
0.05
geometry_depth_convention
z_depth
geometry_metric_depth_max
700
geometry_metric_depth_min
2.5
normalization_max_depth_m
700
num_inference_steps_depth
20
vae_decoder_learning_rate
0.000001
valid_max_depth_m_vkitti2
650
visibility_apply_to_depth
false
camera_residual_logit_clip
4
depth_head_hidden_channels
64
depth_resize_interpolation
nearest
depth_smoothness_normalize
true
detach_rays_for_point_loss
true
enable_efficient_attention
true
lambda_visibility_preserve
0.05
visibility_apply_to_dehaze
true
visibility_apply_to_fusion
false
visibility_hidden_channels
32
visibility_target_quantile
0.9
weight_decay_depth_decoder
0
weight_decay_geometry_head
0
camera_require_for_training
true
camera_sine_num_frequencies
null
checkpoint_selection_metric
valset/muses/mae
cross_task_fusion_dilations
[1,2,4]
depth_decoder_learning_rate
0.00002
emit_camera_full_resolution
true
geometry_eg_ssi_edge_source
hazy
geometry_encoder_input_mode
hazy_hazy
geometry_head_learning_rate
0.00004
geometry_output_uncertainty
true
geometry_ray_representation
unit
gradient_accumulation_steps
1
validation_geometry_enabled
false
weight_decay_dehaze_decoder
0
camera_output_representation
angles
camera_valid_for_approximate
false
cross_task_fusion_directions
["rgb_to_depth","depth_to_rgb"]
dehaze_decoder_learning_rate
0.00002
depth_head_context_dilations
[1, 2, 4]
depth_smoothness_edge_source
clear
depth_smoothness_edge_weight
10
geometry_head_residual_scale
0.05
lambda_depth_edge_smoothness
0.002
lambda_rgb_depth_consistency
0.1
weight_decay_geometry_camera
0
depth_diagnostics_edge_source
dehazed
depth_diagnostics_edge_weight
10
depth_head_base_upsample_mode
bilinear
geometry_camera_learning_rate
0.00004
geometry_head_hidden_channels
128
geometry_sine_num_frequencies
64
visibility_decoder_grad_scale
1.0
camera_approximate_fov_degrees
60
checkpoint_selection_direction
min
cross_task_fusion_learned_gate
true
geometry_use_deterministic_vae
true
native_crop_lower_region_ratio
0.05
depth_multires_downscale_factor
2
lambda_depth_gt_edge_smoothness
0
validation_geometry_camera_mode
predicted
validation_geometry_output_mode
geometry_head
camera_include_input_in_encoding
null
camera_intrinsics_metadata_scope
intrinsics
cross_task_fusion_gate_init_bias
0
cross_task_fusion_sender_lowpass
false
depth_head_use_depthwise_context
true
geometric_pairs_views_per_sample
2
geometry_condition_on_visibility
false
geometry_detach_encoder_features
true
cross_task_fusion_hidden_channels
64
depth_diagnostics_normalize_depth
true
native_crop_centered_region_ratio
0.0
native_crop_lower_region_position
center_bottom
rgb_depth_consistency_edge_weight
4
rgb_depth_consistency_weight_mode
hallucinated
geometric_pairs_emit_pair_metadata
true
geometry_include_input_in_encoding
false
lambda_depth_multiscale_smoothness
0.001
depth_ensemble_regularizer_strength
0.02
native_crop_centered_region_position
center
cross_task_fusion_attention_kv_tokens
1024
cross_task_fusion_attention_max_block
-1
depth_diagnostics_road_edge_threshold
0.08
geometry_head_detach_camera_embedding
true
depth_diagnostics_road_bottom_fraction
0.4
geometry_encoder_no_grad_when_detached
true
cross_task_fusion_sender_lowpass_kernel
3
depth_diagnostics_high_frequency_kernel
5
validation_geometry_compute_edge_metrics
true
validation_geometry_compute_point_metrics
false
validation_geometry_max_points_per_sample
20000
validation_geometry_save_geometry_samples
true
validation_geometry_compute_camera_metrics
true
validation_geometry_compute_resolution_sweep
false
validation_geometry_max_point_metric_samples
8
validation_geometry_save_point_cloud_samples
false
validation_geometry_compute_uncertainty_metrics
true
Raw JSON
{
  "clip": "true",
  "crop": "center",
  "seed": 42,
  "tracker": "wandb",
  "use_ema": "false",
  "gpu_type": "rtx_4090:1",
  "job_name": "sd_train_concat",
  "norm_max": 1,
  "norm_min": -1,
  "run_name": "joint-opt-l2-softinv-tau30-sky0...

Events

Launch Events

0
No launch events recorded.
Euler View - ML Experiment Monitor