Train: MultiSet + Muses Val V2 + Sky-SDF 16ch + C-RADIOv4 ece594b5
Copy of "Train: MultiSet + Muses Val V2 + Sky-SDF 16ch" (config 111, v5) with the frozen C-RADIOv4-H feature-conditioning pathway enabled, and nothing else changed. WHAT IS NEW. model.feature_conditioning injects one frozen, spatially aligned C-RADIOv4-H feature map into the SHARED encoder, by adding a separate zero-initialised conv_f to conv_in's output. A convolution over a channel concatenation is the sum of convolutions over the parts, so this is algebraically the concatenation -- while leaving conv_in's 16-channel width and every pre-feature checkpoint untouched, and being exactly identity at step 0. Both task decoders and the sky decoder see it through the trunk. The intent is a fifth cross-task coupling channel: all four existing channels couple the two predictions to EACH OTHER, which two branches can satisfy by hallucinating the same wrong scene; a shared external anchor makes agreement have to be agreement with the anchor. See docs/feature_conditioning.md and docs/plans/foundation_feature_conditioning.md. DELTAS FROM CONFIG 111. (1) model.feature_conditioning.* added, enabled=true, backbone=c-radio_v4-h, channels=32, autocast_dtype=bfloat16. (2) training.feature_adapter_learning_rate=6e-5 -- REQUIRED when the pathway is on; without it the zero-init conv_f rides in the backbone group and train_joint.py fails fast. (3) training.weight_decay_groups.feature_adapter=0. (4) run.sh exports TORCH_HOME and aborts at startup if the torch.hub cache is cold, because Euler compute nodes cannot download the backbone. (5) segmentation_detach_encoder_features and use_deterministic_vae were unset in config 111 and had to be typed at every launch; pinned here to the values run 2026-09-11_16-19-37_e3fc actually used (both true). UNMEASURED. channels=32 and feature_adapter_learning_rate=6e-5 are reasoned defaults, not optima. No GPU has ever executed this pathway; throughput and peak memory are unmeasured, and one ViT-H/16 forward per image per step at batch_size 24 is a real cost. gpu_type still reads rtx_4090:1 from the source config -- set the A100 at launch. WATCH IN THE FIRST HOUR. rgb.train.stat.feature_contribution_ratio must rise off exactly 0 (conv_f starts at zero, so a flat 0 means the pathway is dead, not that it does not help) and rgb.train.stat.feature_map_std must not collapse toward a constant map. A paired control -- this config with feature_conditioning.enabled=false -- is what makes the comparison attributable.
Output
Failure Metadata
failedContext JSON
{
"reason": null,
"exitCode": 255,
"remotePid": 3486838,
"computePid": null,
"allocationJobId": 16315047
}Datasets
17clear_root_rds/cluster/work/igp_psr/drothenpiele/data/rds/rgb.zip/depth_root_rds/cluster/work/igp_psr/drothenpiele/data/rds/depth.zip/intrinsics_root_rds/cluster/work/igp_psr/drothenpiele/data/rds/calibration.zip/sky_mask_root_rds/cluster/work/igp_psr/drothenpiele/data/rds/semantic_segmentation_2d.zip/train_hazy_root_rds/cluster/work/igp_psr/drothenpiele/data/rds/gloomy_finalval_hazy_root_rds/cluster/work/igp_psr/drothenpiele/data/rds/gloomy_fixed/foggy_rgb.zip/clear_root_vkitti2/cluster/work/igp_psr/drothenpiele/data/vkitti_2.0.3_rgb.zipdepth_root_vkitti2/cluster/work/igp_psr/drothenpiele/data/vkitti_2.0.3_depth.zipintrinsics_root_vkitti2/cluster/work/igp_psr/drothenpiele/data/vkitti_2.0.3_textgt.zip/sky_mask_root_vkitti2/cluster/work/igp_psr/drothenpiele/data/vkitti_2.0.3_classSegmentation.zip/train_hazy_vkitti/cluster/work/igp_psr/drothenpiele/data/vkitti_next/gpu_2/foggy_rgb.zip/val_hazy_vkitti/cluster/work/igp_psr/drothenpiele/data/vkitti_next/gpu_2/foggy_rgb.zip/muses_rgb/cluster/work/igp_psr/drothenpiele/data/muses/frame_camera_trainvaltest.zip/muses_sparse_depth/cluster/work/igp_psr/drothenpiele/data/muses/lidar_trainvaltest.zip/muses_intrinsics/cluster/work/igp_psr/drothenpiele/data/muses/frame_camera_trainvaltest.zip/muses_extrinsics/cluster/work/igp_psr/drothenpiele/data/muses/frame_camera_trainvaltest.zip/muses_sky_mask/cluster/work/igp_psr/drothenpiele/data/muses/gt_semantic_trainval.zip/Execution Artifacts
2Published Exports
Launch-Owned Exports
These semantic handles are what downstream pipelines should resolve against. Auto-published exports come from config-template metadata captured on this launch.
This launch does not publish any exports yet.
Parameters
306Typed Parameters
clear_root_rds(4)depth_root_rds(5)intrinsics_root_rds(6)sky_mask_root_rds(7)train_hazy_root_rds(277)val_hazy_root_rds(196)clear_root_vkitti2(1)depth_root_vkitti2(2)intrinsics_root_vkitti2(12)sky_mask_root_vkitti2(11)train_hazy_vkitti(216)val_hazy_vkitti(238)muses_rgb(77)muses_sparse_depth(76)muses_intrinsics(82)muses_extrinsics(81)muses_sky_mask(75)Simple Parameters
cliptruecropcenterseed42trackerwandbuse_emafalsegpu_typertx_4090:1job_namesd21_cradionorm_max1norm_min-1run_namemultiset-muses-v2-sky-sdf-cradiotmp_size10Gadam_8bittruedata_kindreal_drive_simema_decay0.9999ema_dtypefloat32log_every10max_depth800min_depth1adam_beta10.9adam_beta20.999batch_size24clear_root4depth_modemetric_logdepth_root5image_size[384, 768]joint_modesingle_passlog_imagestruenum_epochs200output_dir/cluster/scratch/drothenpiele/SD21/multiset_muses_v2_sky_sdf_cradiotime_limit2-00:00:00torch_home/cluster/work/igp_psr/drothenpiele/cache/torchmem_per_cpu8Gnum_workers4adam_epsilon1e-8adam_foreachfalseaspect_ratio[1, 2]camera_modelpinholeconditioningconcatlambda_depth1lambda_point0.0min_lr_ratio0.01project_namejoint-dehazinguse_geometryfalsewarmup_ratio0.1weight_decay0cpus_per_task8lambda_camera0.0lambda_dehaze1lambda_eg_ssi0.0learning_rate0.00004max_depth_rds800max_grad_norm1min_depth_rds0.00001val_hazy_root104camera_enabledtruefreeze_encoderfalselog_visibilitytruenum_log_images4use_depth_headtrueval_batch_size10camera_fallbackinvaliddepth_head_typebase_residualenable_xformerstrueeuler_train_dir/cluster/work/igp_psr/drothenpiele/data/out/train/sd21-jointlambda_depth_tv0.0002mixed_precisionfp16prediction_typev_predictionsource_kind_rdsreal_drive_simtrain_hazy_root103visibility_cmapviridiscamera_embeddingsinecamera_emit_raysfalsecamera_mlp_ratio4camera_num_heads4camera_token_dim256depth_noise_typeannealed_multiresfeature_backbonec-radio_v4-hfeature_channels32lambda_log_depth0.0lr_schedule_typeconstant_with_warmupmin_max_quantile0.02no_decay_enabledfalsepretrained_modelsd2-community/stable-diffusion-2-1sky_depth_meters800camera_num_layers2camera_num_tokens4conv_in_init_modejoint_marigoldgeometry_lambda_z0.15geometry_timestep0joint_weight_psnr1joint_weight_ssim10lambda_depth_head0.2lambda_lambda_mse0.0lambda_visibility0.05max_depth_vkitti2300min_depth_vkitti20.00001sampling_strategysource_balancedsource_weight_rds1depth_ensemble_tol0.001lambda_uncertainty0.0model_depth_outputvae_decodesky_labels_enabledtrueval_every_n_epochs1camera_feature_poolavgdepth_ensemble_size1geometry_lambda_phi1lambda_segmentation1num_inference_steps25sampling_epoch_sizeallsave_every_n_epochs2source_kind_vkitti2vkitti2use_visibility_headtruewarmup_start_factor0.001camera_metadata_rootnullconv_in_target_scale0.5773502691896258external_camera_modenonegeometry_loss_robustcharbonnierlambda_vae_depth_rec0.0lambda_visibility_tv0.005segmentation_enabledtruesky_depth_meters_rds800camera_branch_enabledtruecamera_embedding_sizelatentdepth_gradient_robustcharbonnierdepth_multires_levels4encoder_learning_rate0.00001geometry_head_enabledfalsegeometry_lambda_theta1lambda_depth_gradient0.05lambda_geometry_total0.0source_weight_vkitti21use_cross_task_fusiontrueuse_deterministic_vaetruevae_decoder_trainablefalsevisibility_rank_pairs128weight_decay_backbone0weight_decay_no_decay0zero_grad_set_to_nonetruecamera_feature_sourcesautocamera_metadata_formateuler_loadingdepth_ensemble_max_res1024depth_smoothness_spacedepth_headfeature_autocast_dtypebfloat16freeze_cross_attentiontruegradient_checkpointingtrueinference_depth_outputvae_decodelambda_geo_consistency0.0lambda_visibility_rank0use_task_skip_adaptersfalsevisibility_rank_margin0.05camera_embedding_stride8camera_prediction_spaceresidual_intrinsicsdepth_ensemble_max_iter2depth_multires_strength0.9depth_smoothness_scales4geometric_pairs_enabledfalsejoint_weight_delta1_pct0.5keep_last_n_checkpoints5lambda_segmentation_bce0.5lambda_segmentation_sdf1segmentation_latent_std[1,1,1,1]segmentation_noise_typegaussiansegmentation_sdf_robustcharbonniervalidation_depth_outputbothvisibility_target_gamma4visibility_warmup_steps1000cross_task_fusion_blocks[1,2,3]cross_task_fusion_detachfalsedepth_ensemble_reductionmediandepth_normalization_typelog_depthgeometry_camera_encodingsinegeometry_head_num_blocks2joint_weight_abs_rel_pct0.5lambda_segmentation_dice0.5segmentation_depth_cap_mnullsegmentation_latent_mean[0,0,0,0]sky_depth_meters_vkitti2800camera_intrinsics_loadinghierarchicaldepth_diagnostics_enabledtruedepth_head_residual_scale0.05geometry_depth_conventionz_depthgeometry_metric_depth_max800geometry_metric_depth_min1num_inference_steps_depth20segmentation_latent_spacepost_vae_scaling_factorsky_semantic_label_weight1vae_decoder_learning_rate0.000001visibility_apply_to_depthfalsecamera_residual_logit_clip4depth_head_hidden_channels64depth_resize_interpolationbilineardepth_smoothness_normalizetruedetach_rays_for_point_losstrueenable_efficient_attentiontruelambda_visibility_preserve0.05sky_heuristic_label_weight0.5sky_labels_allow_heuristicfalsevisibility_apply_to_dehazetruevisibility_apply_to_fusionfalsevisibility_hidden_channels32visibility_target_quantile0.9weight_decay_depth_decoder0weight_decay_geometry_head0camera_require_for_trainingtruecamera_sine_num_frequenciesnullcheckpoint_selection_metricvalset/muses/maedepth_decoder_learning_rate0.00003emit_camera_full_resolutiontruegeometry_eg_ssi_edge_sourcehazygeometry_encoder_input_modehazy_hazygeometry_head_learning_rate0.00004geometry_output_uncertaintytruegeometry_ray_representationunitgradient_accumulation_steps1segmentation_boundary_scale0.125validation_geometry_enabledfalseweight_decay_dehaze_decoder0camera_output_representationanglescamera_valid_for_approximatefalsecross_task_fusion_directions["rgb_to_depth","depth_to_rgb"]dehaze_decoder_learning_rate0.00003depth_head_context_dilations[1, 2, 4]depth_smoothness_edge_sourcecleardepth_smoothness_edge_weight10feature_conditioning_enabledtruegeometry_head_residual_scale0.05lambda_depth_edge_smoothness0.002segmentation_boundary_weight4weight_decay_feature_adapter0weight_decay_geometry_camera0depth_diagnostics_edge_sourcedehazeddepth_diagnostics_edge_weight10depth_head_base_upsample_modebilinearfeature_adapter_learning_rate0.00006geometry_camera_learning_rate0.00004geometry_head_hidden_channels128geometry_sine_num_frequencies64visibility_decoder_grad_scale1.0camera_approximate_fov_degrees60checkpoint_selection_directionmincross_task_fusion_learned_gatetruegeometry_use_deterministic_vaetruesegmentation_codec_encode_modedeterministicdepth_multires_downscale_factor2lambda_depth_gt_edge_smoothness0segmentation_direct_start_epoch3validation_geometry_camera_modepredictedvalidation_geometry_output_modegeometry_headcamera_include_input_in_encodingnullcamera_intrinsics_metadata_scopeintrinsicscross_task_fusion_gate_init_bias-4cross_task_fusion_sender_lowpassfalsedepth_head_use_depthwise_contexttruegeometric_pairs_views_per_sample2geometry_condition_on_visibilityfalsegeometry_detach_encoder_featurestruesegmentation_depth_output_policyrawsegmentation_inference_threshold0segmentation_sdf_truncation_typeshort_side_fractioncross_task_fusion_hidden_channels64depth_diagnostics_normalize_depthtruesegmentation_head_hidden_channels64segmentation_sdf_truncation_value0.0625segmentation_validation_threshold0geometric_pairs_emit_pair_metadatatruegeometry_include_input_in_encodingfalselambda_depth_multiscale_smoothness0.001depth_ensemble_regularizer_strength0.02segmentation_direct_timestep_weightalpha2segmentation_head_context_dilations[1,2,4]segmentation_latent_standardizationfalsesegmentation_detach_encoder_featurestruesegmentation_probability_temperature0.1depth_diagnostics_road_edge_threshold0.08geometry_head_detach_camera_embeddingtruedepth_diagnostics_road_bottom_fraction0.4geometry_encoder_no_grad_when_detachedtruecross_task_fusion_sender_lowpass_kernel3depth_diagnostics_high_frequency_kernel5segmentation_head_use_depthwise_contexttruevalidation_geometry_compute_edge_metricstruevalidation_geometry_compute_point_metricsfalsevalidation_geometry_max_points_per_sample20000validation_geometry_save_geometry_samplestruevalidation_geometry_compute_camera_metricstruevalidation_geometry_compute_resolution_sweepfalsevalidation_geometry_max_point_metric_samples8validation_geometry_save_point_cloud_samplesfalsevalidation_geometry_compute_uncertainty_metricstrueRaw JSON
{
"clip": "true",
"crop": "center",
"seed": 42,
"tracker": "wandb",
"use_ema": "false",
"gpu_type": "rtx_4090:1",
"job_name": "sd21_cradio",
"norm_max": 1,
"norm_min": -1,
"run_name": "multiset-muses-v2-sky-sdf-cradio",
...