HARMONY300

Leaderboard

Geometry is measured only where a ground-truth mesh exists — the 100 Front3D scenes — and is complete for every method. The all-scenes appearance table below is still filling in — several baselines have not finished running.

Geometry — the 100 Front3D scenes with ground-truth meshes, complete for every method. ★ marks the best value in each column.
Method	Scenes	CD ↓	F@0.1 ↑	F@0.01 ↑	F@0.001 ↑
SAM3D*	100	0.056	81.79	14.44	0.089
Gen3DSR	100	0.057	78.33	10.01	0.050
CAST	100	0.052	85.70	14.39	0.093
3D-ReGen	100	0.060	78.87	12.65	0.088
VIGA	100	0.055	83.90	12.67	0.074
GPT6-Astra	100	0.061	77.44	14.04	0.090
Ours (HARMONY)	100	0.049	89.82	19.47	0.130

Appearance — rendered from each scene's own camera and compared to the reference photograph. Lower is better throughout. ★ marks the best value among the methods scored on all 300 scenes; rows with partial coverage are not eligible until their runs complete.
Method	Scenes	N-CLIP ↓	PL ↓	LPIPS ↓
Gen3DSR	218	0.202	0.052	0.514
3D-ReGen	209	0.150	0.099	0.514
SAM3D	300	0.139	0.065	0.602
CAST	300	0.183	0.087	0.605
VIGA	296	0.184	0.073	0.668
GPT6-Astra	300	0.132	0.057	0.590
Ours (HARMONY)	300	0.109	0.061	0.563

! Several baselines have not finished running. Gen3DSR is evaluated on 218 of 300 scenes, 3D-ReGen on 209 and VIGA on 296, so their appearance averages still rest on different subsets and are not eligible for a ★. Gen3DSR currently posts the lowest PL (0.052) and 3D-ReGen the lowest LPIPS (0.514) overall, but each on only about 70% of the benchmark. These numbers are provisional and will change as those runs complete. The geometry table is directly comparable: every method is measured on the same 100 ground-truth scenes.

What's in the benchmark

Scenes are drawn from real interior photography, so the room-type mix is uneven by design — it follows what indoor photographs actually contain rather than a balanced synthetic split.

Room types

Living room144

Bedroom114

Dining room17

Office8

Bathroom8

Other8

Per-scene assets

scene_full.glbTextured scene, objects as separate meshes
render_blender.pngCycles render from the reference camera
camera_vggt.jsonRecovered pose, look-at and HFOV
input photoThe single monocular image

Difficulty tiers

Tiers grade the reconstruction problem, not the photograph's quality: how many objects occlude one another, how much of each is cut off by the frame, and how oblique the camera sits to the room's axes.

Easy

100

Frontal or near-frontal camera, objects largely unoccluded and fully in frame.

Medium

100

Oblique views, partial occlusion between pieces, some objects clipped at the frame edge.

Complicated

100

Dense clutter, heavy inter-object occlusion, strong perspective, foreground pieces cut off.

Submit to the leaderboard

Evaluated a method on HARMONY300? Send the numbers and they will be added here. Open a discussion on the dataset page with the details below — that keeps the submission, the reply and any follow-up questions in one public thread.

What to include

methodName, and a paper or code link
scenesHow many of the 300 you evaluated, and which split
geometryCD, F@0.1, F@0.01, F@0.001 on the Front3D 100
appearanceN-CLIP, PL, LPIPS from the input viewpoint
rendersOne render per scene, named by scene id

Ground rules

Render from each scene’s own camera in camera_vggt.json, so appearance is measured against the reference photograph from the viewpoint it was taken from.

State your scene count. A method evaluated on a subset is welcome — partial coverage is listed as such rather than excluded, because a score averaged over an easier subset is not comparable to one over all 300.

Submit your results Get the dataset

Citation

HARMONY300 accompanies HARMONY: Hierarchical Agentic Reasoning for MONocular Image-to-Scene Synthesis. If either is useful in your work, please cite the paper.

@article{sun2026harmony,
  title   = {HARMONY: Hierarchical Agentic Reasoning for Monocular Image-to-Scene Synthesis},
  author  = {Sun, Shufan and Wang, Chen and Gu, Jiatao and Liu, Lingjie},
  journal = {arXiv preprint arXiv:2609.26793},
  url     = {https://arxiv.org/abs/2609.26793},
  year    = {2026}
}

Paper: arXiv:2609.26793