Aggregating LLM-Based Weak Verifiers
for Spatial Layout Generation

In submission

Aggregating LLM-Based Weak Verifiers for Spatial Layout Generation teaser figure
Figure 1. We introduce a system to verify the outputs of spatial layout generators against specific task descriptions. Our approach builds a collection of LLM-based weak verifiers using a layout verification DSL, and aggregates them into a single strong verifier. We apply our verification pipeline to several 3D and 2D layout domains. The resulting task-specific strong verifiers mark various layouts as positive (green check) if they match the task description and negative (red X) otherwise. For each negative example we include a brief note on what violates the task description. We show how these verifiers can be used automatically in conjunction with a layout generator to produce layouts that match the task descriptions (see Verifier-Guided Layout Generation below).

Abstract

LLM-based layout generation has shown promise in producing high-quality spatial layouts at scale from natural language descriptions. However, directly prompting LLMs to produce layouts, even with structured primitives, often results in layouts that do not match a user's criteria. While human-in-the-loop verification can help, it is often expensive and time-consuming. We also find that the status quo LLM-as-a-judge approach is unreliable in spatial domains, particularly in 3D. We present a system that overcomes these limitations and produces strong verifiers for spatial layout generation tasks by aggregating LLM-generated programmatic weak verifiers. We find that the strong verifiers produced by our pipeline outperform the status-quo approach of using a set of LLM judges to directly check whether a layout matches a task description, raising F1-scores by up to 6.5x across a variety of layout design tasks over three different domains. We also demonstrate that verifier-guided layout generation using natural language feedback from our strong verifiers improves layout quality of a base layout generator from 37.2% to 77.3% (a 40.1 point improvement) according to human evaluation.

Method

The four stages of our verification pipeline
The four stages of our verification pipeline. 1. Dataset generation. We first curate a dataset of training examples from a generator or an existing dataset. 2. Dev set labeling. The user then labels a small random subset (about 10) of these examples as positive or negative, possibly providing notes alongside each label, e.g. “the piano is not in the corner.” 3. Weak verifier generation. We provide an LLM with the user prompt, the labeled dev set examples, and documentation of our DSL and query it to generate a set of weak verifiers. 4. Weak verifier aggregation. We run all the weak verifiers on our entire training dataset and aggregate them into a single strong verifier.

Our verification pipeline takes a user's task description of the desired layout as input and outputs a strong verifier that evaluates whether a generated layout matches the prompt. Rather than directly using an LLM as a judge, our approach is to use the LLM to generate many task-specific verifier programs. To help the LLM write programs with complex spatial logic, we design a verification domain-specific language (DSL) that defines low-level spatial layout queries. To make weak learning work in our setting, we encourage our LLM-based weak verifier generation to produce diverse verifier programs, and then filter for a subset of higher-quality weak programmatic verifiers before aggregating their responses.

Verification Results

Our experiments over three different layout domains (3D rooms, 2D posters and 3D objects) find support for three main claims: (1) Using our programmatic weak verifiers in combination with any of the aggregation techniques outperforms directly using LLMs as judges by up to 6.5x in F1-score; (2) Weaver aggregation generally outperforms the other aggregation methods with as few as 10 labeled layout examples; (3) Using our strong verifiers in verifier-guided layout generation with Detailed feedback improves a base layout generator from 37.2% to 77.3% (a 40.1 point improvement) according to human evaluation.

3D Rooms (F1-Scores ↑)
Verifier TypeWeaverNMLRTop-1
LLM Vision Judges0.270.350.120.34
LLM Judges0.290.490.110.43
Ours0.800.680.720.77
2D Posters (F1-Scores ↑)
Verifier TypeWeaverNMLRTop-1
LLM Vision Judges0.460.540.350.61
LLM Judges0.210.180.230.23
Ours0.790.620.690.74
3D Objects (F1-Scores ↑)
Verifier TypeWeaverNMLRTop-1
LLM Vision Judges0.400.580.490.56
LLM Judges0.360.540.340.44
Ours0.710.650.640.69
Comparison against aggregating LLM judges. Across all domains, our programmatic weak verifiers outperform using LLM Judges as weak verifiers, regardless of the aggregation method being used.

Verifier-Guided Layout Generation

In addition to verification, we also use our strong verifiers for verifier-guided layout generation—a process that automates the iterative generate-then-verify loop. To explore how the information within the verifier programs can help this process, we investigate two different forms of verifier feedback in this loop: Binary feedback, which simply uses the True/False verification decisions; and Detailed feedback, which uses natural language messages explaining the decisions. Since our feedback messages come from many different weak verifiers of varying quality and logic, we contextualize the individual feedback messages with a reliability score defined using the learned aggregation weights.

Human Evaluator Pass Rate (↑)
Feedback Type3D Rooms2D Posters3D ObjectsAvg.
Base (No verifier)26.2%41.1%44.3%37.2%
LLM+Vision49.2%70.8%40.8%53.6%
Binary (Ours)70.5%63.1%53.6%62.4%
Detailed (Ours)76.9%83.7%71.2%77.3%
Number of Iterations (↓)
Feedback Type3D Rooms2D Posters3D ObjectsAvg.
Base (No verifier)--------
LLM+Vision3.011.951.902.29
Binary (Ours)3.485.774.504.59
Detailed (Ours)2.732.491.942.39
We compare feedback from an LLM+Vision judge against Binary feedback and Detailed feedback from our Weaver verifier. For each domain, we report the human evaluator pass rate (left) and the number of iterations required to generate a layout (right).

Bibtex

@article{zhang2026weakverifiers,
    author = {Zhang, Sharon and Jones, R. Kenny and Wu, Jiajun and Agrawala, Maneesh},
    title = {Aggregating LLM-Based Weak Verifiers for Spatial Layout Generation},
    journal = {arXiv preprint arXiv:2606.05268},
    year = {2026}
}