Airbnb Details CASTLE, a Way to Train Natural Language Search Before Any Real Queries Exist

Airbnb has published the method behind its natural language search, which matters to hosts because it shapes how a listing gets matched to a guest who types something like "romantic cabin near skiing."

The framework is called CASTLE (Contrastive And Seed-guided Training for natural Language sEarch). It was built to solve a cold-start problem: a search model needs training examples, but before launch there are no real natural language queries to learn from. The paper behind it was accepted at CIKM 2026.

How it works

CASTLE creates both the queries and the relevance labels from de-identified booking data.

• Booked versus viewed. When a guest books a listing after looking at alternatives, the booked listing counts as the preferred one. Each booked listing is paired with a randomly chosen listing the guest viewed but did not book.
• Inputs. The model sees amenities, property details and a one-sentence summary of recent reviews, with personal information removed.
• Labels by construction. An LLM writes a query that fits the booked listing but not the other one. Because the preference comes from real booking behavior, the labels need no later judgment by a model.
• Three prompt styles. Template-based prompts add structural variety. Few-shot prompts use about 500 seed queries from user research. Attribute-based prompts focus on distinguishing features such as a fireplace or distance to a ski resort. Together they counter "query collapse," where an unguided LLM keeps repeating the same short phrases.
• Virtual Judge. An LLM evaluator labels extra query and listing pairs for listings that have no booking history.
• CASTLE-R. A variant seeded with real queries, for production settings where some traffic already exists.

What Airbnb measured

Airbnb compared the generated queries with real ones using KL divergence, where a lower number means a closer match.

• CASTLE scored 1.01 overall. The best prior method (Promptagator) scored 9.33, and prior approaches ranged up to 26.2. That is a 9.2 times improvement.
• On attribute types, CASTLE scored 0.08, slightly better than the hand-written survey seed queries at 0.09.
• The spread in query length was 2.48 words, close to real users at 2.49. Baseline methods (InPars and Promptagator) managed 0.99.
• Three subject matter experts reviewed 200 sampled examples and agreed with CASTLE's labels 91 to 93 percent of the time.

Real guests typically search with short phrases of four to eight words, and CASTLE is built to match that rather than produce long, wordy queries. Because generation is tied to listing characteristics, it also over-samples rare but important intents, which Airbnb says keeps helping after real traffic arrives.

Why operators should care

The method leans on booking behavior and on listing content such as amenities and review themes. That suggests specific, accurate amenity data and well-described listings are what natural language search will match on. Airbnb lists multi-turn conversational search as future work, for example refining "cabin near skiing" to "one that allows dogs."

Source: Airbnb Tech Blog.