Core Concepts

The foundational ideas behind TasteBrain: a shared embedding space, lightweight per-user models, living data, and grounded language.

The embedding space

TasteBrain is a transformer-based, multimodal model derived from Qwen-Omni and trained at scale on ~20M curated product images across ~7M products. The model fine-tunes attention layers and uses a custom projector to map per-token hidden states into two representations:

This shared embedding space represents the global geometry of taste. Styles, brands, materials, form factors, and aesthetic signals form coherent regions. Proximity in this space means aesthetic similarity — regardless of whether the inputs are images, text, audio, or products.

Cross-modal queries

Because every modality is projected into the same 128-dimensional space, you can combine inputs freely:

• Image → products — "find me products that look like this"
• Text → products — "minimalist Scandinavian desk lamp"
• Image + text → products — "like this room, but in warm earth tones"
• Product → products — "more like this" from a product handle
• Audio → products — map sonic aesthetics to visual products

Multiple query inputs are combined in embedding space before retrieval, allowing nuanced multi-signal searches that no single modality could express alone.

Per-user taste models

On top of the shared embedding space, TasteBrain learns lightweight per-user models that identify each individual's personal taste manifold. These models capture:

• Attraction — regions of the embedding space the user gravitates toward
• Aversion — regions the user consistently rejects
• Drift — how the user's taste evolves over time

Crucially, per-user models do not retrain the backbone. They operate as learned offsets on top of the shared geometry, which means:

In the API, personalization is activated simply by passing a user_id to the Prism search endpoints. Without a user_id, results come from the shared space alone (equivalent to using the Shopkeep API).

Crawling infrastructure

The embedding space is only as useful as the data it covers. Bestomer operates proprietary crawling infrastructure with two key properties:

Weekly revisits

Every indexed retail domain is revisited weekly. This tracks catalog changes — new arrivals, discontinued items, price updates, seasonal rotations — so the product corpus stays current. Stale data degrades aesthetic relevance, so freshness is a first-class concern.

Cross-domain coverage

The system goes beyond retail products:

This breadth is what allows taste to be modeled as a living, evolving system rather than a static product catalog. When a new design trend emerges in architecture, TasteBrain can connect it to products, restaurants, and music that share the same aesthetic signal — before anyone writes a tag or a label.

Sentiments

Sentiments are user reactions to products that train the per-user taste model. Each sentiment is a data point that adjusts the user's personal manifold:

Sentiments are recorded in the Level 2: Persona Services tier and fed back into the per-user model in real time. They're the primary training signal for personalization.

Captures

Captures are media inputs — images, screenshots, social media posts — that express aesthetic preference without words. When a user uploads or saves an image, it's projected into the embedding space and becomes a query anchor.

Sources

• Device uploads — photos from camera roll (via iOS integration or web upload)
• Social sync — saved posts from Instagram, Pinterest, TikTok
• Product screenshots — images of specific products the user wants to find
• Scene captures — interiors, environments, fashion looks, food presentations

At Level 1, captures are used as stateless query inputs. At Level 2, they're persisted and clustered to build richer user profiles over time.

Windows

Windows are personalized collections of products organized around an aesthetic theme. They exist at the Level 2 tier and can be:

• Auto-generated — the system clusters a user's sentiments and captures to create themed collections
• Manually curated — users add captures and refine with explicit sentiments
• Scoped — filtered by brand, retailer, or recipient (for gift-giving)

Each window represents a coherent region of the user's taste manifold. As the user interacts, windows evolve — new products appear, less relevant ones fade, and entirely new windows can emerge when the system detects a shift in the user's aesthetic interests.

Tasteboards

Tasteboards are visual summaries of a user's aesthetic DNA. Unlike windows (which are product collections), tasteboards are computed visualizations of the user's position in the embedding space — showing the aesthetic axes they occupy.

• User-facing — help users understand and share their own taste profile
• Brand-facing — show brands which aesthetic regions resonate with specific user segments
• Collaborative — share taste profiles with friends, stylists, or gift-givers

Grounded language generation

The same multimodal backbone is being extended to generate natural-language explanations of why products match a query or a user's taste. Using VQA-style (Visual Question Answering) supervision, the system learns to produce grounded descriptions:

This unifies representation, personalization, and explanation in a single system — the model doesn't just find relevant products, it can articulate the aesthetic reasoning.

How it all connects

Next steps

• Level 0: The Model — architecture details for the Qwen-Omni backbone
• Prism API — start making personalized searches
• Shopkeep API — non-personalized search endpoints
• Level 2: Persona Services — Windows, Sentiments, Tasteboards, and B-Shop