Shop the look

POST /v2/agent/match takes a photo of someone dressed and returns your products in it. The shop-the-look models read the photo once and find every item the person wears: garments, shoes, bags, jewellery, eyewear and hats. They match each one against your store's products and return the best product with the other high-scoring ones behind it.

A call takes about a second. No language model runs: the models are trained for this job and run on our servers.

Shop the look is experimental. Its request and response fields may change, and its matches are still improving.

What it is for

• Shop the look. A shopper uploads a street-style photo or a screenshot and gets your products back, piece by piece.
• Complete the look. On a product page, send the product's own model photo with its itemId and you get everything else the model is wearing, without the product itself. The complete-the-look strip does exactly this.
• Catalogue coverage. Run reference outfits through it to see how much of a look your range can reproduce.

Sending the photo

Send it inline as a data: URI (PNG, JPEG or WebP) or as imageUrl, a public URL we fetch. About 1024px on the long side is plenty, and an inline photo over 8,000,000 characters is refused with 413.

async function toDataUrl(file, maxSide = 1024) {
  const bitmap = await createImageBitmap(file);
  const scale = Math.min(1, maxSide / Math.max(bitmap.width, bitmap.height));
  const canvas = document.createElement('canvas');
  canvas.width = Math.round(bitmap.width * scale);
  canvas.height = Math.round(bitmap.height * scale);
  canvas.getContext('2d').drawImage(bitmap, 0, 0, canvas.width, canvas.height);
  return canvas.toDataURL('image/jpeg', 0.85);
}

const API = 'https://stylor.ai/api';

async function shopTheLook(image, { store, country } = {}) {
  const res = await fetch(`${API}/v2/agent/match`, {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${PUBLIC_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ image, store, country }),
  });
  const body = await res.json();
  if (!res.ok) throw new Error(body.error || `HTTP ${res.status}`);
  return body; // { garments, indexed }
}

imageUrl must be a public http(s) URL on a domain name. IP addresses and internal hosts are refused, and redirects are not followed.

Which store is searched

One store's products are searched:

• itemId: the photo is that product's own, such as a product page's model shot. The product's store is searched, and the photo is read together with the product's title. The product itself is never returned, and neither are its other colourways or the other pieces of a set it belongs to.
• store: a storefront URL or domain. The search is narrowed to the datasets of that store.
• Neither: datasetIds is searched, or every active dataset when it is omitted.

A store's products are searchable once they are indexed, which happens after each sync. indexed: false means none of them is indexed yet.

Reading the result

{
  "indexed": true,
  "garments": [
    {
      "slot": "jacket",
      "family": "outerwear/jacket",
      "colour": "black",
      "box": [0.18, 0.21, 0.83, 0.58],
      "match": "exact",
      "item": {
        "id": "8f2c1a9e-4b7d",
        "itemId": "8f2c1a9e-4b7d-4c1e-9a3f-2d6b7e5c1a90",
        "name": "Moto Leather Jacket",
        "brand": "Acme",
        "price": 340,
        "currency": "USD",
        "url": "https://shop.example.com/products/moto-leather-jacket",
        "image": "https://stylor.ai/api/v1/image/org_5f3a/ds_9c21/8f2c1a9e-4b7d/0.jpg/-/resize/600x/-/format/jpeg",
        "slot": "outerwear",
        "family": "jacket",
        "colour": "black"
      },
      "alternates": [
        { "itemId": "1b7e33c0-9a2d-4f8e-b5c6-7d8e9f0a1b2c", "name": "Biker Jacket", "price": 290, "currency": "USD" }
      ]
    }
  ]
}

• One entry per item, in the order the model is most sure of them. An item the store sells nothing like is left out rather than matched to something wrong.
• item is the best product. alternates are the next highest-scoring ones, best first, with a product's colourways shown once. Ask for 0 to 4 with alternates (the default is 4).
• match says how the top product relates to the photo. exact means the model is sure it is the same product: on stores it never trained on, 98.6% of its exact calls are right. similar means it is the closest the store sells, a look-alike, or the exact product when the model is not sure enough to say so.
• box is where the item is in the photo, as [x0, y0, x1, y1] fractions of its width and height. Use it to draw a hotspot over the item.
• slot is a word to show a shopper ("jacket", "sneaker"). family is what the model read, in the product taxonomy.

item.image is a 600px JPEG URL ready to use as an image source.

Prices

Send country (on a Shopify storefront, Shopify.country) and every card is priced in that shopper's market. A product that market does not sell is replaced by its next alternate that it does. See Prices in the shopper's country.

After the match

The itemId values are full catalogue ids. A matched outfit can go straight to POST /v2/agent/looks to be rendered as one image, or into a /v2/agent/chat conversation as context.

Errors and retries

Validation failures come back as JSON with a 4xx status. The reference lists the exact messages. The common ones are:

Each call uses one unit of api.v2.agent.match quota. On 429, honour Retry-After.