# Placement Experiments and Holdouts

Built-in A/B tests and standing holdouts per placement so every layout, experience, or parameter change is read as lift, not as a trend.

Every placement keeps a standing holdout, a matched group of viewers who don't see it. Experiments split viewers between versions of one change. Genuin reads the difference as lift, so you see whether a change helped before revenue moves.

[Octo](https://begenuin.com/developers/get-started/any-business/octo-operates-your-brand-control-center-octo-proposes-you-approve), Genuin's Supervised Intelligence interface in Brand Control Center (BCC), proposes this capability when it finds a fit on your sites and apps. It completes the steps under Set it up from your brand context. Nothing goes live until you approve it. It's part of [Onsite](https://begenuin.com/developers/browse-by-products/onsite/onsite-what-it-does-and-where-to-start), and [Measure engagement](https://begenuin.com/developers/get-started/any-business/measure-engagement-watch-time-lift-and-cohorts) explains how lift is counted against a holdout.

## When to use it

Run an experiment before you change a placement's position, experience, or feed settings. Keep the holdout on permanently, for lift reporting.

| Variable | What you can compare |
| --- | --- |
| Position | Places on the page, such as after paragraph three or at the end of an article |
| Experience | Shorts, Shoppable, Community, or Live & Q&A |
| Layout | Layouts such as feed, carousel, or inline |
| Feed parameter | Exploration rate, freshness weight, repeat window, pinned slots, cohort use, or paid pacing |

## How it works

- The holdout takes a share of each placement's viewers, 5 percent by default.
- Lift is the difference between viewers who saw the placement and the matched holdout. It's measured in watch time, pages per session, or return rate.
- Each experiment tests one variable at the split you choose, with a set duration and a minimum sample.
- Results show lift with confidence intervals, so a change reads as lift, not as a trend.
- The same holdout isolates [Maya](https://begenuin.com/developers/browse-by-products/supervised-intelligence/supervised-intelligence-what-it-does-and-where-to-start#maya-genuin-s-orchestrator)'s contribution, so the [Relevance dashboard](https://begenuin.com/developers/get-started/any-business/reading-the-relevance-dashboard) reads lift directly rather than inferring it.
- Holdouts also support incrementality, the holdout-based model in [Attribution](https://begenuin.com/developers/get-started/any-business/attribution-closed-loop-commerce-and-conversion-forecasts).

## Set it up

Octo proposes the experiment with these steps already done. You review the result and confirm it.

1. Open the placement under **Network** > **Grid** and choose **Experiment**. Learn how the grid works in [The Hierarchical Grid](https://begenuin.com/developers/get-started/any-business/the-hierarchical-grid-one-spine-for-inventory-layout-and-reporting).
2. Pick the variable and the split. The variable is a position, an experience, a layout, or a feed parameter. Learn about feed parameters in [Personalization and relevance](https://begenuin.com/developers/get-started/any-business/personalization-and-relevance-how-maya-ranks-every-feed).
3. Read the results in the Relevance dashboard, and promote the winner with one click. Learn how in [Reading the Relevance dashboard](https://begenuin.com/developers/get-started/any-business/reading-the-relevance-dashboard).

## Settings

| Setting | Default |
| --- | --- |
| Holdout share | 5 percent per placement |
| Experiment duration | Set for each experiment |
| Minimum sample | Set for each experiment |

Set an experiment's duration and minimum sample when you create it under **Network** > **Grid**. The holdout share is also part of the [Relevance Dashboard](https://begenuin.com/developers/browse-by-products/supervised-intelligence/relevance-dashboard) configuration.

## What it reports

The Relevance dashboard shows each experiment's lift, with confidence intervals, on these measures. [Measure engagement](https://begenuin.com/developers/get-started/any-business/measure-engagement-watch-time-lift-and-cohorts) defines each one.

- **Watch time**: watch time per user per minute, the engagement metric Maya is measured on.
- **Completion**: the share of plays that reach 90 percent.
- **Follows**: follows per thousand views.
- **Revenue**: what the placement earned.

## Build on it

The Web SDK renders the placement you test. [Add a Genuin placement to your website](https://begenuin.com/developers/developer-resources/sdk/how-do-you-configure-the-genuin-web-sdk-placement-on-your-website) covers the placement and style IDs, and [Embed the Genuin Web SDK on your website](https://begenuin.com/developers/developer-resources/sdk/how-do-you-embed-and-configure-the-genuin-web-sdk-on-your-website) covers loading the script. [Customize the Genuin video player](https://begenuin.com/developers/browse-by-products/control/how-does-a-brand-customize-the-genuin-video-player-controls) sets sizes, tile layouts, and the profile menu.

[Receive engagement events in your own analytics](https://begenuin.com/developers/developer-resources/sdk/receive-engagement-events-in-your-own-analytics) sends what viewers watch and do to your own analytics tool and warehouse.

The experiment and report calls in the [Business Account API](https://begenuin.com/developers/get-started/any-business/business-account-api-provisioning-oauth-and-the-resource-map) are proposed, not live. [Live API today and the Business Account API roadmap](https://begenuin.com/developers/developer-resources/guides/live-api-today-and-the-business-account-api-roadmap) maps what you can call now to what's planned.