Test up to eight complete subject-and-body versions on a slice of your audience, then send the winner to the rest
Write several complete versions of the email, send them to a test group, then send the best one to everyone else.
A subject-line A/B test changes only the subject. Full versions let you change the whole email: subject and body together. You write up to eight versions, a slice of your audience receives them split evenly, and the rest of the list is held back. When you promote a winner, the held recipients receive that version. The section lives on the A/B test step of the campaign composer, under the subject-line test, and it is described there as: send different subject lines and bodies to a slice of your audience, then send the winner to everyone else.
Versions attach to a saved draft and each one starts as a copy of the campaign's own subject and body.
Each version row has a Pause button (which reads Resume once paused) and a Remove button. A paused version keeps its copy but stops taking a share of the test group, and the word Paused appears beside its heading. Removing asks you to confirm first: Remove this version? warns that the version and its copy are removed from the campaign and that it cannot be undone, with Remove version and Keep version as the choices. Saving each version is a separate action, so editing one version never discards what you have typed into another.
Before you can start, the section explains anything standing in the way:
How much of the audience is tested, for how long, and which number decides.
| Field | Description |
|---|---|
| Test group size | The share of recipients that receives the test, split evenly across the active versions. Moves in steps of 5 percent, from 5 to 50. The line under it reads back the current value as a percentage of recipients. |
| Test length in hours | How long results are collected before the campaign decides on its own. Counted from when the test group has finished sending, between 1 and 168 hours. |
| Deciding metric | Which number picks the winner: Reply rate, Open rate, or Click rate. Reply rate is the default, because a full version is a pitch and replies say more about it than opens do. |
Under the settings, a live line shows the split as you change it: how many recipients are in the test group, how many that is per version, and how many are held for the winner. If the split leaves each version with a small audience, a quiet sentence appears beside it. Below about 250 recipients per version it says that 250 per version is where an early signal becomes usable and 1,000 is where a reply rate difference is worth acting on, and that differences below that are usually noise. Between 250 and 1,000 it says the sample is enough for an early signal but not yet worth acting on. If the test group is too small for the campaign to decide at all, a second line explains that a winner can only be decided once every version has at least 30 delivered sends, and gives the smallest test group your number of versions needs. None of these lines block anything: they are advice, and you can send anyway.
The per-version numbers and the Promote action live on the campaign detail page.
Open the campaign and look for Full version test. A chip beside the heading reads Running or Winner promoted, and a summary line underneath states the deciding metric, the test group percentage, the test length in hours, and the date the clock started from. If the campaign extended the window once, the line ends with a note that the window was extended once.
The table has one row per version, with these columns:
Under the table, a line states which version is ahead on the deciding metric and whether that lead is larger than what chance produces at the sample you have so far. A winner is decided once every version has at least 30 delivered sends, the test has gathered enough of the deciding signal in total, and one version is ahead by more than chance. The signal is counted in the deciding metric itself: replies for a reply-rate test, opens for an open-rate test, clicks for a click-rate test.
Promoting is a decision you make. If nobody promotes a winner before the test window closes, the held recipients are released automatically and you are told which version was chosen and why. If the automatic check is unavailable, the footnote says so and asks you to promote a version yourself instead of promising a release that will not happen.
Selecting Promote asks Promote this version? and explains that the version is sent to every recipient held back from the test, that the other versions stop sending, and that it cannot be undone. The buttons are Promote version and Keep testing. After a winner is promoted the composer stops accepting edits to the versions and says a winner was promoted and only the winning version sends now.
The campaign decides on its own once the test window closes, and tells you what it did.
Promoting by hand is the main path, but the held recipients are never left waiting forever. When the test length has passed, one of these notifications arrives:
These appear in the app. They are listed as Variant test decision in your notification settings, where you can turn them off, or turn email on for them. When email is on they arrive in a digest rather than as an immediate message.
A few details worth keeping in mind.