Help Center

Docs

A/Z Testing Steps

Give a cold-sequence step up to eight variants (A to H), let Laureo pick the winner on reply rate or positive replies, pause the losers, and read the exact rule it used

Up to eight variants of a step

Test subject lines or bodies against each other.

An A/Z test lets you give any step in a cold sequence more than one variant, up to eight (A through H). Laureo splits your leads evenly across the active variants and reports how each one is doing, so you can keep the variant that earns the most replies. Letters are stable identities: remove C and the rest stay A, B and D, and a letter is never reused once its variant has sent. It is an Ultra capability, built into the sequence builder.

Only step 1 has a subject to test
Every variant of step 1 gets its own subject field and its own body. Later steps are follow-ups on the same thread, so they carry no subject at all; there the test is the body alone. Each variant has the same plain and rich text choice a step has, and each one is set on its own.
Each step is tested on its own
A lead is assigned one variant the first time a step sends to them and keeps it for that step's whole thread. Different steps are tested separately, so the same lead can be on variant A at step 1 and variant B at step 2. Threading is not affected: follow-ups reuse the subject from step 1.

Add a test to a step

From the step in the builder.

1

Open the sequence and pick a step

In the builder, find the step you want to test. Below the step card, choose "Add A/Z test".
2

Variant A starts from your current copy

Variant A is seeded with the step’s existing body so you keep what you already wrote. Variant B arrives with an empty body, ready for your challenger. On step 1 both variants start on the step’s current subject, so a body-only test needs no subject edit at all.
3

Write the challenger

Edit Variant B. Each body carries the same two writing controls the step itself has, "Variables" and "Word variations", and the Variables menu offers exactly the same fields and booking pages, so a variant can never reach for a token the step could not. Every variant also gets the same unsubscribe footer and postal address as a normal step.
4

Saving looks after itself

The test saves on its own shortly after you stop typing. The card says "Unsaved changes" while an edit is pending and "Saved" once it lands, and "Save variants" flushes it immediately if you would rather not wait.
5

Add more variants if you want

Choose "Add variant" to append variant C, then D, up to eight in all. Each variant needs 250 sends before you can pick a winner, so the Add control states the cost as you go.

Removing one variant

A variant that has sent is paused, never erased.

Each variant has its own remove control. A variant that has never sent is removed for good. A variant that has already sent is paused instead, so its results stay in the report. Letters are never reused: remove variant C and the rest stay A, B and D. A test needs at least two variants, so the last two cannot be removed one at a time; "Remove A/Z test" takes the whole test away and erases its results.

A new variant does not send until it has a body. It sits paused, with a line on the card saying so, and the first save with a body starts it.

Watch the results

Reply rate, per variant, from exact counts.

Once the sequence is live, each variant shows exact counts: how many sends it has, how many replies it earned, and the reply rate, plus how many opened and how many clicked. Cold outreach is judged on replies, not opens, so reply rate is the number the builder puts front and centre. The counts are exact, not estimates, so the comparison is trustworthy.

Opens and clicks need a verified tracking domain
The open and click columns only fill in when you have verified a tracking domain and turned on open or click tracking for the sequence. Without that, and on plain-text-only sends, they stay at zero. Reply rate is always exact, because replies are detected regardless of tracking, so judge the test on reply rate.

Promote the winner

Once each variant has enough data.

  • A minimum sample is required. The “Make this the winner” control stays disabled until every variant has at least 250 sends. An under-sent variant cannot tell you it is a loser, so all of them, whether two, three, or eight, must reach that floor first.
  • You choose the winner. The builder highlights the variant with the highest reply rate once every variant passes 1,000 sends, the point at which a reply-rate difference is trustworthy. Below that it stays quiet and shows a small-sample note. The decision is yours by default. If you turn on auto-optimize, Laureo makes it for you once a day and tells you exactly why, and you can change it or reopen the test whenever you like.
  • Underperformers are flagged, not retired. Once variants clear the 1,000-send confident floor, one that falls well behind the leader picks up a muted “Underperforming” hint. It is a suggestion only: with auto-optimize off, nothing stops sending until you say so.
  • The other variants stop. When you make one variant the winner, the rest are paused and their results stay in the report, and every future send of that step uses the winner. Reopen the test to run another one.

Let Laureo pick the winner

Opt in per sequence and Laureo narrows the test for you, once a day, on the metric you chose.

This is off unless you turn it on, and you turn it on one sequence at a time. It lives in the builder under Schedule and sending, in the Variant testing block, as a switch called “Pick the winning variant for me”. Turning it on covers every A/Z test in that sequence, step by step. Leave it off and nothing about your test changes: you read the numbers and you pick the winner yourself.

With the switch on, Judge variants on sets the metric the decision is taken on. Positive replies counts only the replies classified as interested or a booked meeting, and falls back to all replies while a sequence has none. All replies counts every human reply, whatever it says. Opens and Clicks are offered only once the matching tracking is armed for this sequence, because an untracked send records a zero and a metric that reads zero on every variant would decide on nothing. When one of those two is not available the reason is written beside it, so a greyed choice never sits there unexplained.

The rule, in the words the builder uses
A variant is paused when every variant has at least 250 sends, the variants have been running 5 days, it is at least 2 percentage points behind the best one, and the gap is unlikely to be chance. Everything else keeps sending, so with eight variants two strong ones can both stay on.
  • Laureo looks once a day, not on every send. The check runs in the background and judges any one step about once a day, so a test that was looked at a few hours ago waits until tomorrow rather than being re-decided on a handful of new replies.
  • Losers pause, and nothing is deleted. A paused variant stops sending, keeps its copy, and keeps every send and reply it collected, so the report still adds up. No lead already in a thread is moved to another variant.
  • The reason is written down. The panel shows the decision in plain words with the numbers behind it, naming each paused variant separately: its own rate, the rate of the variant that beat it, and the p-value for that one comparison. If the metric had to fall back, the sentence says so and why.
  • You get a notification. One notification per decision goes to whoever created the sequence, naming the sequence and the step and carrying the same sentence, so a decision taken overnight is never something you have to go looking for.
  • You stay in charge. “Make this the winner” sits on every variant card, including one a decision paused, so you can overrule the result in one gesture; it asks for the same 250 sends on every variant, which an automatic decision has already met. Reopen this test puts every variant back to sending and clears the decision, with no sample floor to wait for, because starting a test again takes a restriction away rather than declaring anything.

Laureo re-checks a test every day until it decides, so treat a decision as a strong signal rather than a laboratory result.

The floors do not move when you turn this on. A pick you make by hand still needs 250 sends on every variant, and the builder still waits for 1,000 sends on every variant before it calls a leader for you. The automatic path uses the same 250 sends and adds a 5 day wait after the first send, because a cold reply arrives days after the email does.

Laureo does not force a single winner. It pauses only the variants that are clearly behind the best one, so if you are running eight and two of them are both doing well, both keep sending. The panel says “Leading: B, D” while more than one is still in play and “Winner: B” once only one is left. A test can narrow twice: eight down to two this week, two down to one later, as more replies land. You can do the same thing yourself with “Keep these”: tick the variants you want to carry on with and the rest pause. As with everything here, nothing is deleted and you can reopen the test.

Writing checks on a variant are advice, not a gate
Each variant body runs the same writing checks a step does, and the findings appear under that variant. They stay advisory: the launch check reads the step's own subject and body, so when a variant is what sends, its links, its subject and its formatting are yours to check. See Building Sequences.
Give a test room to breathe
Cold reply rates are low, so a variant needs a meaningful number of sends before a difference means anything. You can pick a winner by hand once every variant passes 250 sends, but the builder only calls a leader once every variant passes 1,000, the point where the reply-rate difference is trustworthy. Resist deciding on the first few replies, and lean on the 1,000-send read before you commit.

Related Articles

Building SequencesAnalyticsDeliverability
Previous
When will my sequence start sending?
Next
Reply Handling