A/B testing copy and segments
Split a campaign's leads into weighted buckets so each bucket gets its own email sequence and AI-written copy, then compare results to find the message that converts best.
Overview
A/B testing in Ken lets you run multiple messaging variations inside a single campaign. Instead of spinning up separate campaigns and splitting your sending footprint, you define two or more named buckets - each with its own percentage weight - and Ken assigns your qualified leads across them automatically. Every bucket gets its own email sequence and personalization prompts, so the AI writes copy tuned to each angle. When you find a losing variant, you can delete it and Ken redistributes its unsent leads to the surviving buckets proportionally.
The feature is available to all campaigns. Find it by opening any campaign, clicking the Actions tab, then selecting A/B Testing in the Segmentation section of the sidebar.
Before you start
- You need edit access to the campaign.
- A/B Testing is mutually exclusive with Manual Segments. If the campaign has active manual segment filters, clear them before switching to A/B mode.
- Percentages must total exactly 100 before you can launch. You can save buckets at any partial total while you are still editing - Ken only enforces the 100% rule at launch time.
How to set up A/B buckets
- Open the campaign and click the Actions tab.
- In the Segmentation section of the left sidebar, click A/B Testing. The panel loads with one empty row.
- Enable the A/B Testing toggle if it is not already on.
- For each bucket you want, fill in two fields:
- The name field (placeholder text: "Bucket A") - enter a descriptive name such as "Pain-point angle" or "ROI angle".
- The percentage field - enter a whole number from 1 to 100.
- To add more buckets, click Add Segment. You can have as many buckets as you need.
- To remove a bucket, click the trash icon to its right. If the bucket already has contacts assigned to it, a confirmation dialog ("Remove segment?") appears before the removal is staged.
- Watch the Total counter below the bucket list. It shows your running sum and turns blue when it hits exactly 100%. If the total is not 100% it shows "Must total 100% to launch."
- Click Save Segments to commit your changes.
Editing buckets after launch
Once a campaign is sending, you can still rename buckets and adjust percentages. Ken validates the projected state of the bucket set before saving, so any edit that would break the 100% invariant is rejected before it touches the live sending platform.
How to read the results
Open the Analytics tab on the campaign. Each A/B bucket appears as its own row in the segment breakdown table. The columns show:
- Contacts - leads assigned to that bucket, with its share of all ready contacts.
- Emails Sent - outbound sends from that bucket.
- Progress - delivered rate (delivered / sent).
- Replies - all inbound replies.
- Positive Replies - replies classified as positive intent.
- Clicks - unique human clicks (bot and security-scanner traffic is filtered out).
Expand any segment row to see per-email-step and per-version breakdowns within that bucket.
How to retire a losing variant
- Go to the Actions tab and open A/B Testing.
- Click the trash icon next to the bucket you want to remove.
- Confirm the removal in the dialog.
- Adjust the remaining buckets so their percentages still sum to 100.
- Click Save Segments.
Contacts from the removed bucket who have not been sent yet are redistributed across the surviving buckets in proportion to their percentages. Contacts already sent through the sending platform stay mapped to the original bucket - their sending history is preserved and the remote campaign for that bucket is paused automatically.
How it works
When you save a set of A/B buckets, Ken stores each as a named segment with its assigned percentage weight. Lead assignment happens in two stages:
Before AI personalization. When the personalization worker runs, it first assigns every unplaced eligible contact to a final bucket based on the saved percentages. It uses a cumulative-fraction algorithm - contacts are shuffled randomly, then the list is cut at proportional breakpoints derived from the sum of weights rather than a flat 100 value. The last bucket always absorbs the remainder, so it never rounds down to zero contacts. After a contact is placed, a durable marker prevents the worker from reshuffling it on the next tick. Personalization then runs against the contact's assigned bucket, so the AI resolves the correct sequence and prompts for that bucket from the start.
If the campaign's active buckets do not sum to 100 at the time the personalization worker runs, the worker skips the campaign entirely and logs a warning. No contacts are personalized until the split is fixed.
At launch. Before the campaign connects to the sending platform, Ken validates that all active bucket percentages sum to exactly 100. If this check fails, the launch is aborted and no remote send configuration is created or updated until the split is corrected.
On bucket deletion. Only contacts that have not yet been exported to the sending platform are redistributed. Already-sent contacts remain on the soft-deleted bucket so their remote mapping stays intact. The redistribution runs inside a serialized per-campaign operation to prevent two concurrent deletions from stranding contacts.
The percentage fields accept any value from 1 to 100 per bucket while you are editing. The system does not reject a mid-edit state where the total is not 100 - it only enforces the rule at the single gate that matters: launch.
Troubleshooting and debugging
Symptom: "Must total 100% to launch" error at launch
- Likely cause: bucket percentages do not sum to exactly 100.
- What to check: open Actions > A/B Testing. The Total counter shows the current sum. Adjust one or more buckets until the counter turns blue and reads "100%", then click Save Segments before relaunching.
Symptom: A/B Testing toggle or sidebar item is missing
- Likely cause: the campaign may have active manual segment filters, or the campaign is in a state that does not allow segmentation changes.
- What to check: check the Manual Segments panel for any active filter rows. Clear them, save, then return to A/B Testing.
Symptom: Contacts are not appearing in any bucket on the Analytics tab
- Likely cause: the campaign has not yet sent any emails, or the personalization worker has not yet run its assignment pass.
- What to check: confirm the campaign is in Sending or Scheduled status. If it is, wait for the next worker tick (a few minutes) and refresh the Analytics tab. A bucket with zero contacts may mean the percentage is too small for the current list size - on a short contact list, a very small percentage bucket may receive zero contacts because the proportional slice rounds down (only the last bucket in the list is guaranteed at least one contact).
Symptom: Deleted a bucket but its contacts did not move to the surviving buckets
- Likely cause: all of that bucket's contacts had already been exported to the sending platform, so redistribution deliberately left them in place.
- What to check: on the Analytics tab, check the deleted bucket's "Emails Sent" column before deleting. If it already has sends, those contacts are intentionally preserved on the old bucket. Only contacts with zero sends are redistributed.
Symptom: Campaign launch fails after editing A/B percentages, or campaign stops personalizing new contacts
- Likely cause: the bucket percentages no longer sum to 100. The system enforces a strict 100% check before connecting to the sending platform. If the sum is wrong at launch time, the launch is aborted. If the sum is wrong when the personalization worker runs, the campaign is skipped until the split is corrected.
- What to check: open A/B Testing, verify the Total counter reads 100%, save, then re-launch the campaign. If the campaign was already sending and contacts stopped moving, saving a valid 100% split is enough - the next personalization tick picks the campaign back up automatically.
Symptom: Save Segments returns an error about clearing segment filters first
- Likely cause: you are trying to switch from Manual Segments mode to A/B mode while manual filters are still active.
- What to fix: open Manual Segments, remove all filter rows, save, then return to A/B Testing.
FAQ
Can I run more than two buckets? Yes. Ken supports any number of named buckets as long as all percentages sum to exactly 100 at launch. Add more buckets with the Add Segment button.
Do I have to hit exactly 100% while I am editing? No. You can save buckets at any percentage total while you are setting up. The 100% rule is only enforced at launch and by the personalization worker - both gates check independently so an in-progress edit can never push a broken split to the live sending platform.
What happens to contacts mid-flight when I delete a bucket? Contacts that have already been sent to are not moved - their sending history and remote campaign mapping are preserved. Only contacts that have not yet been sent are redistributed across the remaining buckets in proportion to their weights.
Can I rename a bucket after the campaign has launched? Yes. Renaming a bucket does not affect assignment, sending, or analytics attribution.
How are contacts assigned to buckets if my percentages do not divide evenly? Ken uses a cumulative-fraction algorithm. Contacts are shuffled randomly, then the list is sliced at proportional breakpoints based on the sum of all active weights. The last bucket absorbs any remainder, so every contact is placed and no bucket rounds down to zero even if its share is very small.
Can I switch a campaign back to no segmentation after setting up A/B buckets? Yes. Disable the A/B Testing toggle and save. The campaign returns to standard (unsegmented) mode. Any unsent contacts return to the unassigned pool.