Docs
Another common use case for testing within Chargebee Retention is presenting a random set of offers that are contextual to a specific audience. A few good examples are:
In each of these examples above, it may make sense for your business to have a subset of offers you only expose to this specific Audience vs the entire cancel population. Pauses won't make sense for most annual customers but are great for Monthly customers for example whereas you may want to show a stronger % discount to High LTV customers than you show to Low LTV customers. This is a great opportunity to create targeted Offer tests within Audiences.
The steps to set these tests up are very similar to the Random Offer Test, except instead of creating one set of Offers / Pages, we create one for each Audience we choose to target. For this example, we are going to set up a simple Monthly v. Annual Audience split and test different offers within them. We will take the following steps:
We cover how to set up an Audience in this article, so will skip that here. For the purposes of this test, we will create a Monthly and an Annual Audience.
We cover updating Pages in Chargebee Retention in this article. When creating your control page you will want to make sure that you are creating a generic page that generally represents your existing in-house cancel flow. This will help us monitor performance as the tests evolve and ensure we are not biasing the initial test control with too many improvements. When you have multiple Audiences, it may make sense to have a control per-audience. This applies if your page has different elements on it for the various cancel Audiences. In this case (monthly v. annual) the offers will change but the page will not. Therefore we will create a single control page that we will target within each of the two Audiences.
Once you have a control page, it is time to navigate to the Chargebee Retention Offer Library and create three to four different Offer Types.
Targeting multiple Audiences lets you set up different sets of Offers per-audience. This makes sense in many cases because the incentives you are willing to give up maybe contextual to that specific Audience. In our example case, we don't want to show Skip Offers to Annual customers and likewise don't want to show Downgrade to Monthly to a Monthly customer. Therefore we set up two different sets of Offers, one for Monthly and one for Annual. Our Offer Mix will look something like this:
Annual
Monthly
After creating your Offers, you should navigate back to the Pages tab. Here we will create 5 pages, one for each of the offers in our Annual and Monthly Audience.
In this case, your Pages tab would look something like the above. Now you are ready to go add these Audience splits in the targeting tab to set up your test.
One of the immediate opportunities to generate lift in Chargebee Retention comes by layering two offers within a single Page. This allows you to present an initial offer (say a Skip) in the Loss Aversion card that everyone on the page sees. Then when they select a reason, they will be presented with a second often stronger offer such as a Discount of full Pause. Several of our customers leverage this by applying a Loss Aversion offer right from the beginning of their initial test.
To create a Loss Aversion Offer, navigate to the Offer Library to create a new Loss Aversion Offer. Select the Goal you would like, in this case, we will choose a Skip. Under the Layout section scroll to the bottom and select the Loss Aversion CTA Card.
Once you have created this card, you will place it in the Card B slot for all of your Pages except for the control. Setting this test up with a Loss Aversion card on each Page beside the control allows us to isolate the impact of the Loss Aversion Offer against the Control, without distorting the Random Offer test by showing the same Loss Aversion Offer to every customer that sees a Random Offer.
Now it is time to navigate to the Targeting Tab and configure the routing for your test. Select the "+ Target Audience" button and add your first Audience to the Tree.
Note we will prioritize the Audience logic from top to bottom, so put your highest priority audiences first.
From there you will add each of your relevant pages under the Monthly and Annual Audiences. In this case, we will set a control within each audience vs. at the top level. This allows us to report on the control per audience vs. having it separated at the top.
Whenever you have Audiences defined in a test you need to set a fallback Page in Chargebee Retention that will show if the fields used to define the Audience are not present. In this case, we will simply set the control as the fallback and publish our new test.
Once your test is live, you will be able to use the Experience Performance Report to understand how each page is performing.
If you have any further questions about how to set up a test in Chargebee Retention, please contact your Technical Account Manager or retention-support@chargebee.com.
