An Antidote to Conversion Overwhelm: A Pro’s Checklist For Conducting CRO Research

It’s no secret that conversion optimization, when done correctly, provides insane ROI.

The problem is, many people jump in and start testing random things (button colors, fonts, etc.). They find they’re consistently running inconclusive tests, and they get disillusioned or lose organizational support for running experiments.

It doesn’t have to be that way, though.

As Abraham Lincoln once (probably) said, “Give me six hours to chop down a tree and I will spend the first four sharpening the axe.”

Source: Image Source

What we call conversion research is the sharpening of the axe, and it’s crucial for effective SaaS optimization.

Start With Best Practices But Don’t End There

If you’re just starting out or don’t have the traffic to test, best practices aren’t a bad place to begin with. (though they get a bad rep).

They’re not always optimal, but at least they’re prototypical (your users will be used to them). And some of them are supported by a bit of empirical evidence as well.

How do you find best practices that actually work? Some great observational research has been done by Hiten Shah recently, particularly focusing on SaaS.

Otherwise, we advocate using a framework of sorts. We’ll go further into heuristic analysis later, as part of a rigorous optimization process, but you can also use heuristics to optimize your set if you don’t have enough traffic to test.

What are some of these heuristics?

Speed

How fast your site loads is important, and it’s often low hanging fruit in terms of conversion optimization. We wrote an article about this a while ago, summarizing the research on the topic as follows:

• “57% of visitors will abandon a page that takes 3 seconds or more to load.”
• “47% of people expect a web page to load in 2 seconds or less.”
• Even a mere 1 second delay can result in “11% fewer page views, a 16% decrease in customer satisfaction, and 7% loss in conversions.”

Increasing page speed will almost always improve your conversions.

Focus

Anything that isn’t helping lead users towards a goal is distracting them.

Though SaaS websites have more recently embraced the trend of minimalism (generally a good thing), the focus heuristic is usually completely ignored by larger companies with multiple people fighting for space on the home page.

The focus principle is especially important on landing pages and when you get closer to the point of purchase. According to Murphy’s law of conversion optimization, “anything that can distract a user from purchasing, will distract them.”

Unless you’re Ling’s Cars, you probably don’t want your site to look like a Christmas tree:

Clarity

As the saying goes, it doesn’t matter how many psychological triggers you use, how beautifully your site is designed, or how usable the checkout is – if no one knows what the hell you’re selling, it’s hard to convert them.

There are many copywriting heuristics you can use to analyze this on your own, or you can conduct a 5 Second Test. This is where you show users a glimpse of your page for only five seconds and see how well they can explain what it is you do.

Motivation

According to Marketing Experiments, “The motivation of the user is the single most important factor affecting conversion.”

Source: Image Source

A user’s motivation is almost wholly intrinsic. They landed on your website for a reason.

Your job, then, is to discover that reason and to communicate in a way that resonates with their intrinsic motivation.

Trust & Proof

How believable is your offer? How much do people trust your website and your company?

You wouldn’t believe how many people are still afraid to enter their credit cards online, and if they don’t already know your company/site well, then you have an uphill battle.

Source: Image Source

Of course, there are so many ways to increase trust on your website. Just a few:

• Social proof (many iterations of this
• Trust symbols
• Guarantees
• Transparent company information

Usable

Even if a user has high motivation, if they can’t use your site, it’s hard to convert. This is conveyed well by BJ Fogg’s Behavior model:

Source: Image Source

The higher the motivation, the more usability problems you can get away with. For instance, if you need to renew your vehicle registration for the DMV or pay your mortgage, you’re going to complete the task no matter how difficult.

But if you want a new pair of socks and the site sucks and is hard to use, you’ll probably go elsewhere.

Once your site is at a solid baseline, you may want to jump into running a ton of A/B tests. But first, to make them more effective, it helps to do some preliminary customer/conversion research.

Conducting Your Own SaaS Conversion Research

Following best practices isn’t optimization; it’s just a starting point. To truly optimize your SaaS site, you’ve got to learn about your customer – her problems, motivations, questions, and how she interacts with your site.

That process begins with conversion research.

I say process, because it’s not a crapshoot. You should be able to teach and repeat this process.

Now, everyone has their own, but I’m going to outline one we developed at ConversionXL, a very effective one we call ResearchXL.

ResearchXL: A Conversion Research Process to Create Better A/B Tests

There are six steps in our process:

• Heuristic analysis
• Technical Analysis
• Digital analytics analysis
• Mouse tracking tools
• Qualitative surveys
• User testing

It seems pretty intimidating, but it’s actually quite an easy process to follow once you dive into each individual prong.

Heuristic Analysis is a structured, expert walk-through of your site. I already mentioned that bit above, when talking about best practices, but it’s important to note that in this section you’re not just implementing changes based on opinions. You’re simply looking for opportunity areas to investigate further.

In addition, this is structured. You’re looking for particular heuristics. I mentioned a list above for best practice. There are other frameworks as well. Here’s what we look for:

• Relevancy
• Clarity
• Value
• Friction
• Distraction

Just remember as you do the walkthrough – whatever you write down is merely an “area of interest”.

When you start digging in the analytics data and putting together user testing plans and what not, make sure you investigate that stuff – with the intention to validate or invalidate whatever you found.

Technical Analysis is about picking up the low-hanging fruit in terms of technical fixes.

Bugs – if they’re around – are your main conversion killer. You think your site works perfectly – both in terms of user experience and functionality – with every browser and device? Probably not.

First, do cross-browser and cross-device testing. To do this, open your Google Analytics and go to Audience -> Technology -> Browser & OS report. Here you can investigate if a certain browser or device is converting less than average. Of course, this is mere correlation, but it’s a starting point for pinpointing bugs and UX problems (especially if a browser is drastically less).

Then do speed analysis. Just hop onto Google Analytics and do this: Behavior → Site Speed → Page Timings. Turn on the ‘comparison’ to easily spot slower pages.

Write down any URLs with sub-optimal speed. Use Google PageSpeed Insights and enter every URL you wrote down, and it will list all the identified issues.

Web Analytics Analysis is probably what you’re most familiar with. First, we want to check that things are set up correctly – almost every analytics setup we’ve seen has been broken in some significant way.

Second, we want to analyze what people are doing on your site. While analytics can’t give you the “why,” it can give you the “what,” “where,” and “how much”.

For your analytics rig, Google or Adobe Analytics is a must. But there are so many ways to handle your data that it would be embarrassing to try to write about it in a paragraph or two. I will mention, however, that there are some new out-of-the-box tools that are showing a ton of promise, especially for SaaS companies.

Personally, I love Amplitude and Heap. Both make it incredibly easy to find behavioral cohorts, correlative metrics, and with Heap, you can set all of this up post-hoc (so it collects all the data and you can structure the reports the way you want them later).

Here are a few resources for furthering your analytics prowess:

1. 5 Essential Analytics Tools For Subscription Businesses
2. Google Analytics Hacks for Subscription Businesses
3. Google Analytics for Conversion Optimization
4. Google Analytics 101: How To Configure Google Analytics To Get Actionable Data

Mouse Tracking Tools are another popular tool. Tools like HotJar or CrazyEgg let you see where and how users are interacting with your site. Are they clicking a button that they aren’t supposed to? Remove it or make it clickable. Are they not scrolling far enough to reach your CTA? There are many ways to fix it, but now you know.

Heatmaps can’t tell you everything. But what they are wonderful at is providing an at-a-glance for specific pages, and they’re really good at communicating these things to executives and across teams. Their visual nature makes them easier to understand than an analytics dashboard or a spreadsheet.

The best part of a heat maps tool is usually the session replay, where you can watch people actually use your site (creepy, right?).

Recommended tools: Inspectlet, Hotjar, or Clicktale.

Qualitative Surveys are awesome at digging into the ‘why’ of your problems. Why aren’t users converting? Why do people love your brand? What problems are they trying to solve?

In my mind, there are two broad types:

• On-page surveys
• Customer surveys

On-page surveys can be implemented through tools like Marketizator or HotJar. They’re great at targeting specific web visitors (not just customers) at a specific time and on a specific page. So you can really weed out insights at the moment of truth.

You’ve probably seen these pop-up surveys before. There’s a whole strategy to conducting these. You don’t want to ask questions just for the sake of doing so, just as you don’t add UI elements just for their own sake.

It’s about accomplishing specific goals. Read this article for more info.

Customer surveys are sent to your current customer list and are supposed to gather different information.

Send an email survey to recent first-time buyers. These are people who still freshly remember their purchase, and the friction they experienced in the buying process. Only survey those people who have no previous relationship or experience with you that might affect their responses (you want to filter out repeat buyers or people who bought a long time ago).

User Testing is my favorite conversion research method. It’s an oldie but goodie, coming from the user experience and design space. Thankfully, we have new technology and tools that allow us to conduct remote user testing (as opposed to recruiting people to come into a lab).

The quality of the tasks you give testers will determine the quality of insight you will get. In most cases you want to include 3 types of tasks in your test protocol:

• A specific task
• A broad task
• Funnel completion

Prioritization of Tests and a Continuous Process

Once you’ve done the conversion research, “what to test” suddenly becomes a non-issue. Instead of brainstorming random ideas, you’ll have a huge list of opportunity areas and testing ideas.

Now you just have to prioritize them in some cohesive way, because nobody has unlimited traffic (except Amazon).

There are many frameworks to do this, maybe with great acronyms like the PIE or ICE frameworks.

You can discover your own prioritization framework, and you can optimize the framework as much as the site itself. Most people introduce variables like “ease of implementation,” “how much impact will it have,” and “confidence of a lift” into the mix.

The great thing about this prioritization is that once you start running test, you begin to learn from them – winners and losers. And you start to spin new hypotheses.

So really, optimization (growth) is never done. Testing becomes its own form of research, a sort of intellectual property that becomes a competitive advantage as you learn more about your customers (h/t to create Craig Sullivan for the IP reference.

Conclusion

Everyone wants to get huge wins on A/B tests, but nobody wants to do the heavy lifting: user research.

What I’ve outlined above is our framework for conversion research. It’s just one framework out of any amount of possible methodologies. There are many out there that are faster to go through, many out there that are more thorough. Some involve eye-tracking, biometric research, or limbic mapping. Some involve predictive modeling.

The point is, you don’t have to go into testing blind. And if you go in with some data in hand, you tend to have better results, and you grow your business exponentially, as experimentation is meant to do.