Sites often get money from advertising. Sometimes they get it from subscriptions. And sometimes they rely on donations.

The news business has changed a lot recently, with revenues for newspapers falling, news publishers are trying to figure out how to monetise news online. In light of that, there are some news outlets that are toeing a line between selling their news subscriptions and making what sometimes sound like pleas – asking visitors to subcribe on the basis of keeping the endangered news-business flourishing – making it sound like a rare nocturnal mammal.

This is how The Guardian is currently doing that:

First of all – it’s delightfully British isn’t it?

I think it’s a message that could do with some tweaking.

1. Headline – not descriptive enough. It’s almost apologising for existing. <imagine this being read in a Hugh Grant-like voice> “Look, I wouldn’t normally ask you to subscribe to our newspaper under normal circumstances, and I hate to interrupt your day, but since you are already reading I thought..”
2. The brain vs the gut. Has Trump’s victory taught people nothing? Appealing to the brain isn’t always the best approach, you have to appeal to the gut. Do people want good news? Of course. But they don’t truly care about the well-being or livelihood of the journalists that work at The Guardian, so don’t write copy that reads like an Amnesty International appeal-a-thon. Go for the gut! Show value.
3. Don’t have 2 calls-to-action. Surely both Become A Supporter and Make A Contribution require me to spend money and will result in me getting access to something. So don’t split them up and add to the friction that the user will already experience.

In addition to that little call-out on the page, there was another one very close-by!

This banner is a bit better – it stands out a bit more, to be sure. It has a single CTA and it states the price. I think the should copy should be about the reader, not the writer. The reader doesn’t really give a shit about the writer. How about some reader-centric headlines:

a. Would You Prefer A World Where Every News Source Were Owned By 1 Mogul?

b. Live The Guardian – No Ads, All Access.

c. Smart People Read Real News.

d. You Have The Power To Keep The Facts Alive.

Also you can separate the sting of paying for something a bit by reformatting how the price is communicated. I would wager that:

“Just A Tenner A Month”

is perceived as being more affordable than:

“$10”

That’s why menu’s in restraunts all quote prices without dollar signs and decimals.

I invent things sometimes. Here’s another – inverse goals.

Confirmation bias is certainly a danger in optimisation. Anyone can fall into the trap of only seeing test results the way they want to. So a nice insurance policy against that is to set inverse goals.

Let’s say you have an experiment which adds an extra button into the header section. Your goal for that experiment is clicks. Those clicks will move someone deeper into the funnel and closer to the downstream conversion point. You run the test and get a nice 20% lift in people. Sweet!

But wait – any time you add something to a page, you are taking away from other things. Attention is zero-sum. So if you’re only measuring the element that you have added, you’re very likely to get a positive result. Have you measured the other stuff on the page – the things that will suffer from the introduction of more stuff onto a page. If you don’t, you’re not getting the full picture.

You have to set inverse goals – measure indicators around the thing that you’re changing and make sure you’re not only seeing what you want to see.

One of the funnest parts of testing is the post-test analysis. You get to dig around, see which segments behaved differently, what they did and see if there is any evidence for why they did it and what steps may come. You look at total data, you look at data segmented by device, segmented by visitor type, segmented by landing pages, drill down, refine, rinse, repeat.

It’s laborious, but it can be fun. If you’re not psycho and don’t enjoy digging around looking for the unknown, then I have just the tool for you – the Adobe Analytic segment comparison tool.

As a general rule, I hate Adobe Analytics. It’s clunky, unwieldy, inefficient. I’m a Google Analytics fan-boy till the bitter end as it is superior in every way. Well, except for this one way. GA doesn’t have a good tool for automagically finding discrepancies between segments. Adobe does.

After your test has wrapped, head to the Segment Comparison report and drop in your experiment control as 1 segment, and the experiment variation as the other segment. Then press go and let the computers do all the work. You’ll get a sweet little report that shows the metrics and dimensions that are most discrepant between these 2 segments.

With the below example that I stole from Adobe.com, we can see that 100% of Hiking Sandal Purchasers were using Google Chrome, and only 19.6% of all purchasers used Chrome, which is an 80% discrepancy and the dimension which is most different between these 2 segments.

There you go, a pro tip to embiggen your post-test analysis skillz.

Happy analysing.

There are some basic health-check metrics I look at after a test has wrapped, just to make sure, before I spend 4 hours crawling through the data, that the data collected normally and the test ran as planned.

The basics are – daily unique visitors, daily conversion rate and cumulative conversion rate. With these 3 things you can get a good overview of data collection and test administration.

When checking daily unique visitors, it is nice if the data is evenly distributed across the test period, but not really important. If it isn’t, it’s just good enough to take that into account when you’re doing your analysis.

Checking daily conversion rate will just flag unusual days or activity or trends that are helpful to be aware of.

Cumulative conversion rate is an interesting one. I have seen on quite a few occasions cumulative conversion rates that just keep going up. A common-ish issue with tests is new visitors not being allocated to buckets properly. Usually it happens when a variation is paused or deleted and users who have been allocated to that variation keep coming back and having their activity recorded in analytics. So while new visitors aren’t being added to the test bucket (variation / experience), the returning visitors are still in that bucket, and will continue to come back to the site. And since returning visitors are more and more likely to convert with each visit, as time goes by, the conversion rate for that variation just keeps going up.

Little pro tip. Happy analysing.