How Removing a Featured Newsletter Article Increased CTR 13%

[ 0 By

Newsletters are an email marketing staple. Whether it’s Tuesdays at 3pm, every third Monday at 11am or every morning at 9am, the process of assembling a newsletter is a familiar one to most of us.

Although marketers are increasingly questioning their merits, newsletters still retain value. They circulate a variety of content to users and allow marketers to easily maintain consistency in their email cadence. Ideally, they also have targeted, valuable content and aren’t just being sent for the sake of being sent—but that’s a subject for another post.

Many of you know Litmus for our monthly newsletter, which features email tips, guides and case studies. Did you know we also publish a second monthly newsletter called Email Design Monthly? It was launched as a Litmus hack week project and has retained an extremely active audience ever since.

Email Design Monthly is a Litmus-curated collection of email resources from around the web. We want to provide marketers and designers with the best information available (whether it’s on our site or not), and this newsletter allows us to do so each month.

TESTING FEATURED ARTICLES

As Email Design Monthly has evolved, we’ve run a few A/B tests to find the optimal way to deliver this kind of content. As originally designed, the newsletter includes one featured article, followed by a marketing section and a design section.

email-design-monthly1

To increase performance, we wondered what would happen if we removed the featured article concept, giving equal visual importance to all articles. We anticipated two potential outcomes:

  • Without an eye-catching main article, more subscribers would dive deeper into the newsletter to find something that interested them.
  • The lack of hierarchy could overwhelm readers, resulting in less click activity.

To test this egalitarian concept, we sent two versions of our most recent edition. One version included a featured article, and the other did not. In the second version, the “featured” article was still the first article listed, but it was bundled in with the other marketing articles:

edm-featured2

Version A (w/ Featured Article)

edm-no-feature2

Version B (w/o Featured Article)

WHAT HAPPENED?

Although we don’t always see dramatic differences in our A/B tests, this experiment gave us some clear results:

  • When designated as featured, the first article received 36% more clicks.
  • When the featured concept wasn’t present, each of the other articles received 9% more clicks on average.
  • The version with no featured article got 13% more people to click at least one article.

These click rate disparities suggest that featured articles play a major role in a subscriber’s perception of the entire newsletter. If the featured article isn’t appealing, readers are less likely to look beyond that article to find something of interest.

If there is no featured article to influence their perception, subscribers seem more likely to skim to find an article that interests them. This version had much deeper engagement at the bottom of the newsletter compared to the one with the featured article.

SHOULD WE STOP FEATURING SPECIFIC ARTICLES?

Not necessarily, as results weren’t completely in the test variant’s favor. The featured article version won on an interesting metric: the number of links clicked by each subscriber. On average, subscribers who received that version clicked 3 links, while their peers in the non-featured group each clicked only 2.5 links.

That particular result suggests that subscribers who saw the featured article were likely to click it purely on principle of it being featured. They came back to the newsletter to keep clicking on other articles.

For the purposes of choosing a winner, the 13% unique click-through rate increase on the version with no featured article is the key metric. I’d much prefer that more people find at least one article appealing so that they continue to find value in the publication and don’t unsubscribe from lack of interest. A next step will be to increase total clicks in that version while still maintaining a high unique click-through rate.

Remember, these results could obviously vary from month-to-month, company-to-company and even article-to-article. The only way to know what impact a featured article could be having on your newsletter is to test it! When you do, evaluate performance based on these metrics:

  • Unique click-through rate: What percentage of subscribers click at least one article?
  • Total clicks: How many clicks overall did you generate?
  • Clicks per clicker: Divide the number of total clicks by the number of unique clicks to get the average number of links each clicker clicked.
  • Difference in click-through rate on the featured article.
  • Difference in the click-through rate on each of the other articles.

SUBSCRIBE TO EMAIL DESIGN MONTHLY

Not a current subscriber? Sign up below! We’ll send you our favorite email marketing resources from around the web each month. (Plus, you’ll get to be a part of our test next time!)