KMStatus Quo vs. Innovation: An End-of-Year Experiment
Hey all!
So we (Litmus) just sent a 2016 Year in Review email. You can view the email here: https://litmus.com/builder/2c72caf
We have received a few emails and tweets about the email being "broken" or feedback that we "didn't test." It's probably not surprising that we take testing really seriously and tested this email thoroughly—sending it was not a mistake.
Yes, this email was tested. The design choices made in this email were intentional.
With that in mind, and knowing that there may be a perception that this email appears "broken" in some legacy email clients, we wanted to create a discussion around the strategy and techniques behind the email.
In the spirit of innovation, we increased the scale of this email to be 1440px at its widest - the widest email we've ever sent at Litmus. We wanted to be big (literally) and bold with our design to highlight our end of year stats. Creating an email at this scale presents several challenges in developing around email client quirks and requires certain tradeoffs.
For Outlook desktop email clients (where media queries aren't supported), we decided to stick with the 1440px wide design. Ideally, we would have built a targeted ~800-1000px version for Outlook, but were under a tight deadline and decided to stay with the original 1440px design. This meant that many Outlook users would not see the entirety of the email initially and would need to resize their window to see the email in its entirety. Frankly, we didn't anticipate that Outlook users would assume the email was broken. Sending this email provided a lot of insight about how our users respond to expectations about email width. Outlook desktop users amount to ~13% of our audience, which is our third most popular client behind Apple Mail (~33%) and Gmail (~32%).
The email was sent without some Outlook.com fixes as well (my personal fault on that one - the only true mistake in this email) but the Builder link above shows the email with said fixes. End users for the final send simply saw the fallback hybrid stacked structure. Not to discount those users, but luckily Outlook.com only accounts for ~1% of our opens.
Since we have a limited audience, we decided not to support international webmail clients, such as Mail.ru and Web.de, with limited CSS support - those clients received the fallback hybrid stacked structure and accounts for less than 1% of our opens.
For any email client that supported media queries (such as Apple Mail and Gmail), the email was fully responsive and looked great from a small 320px mobile view up to the complete 1440px view. It even looked great on mobile email clients that don't support media queries like Gmail App Android (Non-Gmail Account). Our intention was to experiment with an innovative infographic-in-email format. Our goal was to focus on our core audience by creating a progressively enhanced experience for most - while still trying to account for solid fallbacks for the rest of our audience.
So, what are your thoughts on our approach?
- What constitutes an email being "broken" for users?
- Is having a 1440px wide design inherently "broken" for users?
- Is an email like this worth the experiment?
I wouldn't say users having to scroll counts as being broken, but it is a substandard experience for those users. I think your idea of "a targeted ~800-1000px version for Outlook" is the perfect solution and it's a shame you ran out of time to implement that.
I'd say the experiment was well worth it. It's probably the best looking email I've seen this year. You're aware of the technical issues you had and you already have a solution in mind for that so next time it'll work better.
Time is always the enemy, it's a balance between building something awesome, and something safe. Im my opinion, these big showpiece type emails should sway more to the awesome and the everyday notification type message should be more on the safe side.
The issues you pointed out for outlook.com is what I would consider as broken and it's obvious you're aware of that. It's something you would have to "fix" to make right. I don't think I would put your outlook issue into the broken category though. Yes it looked off but it's not broken. "Fixing" outlook isn't the correct wording here, making the email look better in outlook is. Ultimately it all comes down to are you fixing something or making it better.
Experiments are ALWAYS worth it. Being brave and innovative is really what pushes email past the limits that are placed around it. Let's have fun with the things we truly love and find out what it's capable of. Let's be the ones that spark crazy and interesting things that are going to be perfected and will eventual end up in our inboxes from people that were inspired from the experiments we ran with.
My $.02
An email is "broken" when content can't be accessed and when an image is shifted in a way that makes it feel like the quality has been reduced.
No, but it's not something I would risk for sales-oriented emails. People may not go the extra mile to open in full view.
YES! This email and the report itself is a celebration, both of Litmus and of email as a whole. While the ultimate goal is to get people to click through to the report, I think it was paramount to show how far email clients have come in 2016. Hopefully, the Outlook team will see this email and this thread to understand that it is the primary detractor to email at this stage. I'm optimistic that the open communication amongst the team there and the community will lead to great things.
Congratulations on a killer year, having an awesome team, and on creating another email that sets the bar for what I'd like to achieve with my work.
If there's one thing I've learned over and over again at Litmus, it's that you can't be afraid of failure if you want to innovate. And that extends to the perception of failure. We've accepted that we need to take risks (as we've done today) in order to move the industry forward. Sometimes that means acknowledging that ~13% of our audience may think an email is broken, and being okay with that. It's uncomfortable and scary, but then again so is any type change—and change is what fuels innovation.
Our audience (all of you!) have really high expectations. We've set a high bar for excellence and quality in email. I'm incredibly proud of that, and am even more proud to continue pushing the bar higher.
Thanks for your support along the way!
Emails like this are how we push forward innovation. As far as being broken, aside from the outlook.com stuff it's not broken, it's just progressive enhancement. Email doesn't have to, and shouldn't, look the same everywhere - it's far better for everyone to make the best experience you can for each user and their app/device combo.
Progressive Enhancement is our best approach, but I guess Litmus is in a bit of a unique situation in that people will go and look at other email clients and see what happens, in a way that wouldn't be a problem for a company outside the meta-email space.
With EmailWeekly we had to add a bit of a caveat at the bottom, to explain that it's our space to test out new things and it's not perfect. We're all capable of making a working email to a high standard, but that comes from having a playground to test out things and occasionally make mistakes. Of course, it's great to get feedback so that we can work on improvements. Things we've learnt from EW include: advancements in Hybrid code, edge case bug fixing for our CDN, challenges producing wider email, recoding templates whilst maintaining content in our CMS, lots of process things, and we even found an edge case bug in Outlook that got reported and fixed. Those are all things that we wouldn't have found out if we'd had stuck to a functionally perfect but unadventurous codebase.
also, 1440px isn't enough, the wider the better!
There's clearly tension in email design between playing up the most advanced and engaging experiences and playing down to the lowest common denominator on code support. Fallbacks are clearly the answer, although that still means that some subscribers are having (potentially wildly) different experience than others. Based on our State of Email Production research, marketers seem to be a peace with this decision, since only a quarter of email marketers want their emails to look good in every email client. Everyone else is fine with small, less-than-optimal differences. Clearly, for some of our subscribers, the less-than-optimal differences weren't considered small.
I opened the email in Apple Mail on my iPhone SE, and I definitely had this "broken" feeling. I think it comes down to two things:
Since I only opened this email on mobile, I didn't even realize there was something big to see on desktop get to experience (which is really nice now that I see it).