See The Results of a 3-Week Email List Growth Experiment

Over the course of 21 days I did an email list growth experiment to try to grow my audience. I tried three different email signup techniques on JasonDoesStuff.com.

The three email list growth tactics I used were:

1. My standard email signup forms (the control)
2. An exit intent pop-up
3. The website takeover (AKA welcome mat)

My 3-week email list growth experiment started with this tweet:

Today marks the start of a 3-week experiment. I will report back with findings in about a month. This vague tweet is content for the recap 😁

— Jason Zook (@jasondoesstuff) February 3, 2016

I kept this growth experiment as contained as possible, testing only a few variables

The three email list growth tactics would follow the following schedule:

Week #1 (Feb 2 – 9) Baseline Tracking: The control. Nothing new, just tracking email signups. This would be the baseline for overall comparison.

Week #2 (Feb 9 – 16) Exit-intent Popup: I’d heard (and seen) so many people saying the exit-intent popup works really well at getting people to join their email list. If you aren’t familiar, this popup appears only when someone moves their mouse away from your website, to the search bar or anywhere else to leave. It’s a last-minute “before you go!” call to action. (Going in, I assumed this one would be the clear winner. Keep reading to find out if I was right… #tease)

Week #3 (Feb 16 – 23) Welcome Mat: You’ve probably seen the welcome mat (or website takeover) on other people’s websites before. It rolls out to greet you as soon as you arrive on a site, and it takes over the screen with a clear call to sign up for an email list before doing anything else.

Now, before we dive into details, I should share that this was also kind of a good vs. evil experiment for me. I’m not a fan of popups, but I don’t mind the welcome mat, so I was curious to see how others would respond to them on my site.

Here’s what my normal week looks like for JasonDoesStuff.com, my email list, and how I “promote” my website and articles.

JasonDoesStuff.com Strategy and Averages

On an average week in the last year (before these tests):

• JasonDoesStuff.com was visited by 4,000 unique visitors per week. Of those visitors, about 2/3 come from organic search.
• The Action Army (my previous email list) would gain 5 new subscribers per day, or approximately 35 new subscribers per week.

Weekly content sharing strategy I was using at the time:

• Late every Sunday night, I posted a new article to JasonDoesStuff.com
• On Monday morning, I emailed that article to the Action Army list, which had ~9,000 subscribers
• Also on Monday morning, I shared the article on Twitter and Facebook
• On Monday evening, I tweet the article link again
• On Wednesday morning, I syndicate the same article to my profile on Medium.com (typically, this includes tweeting the link to the article on Medium)

I share this schedule because I made sure to keep it the EXACT same throughout the 3-week experiment. I wanted to try to keep as many variables out of the equation as possible.

---

The 3 weeks of my email list growth experiment

Week #1 was the control group. The starting point of my email list growth experiment.

From February 2 to February 9, I simply went about my business. I wrote and shared my article following the strategy I mentioned a moment ago. I kept to the same content-sharing schedule. I kept things humming along normally.

These are the results from Week #1:

JasonDoesStuff website traffic Feb 2 – Feb 9: 3,676 website visitors
Total new Action Army subscribers Feb 2 – Feb 9: 39 new subscribers

These numbers are pretty close to my weekly averages. No surprises here!

Week #2 was the exit-intent popup

From February 9 to February 16, things got interesting. I was nervous to have a popup on my website. Again, I’m pretty vocal about not liking popups, but for the sake of science (and because it’s an exit popup), I was willing to give it a shot.

When I started thinking about this experiment, I remembered seeing a popup that didn’t make me shudder in disgust. It was simply designed, and for once, there was no spammy copy. It was on the website Minimums.com, and here’s what it looked like:

I reached out to the folks at Minimums, and they told me they used a customized popup on Picreel. I’d never heard of Picreel, but if it was good enough for their well-designed website, I figured it would be good enough for mine.

I signed up for the free 14-day trial on Picreel and got to working on my first “campaign.” It was super easy to set up, and I have to give their team a big customer service shout out. My custom font wasn’t rendering in the popup. Not only did they troubleshoot through the code on my website to fix the problem, but they also installed my custom font (Canaro bold, for all the font fans) on their servers. They deserve any and all attention from this article.

Anyhoo, here’s the popup I designed:

*Important note: I made sure to write copy that I could replicate for each version of this experiment.

Once I was done creating the popup in my Picreel account, all I had to do was copy a couple lines of javascript from their website and add it to the code on JasonDoesStuff. I opted to add it to the header.php file in my WordPress editor, which is where I have code for analytics and other random things. Plus, it ensures the popup would show up on every page of my website. From there, it was a simple flip of the switch in my Picreel dashboard, and the popup would start showing up.

Ready for the email list growth results from Week 2??

JasonDoesStuff website traffic Feb 9 – Feb 16: 3,501 website visitors
Total New Action Army subscribers Feb 9 – Feb 16: 63 new subscribers (18 from the exit-intent popup)
Estimated total annual additional subscribers with the popup: 936 new subscribers (18*52 weeks)

To be 100% honest, I was surprised to gain only 18 additional subscribers with the exit-intent popup. I’ve heard of huge spikes in subscriber numbers. I certainly wasn’t unhappy with adding 61% more subscribers during the week, but I thought it would have been more.

With Week Two in the books, I was happy to pause my campaign in Picreel and move on to the last week of the experiment.

What happened in Week #3, the website takeover (AKA welcome mat)

I wasn’t sure what to expect in the final week of my experiment. I will admit, I do enjoy the experience of the website takeover (aka welcome mat) much more than a popup. So I was happier to move on to this test.

What tool did I use for the takeover? Many of you may be familiar with SumoMe, but my website takeover was a custom job by me and my good friend (and Internet BFF) Paul Jarvis. While I appreciate that the SumoMe plugin makes it super easy to create a website takeover (and a slew of other list-building and traffic-increasing tools), it’s just not the custom experience I wanted on my website.

I dusted off my copy of Adobe Photoshop and whipped up the website takeover design myself. I was very careful to use the same headline and copy as I had in the exit-intent popup. I picked a simple image for the background that wouldn’t be too distracting, and here’s what I ended up with:

As I mentioned, I’m fortunate to have a good friend (Paul) who spent an hour helping me get the website takeover working on my site and in my MailChimp account the exact way I wanted it to work.

Similar to the popup, the additional code of the website takeover lived in the header.php file. Every page of my site would feature the website takeover.

And the results of Week 3 of my email list growth experiment??

JasonDoesStuff website traffic Feb 16- 23: 3,745* website visitors
Total new Action Army subscribers Feb 16-23: 84 new subscribers (45 from the takeover)
Estimated total annual additional subscribers with the popup: 2,340 new subscribers (45*52 weeks)

The website takeover more than doubled the results of the exit-intent popup! Truthfully, this result made me very happy. I enjoy the experience of the website takeover way more than the popup (which is a key takeaway I’ll get to in a moment).

The number I really enjoy here is the estimated total annual additional subscribers. If you read my State of the Union article, you know that the Action Army list grew by ~2,000 subscribers in all of 2015. If I continued doing things the same way in 2016, I might add another 2,000 in 2016. But by adding the website takeover, I could add 4,340 people to my list instead! Yay for doubling email list growth!

*I want to point out that there was a tiny bump in traffic in the third week, resulting from a feature about me on another website. But if we do some simple math, we can see that an additional 200 visitors wouldn’t make that big of a difference. Based on my weekly averages, I get a new subscriber ever 114 visitors (total visitors divided by total new weekly subscribers). So the extra 200 visitors would have netted only TWO subscribers based on my averages.

---

My 3-week email list growth experiment takeaways

This was a very small experiment that was meant to test the fewest possible variables. I completely understand that what works on my website may vary greatly from what works on yours. That’s exactly why you’d want to run your own experiment and let the results help you make a decision.

Test your email list growth tactics and assumptions

I said it at the beginning: I thought the exit-intent popup would be the clear winner, just based on what worked on other people’s websites. My tests proved that assumption wrong.

You may have assumptions about what your website visitors (and future email subscribers) like and don’t like, but unless you test things, you’ll never actually know. It’s great that tools like Picreel and SumoMe exist to help us accurately test our assumptions.

Remember: Use as few variables as you can to get the most accurate test results!

Results are great, but do what feels right

I mentioned that I’m not a fan of popups. I just really don’t enjoy their interruptive nature. To me, it feels like a very abrupt screeching of the tires when I’m on a website and a popup smacks me in the face. A small inferno of rage boils up inside of me every time this happens.

At the beginning of this experiment, I told myself that no matter the numbers, I wouldn’t keep the popup on my site. Even though it’s only an exit-intent popup, it just doesn’t jive with me and the experience I want to give my website visitors. Luckily, the data helped reinforce this decision!

Make sure all your other ducks are in a row with your business

I recently wrote an article about sprinkles, and I will openly admit this experiment is an example of something I want to dissuade people from until they have their product, promotion, and customer service locked in place.

Yes, experiments like this can help you make your products, promotion, and customer service better—but they can also be a huge distraction from working on those important foundational things.

It’s been two years since I launched JasonDoesStuff.com, and I’m just now feeling like I have a content schedule and sharing plan in place that I like. I’m excited to share it with as many of you as will join me on this crazy adventure.

That’s it for my email list growth experiment!

I hope you enjoyed this behind-the-scenes look at my little email list growth experiment. Again, this was specific to a few tech tools we’ve all heard so much about, and may have strong feelings about, too. I wanted to see if all the hype about them was warranted, and what kind of effects I might see from very little upfront effort.

I’m glad I did it, and for the foreseeable future, the welcome mat will be my email growth strategy of choice. It works AND I feel great about it. That’s the winning combo in my mind.