Our Approach to Evaluating Email Subscriber Performance in eCommerce
In this article, we share the key metrics, frameworks, and visualizations we use to analyze email subscriber performance.
Recently, we published an article offering a concise explanation of the role analytics plays in the eCommerce space. The article also provides an overview of key eCommerce objects that we believe are crucial to monitor regularly to accurately assess performance across important areas. Today, we’ll take a deep dive into one of these key areas.
Most eCommerce businesses engage in email and, to a lesser extent, SMS marketing. These channels are often referred to as "owned channels" because marketers have much greater control over them compared to paid channels like Facebook or Google Ads.
In this article, we'll take a deep dive into evaluating the performance of eCommerce owned channels (email, SMS) subscribers. We’ll cover the most important metrics to track, discuss how to compare the performance of owned channels subscribers over time, and highlight common pitfalls to avoid when analyzing subscriber performance.
A second article will be published in the future, focusing on the performance of owned channel campaigns. But for now, let's dive deep into evaluating the performance of subscribers in owned channels.
Owned Channel Subscribers: Key Metrics
Before discussing the metrics used to analyze owned channel subscriber performance, it’s important to first define what we mean by an "owned channel subscriber."
An owned channel subscriber is someone who has opted in to at least one owned channel, such as email, SMS, or push notifications. This subscriber count is distinct from the number of customers, as a subscriber may not necessarily have made a purchase with the brand.
Note: We recommend calculating each of the following metrics separately for each channel (e.g., Email, SMS, Push Notifications). In the next section, we’ll use email subscriber performance as an example when introducing various metrics, but all of these metrics are fully applicable to other channels as well.
When analyzing your email subscriber list, the first question that likely comes to mind is whether your list is growing or shrinking. To determine this, you need to consider three key groups that contribute to overall list growth:
New Subscribers
This group includes individuals who have joined your email list for the first time within a specified time period.
Re-Subscribers
These are people who were previously subscribed to your email list, unsubscribed, and have now decided to subscribe again.
It’s important to track the growth of new subscribers and re-subscribers separately, as they represent different cohorts. This distinction can influence your email marketing approach. For example, a welcome email series might focus on introducing the brand to new subscribers, whereas for re-subscribers, it could highlight new products introduced since they last unsubscribed.
Unsubscribers
These are individuals who have chosen to unsubscribe from your email list during the specified time period.
Email List Growth
To measure your email list growth over a selected time period, use the following formula:
Email List Growth = New Subscribers + Re-Subscribers - Unsubscribers
We also calculate the average daily values for all metrics, providing a clearer understanding of daily performance over a selected time period, rather than just evaluating the performance of the entire period.
When analyzing trends, we typically begin with the reporting dashboard we've developed. It allows us to compare two different time periods, helping us assess whether each metric is improving over time.
As shown in the screenshot, there are four columns. The first column displays all the metrics we've discussed so far for the selected period (1/1/2023 - 12/31/2023).
The second column shows the same metrics for the previous period, which is set using the "Days before" parameter on the right. It's currently set to 365 days, so we're comparing the selected period to the period 365 days prior (1/1/2022 - 12/31/2022).
The third column displays the difference between these periods in absolute terms, while the fourth shows the difference in percentage terms.
Looking at the data, we acquired 10.77% fewer new subscribers compared to last year, but we've seen a significant improvement in re-subscriptions, up by 54.88%. However, more people are unsubscribing this year (34.07% compared to last year), resulting in an overall subscriber base decline of 26.27%. Not an ideal situation, to say the least.
Adjusting Time Periods
A great feature of this report is the flexibility to compare any time period by adjusting the parameters on the right. For example, to compare December 2023 performance with December 2022, we simply change the "Date From" to 12/1/2023 (see the screenshot below).
When comparing December 2023 to December 2022, we can see that our email list shrank by 18.57%.
“Days before” parameter
If we want to compare December 2023 to the previous 30 days instead of a year-over-year (YoY) comparison, we just adjust the "Days before" parameter from 365 to 30.
In this case, we collected significantly more emails from new subscribers (165.68%) and improved on re-subscribes and unsubscribes as well, resulting in a remarkable 715.92% increase in email list growth compared to the prior 30-day period.
We find it very useful to compare two different time periods in this way; however, identifying positive or negative anomalies on specific days can be challenging with this report alone. That’s why we also value a report that displays the daily performance of each metric over time.
For example, here is a screenshot of a report showing the daily number of new email subscribers, with the orange line representing new subscribers in 2024 and the blue line in 2023.
There are two interesting observations we can make immediately:
First, we can clearly identify daily peaks in new email subscriber acquisition. What occurred on these days? Did we launch a new email acquisition campaign? Is our data tracking accurate? Or perhaps we’ve attracted new 'bot' subscribers, which could harm our domain’s reputation. This is certainly a question worth investigating further.
Secondly, we observed a significant underperformance in new email subscriber acquisition during 2024 compared to 2023, between the points marked in the screenshot below.
This decline was due to a change in the checkout process that significantly reduced the number of new email subscribers gained during checkout. We've dedicated a separate article explaining why this change led to the negative effect and offering suggestions on how to address it.
The negative effects of this change were only resolved couple of months later, resulting in a considerable loss of new subscribers during that period—a loss with a real impact on revenue.
Had we had this type of report earlier, we likely would have detected the issue much sooner.
A similar issue occurred with another client when a change was made in the checkout process, resulting in a significant drop in new email subscribers. Due to the lack of this type of report, the issue went unnoticed for nearly eight months. You can read more about this change in this article:
Email Health Segmentation
We believe that it's not enough to simply analyze the size of your email subscriber list—its health is equally important. This is where email health segmentation becomes essential.
Email health segmentation categorizes your entire subscriber list into various segments based on engagement levels and the length of time each person has been subscribed to your newsletter.
Here are the typical email health segments we set up for our clients:
(a) New – First-time subscribers who joined within the last 31 days
(b) New (Resubscribed) – Re-subscribers who rejoined within the last 31 days
(c) Active – Subscribers who clicked on an email campaign within the last 91 days
(d) Lapsing – Subscribers who clicked on an email campaign within the last 181 days
(e) Lapsed – Subscribers who last clicked on an email campaign more than 180 days ago
(f) Passive – First-time subscribers who joined within the last 91 days but have never clicked on an email campaign
(g) Inactive – Subscribers who have never clicked on an email campaign
These groups are mutually exclusive, meaning each subscriber can only belong to one category. When categorizing subscribers, we start from the top and work our way down through the segments.
The best practice is to send regular newsletters only to segments (a) through (c). The reason is simple: sending emails to unengaged subscribers can negatively impact your domain's overall health.
Email providers like Google and Yahoo track recipient engagement. If many recipients don’t engage with your emails, they may lower the trustworthiness of your domain. This can result in your emails landing in the promotions tab—or worse, the spam folder—rather than the main inbox.
There’s a real cost to sending emails to disengaged subscribers, as it can harm inbox placement for everyone on your list.
That's why we recommend regularly sending newsletters to groups (a) through (c), and only occasionally targeting the remaining groups with specific campaigns.
When it comes to analyzing these segments, we typically rely on two key reports.
The first view, similar to the ones we've discussed earlier, compares the size of email list segments on two different days. The first column represents the last day of the selected period, while the second column shows the last day of previous period, which can be adjusted using the "days before" parameter. The third and fourth columns show the differences between the two periods in absolute and percentage terms.
As you can see, the portion of our email subscriber list that receives regular newsletters has also decreased year-over-year (YoY) by 16.66%.
Similar to the previous reports, you can compare different time periods by adjusting the "Date To" parameter or the "Days Before" setting.
The second report we like to analyze shows how your email subscriber list evolves over time broke down by email health segments. In the example below, we can spot an interesting insight.
When examining the overall email list size, we can see steady growth over time (see the screenshot below).
However, if we focus specifically on segments (a) to (c)—the subscribers who receive regular email campaigns and engage with them most often —we notice that this list peaked around mid-December 2022 and then steadily declined from mid-May 2023 until the end of the year.
This highlights the importance of not only tracking overall email list growth but also monitoring how email health segments evolve over time.
If you only look at overall list growth, you might conclude that your email list is growing well. In reality, the number of subscribers actively engaging with your emails could be shrinking.
Subscriber Performance Metrics
The final group of metrics we’ll discuss today focuses on the actual performance of email subscribers. There are three key metrics we consider most important in this category:
Revenue per Subscriber
This metric represents the average revenue generated by a subscriber over the duration of their subscription. It’s an excellent way to measure whether the value of your email subscribers is growing or shrinking over time.
Subscriber Conversion Rate to Purchase
This is the percentage of subscribers who make at least one purchase after signing up for the newsletter.
By default, we exclude all purchases made within 24 hours of subscribing. The reason is to filter out purchases that were likely not influenced by email marketing, as they would have happened regardless of the subscription.
Without this adjustment, certain email acquisition channels would show inflated metrics compared to others. For example, customers who subscribe at checkout often make a purchase immediately after subscribing. While this purchase wasn’t influenced by email, it would artificially boost the subscriber conversion rate compared to subscribers who sign up through a pop-up banner.
By excluding these early purchases, we ensure a more accurate comparison between different cohorts (checkout subscribers vs. email pop-up subscribers), giving us an "apples to apples" view of subscriber performance.
(Active) Subscriber Retention Rate
This metric has two reporting variants.
The overall Subscriber Retention Rate is defined as the percentage of subscribers who remain subscribed after signing up for a newsletter—in other words, the portion of our audience still reachable by email.
However, we find it more insightful to report on the Active Subscriber Retention Rate, which measures the percentage of subscribers who not only remain subscribed but are also actively engaging with our newsletters. This Active Retention Rate is our default metric and is prioritized in most of our reports over the general subscriber retention rate.
From our perspective, this metric more accurately reflects the number of subscribers we've retained through the email communication channel, as it captures those actively engaging with our emails.
Now, let’s take a look at the report on Revenue per Subscriber based on the month customers subscribed to the newsletter. Pay particular attention to the last column, which shows the average revenue per subscriber over their lifetime. As you can see, this KPI has been decreasing significantly over time.
For example, the average lifetime revenue per subscriber for those acquired in January 2023 is $167.40, compared to $104.40 for subscribers acquired in January 2024, and only $11.34 for subscribers acquired in October 2024.
Why such a sharp decline? The explanation is simple: time. Subscribers acquired in October 2024 have only been subscribed for a few days, while those acquired in January 2024 have been subscribed for 9-10 months, and subscribers from January 2023 have been subscribed for 21-22 months.
It’s much more likely for subscribers to make one or multiple purchases over 22 months than within just a few days. A person who places 3 orders in 10 days is an outlier, but someone placing 3 orders over 2 years is not.
Therefore, comparing the overall average revenue per subscriber across different time periods (like January 2023 vs. January 2024) is not a fair comparison—we're not comparing apples to apples.
The good news is that there is a way to make an accurate comparison between these cohorts and determine whether January 2024 subscribers are performing better than those from January 2023.
We do this by evaluating average revenue per subscriber after specific time periods from when they became subscribers. As shown in the screenshot above, we report average revenue per subscriber after 7 days, 1 month, 3 months, 6 months, 12 months, 24 months, and 36 months.
When comparing the performance of January 2023 versus January 2024, it makes more sense analytically to ask the following question: What was the average revenue per subscriber for these cohorts after 3 months of being subscribed?
As shown in the screenshot below, for the January 2024 cohort, the average revenue per subscriber after 3 months was $62.59, compared to $58.18 for January 2023 subscribers. This means that the January 2024 cohort is performing 7.58% better in terms of average revenue per subscriber after 3 months.
The final point we want to address is the presence of empty values in many of the columns shown in the report. Why are they there?
The answer is straightforward. No one who subscribed in January 2023 has been subscribed for a full 24 months yet (this article was written on October 16, 2024). So, we cannot calculate the revenue per subscriber after 24 months if they've only been subscribed for 21 months.
Revenue per Subscriber after 24 months will be calculated for the January 2023 cohort in January 2025.
We have developed similar reports for two other key metrics. Below is an example of the report for the subscriber conversion rate to purchase.
Just like with revenue per subscriber, the subscriber conversion rate to purchase after 3 months is higher for the January 2024 cohort compared to the January 2023 cohort (11.17% vs 10.22%). This means that subscribers acquired in January 2024 are more likely to make a purchase than those acquired in January 2023.
Lastly, we have the report for the Active Subscriber Retention Rate metric:
By changing the setting on the right from 'Active' to 'All Subscribers,' you can easily adjust the metric to include all subscribers who remain subscribed to the email newsletter, regardless of their engagement level. This contrasts with the default calculation, which requires active engagement with our emails.
So far, we've analyzed Revenue per Subscriber, Subscriber Conversion Rate to Purchase, and Subscriber Retention Rate, all segmented by the subscriber acquisition date. However, there’s another valuable dimension to consider: breaking these metrics down by acquisition strategy or source.
Here’s an example of how these three metrics might look when broken down by acquisition source:
Breaking down these metrics by acquisition strategy or source is important because it helps uncover deeper insights into the quality and behavior of subscribers. For example, in our experience, strategies like giveaways can be highly effective at quickly acquiring new subscribers. However, the revenue per subscriber from these strategies tends to be significantly lower compared to other sources.
This is likely because a large portion of giveaway subscribers sign up primarily for the chance to win, with little to no intent to make a purchase. As a result, you're collecting subscribers of lower quality who are less likely to convert into paying customers.
If you're using multiple strategies to acquire email subscribers, it’s crucial to have these reports in place. This allows you to evaluate the performance of each acquisition tactic and optimize your efforts based on subscriber quality and revenue potential, not just the volume of new signups.
If you found this post valuable…
We hope you found this article valuable. If so, please consider subscribing (for free!) to receive updates on our latest publications.
If you'd like to explore any topics mentioned in the article further, or if there's anything else we can assist you with, feel free to schedule a meeting with us by clicking the button below:
P.S.: We’d be happy to provide you with a full demo of our email subscriber reporting! There are a few interesting dashboards and concepts we’ve developed that didn’t make it into this article.