Category Archives: Email Automation

Web Tip – How to Measure Statistical Significance of Your A/B Tests

This week, we welcome a guest post from our Digital Project Manager, Anna Pleshakova.

 Anna is responsible for behavior-driven email marketing campaigns and strategy for B2B and B2C clients and helping them navigate emerging digital trends. She also works on web development and data integration projects for clients.

As marketers, we always want to be testing to see what creative content fits our audience best. Is it a sneaky subject line that entices email opens? Or a particularly good call to action that gets the click on a landing page?

Multivariate, or A/B testing, is a great way to test and see what works and doesn’t. “Success” can be measured in numerous ways, but the results should be significant. If you’re scratching your head because we lost you at multivariate, read on to learn what A/B testing is and how you can measure the statistical significance of your tests.

A/B or Multivariate Testing

A/B Testing is a method of comparing two, or more if you’re using A/B/C/D testing, versions of web pages, emails or other assets to determine which one performs best (has a better conversion rate).

An A/B test works by splitting your audience into two samples: a “testing” sample, generally 20% – 40% of your audience, and a “remaining” sample. The marketer sets the metric that determines the “winner”. This metric is open rate, click rate, effective rate, or something similar, to define success. Then an end date and time is set for the conclusion of the testing.

When performing A/B testing on emails, the first thing you will do is send both the test and control versions of your email to the testing sample audience. After the test window ends, the “winner” is sent or shown to the remainder of the audience.

The A/B testing process for webpages is a little different than testing emails:

    1. Study Your Website Data to Observe User Behavior
      Start by using a tool, such as Google Analytics or crazy egg, to view data for each of your webpages. Look at your bounce rates, conversion rates, where visitors are clicking on each page to see if there’s a disconnect and identify the goal you want to achieve.
    2. Create a Hypothesis
      Once you see the bounce and conversion rates, you will be able to come up with a hypothesis. Perhaps, there is less conversion on one page more so than on other pages. It could be that the size of a button is too small and people find it hard to click on mobile, or the call to action isn’t enticing or informative enough.
    3. Test your Hypothesis
      After you create a hypothesis, develop your test version(s) for an A/B test you create a second version of your targeted page and make changes you believe will lead to an improvement of your end goal. Half your traffic is shown the original version of the page (control) and half are shown the modified version.
    4. Analyze Data and Draw Conclusions
      Review the data and see if there is a clear winner. The winner can sometimes be the test version or the control.

The statistical significance of the test results is imperative for drawing meaningful conclusions. Read on to read what statistical significance is and why it’s important in testing.

Statistical Significance

According to Optimizely, “statistical significance is a way of mathematically proving that a certain statistic is reliable. When you make decisions based on the results of experiments that you’re running, you will want to make sure a relationship actually exists.”

In other words, unless your tests are statistically significant, you will not be able to back up your claims of one version winning over another.

Statistics crash course:

  • Null Hypothesis: the email or variable you are testing
  • Alternative Hypothesis: the other email or variable
  • P-value (a number between 0 and 1) is used to help determine the significance of your results
    • P-value less than or equal to 0.05, null hypothesis rejected
    • P-value more than 0.05, null hypothesis accepted

If you run an A/B test with the outcome of a significance level of 95% (aka your p-value is equal to 0.05) or more, this means that you are 95% confident that the winner is real. It also means that there is a 5% chance that the winner could be an error caused by randomness. An error rate of more than 5% is considered too high.  You are not going to be 100% right all the time, but you can ensure you are as accurate as possible by keeping a break point of 5% in mind.

Measuring Statistical Significance

Are you thinking of calling up your old statistics professor for help? Put the phone done, we’ve got you covered.

There are some awesome free (and accurate) statistical significance calculators online that keep you from digging out your calculator, but allow you to run the calculations yourself. All you need to do is plug in the number of people in each test and the number of conversions (opens, clicks, downloads, views, etc.) each test received.

Here are our favorite resources:

A/B Test Guide
GetDataDriven
Neil Patel

One Last Thing

We want to stress how important it is to not make assumptions while A/B testing. Significance percentages give you confidence that your results, which ultimately power changes to your website or other assets, will be worthwhile and reliable.

We encourage you to start testing early and often with the help of a significance calculator. If you’ve already been testing, but haven’t been calculating statistical significance – use the easy and free tools we linked above!

Email Tip – Mystery Emails We Love

Is your interest piqued at the slightest bit of mystery? If your answered “yes,” the same goes for your audience.

Mystery emails are used to pique interest, add urgency, and increase open and click rates. There are many use cases for this- maybe you’re promoting a sale, or unveiling a new product or service. The same goes for all your other emails; the subject line, creative, copy, and call-to-action all play a role in getting the reader to click through to the landing page. In the case of mystery emails, you’re going to be vague and “tease out” what the landing page will reveal.

Continue reading Email Tip – Mystery Emails We Love

Email Tip – How email personalization increased offline event attendance



Who:

Pacers Running sells running gear and hosts races to promote a running community for their customers.

Challenge

Pacers Running needed to drive participation at the George Washington Parkway Classic 10 Mile & 5K (GWPC).

Solution

Whereoware designed a hyper-personalized, automated email program for Pacers to drive participation at the GWPC.

By integrating multiple race registration and tracking systems, the personalized emails reference each contact’s relationship with Pacers, place in the race lifecycle (pre-registration, post-registration, and race lead-up), and more.

See all the unique datapoints Pacers used to personalize their audience’s experience and boost attendance at the George Washington Parkway Classic 10 Mile & 5K.

Email Tip – Father’s Day Emails We Love

Father’s Day is just a few short weeks away and we have started to receive numerous emails in anticipation of the holiday.

Check out some of our favorite below and get some gift ideas while you read!

Gift Guides

Gift guides make it easy to cut through the clutter on a website and see what stand out items the retailer thinks are perfect for a holiday. We like that Kiehl’s shows both individual and kits that would make great gifts for dad. And we like the top 5 gifts guide from Mark & Graham – especially the S-curve layout of the email!

Segmentation

Is your dad the creative type? Or is he more of a sports guy? Emails that segment gift suggestions to a certain “type” of guy help you to determine where to focus your interest. Is your dude on the cusp or more than one type? Mix and match gifts from different segments for a personalized gift! We like how Bergdorf Goodman and Nordstrom Rack approached this email style.

Promotions and Deals

Who doesn’t love a sale? Cabela’s and Artifact Uprising sent emails highlighting the deals and site-wide sales they have. This is especially great if you need to buy multiple gifts for the men in your life celebrating Father’s Day. Paper Culture sent an email about a specific product they are running a special on that they think would make a great Father’s Day gift.

Gift Cards

Gift cards give you a lot of flexibility in your gift giving. Services like Stitch Fix offer gift cards that can be emailed, printed, or mailed to the recipient to fit your needs. Ahead of the game? Get a physical gift card gift boxed and sent to the recipient or sent to you to give to them in person. Looking for a list minute gift? Email them the gift card with the retrieval code.

Marketing Tip – Relational Tables 101

Are you an e-commerce company that sells hundreds of products in various iterations and your customers come back to order from you time and time again? Are you pulling your hair out trying to house all the data you have on your customers into one database? Stop. Relational databases and tables will become your new best friend.

Ali Nazam

We welcome Digital Marketing Manager, Ali Nazam to explain what a relational database is and why as an e-commerce marketer you should use them.

Continue reading Marketing Tip – Relational Tables 101

Marketing Tip – MailChimp 101

So, What Exactly is MailChimp?

MailChimp is an email service provider (ESP) for small to midsize companies with a range of communication needs. MailChimp offers an easy to use user interface with capabilities that personalize the experience for the sender and recipient.

How to Send an Email in MailChimp

MailChimp provides numerous campaign types and templates to customize for your needs. No HTML or CSS coding is necessary to use MailChimp with easy plug-and-chug systems in place. Follow along to see how to send an email:
Step 1
Locate and navigate to “Campaigns” in the top menu. Click “Create Campaign” to get started. Select the type of campaign you would like to build. In this case, we’ll start small. Let’s choose “Regular Campaign.”

MailChimp-Campaigns

Step 2
Choose your list. This is pretty simple – just select which of your lists you would like to send to from the drop-down menu. You then see the option to choose segments. This is where you would apply a specific set of rules to a targeted audience – or, just send to the whole list. (Hint – segmenting does wonders!)

MailChimp-Sender

Step 3
Complete your mailing settings. This includes all of the basics: mailing name, subject line, from name, from email address and tracking. MailChimp provides the option to personalize the “to” field based on data, or check different methods of tracing. The screenshot below shows the generic options – which suffice for a typical send.

MailChimp-MailingSettings

Step 4
Now, the fun begins! Don’t get overwhelmed with the mailing template options, go into this step with a general idea of how you would like to layout your email.

You may choose from basic templates, themes, saved templates (from previous sends), set campaigns, or code your own. Once you have your bearings within MailChimp, you can create a saved template with standardized assets such as your logo, header, and footer. For all future sends, you can select that saved template rather than starting from scratch each time.

For now, we will select a simple 1:2 Column Banded template.

MailChimp-Templates

Step 5
Design to your heart’s content! This is your chance to play with the drag and drop functionality within the platform. Pull different content blocks from the right side and insert images and copy to make the email your own. The template you start with is just a jumping off point, customize it to your needs.

The right panel also offers you styling options per content block. Anytime a content block (image, text, column, etc.) is selected on the left, it’s settings will appear on the right. That’s where you can style based on color, size, linking and more.

Here’s a simple before and after:
Before

MailChimp-Template-Before
After

MailChimp-Template-After

Step 6
You’re almost there! Let’s quickly pause and review what we’ve accomplished. You chose the send list, completed the mailing settings, selected and styled the template. Now you can finally press the “send” button to send now, or you can schedule the mailing for a future send time! Always use an email checklist and send with a buddy to ensure your send is successful.

MailChimp-Send

Is MailChimp Only Used for Sending Email?

Nope! MailChimp has many integrations that can be set up to extend its functionality to fit your needs. Let’s break down some of the popular options:

E-commerce Integrations
The more you know about your customer’s shopping behaviors the better you can target and personalize the customer journey. MailChimp has integration capabilities with numerous e-commerce, such as Shopify, Magento, and ShipStation to name a few.

CRM Integrations
Not only is it important to track online behaviors, but it is also necessary to track offline behaviors customers have with sales or customer service reps. Connect this data with MailChimp, through CRM integrations, such as Salesforce or SugarCRM.

Social Integrations
Facebook
Build single image or carousal image Facebook ads, using the same plug-and-chug tools that you use for email. There is no extra fee for this feature within MailChimp, just pay what you would for Facebook sponsored content. Not only can you setup and trigger Facebook Ads within MailChimp, you can set up custom audiences, and access holistic campaign reporting.

Twitter
Easily integrate Twitter information in your campaigns or post to Twitter all within MailChimp.

Is MailChimp Right for Me?

The best ESP for you depends on your needs (and wants) for the tool and your budget. MailChimp is a great resource if you have a smaller in-house team, who are unfamiliar with coding templates from the ground up. We’d love to talk to more about how MailChimp or another ESP would be best to fit your organization.

Email Tip – 6 Steps to Streamline Email Creation Checklist



6steps-email-teaser
So you want to send an email to drive a sale or promote an event you’re hosting, but where do you start? You know you want it to spark an interest and more importantly, a conversion. According to Monetate, “email marketing drives more conversions than any other marketing channel, including search and social.”

Follow our 6 Steps to Email Creation checklist to help streamline the process and ensure your email has value.

 

Email Tip – Use Email with In-Store Interactions for an Omni-Channel Experience

As email marketers we sometimes get stuck in the digital world. But brands who have brick and mortar locations can also leverage email for in-store events and in-store only promotions. Having an in-store event and want to drive foot traffic? Send a series of email invites! Is it too close to ship before a holiday? Send an email with a coupon that’s only good in-store. According to CMO, sixty-five percent of U.S. shoppers will browse online and buy in-store during the holidays.

Continue reading Email Tip – Use Email with In-Store Interactions for an Omni-Channel Experience