The Unspoken Reality of Net Neutrality

Posted by rjonesx.

It's not very often that Moz as a company openly advocates for a particular political position. While we've always supported our employees' choices to be vocal about the issues for which they're passionate and have done our best to live by the TAGFEE values (as imperfect as that attempt may be), we have rarely directed the attention of our customers or our readers toward a particular end. Today, we break with that tradition to join hands with countless organizations across the web in a Day of Action in support of net neutrality.

Net neutrality is a fairly simple principle: that Internet service providers should enable access to all content and applications regardless of the source, and without favoring or blocking particular products or websites.

At face value, net neutrality seems to affirm the basic principles of free speech which most of us hold dear. If the FCC moves forward in retracting policies that protect the Internet in the interest of the public good, it is reasonable to suspect that these freedoms will be curtailed.

This curtailed freedom is often described in terms of small or independent content producers who will be cut out of this new caste-like system of Internet access. However, I would like to take a moment to shed light on different vital services which are likely to suffer without the protections provided by net neutrality.

1. 911 call centers

Over 65 million Americans rely on Voice-Over-IP (VOIP) for their home phone service, and in 2009, the Congressional Research Service called for 911 call centers to migrate to IP technology in modernizing their infrastructure. Both families and the call centers themselves depend on unfettered bandwidth. When you call 911, seconds matter, and the quality of that bandwidth determines that speed. Without net neutrality, that bandwidth and those seconds are put to the highest bidder.

2. Clinical Video Telehealth for our veterans

In 2016, the Department of Veterans Affairs served over 677,000 veterans in rural and under-served areas via telehealth. The VA's Clinical Video Telehealth (CVT) is a true innovation, connecting their best doctors with their neediest patients. Unfortunately, this critical health technology relies on the same network backbone as any Internet service. Who will pay the increased tolls to ensure that serving our veterans remains a priority on these networks?

3. Online education for K–12 students and their teachers

Finally, by 2014, 75% of all US districts offered some form of online education for K–12 students. More than 2.7 million students took advantage of this blended ed-tech, while 315,000 students were enrolled in full-time online education. The same technology you might use in your workplace to hold sales calls is used by teachers to meet with parents and students across the country, delivering education to those who are difficult to serve otherwise. It's annoying when Netflix buffers, but it is tragic when a child can't communicate effectively with his or her teachers.

These are just three of countless examples of how the Internet has come to provide vital services to our veterans, our children, and our communities. Without the basic protections net neutrality affords, these vital services, and so much more, are at risk. Net neutrality advocates are correct to be concerned about the free and unfettered speech of sites across the web if the Internet is left unprotected, but we should not pretend that only our words are at stake. Our safety, our veterans, and our children are on the line, too.

If you're interested in learning more and taking action, take a look at Battle for the Net.

Thank you for your consideration.


Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!

This Is What They Search For: The Most Popular US Industries & Traffic Shares

Posted by Alex-T

After storing this idea in mothballs for quite a while, I finally decided to conduct an analytical study that would breakdown the most popular industries in the US based on the number of monthly online visitors. Special thanks to the SimilarWeb team, who helped me with the convoluted process of assembling data on the industry traffic distribution across 1,000 top-visited US domains.

The purpose of this research isn’t just to share some general trends and observations that will leave you thinking, “Sounds interesting, but what’s next?” I’ve also included a bunch of actionable ideas based off of the data I went over myself.

For those of you wondering whether it’s worth it to read this article in its entirety, below are the key findings:

  1. Google, Facebook, YouTube, Yahoo, and Amazon own 32.34% of the total US traffic market. These five online giants decide which sites we're going to visit next and what ads we see.
  2. The top five industries in the US are Internet and Telecom, Arts and Entertainment, News and Media, Shopping, and Adult Entertainment. Altogether, these businesses control 82.55% of the US market share.
  3. In the Internet and Telecom Category, search engines and social network sites get up to 95% of the traffic share.
  4. Google, Yahoo, and Bing are the most visited search engine sites in the US However, that doesn’t necessarily mean that people use Yahoo as a search engine.
  5. Wikipedia.org has over 4.7 billion monthly users, with 86% of those users coming from organic search. Wikipedia is known to be a traffic-generating site.
  6. In the Shopping category, 74% of market traffic is split between Amazon (51.24%) and eBay (22.01%).
  7. YouTube promotes the Gambling industry more than any other.
  8. In the Business sector, industries like Marketing, Advertising, and E-commerce have the smallest share compared to other subcategories.
  9. The tourism industry is extremely competitive; however, it has a diverse range of small- and medium-sized players, since the top domains occupy only 17% of the total market share.

82.55% of all US online traffic is shared among five industries

Over 80% of all US online surfers are divided among five industries, while the rest of the traffic (15%) is spread across more than 15 other niches. Among the top five leaders are Internet Telecom (45.9%), Arts and Entertainment (12.35%), News and Media (9.35%), Shopping, and the Adult industry.

I expected to see the Shopping industry at the top of the list with a much higher percentage of traffic, but it may not have made it to the top three because SimilarWeb defines YouTube as part of the Arts and Entertainment industry, which drives over 36% of traffic in this category.

The Reference category is represented mostly by Wikipedia with 1.32% of all US traffic. I can see how one day Wikipedia may be acquired by either Google or Facebook, jacking up their traffic and sales. Currently, Wikipedia is still a non-profit organization, and hopefully things will stay the way they are.

Over 30% (32.34%) of all US online traffic is controlled by five websites

In most cases, these five websites control what information we consume on a daily basis. Even more important, they also determine what sites we visit next and what kinds of ads we see. Here’s a list of the top five sites with their traffic market share:

  • Google – 16.41%
  • Facebook – 6.56%
  • YouTube – 4.91%
  • Yahoo – 2.55%
  • Amazon – 1.91%

And yes, all the websites listed above offer advertising opportunities. If your site doesn't have any visibility on Google and Facebook, you're missing the majority of your audience because 67.4% of all US users search on Google, and Facebook gets 68% of all active web users. Without a doubt, Facebook may not be the right fit for all business types, but it is a must-have SMM channel for B2C products.

Keep reading to find out what I discovered about the top 10 categories as well as what kinds of subcategories they sprout into.

Please note that one of the categories has been left aside for the reason that is has no subcategories. Something tells me you’re well aware that Pornhub has the biggest market share in Adult Entertainment.

Internet and Telecom

According to the US Department of Commerce and the Bureau Of Economic Analysis, in 2015 the Information Industry was the largest contributor to the US economy’s 1.4 percent growth, adding close to $900 billion in value.

On the graph below you can see that over 41 percent of US traffic is shared between search engines and social network sites, which are getting most of the juice.

What I find really interesting is that SimilarWeb doesn’t recognize Yahoo as a search engine, and puts it in the News and Media category instead. That’s why, if you check the top search engines in the US, you won’t find Yahoo listed among them, but you’d be surprised to find Baidoo.com ranking number five. That was quite a gem for me to discover even though the Chinese-speaking population in the US is remarkably high. This may be something digital marketers should pay close attention to, especially if they work for big international companies.

Another finding that really left me clueless is that the least popular Russian email agent, Mail.ru, appears to be among the top industry players — and Yahoo’s email agent still wasn’t there.

Google, Yahoo, and Bing are the most visited search engine sites in the US

Before I even started sifting through the data I gathered, I confidently assumed that I’d find Bing in second place. Turns out, the second-most visited search engine site is Yahoo.com.

So, does this mean that Yahoo is used more actively by online surfers than Bing? If you base your answer solely on the collected data, then the answer to the question is yes. But it’s not that simple.

Yes, it’s true that users visit Yahoo more frequently than they do Bing, but that’s not because they want to search for something on Yahoo. First of all, there's a large group of people still using YahooMail (even though it’s 2017), and some people simply prefer checking for weather updates and news reports on Yahoo. With that being said, if we look at ComScore’s latest search engine popularity report, we will find that Yahoo is used as a search engine by 12.2% of all online US traffic and Microsoft is popular among 21.4%. But, realistically, Yahoo’s share of the search market is even smaller, since the majority of their search results are powered by Bing.

So if you’re considering Yahoo as a platform for promoting your product or service, check the demographics data around what kind of businesses typically advertise on Yahoo.

Speaking of demographic insights, I was struggling to find fresh ComScore reports (the last one was released more than three years ago), so I had to use Alexa.com. This isn’t the best and most accurate tool because the company gathers data from its own SEO toolbar, but it’s better than nothing.

Here’s what Alexa.com reports about Yahoo’s user demographics:

  • There are more female users than male
  • Most users are college-aged
  • Following the previously mentioned trend, the top browser location is a school or a college

In order to determine which industries are advertising on Yahoo, I used Yahoo Ad Insights’ Industries report, which includes such businesses as:

  • Automotive
  • Consumer Packaged Goods (CPG)
  • Entertainment
  • Finance
  • Retail
  • Tech and TELCO
  • Travel

And here I stumbled upon another controversial fact. Data from Alexa.com shows that the dominating age group consists of students who, in my opinion and judging from my own experience, can barely afford products that fall into industries like Finance or Retail. If you happen to have experience using Yahoo for advertising, I’d love to hear your thoughts.

Bing owns 0.48% of all traffic and 30% of the search engine market in the US

If we compare Bing with Yahoo, the former gets 3.35 times less traffic than Yahoo does. But as we have just discovered, Bing gets two times more search market queries compared to Yahoo. This means that it provides a lot more advertising opportunities for businesses. Also, the majority of Yahoo searches are powered by Bing, which means that once you’re ranking well in Bing, you automatically become visible in Yahoo.

All in all, Bing can boost traffic to your business by 30% and you don’t even need to invest in a new market or launch a new product or service. There’s no doubt you'll need to put some effort into that process, but if you currently have a steady traffic flow from Google, then you’re already receiving visitors from Bing as well. You just need to analyze what exactly is going on with your Bing traffic, and find the right ways to take advantage of it. Here’s a great read supported with a video by John Lincoln who talks about SEO for Bing.

If you’re still not sure whether you should care about traffic coming to your site from Bing, here’s a great example. Searchengineland.com receives a little over 10% of Bing.com’s one million organic traffic visitors on a monthly basis:

Arts and Entertainment

As I’ve previously mentioned, this category ranks second and owns over 12% of all US traffic, all thanks to YouTube. Also if we look at the top industry domains, we’ll find that Netflix gets 5.67% of all traffic in this category. I find it interesting that organic traffic isn’t the top referral traffic source for Netflix. Those would be direct (58.54%) and referrals (23.59%). Obviously, you can tell which of the media streaming platforms — YouTube or Netflix — Google gives its royal preference to. It kind of makes sense because all of Netflix’s content is on-demand.

The graph below demonstrates that YouTube gets three times more organic traffic than Netflix:

Digging deeper, we learn Google can't list Netflix’s content in a video featured snippet because Netflix is only accessible with a paid subscription. In some way, Netflix is cornering itself.

The screenshot below shows that Netflix does have visibility in SERPs via the Knowledge Graph, but it’s not getting any traffic from this ranking because the Knowledge Graph doesn’t feature a link to a domain.

The Music and Audio subcategory has its own peculiar numbers. I was surprised to see Pandora as a leader, ahead with two times more traffic and leaving Spotify with only 3.68 percent.

The pie chart below gives you a breakdown of the traffic distribution for other subcategories:

YouTube sends the majority of its traffic to Gambling sites

SimilarWeb shows that somewhere around 5% of all YouTube ad traffic is sent to Bet365.com, one of the largest gambling websites. Using SEMrush, I also checked the list of sites that get the most visitors from YouTube, and I found out that among the top three sites there’s another gambling site: Freelotto.com:

It’s safe to say that if you have a business in the entertainment industry, you should definitely consider YouTube as one of your traffic sources.

News and Media

Findings from the data collected confirm that people still read newspapers online and check them for weather updates instead of checking their phone apps.

My husband reads the news on his laptop during breakfast. Yet it still drives him nuts when I ask him to check for weather updates for me, despite my having all kinds of gadgets. Oh, well — guess old habits die hard. But it looks like I'm not alone in this world, because the majority of users have the same habit:

In case you've been wondering what the "Other" category stands for in this graph, here’s what it means. Currently, SimilarWeb hasn’t come up with a way to categorize those websites — that’s why it has the highest percentage. But among the most popular sites I found two prominent newspapers: Dailymail.co.uk and Theguardian.com.

Take a look at the screenshot above. Both The New York Times and Washington Post are among the top 5 sites in Newspaper subcategory.

The screenshot below demonstrates top 5 countries that bring traffic to Dailymail.co.uk. As you can see, there’s more traffic coming from the US than from anywhere else in the world:

It's something to keep in mind if you’re searching for the most popular US newspapers online.

Shopping

Online shopping is an integral part of the e-commerce industry, which is, in fact, one of the fastest-growing markets in the US. In the past few years, the e-commerce share of the overall US retail market has grown from 6.6% in 2014 to 8.5% in Q1 2017. However, even though most retail purchases are made online, there’s a big group of people who are inspired to purchase a product offline after visiting a website. Statista reports that the number of web-influenced offline retail sales is 20% higher than non-web influenced sales. This means that for physical stores that don’t have an online representation, establishing their web presence is a must because the conversion process in most cases starts online.

There’s a 74% chance it will either be Amazon or eBay

The subcategory of Shopping called “General Merchandise” accounts for over 60% of web traffic, and is owned by Amazon (51.24%) and eBay (22.01%). The rest of the subcategories can be found in the pie chart below:

When shopping for goods in the Home and Garden category, North American users most likely check Homedepot.com, which gets 20.29% of all traffic in this subcategory, or Lowes.com, which is a go-to place for 10.55% of all users. Interesting fact: the traffic source that drives the most visitors to both sites is organic search results, which brings over 40% of monthly visitors.

Computer and Electronics

The data confirms that Microsoft has more monthly online visitors than Apple. Microsoft’s traffic share is a little over 15%, with Apple being left behind with only 3.28%. However, this doesn’t affect Apple’s sales at all, and it serves as proof of the fact that investing in your brand authority and focusing more on the quality of your product will make you stand out.

Based on R&P Research, Apple net profits surpassed those of Microsoft in 2011. Apple made $25.9 billion in net profits in 2011, and Microsoft saw $23.2 billion. From then on, Apple has outplayed Microsoft in acquired net profits:

If you’d like to dive deeper and learn more about how traffic is distributed across Shopping subcategories, then take a look at the graph below:

Reference category

I’m sure it’s not news to you if I tell you that Wikipedia’s traffic share in the Reference category is 44.55%. When it comes to subcategories, directories such as Yelp, Yellowpages, and Whitepages get over 85% of Internet traffic. It’s funny how I, as a teenager growing up in Russia, used to flip through the Yellow Pages — one of the most popular print directories for finding various companies. Any time I needed to find a store, I’d open up this book and navigate my finger through finely printed lines of text.

Now, you can only find hard copies of the Yellow Pages gathering dust somewhere in a small-town office.

The pie chart below gives a more detailed overview of how traffic is distributed across all the subcategories:

Wikipedia can bring you relevant users from search

Wikipedia.org has over 4.7 billion monthly visitors, and 86 percent of those visitors come from organic search. You should definitely see Wikipedia not only as an authoritative source with high-quality links, but also as a traffic generation channel.

For instance, according to SEMrush’s Traffic Analytics tool, SEJ receives more than 300 visitors from Wikipedia on a monthly basis:

Wikipedia is the best option for well-established businesses that really want to increase their online traffic, but suffer from an obnoxiously high level of competition in Google. To make this happen, your business has to have enough authority on the web; otherwise it will take forever. Prior to suggesting that experts link to your content, you have to make sure your brand is recognized. The type of content you want to end up under the “References” section on Wikipedia should be of the exact same quality as everything you read on that website.

Pay attention to the visibility of Wikipedia's pages in SERPs for a keyword you’re targeting

To check that, go to SEMrush and check the domain for keywords:

You can also type in the following query to Google:

site:wikipedia.org your keyword + “dead link”

This will show you all articles on the web with dead links. If you’re looking to learn more about how Wikipedia can help you with your SEO efforts, here’s a post that I’ve recently come across that has tons of actionable advice.

Business Industry

In this category, the largest proportion of traffic is divided between Zillow.com (3.65%), USPS.com (2.50%), and UPS.com (1.69%).

Marketing, Advertising, and E-commerce have the smallest share compared to other subcategories:

Moving further, while looking at the leading sites in Marketing and Advertising, I found that advertising networks are getting the highest number of visitors. Among those sites are Dotomi.com (2.45%), Traffichaus (2.60%), and Innovid (2.63%). In addition, VigLink recently published a study in which they confirm that the demand for ad network is constantly growing, and advertisers are looking to connect with publishers and take advantage of affiliate marketing traffic.

Career and Education

In this niche, Indeed.com and Instructure.com attract the majority of visitors. The latter is an Ed Tech company which develops educational software; the majority of its traffic comes from referrals (61.7%).

The Universities and Colleges subcategory, along with listing Ivy League universities, mentions Purdue University, which, for some reason, happens to rank only 92 in the QS World University Rankings for 2016–2017, but is number 13 in terms of traffic.

We wanted to see which channel brings the most traffic to the world famous universities (ranked by the QS World Universities) compared to Purdue University, and find out the reasons for success in getting online traffic for both Purdue and other world-renowned universities.

All the universities in the screenshot above rank the highest, even though Purdue University is only number 13 when it comes to online traffic. Yet the screenshot clarifies a lot; Purdue University receives much of its traffic from organic search, which contributes greatly to its online visibility.

The is the first industry in which I’ve noticed traffic being distributed equally among all subcategories:

Travel

The travel business is extremely competitive; however, it is made up of a large diversity of small- and medium-sized players, because the top industry domains only have a little over 17% of the total market share. However, a subcategory such as Airlines and Airports has a few major players that get the majority of visitors 45% of the traffic (in the travel industry), shared among the following businesses:

  • Southwest.com – 14.74%
  • American Airlines – 10.68%
  • Delta – 10.78%
  • United Airlines – 9.12%
  • JetBlue – 4.5%

If you look at the graph below, which shows the traffic distribution for the different subcategories, you’ll see that, in general, traffic within the Hotel and Accommodation sector is higher than for airline- and airport-dedicated websites.

The reason for this might be because one of the most common means of traveling in the US is by car.

Speaking of general marketing trends in the travel industry, one of the top sources that brings traffic to that niche is affiliate marketing. For instance, Kayak.com is one of the biggest affiliates of Southwest.com, bringing over 15,000 visitors on a monthly basis:

If you’re interested in learning more about the current state of affiliate marketing in the travel industry, we’ve recently conducted a comprehensive study. We analyzed the top affiliate programs and sites using the SEMrush Traffic Analytics report. We also asked 50 well-known affiliate marketing experts about general affiliate marketing trends and incorporated their answers into our research.

Final thoughts

As always, it flatters me that you've taken the time to familiarize yourself with results of my hard work, and I hope that you now have a good understanding of the current state of traffic distribution across the most popular US industries. Before I began my research, I thought that a good portion of all Internet traffic was controlled by Google, Facebook, YouTube, etc. But it turned out that the top five most visited sites only get a little over 30% of all US traffic. On the other hand, findings from this data prove once again that establishing your business on Facebook, YouTube, and Google is essential to its long-term success.

As for the industry traffic distribution across top-visited domains, what springs to mind is Maslow’s hierarchy of needs. Categories like Adult Entertainment, Internet and Telecom, Shopping, and News and Media mostly serve our basic psychological needs. And moving down the list of industries, you’ll find that Business and Industry, Reference, Career and Education, and Travel get less searches because, apparently, not that many people nowadays have time to take care of their self-actualization needs.

As always, I’d love to hear your thoughts about my research as well as any ideas on what I could have covered but didn’t. Let me know if you were able to put any of the aforementioned conclusions into practice.


Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!

Half of Page-1 Google Results Are Now HTTPS

Posted by Dr-Pete

Just over 9 months ago, I wrote that 30% of page-1 Google results in our 10,000-keyword tracking set were secure (HTTPS). As of earlier this week, that number topped 50%:

While there haven't been any big jumps recently – suggesting this change is due to steady adoption of HTTPS and not a major algorithm update – the end result of a year of small changes is dramatic. More and more Google results are secure.

MozCast is, of course, just one data set, so I asked the folks at Rank Ranger, who operate a similar (but entirely different) tracking system, if they thought I was crazy...

Could we both be crazy? Absolutely. However, we operate completely independent systems with no shared data, so I think the consistency in these numbers suggests that we're not wildly off.

What about the future?

Projecting the fairly stable trend line forward, the data suggests that HTTPS could hit about 65% of page-1 results by the end of 2017. The trend line is, of course, an educated guess at best, and many events could change the adoption rate of HTTPS pages.

I've speculated previously that, as the adoption rate increased, Google would have more freedom to bump up the algorithmic (i.e. ranking) boost for HTTPS pages. I asked Gary Illyes if such a plan was in the works, and he said "no":

As with any Google statement, some of you will take this as gospel truth and some will take it as devilish lies. While he isn't promising that Google will never boost the ranking benefits of HTTPS, I believe Gary on this one. I think Google is happy with the current adoption rate and wary of the collateral damage that an aggressive HTTPS ranking boost (or penalty) could cause. It makes sense that they would bide their time..

Who hasn't converted?

One of the reasons Google may be proceeding with caution on another HTTPS boost (or penalty) is that not all of the big players have made the switch. Here are the Top 20 subdomains in the MozCast dataset, along with the percentage of ranking URLs that use HTTPS:

(1) en.wikipedia.org — 100.0%
(2) www. amazon.com — 99.9%
(3) www. facebook.com — 100.0%
(4) www. yelp.com — 99.7%
(5) www. youtube.com — 99.6%
(6) www. pinterest.com — 100.0%
(7) www. walmart.com — 100.0%
(8) www. tripadvisor.com — 99.7%
(9) www. webmd.com — 0.2%
(10) allrecipes.com — 0.0%
(11) www. target.com — 0.0%
(12) www. foodnetwork.com — 0.0%
(13) www. ebay.com — 0.0%
(14) play.google.com — 100.0%
(15) www. bestbuy.com — 0.0%
(16) www. mayoclinic.org — 0.0%
(17) www. homedepot.com — 0.0%
(18) www. indeed.com — 0.0%
(19) www. zillow.com — 100.0%
(20) shop.nordstrom.com – 0.0%

Of the Top 20, exactly half have switched to HTTPS, although most of the Top 10 have converted. Not surprisingly, switching is, with only minor exceptions, nearly all-or-none. Most sites naturally opt for a site-wide switch, at least after initial testing.

What should you do?

Even if Google doesn't turn up the reward or penalty for HTTPS, other changes are in play, such as Chrome warning visitors about non-secure pages when those pages collect sensitive data. As the adoption rate increases, you can expect pressure to switch to increase.

For new sites, I'd recommend jumping in as soon as possible. Security certificates are inexpensive these days (some are free), and the risks are low. For existing sites, it's a lot tougher. Any site-wide change carries risks, and there have certainly been a few horror stories this past year. At minimum, make sure to secure pages that collect sensitive information or process transactions, and keep your eyes open for more changes.


Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!

How To Support Data with Real-Life Interviews – Whiteboard Friday

Posted by rcancino

With all the data that today's marketers can access, there's often still no substitute for the quality of information you can get from interviewing real people. In today's Whiteboard Friday, we welcome Rebekah Cancino -- a partner at Phoenix-based Onward and #MozCon 2016 speaker -- to teach us the whys and hows of great interviews.


Click on the whiteboard image above to open a high resolution version in a new tab!

Video Transcription

Hi, Moz fans. I'm Rebekah Cancino. I'm a partner at Onward, and I lead content strategy and user experience design. Today I'm here to talk to you about how to support the data you have, your keyword data, data around search intent, analytics with real life user interviews.

So recently, Rand has been talking a little more about the relationship between user experience design and SEO, whether it's managing the tensions between the two or the importance of understanding the path to customer purchase. He said that in order to understand that path, we have to talk to real people. We have to do interviews, whether that's talking to actual users or maybe just people inside your company that have an understanding of the psychographics and the demographics of your target audience, so people like sales folks or customer service reps.

Now, maybe you're a super data-driven marketer and you haven't felt the need to talk to real people and do interviews in the past, or maybe you have done user interviews and you found that you got a bunch of obvious insights and it was a huge waste of time and money.

I'm here to tell you that coupling your data with real interviews is always going to give you better results. But having interviews that are useful can be a little bit tricky. The interviews that you do are only as good as the questions you ask and the approach that you take. So I want to make sure that you're all set and prepared to have really good user interviews. All it takes is a little practice and preparation.

It's helpful to think of it like this. So the data is kind of telling us what happened. It can tell us about online behaviors, things like keywords, keyword volume, search intent. We can use tools, like KeywordTool.io or Ubersuggest or even Moz's Keyword Explorer, to start to understand that.

We can look at our analytics, entry and exit pages, bounces, pages that get a lot of views, all of that stuff really important and we can learn a lot from it. But with our interviews, what we're learning about is the why.

This is the stuff that online data just can't tell us. This is about those offline behaviors, the emotions, beliefs, attitudes that drive the behaviors and ultimately the purchase decisions. So these two things working together can help us get a really great picture of the whole story and make smarter decisions.

So say, for example, you have an online retailer. They sell mainly chocolate-dipped berries. They've done their homework. They've seen that most of the keywords people are using tend to be something like "chocolate dipped strawberries gifts" or "chocolate dipped strawberries delivered." And they've done the work to make sure that they've done their on-page optimization and doing a lot of other smart things too using that.

But then they also noticed that their Mother's Day packages and their graduation gifts are not doing so well. They're starting to see a lot of drop-offs around that product description page and a higher cart abandonment rate than usual.

Now, given the data they had, they might make decisions like, "Well, let's see if we can do a little more on-page keyword optimization to reflect what's special about the graduation and Mother's Day gifts, or maybe we can refine the user experience of the checkout process. But if they talk to some real users -- which they did, this is a real story -- they might learn that people who send food gift items, they worry about: Is the person I'm sending the gift to, are they going to be home when this gift arrives? Because this is a perishable item, like chocolate-dipped berries, will it melt?

Now, this company, they do a lot of work to protect the berries. The box that they arrive in is super insulated. It's like its own cooler. They have really great content that tells that story. The problem is that content is buried in the FAQs instead of on the pages in places it matters most -- the product detail, the checkout flow.

So you can see here how there's an opportunity to use the data and the interview insights together to make smarter decisions. You can get to insights like that for your organization too. Let's talk about some tips that are going to help you make smarter interview decisions.

So the first one is to talk to a spectrum of users who represent your ideal audience. Maybe, like with this berry example, their ideal customer tends to skew slightly female. You would want that group of people, that you're talking to, to skew that way too. Perhaps they have a little more disposable income. That should be reflected in the group of people that you're interviewing and so forth. You get it.

The next one is to ask day-in-the-life, open-ended questions. This is really important. If you ask typical marketing questions like, "How likely are you to do this or that?" or, "Tell me on a scale of 1 to 10 how great this was," you'll get typical marketing answers. What we want is real nuanced answers that tell us about someone's real experience.

So I'll ask questions like, "Tell me about the last time you bought a food gift online? What was that like?" We're trying to get that person to walk us through their journey from the minute they're considering something to how they vet the solutions to actually making that purchase decision.

Next is don't influence the answers. You don't want to bias someone's response by introducing an idea. So I wouldn't say something like, "Tell me about the last time you bought a food gift online. Were you worried that it would spoil?" Now I've set them on a path that maybe they wouldn't have gone on to begin with. It's much better to let that story unfold naturally.

Moving on, dig deeper. Uncover the why, really important. Maybe when you're talking to people you realize that they like to cook and by sharing a food item gift with someone who's far away, they can feel closer to them. Maybe they like gifts to reflect how thoughtful they are or what good tastes they have. You always want to uncover the underlying motives behind the actions people are taking.

So don't be too rushed in skipping to the next question. If you hear something that's a little bit vague or maybe you see a point that's interesting, follow up with some probes. Ask things like, "Tell me more about that," or, "Why is that? What did you like about it?" and so on.

Next, listen more than you talk. You have maybe 30 to 45 minutes max with each one of these interviews. You don't want to waste time by inserting yourself into their story. If that happens, it's cool, totally natural. Just find a way to back yourself out of that and bring the focus back to the person you're interviewing as quickly and naturally as possible.

Take note of phrases and words that they use. Do they say things like "dipped berries" instead of "chocolate-dipped strawberries?" You want to pay attention to the different ways and phrases that they use. Are there regional differences? What kinds of words do they use to describe your product or service or experience? Are the berries fun, decadent, luxurious? By learning what kind of language and vocabulary people use, you can have copy, meta descriptions, emails that take that into account and reflect that.

Find the friction. So in every experience that we have, there's always something that's kind of challenging. We want to get to the bottom of that with our users so we can find ways to mitigate that point of friction earlier on in the journey. So I might ask someone a question like, "What's the most challenging thing about the last time you bought a food gift?"

If that doesn't kind of spark an idea with them, I might say something even a little more broad, like, "Tell me about a time you were really disappointed in a gift that you bought or a food gift that you bought," and see where that takes them.

Be prepared. Great interviews don't happen by accident. Coming up with all these questions takes time and preparation. You want to put a lot of thought into them. By asking questions that tell me about the nature of the whole journey, you want to be clear about your priorities. Know which questions are most important to you and know which ones are must have pieces of information. That way you can use your time wisely while you still let the conversation flow where it takes you.

Finally, relax and breathe. The people you're interviewing are only going to be as relaxed as you are. If you're stiff or overly formal or treating this like it's a chore and you're bored, they're going to pick up on that energy and they're probably not going to feel comfortable sharing their thoughts with you, or there won't be space for that to happen.

Make sure you let them know ahead of time, like, "Hey, feel free to be honest. These answers aren't going to be shared in a way that can be attributed directly to you, just an aggregate."

And have fun with it. Be genuinely curious and excited about what you're going to learn. They'll appreciate that too.

So once you've kind of finished and you've wrapped up those interviews, take a step back. Don't get too focused or caught up on just one of the results. You want to kind of look at the data in aggregate, the qualitative data and let it talk to you.

What stories are there? Are you seeing any patterns or themes that you can take note of, kind of like the theme around people being worried about the berries melting? Then you can organize those findings and make sure you summarize it and synthesize it in a way that the people who have to use those insights that you've gotten can make sense of.

Make sure that you tell real stories and humanize this information. Maybe you recorded the interviews, which is always a really good idea. You can go back and pull out little sound bites or clips of the people saying these really impactful things and use that when you're presenting the data.

So going back to that berry example, if you recall, we had that data around: Hey, we're seeing a lot of drop-offs on the product description page. We're seeing a higher cart abandonment rate. But maybe during the user interviews, we noticed a theme of people talking about how they obsessively click the tracking link on the packages, or they wait for those gift recipients to send them a text message to say, "Hey, I got this present." As you kind of unraveled why, you noticed that it had to do with the fact that these berries might melt and they're worried about that.

Well, now you can elevate the content that you have around how those berries are protected in a little cooler-like box on the pages and the places it matters most. So maybe there's a video or an animated GIF that shows people how the berries are protected, right there in the checkout flow.

I hope that this encourages you to get out there and talk to real users, find out about their context and use that information to really elevate your search data. It's not about having a big sample size or a huge survey. It's much more about getting to real life experiences around your product or service that adds depth to the data that you have. In doing that, hopefully you'll be able to increase some conversions and maybe even improve behavioral metrics, so those UX metrics that, I don't know, theoretically could lead to higher organic visibility anyway.

That's all for now. Thanks so much. Take care.

Video transcription by Speechpad.com


Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!

Introducing Progressive Web Apps: What They Might Mean for Your Website and SEO

Posted by petewailes

Progressive Web Apps. Ah yes, those things that Google would have you believe are a combination of Ghandi and Dumbledore, come to save the world from the terror that is the Painfully Slow WebsiteTM.

But what actually makes a PWA? Should you have one? And if you create one, how will you make sure it ranks? Well, read on to find out...

What's a PWA?

Given as that Google came up with the term, I thought we'd kick off with their definition:

"A Progressive Web App uses modern web capabilities to deliver an app-like user experience."
Progressive Web Apps

The really exciting thing about PWAs: they could make app development less necessary. Your mobile website becomes your app. Speaking to some of my colleagues at Builtvisible, this seemed to be a point of interesting discussion: do brands need an app and a website, or a PWA?

Fleshing this out a little, this means we'd expect things like push notifications, background sync, the site/app working offline, having a certain look/design to feel like a native application, and being able to be set on the device home screen.

These are things we traditionally haven't had available to us on the web. But thanks to new browsers supporting more and more of the HTML5 spec and advances in JavaScript, we can start to create some of this functionality. On the whole, Progressive Web Apps are:

Progressive
Work for every user, regardless of browser choice because they're built with progressive enhancement as a core tenet.
Responsive
Fit any form factor: desktop, mobile, tablet, or whatever is next.
Connectivity independent
Enhanced with service workers to work offline or on low quality networks.
App-like
Feel like an app to the user with app-style interactions and navigation because they're built on the app shell model.
Fresh
Always up-to-date thanks to the service worker update process.
Safe
Served via HTTPS to prevent snooping and ensure content hasn't been tampered with.
Discoverable
Are identifiable as "applications" thanks to W3C manifests and service worker registration scope allowing search engines to find them.
Re-engageable
Make re-engagement easy through features like push notifications.
Installable
Allow users to "keep" apps they find most useful on their home screen without the hassle of an app store.
Linkable
Easily share via URL and not require complex installation.
Source: Your First Progressive Web App (Google)

It's worth taking a moment to unpack the "app-like" part of that. Fundamentally, there are two parts to a PWA: service workers (which we'll come to in a minute), and application shell architecture. Google defines this as:

...the minimal HTML, CSS, and JavaScript powering a user interface. The application shell should:
  • load fast
  • be cached
  • dynamically display content
An application shell is the secret to reliably good performance. Think of your app's shell like the bundle of code you'd publish to an app store if you were building a native app. It's the load needed to get off the ground, but might not be the whole story. It keeps your UI local and pulls in content dynamically through an API.
Instant Loading Web Apps with an Application Shell Architecture

This method of loading content allows for incredibly fast perceived speed. We are able to get something that looks like our site in front of a user almost instantly, just without any content. The page will then go and fetch the content and all's well. Obviously, if we actually did things this way in the real world, we'd run in to SEO issues pretty quickly, but we'll address that later too.

If then, at their core, a Progressive Web App is just a website served in a clever way with extra features for loading stuff, why would we want one?

The use case

Let me be clear before I get into this: for most people, a PWA is something you don't need. That's important enough that it bares repeating, so I'll repeat it:

You probably don't need a PWA.

The reason for this is that most websites don't need to be able to behave like an app. This isn't to say that there's no benefit to having the things that PWA functionality can bring, but for many sites, the benefits don't outweigh the time it takes to implement the functionality at the moment.

When should you look at a PWA then? Well, let's look at a checklist of things that may indicate that you do need one...

Signs a PWA may be appropriate

You have:

  • Content that regularly updates, such as stock tickers, rapidly changing prices or inventory levels, or other real-time data
  • A chat or comms platform, requiring real-time updates and push notifications for new items coming in
  • An audience likely to pull data and then browse it offline, such as a news app or a blog publishing many articles a day
  • A site with regularly updated content which users may check in to several times a day
  • Users who are mostly using a supported browser

In short, you have something beyond a normal website, with interactive or time-sensitive components, or rapidly released or updated content. A good example is the Google Weather PWA:

If you're running a normal site, with a blog that maybe updates every day or two, or even less frequently, then whilst it might be nice to have a site that acts as a PWA, there's probably more useful things you can be doing with your time for your business.

How they work

So, you have something that would benefit from this sort of functionality, but need to know how these things work. Welcome to the wonder that is the service worker.

Service workers can be thought of as a proxy that sits between your website and the browser. It calls for intercept of things you ask the browser to do, and hijacking of the responses given back. That means we can do things like, for example, hold a copy of data requested, so when it's asked for again, we can serve it straight back (this is called caching). This means we can fetch data once, then replay it a thousand times without having to fetch it again. Think of it like a musician recording an album — it means they don't have to play a concert every time you want to listen to their music. Same thing, but with network data.

If you want a more thorough explanation of service workers, check out this moderately technical talk given by Jake Archibald from Google.

What service workers can do

Service workers fundamentally exist to deliver extra features, which have not been available to browsers until now. These includes things like:

  • Push notifications, for telling a user that something has happened, such as receiving a new message, or that the page they're viewing has been updated
  • Background sync, for updating data while a user isn't using the page/site
  • Offline caching, to allow a for an experience where a user still may be able to access some functionality of a site while offline
  • Handling geolocation or other device hardware-querying data (such as device gyrpscope data)
  • Pre-fetching data a user will soon require, such as images further down a page

It's planned that in the future, they'll be able to do even more than they currently can. For now though, these are the sorts of features you'll be able to make use of. Obviously these mostly load data via AJAX, once the app is already loaded.

What are the SEO implications?

So you're sold on Progressive Web Apps. But if you create one, how will you make sure it ranks? As with any new front-end technology, there are always implications for your SEO visibility. But don't panic; the potential issues you'll encounter with a PWA have been solved before by SEOs who have worked on JavaScript-heavy websites. For a primer on that, take a look at this article on JS SEO.

There are a few issues you may encounter if you're going to have a site that makes use of application shell architecture. Firstly, it's pretty much required that you're going to be using some form of JS framework or view library, like Angular or React. If this is the case, you're going to want to take a look at some Angular.JS or React SEO advice. If you're using something else, the short version is you'll need to be pre-rendering pages on the server, then picking up with your application when it's loaded. This enables you to have all the good things these tools give you, whilst also serving something Google et al can understand. Despite their recent advice that they're getting good at rendering this sort of application, we still see plenty of examples in the wild of them flailing horribly when they crawl heavy JS stuff.

Assuming you're in the world of clever JS front-end technologies, to make sure you do things the PWA way, you'll also need to be delivering the CSS and JS required to make the page work along with the HTML. Not just including script tags with the <code>src attribute, but the whole file, inline.

Obviously, this means you're going to increase the size of the page you're sending down the wire, but it has the upside of meaning that the page will load instantly. More than that, though, with all the JS (required for pick-up) and CSS (required to make sense of the design) delivered immediately, the browser will be able to render your content and deliver something that looks correct and works straightaway.

Again, as we're going to be using service workers to cache content once it's arrived, this shouldn't have too much of an impact. We can also cache all the CSS and JS external files required separately, and load them from the cache store rather than fetching them every time. This does make it very slightly more likely that the PWA will fail on the first time that a user tries to request your site, but you can still handle this case gracefully with an error message or default content, and re-try on the next page view.

There are other potential issues people can run in to, as well. The Washington Post, for example, built a PWA version of their site, but it only works on a mobile device. Obviously, that means the site can be crawled nicely by Google's mobile bots, but not the desktop ones. It's important to respect the P part of the acronym — the website should enable features that a user can make use of, but still work in a normal manner for those who are using browsers that don't support them. It's about enhancing functionality progressively, not demanding that people upgrade their browser.

The only slightly tricky thing with all of this is that it requires that, for best experience, you design your application for offline-first experiences. How that's done is referenced in Jake's talk above. The only issue with going down that route: you're only serving content once someone's arrived at your site and waited long enough to load everything. Obviously, in the case of Google, that's not going to work well. So here's what I'd suggest...

Rather than just sending your application shell, and then using AJAX to request content on load, and then picking up, use this workflow instead:

  • User arrives at site
  • Site sends back the application shell (the minimum HTML, JS, and CSS to make everything work immediately), along with...
  • ...the content AJAX response, pre-loaded as state for the application
  • The application loads that immediately, and then picks up the front end.

Adding in the data required means that, on load, we don't have to make an AJAX call to get the initial data required. Instead, we can bundle that in too, so we get something that can render content instantly as well.

As an example of this, let's think of a weather app. Now, the basic model would be that we send the user all the content to show a basic version of our app, but not the data to say what the weather is. In this modified version, we also send along what today's weather is, but for any subsequent data request, we then go to the server with an AJAX call.

This means we still deliver content that Google et al can index, without possible issues from our AJAX calls failing. From Google and the user's perspective, we're just delivering a very high-performance initial load, then registering service workers to give faster experiences for every subsequent page and possibly extra functionality. In the case of a weather app, that might mean pre-fetching tomorrow's weather each day at midnight, or notifying the user if it's going to rain, for example.

Going further

If you're interested in learning more about PWAs, I highly recommend reading this guide to PWAs by Addy Osmani (a Google Chrome engineer), and then putting together a very basic working example, like the train one Jake mentions in his YouTube talk referenced earlier. If you're interested in that, I recommend Jake's Udacity course on creating a PWA available here.


Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!