How 400+ beta readers helped us write our best content (PS: you can do the same)

Some background: Why we wrote a SaaS guide in the first place

The XAwards: the 2-day event that inspired the SaaS guide

On May 11-12, Hotjar hosted a 2-day event for early-stage startups in Malta called the 'XAwards’. The organizers, David (Founder and CEO) and Marc (Co-founder), had worked for months selecting 18 participant teams from over 600 who had applied, crafting an agenda of panel discussions and workshops, and securing speakers from companies like Basecamp, Receptive, Pipedrive, Price Intelligently, and SaaStock. Covering the main challenges faced by a young SaaS startup (how to go about pricing, funding, launching, etc.) these sessions were the catalyst for what would eventually become our SaaS guide.

THE XAWARDS PANELS AND VENUE, MALTA

Hotjar gets its first content team

Back then, Hotjar did not have a content team. A few people over the years had published occasional blog posts, some of which had even managed to get some traction, but there were no full-time resources or an overall strategy dedicated to content.

Luckily (for me!), things were changing: I joined Hotjar on May 1st, as their new Content Marketer, and was promptly flown to Malta to attend the XAwards so I could later write a conference round-up. At the end of the month, Content Strategist Louis Grenier joined Hotjar as well.

THE HOTJAR CONTENT TEAM IN 2017: FIO DOSSETTO AND LOUIS GRENIER

THE CONTENT TEAM MEETING IN PERSON AT OUR MARBELLA RETREAT IN JUNE 2017: ME AND LOUIS (LEFT), THE COMPANY'S INFLATABLE UNICORN (RIGHT)

We were now a brand-new 2-person content team who needed to find their feet and direction, build and enact a vision of what content could do for the company, and find the most productive way to use all the material from the XAwards.

Identifying our first content opportunity

The XAwards attendees had been struggling to find a go-to resource on how to launch and grow their business: they had told us so both during the event and in their feedback forms afterwards. A quick round of Googling and checking in with people in our network confirmed that there was indeed a gap in the market for a comprehensive resource aimed at early-stage startups—and since each XAwards session had been recorded, we owned a wealth of fresh, original, authoritative audio/visual content that could fill it.

We discarded the idea of writing a conference round-up and publishing all the videos on one page (you can see that original idea in the image below): we did not think that people would really want to sit through 10+ hours of content, however relevant. Instead, we opted for creating a central round-up page that would link to separate articles, each using a different video as its starting point.

THE VERY FIRST IDEA WE HAD (AND QUICKLY DISMISSED): A ONE-PAGE ROUND-UP POST

June/July: drafts 1 to 4

Being relatively new to the SaaS world, I spent the research phase learning about the most common problems in the life of early-stage SaaS startups, and wrote the first two drafts throughout June. This is what the front page of that Google Doc looked like back then:

NOTE THE TITLE AND IMAGE CAPTION - IT WAS A REAL WORK IN PROGRESS!

At the end of June, Louis and I decided to re-position the piece as a multi-chapter guide and abandon the link to the XAwards. In our mind, this would help us reach a much larger audience—not just those who had participated in the event and/or knew about Hotjar, but anyone in SaaS at the early-startup stage, regardless of their interest in our company.

By mid-July we had a third draft of about 35 Google Docs pages that we shared with the XAwards speakers and a few SaaS experts in our network; following their feedback, I was back at work on draft 4, which took another week and moved us from 35 to 49 pages.

Launching the beta reader concept

The a-ha! moment of inspiration from a Hotjar customer

While all this re-writing was going on, one of Hotjar’s customers, Chris, reached out to let us know he had been experimenting with our (then) newest tool, Incoming Feedback, to get content-specific feedback on a website. Incoming Feedback allows people to take screenshots of a specific portion of a page and leave their comments and feedback about it, and Chris was reporting back that his team was receiving multiple responses from readers on a regular basis, and getting a wealth of information into how their articles were performing.

INCOMING FEEDBACK'S SCREENSHOT-AND-COMMENT FEATURE

This made us think. We had previously used polls to find out what users wanted to read; we also regularly checked through comments and feedback on individual pieces to understand what people thought of them. But we had not thought about using our own tool like one of our customers was doing (ironic, I know)—inviting readers to send us screenshots of specific paragraphs, or even words, they loved or hated or did not understand.

We were excited to explore the potential of Incoming Feedback in relation to content, but chose to take a slightly different approach. Instead of using it on the published guide, we decided to see what would happen if we used a beta reader approach and got a panel of people to use Incoming Feedback on draft 4 before publishing the guide. The idea of beta readers had been championed by Louis during his interview for the job, and implementing it now would help us A) make sure the content would be truly helpful for its intended audience and B) get people involved at an earlier stage, so they could help us spread the word on go-live day.

So here is the process we used (which you are free to copy or adapt to your needs):

1. Identifying a potential audience and reaching out via email

Although the guide was directed at ‘anyone in SaaS at the early-startup stage’, we used a slightly wider selection criteria for our beta readers, retaining the SaaS element but allowing for people at a more advanced level of startup maturity—we figured they would have something useful to say from the vantage point of their experience. From our Hotjar user database, we selected:

• 5700+ Hotjar users in SaaS

• 630+ XAwards applicants

• 60 XAwards participants

and reached out to them with a few variations on the email you see below. We started from a relatively click-baity subject line, ‘A weird request’, to improve the chance that people would at least open the email and see what we were asking. It worked!

In the main body, we tried to delineate the benefits of becoming beta readers for the guide as clearly as possible. We knew we were asking a lot, and we were prepared to give something back in the form of an official acknowledgment in the guide:

2. Collecting beta reader details

The email took interested people to a simple and straightforward Hotjar survey, which would allow us to gather email addresses for the project:

We asked the question about job role because we wanted to make sure our readers would come from a mix of competences and backgrounds. The word-cloud visualization of the answers showed us that yes, we had indeed managed to get a good variety of roles represented:

HOTJAR'S SURVEY WORD-CLOUD GIVES VISUAL PROMINENCE TO THE MOST RECURRING WORD(S) IN THE SURVEY RESULT

3. Opening the draft to beta readers

As people kept signing up to be beta readers, we built a version of the guide on a private URL via Hubspot. At the end of the campaign, the 448 people who signed up were given a week to read through the content and provide their feedback.

On that page, we gave our beta readers clear instructions on what was expected of them. Because some of them would be using Incoming Feedback for the first time, we also made a quick video tutorial to show how to use it:

4. Watching the feedback come through

Over the following week, the beta readers used Incoming Feedback to highlight sections of the guide. I’m not going to lie: I kept checking every 4 or 5 hours just to see the number of comments go up. You can see a selection below organized from love to hate:

LOVING...

DISLIKING...

...AND HATING (OUCH)

Now, if there is one thing I'd like you to take away from this article, it's this:

5. Using the feedback to write a final draft

On August 13, at the end of the beta reader window, around 80 people had left 571 individual pieces of feedback to address. Some macro-trends were immediately obvious: most notably, readers were very eager for more visuals and video clips, as opposed to the full-length videos from the XAwards panels.

WE HEAR YOU, BETA READERS. SHORTER AND FEWER VIDEOS IT IS!

The rest of the feedback took me more than a week. Armed with multiple spreadsheets, I tackled each chapter separately and started sifting through the comments—first to judge whether they were relevant (and most of them were), then to decide whether they were worth implementing, and then to do the research and writing associated with each.

After dealing with all the feedback, and particularly with the requests for more tangible and practical examples, the final draft was 61 pages long. In other words, the beta readers’ comments and suggestions led to me extending the guide by an extra 25%.What would have happened had we not gone through this stage? Best-case scenario, we would have received disappointed comments telling us the published guide was just not that ready or useful; worst-case scenario, we would have completely lost out on our chance to become a trusted authority on the topic for our target readers.

I subjected the piece to a thorough round of copy-editing and requested a final round of internal feedback from the Hotjar team to settle on a title:

This was one of the few cases where, despite strong positive feedback from the beta readers, we went an alternative route. Instead of our (and their) favorite option, You have an MVP: Now what?, we signed off on The Essential Guide to Growing Your Early-Stage SaaS Startup.

Admittedly, it’s a bit of a mouthful and not even remotely as memorable, but we reasoned that it was the most descriptive and self-explanatory title for the completed piece, and might even have long-term SEO benefits.

Results & what’s next

The Essential Guide to Growing Your Early-Stage SaaS Startup went live on August 31. Having hundreds of beta readers who had invested in its creation allowed us to increase word of mouth quite quickly, so the guide made it to #1 on Product Hunt and reached about 15,000 pageviews over the first two days:

NOTE THAT OUR PAID MEDIA SPECIALIST NATALIA ALSO RAN ADS FOR THE GUIDE, SO THIS IS THE TOTAL TRAFFIC AFTER LAUNCH

The guide has since gone on to receive over 100k pageviews, it's ranking well SEO-wise on key terms like 'saas startup', 'saas guide', 'saas product market fit', 'how to launch a saas product', and has been mentioned in major newsletters (StartupWatching, FounderWeekly, The Digital Dub Newsletter, StartupFoundation, ShukiMannNewsletter). They may look like relatively small results to larger content teams out there, but Louis and I were happy with this as a result of our first joint project.

We were particularly happy with the beta reader approach and the results it yielded, to the point where we decided to re-organize our content process and make the beta readers part of it. This time, we picked 50k people from our database and sent them the following email:

There are currently 650+ people who signed up* and have so far been contacted 3-4 times. Most recently, they’ve pitched titles for a “Death by Best Practices” blog post, have reviewed a top-secret project before anybody else out there even knows about it, and we are planning on consulting with them again as we prepare to launch our 2018 content strategy update (we considered having beta-readers beta-read this post about beta-readers… but that felt a little too meta).

We’re hoping this approach will help us continue to produce content that is truly helpful and enjoyable for our readers. It also looks like this is becoming a more widespread approach to content creation, with Moz recently asking its readers to help update their Beginner’s Guide to SEO and Unbounce picking up on our efforts:

A TWEET FROM UNBOUNCE’S MARKETING MANAGER (THANKS, JENNIFER!)

We talk a lot about empathizing with users, and the beta readers program is currently allowing us to practice what we preach. Plus, it’s led us to grow the content team from 2 to almost 700 people! That’s a major win in our books.