August Update: What We're Doing About Bugs

Dmitriy Buterin 31 August 2017 2 comments

August Update“Why does it take so long to fix bugs in your software?”

That’s a very common question we get here. For this month’s update, I thought I’d talk about how bugs get into our software, the process we go through to get rid of them, and what we’re doing to get faster at fixing them.

 

What Is a Bug?

A bug is an error, flaw, or failure of any feature in our software. No software is without bugs, as they’re a natural part of creating and updating software. 

To give you an example of a bug, earlier this year some customers noticed they were unable to change slideshow links. This was not an intended function and we identified a bug that was causing this. We found a solution and released a fix for this bug in March.

 

Where Do Bugs Come From?

The functionality of Wild Apricot relies heavily on other products and technologies to function — mainly web browsers like Google Chrome, Internet Explorer, Safari, and Firefox, which we have no control over. When a browser updates, even a slight change in its code can cause a series of problems in Wild Apricot. This is where most of our bugs come from. 

To combat bugs, we have a Bug Manager who works with all our crews on a regular basis. Usually we identify and fix bugs before you, our customers ever know about them, but some bugs still slip through the cracks, and that’s where our support team comes in. They are the ones speaking with customers every day and collecting as much information as possible about these pesky bugs, so we can do something about them.

 

How Many Bugs Does Wild Apricot Have?

For us, even one bug is too many. But realistically, new bugs are reported to us every day.

In any given week, we receive 1,300 calls and support tickets about our software. Of this, only 30-40 are escalated as potential bugs. After investigation, the majority of these escalated tickets turn out to be known issues or temporary glitches, where only about one quarter turn out to be new bugs identified. 

So, what do these numbers boil down to? This year, clients have helped us identify 414 distinct bugs in Wild Apricot (we’ve fixed about 97% of these already).

This may seem like a lot, but it’s actually a very small amount considering the breadth and depth of our software. Wild Apricot is made up of millions of lines of code and consists of hundreds of functions.

 

How Serious Are Bugs?

The seriousness of bugs is determined by how much Wild Apricot functionality it affects compounded by how many customers it affects. The greater a bug affects these two things, the more of a priority it is to fix it.

To determine how serious bugs are, we’ve developed a scale of five different levels to grade them:

  • Blocker: Bugs that affect major functionality for >50% of all customers. These are the most important bugs to fix. Once identified, all resources in our company are devoted to fixing these bugs as soon as possible. Luckily we very rarely run into Blocker bugs.
  • Critical: Bugs that affect the main functionality for 11-50% of all customers. Once identified, these bugs are usually fixed within 1-3 days and involve unscheduled releases. Depending on the complexity of the bug, it may take a few weeks.
  • High: Bugs that affect the main functionality of our system, not meeting the above requirements. Fixes for these bugs are released with our main software updates.
  • Medium: Bugs that affect supplementary functions, not meeting the above requirements. Fixes for these bugs are released with our main software updates.
  • Low: Bugs that affect supplementary functions for few clients, not meeting the above requirements. Fixes for these bugs are released with our main software updates.

Less than 1% of bugs are Blocker bugs, about 4% are Critical bugs, and the rest fall between Low to High bugs.

 

How Long Does It Take to Fix a Bug?

We wish we could snap our fingers and make bugs disappear, but to fix each one correctly we must put them through a number of stages. Depending on the complexity of the bug, this can take a full month.

Here are the stages to fix each bug:

  • Bug Discovery and Investigation: First, we determined whether or not a reported problem is actually a bug (sometimes it’s a temporary glitch). If it is, then we investigate how widespread the problem is, what functionality it’s connected to, and whether a work-around exists.
  • Project Scope Estimation: Next, we assign a priority level to the bug (Blocker, Critical, High, Medium, Low) and assign it to the appropriate crew. For example, if a bug is found in our events module, our events crew is tasked to look for a solution. The crew then evaluates other urgent items in their roadmap (the list of features and updates they’re working on) and decides when to start working on it.
  • Fix (Solution) Discovery: It usually takes 1-2 days to discover a fix.
  • Fix Testing: Testing is the slowest part of the process. Depending on the complexity of the bug and its fix, this can take weeks. However, testing is also the most important of the process. Any change in one function may actually break another, because the code that makes Wild Apricot work properly is very interconnected. That’s why a lot of time is spent testing the new fix many in a safe environment first. 
  • Fix Release: Once the fix passes all tests, we prepare everything it takes to release the fix to our customers. This usually takes 1-2 weeks. Depending on how serious the bug is, we may release it as soon as it’s ready, or with the next scheduled update of our software.

 

The Number One Thing Helping Us Fix Bugs Faster

The slowest part of fixing bugs is how long it takes to test the fix. To speed up this process, we’ve been investing in something called Automated Testing. Automated Testing runs each new feature through a series of auto-tests and tells us what functions are not working properly from the moment they break.

So far, we have implemented Automated Testing in about 17% of our software and the payoff has been huge. Our average delivery time of bug fixes has been reduced from two months to one month.

We plan to keep implementing Automated Testing to cover 100% of our software, but the process to implement it is quite difficult. Still, our goal with Automated Testing is to reduce the time it takes to deliver a bug fix down to a few days, or even hours.

 

Are Bugs Here to Stay?

Yes. Bugs are a natural part of building software, but with Automated Testing, we’re getting much faster at solving them. I hope this has given you a better understanding of how long it takes to fix bugs and the process we go through to do so. We owe you a big thank-you for helping us identify bugs every day and for your patience as we solve them.

 

Time's Running out for Custom Domains

Do you have a custom domain for your Wild Apricot website?

If so, you should have received several emails from us to update your DNS settings to point to our new IP address – 34.226.77.200.

If you have not yet done this, you need to do so by September 4th. That’s when we will drop our old IP address. For instructions on working with your domain name provider to update your DNS settings, click here. If you don’t know who your domain name provider is, you can click here to find out. 

 

New Feature: Create and Edit Events in Our Mobile App

Last month we added the ability to create, edit, and delete events from Wild Apricot’s mobile app. Watch the short tutorial below to see how.

If you haven’t downloaded our mobile app yet, you can get it for free for iOS or Android.

 

Free Webinar: How the Best Nonprofits Resolve Internal Conflict Quickly and Get on with Their Mission

Free WebinarIs your organization’s mission suffering because of internal conflict, office politics and interpersonal issues? You’re not alone. Many nonprofits experience the same thing, which is why we’ve invited Margaret Sumption, an expert in conflict resolution to join us for a free webinar on Sept 26. Margaret will share the unique approach she has used with nonprofits all across the country to help them turn conflict into growth.

In this free webinar, Margaret will show you:

  • 7 proven communication strategies that prevent conflict in the first place
  • The reframing technique that turns interpersonal tension into creative solutions
  • The best way to rebuild relationships after a conflict

Sign up now and we'll see you on September 26, at 2:00 PM (ET) / 11:00 AM (PT). 

 

Join the Wild Apricot Membership Tribe

We’ve talked a lot about our Facebook Group, The Wild Apricot Membership Tribe since its launch in March. That’s because we've seen a lot of amazing discussions and we want to make sure you're part of them.

If you haven’t joined the 1,456 members (and growing), I really encourage you to check it out and see how other Wild Apricot users can help you run your organization better.

Here’s the link to join, we’ll see you in the group!

 

This Month’s Blog Posts

Have you ever worried your organization could have some hidden legal risks? Often it’s what we don’t know that can come back to haunt us. In one of this month’s blog posts we tackle three common legal traps nonprofits fall prey to and how to avoid them.

Here are the rest of this month’s blog posts:


That's all for this month's update!

Get a Special Report on Simplifying Membership Management

Enter your email and receive this special report in your inbox.
Dmitriy Buterin [Chief Apricot] Dmitriy Buterin [Chief Apricot]

Posted by Dmitriy Buterin [Chief Apricot]

Published Thursday, 31 August 2017 at 10:35 AM

Get a Special Report on Simplifying Membership Management

Enter your email and receive this special report in your inbox.

Comments

  • David Goodrich. American Printing History Association said:

    Thursday, 31 August 2017 at 8:01 PM
    I do hope you fix the bug whereby a member level change is not treated as a renewal that I brought to your attention nearly a year and a half ago. I would think this affects every organization that has multiple member levels.
  • Terry Ibele [Learning Apricot]

    Terry Ibele [Learning Apricot] said:

    Friday, 01 September 2017 at 10:08 AM
    Hi David,

    Thanks again for bringing that to our attention. I've forwarded it on to our support team to investigate.