TESTING, BUGS

Follow-up Testing For a Bug

I participated to a BBST Bug Advocacy online course a while ago. It focused on bug reporting – including investigating bugs and reproducing hard-to-reproduce bugs.

Related to investigating bugs, there was one useful approach on follow-up testing after you’ve found a bug. It consists four (4) types of activities:

  1. Vary my behavior
    • Change what I do as I test
  2. Vary the options and settings of the program
    • Change settings of the application under test
  3. Vary data that I load into the program
    • Different startup files or other data not directly involved in the test
  4. Vary the software and hardware environment
    • E.g. operating system, peripherals, external software that interacts with this application

I’ll give an example of applying this in my current context (insurance company). I noticed a while ago that I got an error with our mobile phone claim form, when I tried to send a claim. As I didn’t know what information on the form was causing a problem or if it was configuration issue, I first tried to

Vary the software and hardware environment

I was using Google Chrome, so I tried out if the problem reproduces with Firefox and Internet Explorer. It did reproduce, so it didn’t appear to be configuration issue.

Next I tried

Vary the options and settings of the program

I was trying out our Swedish form, when problem occurred. So I tried out if it also reproduces with our Finnish form. It did. So language of the form didn’t appear to be the critical condition.

After this I decided to

Vary my behavior

I started to modify how I fill the form:

  • Change city from Åbo to something that doesn’t have Scandinavian characters (didn’t help)
  • Change the money to go to own bank account, and not have beneficiary details filled (didn’t help)
  • Remove one damaged mobile device as I had more than one (didn’t help)
  • Shorten the address as I had maximum [60] characters in it (went through after this! – and further investigation revealed that it let through 50 characters through integration, even if you could type 60 characters to the field)

Conclusion This can be useful approach for further investigating bugs that you’ve found. As long as you don’t forget to also utilize for example logs in your investigation.