2008-12-28

Bug / Problem reporting the right and the wrong way to do it

There is an entire Methodology behind bug/ problem reporting.
There are many tools used for bug tracking and reporting.

all of these do not matter, we need to keep in mind the purpose and the goal for which we report a bug, if we do this then all will be well.
According to the Bugzilla site :(https://developer.mozilla.org/en/Bug_writing_guidelines)
The guidelines to problem reporting are detailed below:

  • Be precise
  • Be clear - explain it so others can reproduce the bug
  • One bug per report
  • No bug is too trivial to report - small bugs may hide big bugs
  • Clearly separate fact from speculation
  • Reproduce your bug using a latest version of the software, to see whether it has already been fixed.
  • When system configuration is relevant specify it.
  • When log files are available provide them.
Following these guidelines will ensure the programmer understands you and has all the information to solve the problem... it obviously does not guarantee bug fix times



Trouble Shooting Guide lines

During the earlier days of my career I found my self constantly being called on Costumer site for trouble shooting reasons .

I would usually discover when arriving there that the so called complex problem is easily solved by a simple and short analysis session.

I tried to lay out some guidelines for others to follow in order to enable them to do the same.

What it basically comes down to is asking the right set of questions in the right order.

the questions which guided me were:
  1. locate the first step/ component to indicate a problem
  2. what is the cpu/ mem utilization?
  3. which application is consuming the most resources?
  4. open the log files
  5. look for the most recent error message
  6. what does it say
  7. what does it mean
When answering these last two questions you can generally solve most problems.
At one time I traveled to one of our costumer sites in the east.
Problem report was system was malfunctioning and missing several input responses.
When arriving on site I noticed that the gate component was running at high CPU utilization (~90% steady state)
when reviewing the logs I noticed a configuration error.
Once I handled this I noticed that the CPU did not drop.
I reviewed the logs again and noticed that the same error message is being printed at a very high rate ( which was causing the high CPU utilization).
I turned to the programmer and requested to reduce the debug level for the specific message (since it was pointing to an not mapped input type and not a system problem)
Once this was accomplished all other problems were gone.
Problem Solved.

If the field Engineer /Support follows this rules he can generally get to the bottom of the problems or at least point R&D in the right direction

This in my opinion is the true meaning of On site Field Engineering, other wise it would mean that the Field Engineer is expected only to perform the "Next, Next, Next, ..." installation type.... Is this what Field Engineers are for ?!?

I Don't think so.


QA On Costumer Site - rules and guide lines

There are very specific criteria to determine when QA is needed on site , as I see it only if these criteria are met then there is a point to assigning QA to the costumer site.

  1. Costumer has specific system inputs which can not be simulated in a lab.
  2. Specific problems can not be simulated in the lab .
  3. Tests did not complete prior to deployment, and costumer is willing to provide a testing environment.
In general as you can see I think there are very few occasions justifying QA on site , how ever since on many occasions system know how is lacking and QA is forced to travel to site . When On site the following rules should be met...
  1. QA has traveled on site for a specific purpose it does not include costumer instructions/ politics on site installations etc...
  2. when reporting from site QA should maintain the same guidelines directing him when reporting a bug from the lab.
      1. be clear and to the point
      2. describe a specific and exact scenario
      3. provide all possible available debug information
      4. describe system configuration with all necessary detail
Once these rules and guidelines are followed we are certain to make the most of the QA visit on site.

When this need actually arises

QA and IT - to install or not to install - that is the question

When dealing with QA yo must remember that there are extensive IT requirements, this is of course product dependent , but for true QA to exist, several things must occur which relate to IT.
First and foremost Product Manager/ System manager must define :
  • on which Platform the Product must run.
  • What are the minimal system requirements for the product.
once the above has been determined it is extremely vital that the QA performs tests for all of the System configuration defined;

And here comes the tricky part who is in charge of the installation and maintenance of these environments.
In the organization in which I am part of w do not have a separate IT group. this creates havoc in my opinion for the following reason.
  1. Not all QA are predefined to have IT capabilities and thus a lot of time is wasted due to insufficient skills.
  2. Since no proper IT exists there are no predefined sets of instructions as to how and what exactly to set up with in the Platform itself
  3. QA time is "wasted" on OS setups.
  4. Since QA are not exactly qualified for this they find it difficult to issue clear instructions to system Field Engineers on how to set up costumer systems, thus creating in consistency of OS types and configuration ... which basically will lead to unexplained problems from specific costumer with out any clear reason.
  5. No one can handle new hardware certification/ old hardware End-Of-Life replacement.
In my opinion it is vital to have a strong IT group in every company , and not just for the sake of handling the tedious tasks of system setup but in order to set up guidelines so that we create a consistency between all of the systems installed among the different costumers