It has been practically observed that there are changes in the functional requirements, when test cases are being executed. It is recommended that we do an impact analysis of the Impact on testing activities. This impact analysis is different from the Impact analysis which is performed by development.
After performing the Impact analysis, the Test manager should inform all stakeholders about the impact of change in requirements. The impact is in terms of schedule and effort. Even if the changes are minor, which the testing team is ready to absorb (i.e. no change in schedule), still a log of these impact analysis forms should be maintained. It has been practically observed that many small changes also add up into substantial impact.
In a dynamic world, there will be change in business requirements. The Testing manager/lead has to ensure that all stakeholders are informed of the impact on testing effort due to change in requirements. This helps in reducing the pressure on the testing team members.
Friday, November 21, 2008
Wednesday, October 22, 2008
What to do if a bug has leaked into production?
The application which your team had tested was released into production. A day after the release, you get a mid-night call, that a priority 1 issue has been reported in the production environment. What do you do in such a scenario?
First of all do not panic. You need to remain calm and composed. There could be different reasons due to which an issue can occur:
First of all do not panic. You need to remain calm and composed. There could be different reasons due to which an issue can occur:
- Application is not deployed or configured correctly
- Incorrect data in production database is causing it
- Due to incorrect understanding by end users, the feature is reported as a bug
- If none of the above, then it could be a bug in the application.
Broadly, your response to such a situation can be divided in two broad categories. One is the immediate corrective action which needs to be taken to minimize the impact of this issue, and second is the root cause analysis for identifying that why it happened and how we can ensure that similar issue does not happen again.
a) Immediate corrective action:
Try to collect all the required information regarding the issue found in production.
- In which particular scenario/condition is the problem happening?
- Is the problem happening consistently in production or is it intermittent?
- What percentage of end users is getting impacted by this issue?
- Is this problem corrupting or losing data which is not recoverable? If yes, what can be done to minimize the data loss?
- Can this issue be replicated in QA environment?
b) Root cause analysis and fixing the process:
One the issue is identified, we need take the following steps so that similar issue does not happen again:
- The QA team tries to simulate the problematic scenario in QA environment. It works closely with Business team, development team and production support team to recreate the scenario.
- If the resolution of the issue requires a patch release, then start planning to test the patch release.
- To identify the root cause of the issue, perform causal analysis.
- Identify corrective actions required so that similar issue does not happen again.
- Implement corrective action
- Be proactive and keep all stakeholders informed at each stage.
The above steps will help in making sure that you do not loose the confidence of the stakeholders.
Wednesday, June 4, 2008
Value addition using Freeware automated tools
Many testing consultants are proud of using and recommending expensive licensed automated testing tools which are provided by big software vendors. The IT companies may get impressed with such testing tools provided by the big software vendors. However, in this approach a major chunk of the Testing budget of these IT companies will be exhausted in procuring the expensive testing tool and very little budget would remain for actual automation of the test cases.
As testing consultants, we need to ensure that we recommend a testing strategy, in which the total cost of the automation solution (including the cost of testing tool) is minimal. This can be achieved by evaluating and recommending freeware testing tools. There are several freeware testing tools which are available for functional, performance, security testing etc.
Many testers are sometimes reluctant to use the freeware tools as they feel that there is a risk associated with the freeware tool as it has limited features. A good approach to mitigate this risk is to create a Proof Of Concept (POC) by automating few test cases using the freeware tool. This will mitigate the risk associated with using the freeware tool. The POC should be demonstrated to the stakeholders to get the required approval.
Another observation is that freeware tools have almost the same set of features as licensed tools, but the freeware tools supports limited platforms or protocols. So, depending upon the Application Under Test (AUT) requirement, we should evaluate and select an appropriate freeware testing tool.
By using the above approach, you can make direct saving in the IT budget of your client.
As testing consultants, we need to ensure that we recommend a testing strategy, in which the total cost of the automation solution (including the cost of testing tool) is minimal. This can be achieved by evaluating and recommending freeware testing tools. There are several freeware testing tools which are available for functional, performance, security testing etc.
Many testers are sometimes reluctant to use the freeware tools as they feel that there is a risk associated with the freeware tool as it has limited features. A good approach to mitigate this risk is to create a Proof Of Concept (POC) by automating few test cases using the freeware tool. This will mitigate the risk associated with using the freeware tool. The POC should be demonstrated to the stakeholders to get the required approval.
Another observation is that freeware tools have almost the same set of features as licensed tools, but the freeware tools supports limited platforms or protocols. So, depending upon the Application Under Test (AUT) requirement, we should evaluate and select an appropriate freeware testing tool.
By using the above approach, you can make direct saving in the IT budget of your client.
Friday, May 23, 2008
Reliability testing – Detecting and fixing memory leaks
Last month, I encountered an interesting situation regarding memory leak in production. I would like to share my experience of how we were able to diagnose and fix the problem.
While working for a client, we released a new version of the application. A strange behavior was observed. On every Monday, the application used to give “Out of Memory exception”. On discussion with the users it was clear that the load on the system was 50 – 60 % higher on Monday. So, definitely there was a memory leak in the system which was causing the out of memory exception.
The client had not invested in any dedicated QA environment. So, it was not possible to simulate or diagnose the problem. We started looking for a freeware tool which could help us diagnose the system in the production environment.
We finally used JConsole which comes along with JDK 5.0. This is a freeware which can be used for monitoring, the memory utilization with time. It clearly shows how many threads are running. It shows at which time the Garbage collection is happening. In fact, it has a provision where you can use it to simulate Garbage Collection.
Using this tool, we could locate the probable portion of the code where the problem could be. Then we did a thorough code walk thru of that portion of the code. We found that the calls to certain functions were being made by starting a new session. This problem was there at multiple places in the application. After making the changes, we could observe the difference in JConsole tool.
More details of JConsole are available at the following location:
http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html
While working for a client, we released a new version of the application. A strange behavior was observed. On every Monday, the application used to give “Out of Memory exception”. On discussion with the users it was clear that the load on the system was 50 – 60 % higher on Monday. So, definitely there was a memory leak in the system which was causing the out of memory exception.
The client had not invested in any dedicated QA environment. So, it was not possible to simulate or diagnose the problem. We started looking for a freeware tool which could help us diagnose the system in the production environment.
We finally used JConsole which comes along with JDK 5.0. This is a freeware which can be used for monitoring, the memory utilization with time. It clearly shows how many threads are running. It shows at which time the Garbage collection is happening. In fact, it has a provision where you can use it to simulate Garbage Collection.
Using this tool, we could locate the probable portion of the code where the problem could be. Then we did a thorough code walk thru of that portion of the code. We found that the calls to certain functions were being made by starting a new session. This problem was there at multiple places in the application. After making the changes, we could observe the difference in JConsole tool.
More details of JConsole are available at the following location:
http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html
Subscribe to:
Posts (Atom)