Automation is executing a process without human intervention. Automation with respect to quality assurance is running tests without human intervention and giving the results with a meaningful analysis.

One of the main benefits of Automation is reduction in cycle time and hence ready for the market earlier. Others being High Productivity with respect to number of tests ran per person day, reduction of human errors in running tests manually. Elimination of routine boring test cycles especially during regression so that employees can do more value added jobs, which helps in employee motivation. Easy and fast repeatability of tests.

Let us examine the each of the above benefits in detail.

Reduction in Cycle time

By automating manual testing using scripting and running in batch mode, one can achieve any where from 8 to 15 times reduction in cycle time for system testing. The same is true even in development testing.

High Productivity

By automating the test cycle there will be a five to ten fold increase in productivity. Grouping tests in such a way that the test scripts can be run over night with fixed test settings, will give high productivity from the perspective of number of tests and test cases executed per person day. Also running the tests in the night time and analyzing the results during the day will result in saving time. The human errors introduced in repeating the tests are eliminated by automating the tests

Easy and fast repeatability of tests

With automated test scripts available, any time we can reproduce the test without wasting time in re creating the test environment or sequencing in which the tests were run etc to isolate problems with Device under test or actual test related issues. Also the tests can be repeated for varying input parameter ranges, boundary conditions, special cases and error conditions.

Ease of Test Management

As the testing cycle at system level keeps growing, the management reports generated are very useful to improve product quality and also improve the development processes. The statistical analysis of various test metrics can be used for value added test cycle management purposes. These are easy to collect with automation and more accurate compared to manual testing or human interpretation.

Automation of network device testing:

In today’s networking industry both the OEMs and ISPs are hard pressed for time for bringing out the product to development and putting the unit to real life network.

The OEMs have very little time to test their product and the buyers have no tolerance for faults. Bad quality products result in network down time which will cause revenue loss and customer anger.

The only way OEMs can guarantee that their product is fit for deployment is through testing or QA validation by Independent testing. Given the competition and time to market shrinking, the test cycle time has to be as low as possible and should be very efficient so that product quality is assured.

The only way is automation of QA activities.

The following method is one of the approaches to surmount the above problem.

Automation can be effectively used for network device testing. The following categories of testing can be automated:

1 CLI (Command Line Interface) Tests

2 Protocol Functionality Tests

3 Interoperability tests

4 Testing the device for stress, stability and performance in simulated real life conditions.

Below is the statistics, derived after the first round of automation. This provides 8x reduction in cycle time. The cycle time can be reduced after further rounds of automation.

Challenges for automation of network device testing

Ease of modification:

Since the devices will be under development stage there are more possibilities of changes in CLI and features. The automation tool must be capable of accomadating these changes without much effort.

Scalability:

The interoperable devices in a setup may be replaced with different vendor devices. The automation tool must provide the facility for replacing the devices and also should have the provision to allow topology changes. The replacement of device should not force modification of the test scripts. The modification to be done to the tool must be minor.

Report Generation:

Since the tests will be run automated, the reports generated must capture all the information required for analysis.

Reliability:

The automation tool must be capable of identifying problems in the test setups. It must be capable of identifying the exact reason of failure and log it into the report.

Tools friendly for automation:

TCL(Tool Command Language) is one of the suitable scripting language for automation.

Using Linux as the platform, the tool can provide the advantage of using freely available utilities like packet generators, packet capture etc.

With strong necessity of putting products early into the markets and stringentrequirements of a highly fault free and reliable internetworking product, the OEMs and service providers are looking for a very efficient and fast QA organization and reduced cycle time. The solution is definitely QA automation. QA automation gives OEM s and Service providers a major saving in dollars and increased customer satisfaction.

Manjula Raju is programme co-ordinator Afila India feedback at This e-mail address is being protected from spam bots, you need JavaScript enabled to view it \n This e-mail address is being protected from spam bots, you need JavaScript enabled to view it "

Issue BG58 Jan06

Related Items:

---

Be first to comment this article

Only registered users can write comments.
Please login or register.