Software Test Automation (TA) has never been so present on the Application Lifecycle Development landscape. Agile methodologies amplified the deployment frequency one order of magnitude, (remember traditional monthly deployment cycles?). That wouldn’t be achievable without quality assurance. That’s where Test Automation (TA) comes into place. But… have you gone further than just automating unit tests?
Unit test automation is the usual suspect for automation, primarily because development teams already waste time doing unit tests, which can easily be translated into automated tests using one of many test frameworks. Although this practice is recommended there are some pitfalls that one tends to overlook:
The true return of investment for test automation begins when it’s put to work for quality and operation teams benefit. This is even more evident for corporate IT, where a heterogeneous body of applications demand a large-scale engagement to tackle the delivery process.
- Usually unit tests are done by the same developer responsible for the code under test (contradicting the best practices)
- Unit tests are not related to user/business requirements, thus can’t be used for acceptance
The true return of investment for test automation begins when it’s put to work for quality and operation teams benefit. This is even more evident for corporate IT, where a heterogeneous body of applications demand a large-scale engagement to tackle the delivery process.
Working together with the quality team:
Working together with the operation team:
Test Automation also has interesting side-effects that will further build up the quality assurance thread:
Test automation will become a central piece of the continuous delivery undertaking. After the tools and technology (GUI / API testing) are tame, test strategy becomes central and TA will thrive.
- Automate the regression test set based on the existent manual test design. The reproducibility of the achieved TA and the ability to execute on demand will save a constant amount of effort by release, which can be shifted for new features testing.
- Create a test data generation process and all the applications that require integrated data for testing will profit. (No more waiting!)
- If your automation framework is built using a modular approach, the quality team will have the ability for (1) performing data-driven tests (using the same test with diverse input data), (2) orchestrating simple tests into complex scenarios (create-update-get-export-delete…) (3) seamlessly execute the same test on different environments thru configuration.
Working together with the operation team:
- Automated Smoke test sets will enable a quick environment validation after deployment
- Automated Sanity test sets can grant the green light for integrated system testing, acting as a quality gate
- On demand tests for patch deployment, will speed up the corrective stream
Test Automation also has interesting side-effects that will further build up the quality assurance thread:
- Requirements standard definition and traceability will become more and more important in order to ensure test coverage. In fact, most of the TA projects I’ve worked the main complexity was due to poor requirement definition and specifications (and not technology), since automation requires explicit and unambiguous definitions. Test Driven Development may come for the rescue, if implemented.
- Test Strategy becomes the central focus, since so many techniques become feasible and effortless for a TA framework. For example Pair wise testing: imagine an interface with 10 fields, each with 10 possible values. The number of possible combinations are 10^10. Even with TA, exhaustive testing would become an issue. Using the Pair wise technique the total required tests to assure 100% coverage is 170, a stretch for manual testing but natural for automated data driven testing. (Curious about this technique: click here)
- Manual testing sometimes seems more like evidence gathering than actual testing. For this kind of tests, manual effort is a waste. TA can validate all fields all the time, take a snapshot and even record the interaction. Expensive manual testing should be an asset reserved for exploratory and acceptance testing.
Test automation will become a central piece of the continuous delivery undertaking. After the tools and technology (GUI / API testing) are tame, test strategy becomes central and TA will thrive.