3. Shift Left – fail early
Unit testing is just one of a number of “developer-side” techniques for detecting defects as early as possible in the development lifecycle. Along with static code analysis, peer code reviews, code coverage analysis and other code-level practices, unit testing catches errors at the earliest possible phase, when they cost the least.
The investment in developer time to create the test in the first instance can be offset by automating the creation process. The key is to focus on typical use cases that affect the behaviour of the system. Combining both “happy path” and “edge cases” makes unit tests even more effective.
Used continuously in a CI/CD cycle, unit testing pinpoints precisely in which lines of code the defect lies. Bugs are fixed before they ever leave the hands of the developer!
4. Clean, atomic code
More than 90% of software development cost is spent on maintenance of existing systems. Creating unit tests as you develop code actually improves the code design and makes the code easier to understand by development teams in the future. As well as being more reliable, unit tested code is simpler, more modular, and hence more reusable. This reduces technical debt and lowers development costs in the long term.
In its extreme form, Test Driven Development (TDM) brings extra clarity even to the definition of requirements by creating tests before the code itself.
5. What about the IBM i?
Modern IBM i applications designed with an ILE architecture already separate the business rules, data persistence and user interface into distinct modules or functions making unit test automation much easier. The entry and exit points of a module are clear so defining a “succeed/fail” status is relatively easy.
However many applications on IBM i still contain sections of source code that were developed as long as 40 years ago and are “too important to touch”. This creates a very precarious situation where the risk and cost of refactoring monolithic or spaghetti code into units or modules is too high to take action. Yes the price of doing nothing is to be outpaced by more agile competition. Luckily, unit testing creates that safety net you need as you modularize your legacy code. Generating and re-running unit tests over “backend” applications as you refactor them makes sure that previous deployments still work when combined with new functionality.
This kind of “Test Driven Maintenance” technique runs unit tests on demand and in batch as a kind of early warning system to prevent regressions emerging as you “untangle” your legacy code.
To be fully adopted by developers, any unit test functionality on IBM i must be integrated within the RDi development environment, and also with standard tools such as Jenkins and JUnit to encourage the sharing of tools between multi-technology teams. The best unit testing solutions on IBM i can automate both the creation and execution of test cases, using parsing technology to search for parameters and their data types, rapidly identifying all the inputs and outputs of the program under test. Cross-referencing and dependency knowledge also help manage test cases and re-use cases between versions.
6. The importance of test automation on IBM i
There are still some extreme cases where IBM i applications cannot be easily refactored and the cost/time of restructuring the code is prohibitive.
Here functional test automation is the far more practical option. Test cases are created automatically as users execute application functions from the standard application UI. Unlike unit testing where the underlying data environment is dynamic, functional test data is static and restored prior to each test run to detect regressions by simple comparison of data, spool output and UI. There is no notion of success or failure, only different.
Of course an optimal testing strategy combines both unit test and functional test automation to minimize the Mean Time to Repair (MTTR) of defects overall. The re-use and sharing of test assets avoids costly wastage of developer time and makes testing a continuous activity, an integral part of the overall CI/CD cycle.
7. Ensuring developer adoption of Unit Testing
In conclusion, we have seen that unit testing facilitates a bottom-up style of testing, by validating individual parts of a program before testing the “sum of its parts“ – an approach that shifts defects left and reduces effort in integration testing later.
Unit tests act as a kind of “living documentation” of the system, enabling an rapid understanding of a unit’s interface. In the case of test-driven development they can even take the place of formal design.
Yet unit testing must be done in conjunction with other forms of testing as it not catch system-level or functional errors and also non-functional aspects such as performance.
The principal challenge with unit testing is the creation of realistic and useful tests, to establish the relevant initial conditions that mirror a normal application execution. Most importantly, unit tests must be maintained in parallel with the application change process itself, to ensure that impacted tests are kept up to date and are executed for every dependent code change. To make the maintenance of unit tests easy, it has to be easy to re-identify new input and output fields as the application evolves. If not, unit tests can become as buggy as the code they are expected to test!
To overcome these challenges unit testing must be highly automated and made an integral part of the continuous delivery cycle:
- Auto-discovery of inputs and outputs to automate the creation of realistic/useful tests
- Integration of unit testing into the version control process
- Auto-execution of tests as continuous part of the CI/CD flow
- Re-use of unit test assets for each program change
- Dependency-driven maintenance/update of unit tests
In short, unit testing is a highly beneficial but rigorous discipline, requiring maximal automation for adoption in the long term !