3. Consistent data is the foundation of functional regression testing
Functional testing is based on executing user-level business functions, just like the test cases that are developed to test and verify that a specific function is operating correctly. Often functional tests are scripted or executed and recorded in a tool (like Arcad Verifier) and the results are checked for consistent values. Once these functional tests are developed they can be used again and again for regression testing, in which one or more functional tests are executed to validate normal application operation after an update to QA. This technique is very dependent on having consistent data so that the functional tests return consistent results and pass.
If the data is changing due to other processes running while the scripted tests are being executed, a false positive can occur that will be interpreted as failure. So here again consistent data = consistent results and enables a much higher level of automation. The importance of having a dedicated set of test data becomes even more apparent.
4. How to ensure the privacy of test data – and refresh it when needed?
The most common reason why businesses do not have multiple sets of test data is because the starting point is production data which can have millions of records and/or sensitive data. Arcad has a tool to handle that case as well (DOT Anonymizer) but that is another topic and we are here to talk about managing that data to your advantage. If you build a good set of test data that is big enough to support all of your test cases but no bigger, it may contain sensitive data that is of high value to your organization and which requires masking and otherwise careful handling. Having a copy of that set aside for refreshing QA environments ideally in a SAVF so it cannot be inadvertently updated is a recommended best practice. Alternatively building a function to refresh that data from production (or using an Arcad tool) is an option and will also provide a means of refreshing test data with more recent production data. Either method means that when a set of test data gets corrupted it is easy to refresh it with good data and, in the case of automation, with consistent values.
This ability to automatically refresh data is valuable for all the testing you do, even developer-driven unit testing which may be dependent on data read from files. Certainly, functional testing by QA needs the ability to refresh their data easily since they are trying to uncover bugs that might corrupt their test data as they are testing. Even using testing tools that manage data updates through a rollback process can leave behind bad data if the QA staff hits a significant error and the testing script process is interrupted.
5. To sum up: continuous test (CT) needs consistent data!
We’ve seen that good consistent data is key to all forms of testing and we should be able to easily restore it when it’s corrupted. It is a bonus if that process can also be used to refresh existing data with more current or realistic values. We need multiple sets of data to support concurrent testing efforts without conflict so this set of data should be a subset of the complete production data. This provides the foundation for our testing methodology and it is the very basis of automating testing.
Consistent data providing consistent results means that not only can the test cases be scripted, but also that defects can be detected automatically during an automated test run. That translates to way fewer manual actions – faster results, less cost overall. It is an often-ignored prerequisite for true continuous test and an optimized DevTestOps cycle.