Background: Metamon has been selected at a number of large corporations over products from the three major mainframe software companies. Although the Metamon product suite has a broad range of functionality, the customers discussed here were primarily concerned with Quality Assurance for both CICS and Batch applications. Each company has a large portfolio of existing mainframe applications. Since these programs have been “field tested” in production for many years, there is a high confidence level that they are stable and work as intended. Historically, these applications have not had 100% code coverage testing because of the labor involved to achieve this level of QA and there has been no pressure to move to this standard. However, when extensions and modifications are made to the code, the IT organizations’ positions change and they need to know if the application will continue to function as required. They use Metamon. products to determine if the changed code has been adequately tested. Unfortunately, traditional tools that measure code coverage make it very difficult and time-consuming to ascertain that changed code gets tested. (Since we are not regular users of the tools from the vendors referenced above, we rely on what our customers have told us in this narrative.) These tools fall into three general categories:
- Tools that “instrument” the code (i.e., insert markers into the code that can be detected at run time.) These tools have the inherent flaw that the code that is being tested is not the same as the code that will go into production. In addition, the “instruments” need to be removed before doing a recompile and then moving into production.
- Tools that use a special compiler option to allow measurement of code coverage. This requires an additional operation and has the same limitation as above in that the code being tested is not compiled the same way as the production code.
- Tools that run on a PC platform and require mirroring of the test environment on the PC. These tools require special skill sets to run, are very time consuming to set up and maintain, and run the applications in a different environment than they would normally run in.
Because of its unique architecture, Metamon provides code coverage data during the normal test runs without the constraints and limitations of the above tools. In addition, results from many test runs can be rolled up to the application and system level and, importantly, new lines of code are identified in this reporting. This approach makes 100% testing of code changes practical for the first time. One can successfully argue that anything less than this level of testing is no longer adequate in today’s world of high customer expectations, regulatory compliance issues and security applications.
Summary: 100% code coverage for new code has long been the “gold standard” for application testing. Rarely has this goal been achieved because of the effort required using the available tools. Even though we are a relatively small company compared to the “big three”, we have been very successful in finding customers that want to implement this level of QA.
Home | Mainframe Debugging Tool | Testing COBOL & Assembler | Mainframe Testing | Software Demos
Competitive Analysis | Software Case Studies | About Metamon | Links | FAQs | Contact Metamon | Site Map