that JTRS capability is delivered to the warfighter.
The Joint Tactical Radio System (JTRS) enables network-centric warfare through the use of advanced, mobile, ad hoc, network-capable JTR devices, and true networking and joint interoperability among all four Department of Defense services across the entire battlespace. Facilitating this interoperable network is a software-defined architecture to permit the porting (or loading) and reuse of a standard suite of software products— including the waveforms used to transmit the data—on a wider variety of hardware configurations.
Employing mature, software-defined radio technologies, the JTRS program is developing more than 10 million lines of code across five ACAT 1D major defense acquisition programs as part of its Increment 1 baseline. This article shares lessons learned, the software metrics framework being used, and a program executive viewpoint of the stability, development progress and quality of JTRS software products.
First and foremost, observations throughout the JTRS development life cycle reveal that all DoD programs, including but not limited to software development, can greatly benefit by requiring systematic product development progress tracking, and quality metrics trended over time. Based on objective measures, these metrics have been required on all JTRS contracts and have proved central to the program management and Joint Program Executive Office (JPEO) tool set, especially at the level of strategic planning, to ensure that capability is delivered to the warfighter.
Early in the JTRS program standup, the JPEO instituted an enterprise software metrics requirements effort. The result of this effort established a baseline of software metrics defined and approved by both the legal and contracts departments. Subsequently, these metrics were added onto each JTRS contract involving software development.
The intention of this set of metrics is to quantify the quality and progress of the software product’s development over time, based on objective measures consistent with the requirements and contained within the contract. The status and trends gathered from these metrics have been essential to determining the step-by-step program status frequently and recurrently, and have provided strategic data to determine adequate tradeoffs to ensure successful completion. This approach to software metrics will work for any type of software development methodology, including incremental, spiral or waterfall.
These objective measures must be defined early in the program. For instance, during the requirements and design phase, the number of requirements and the number of use cases required for design must be estimated. Then, the completion rate of the cases can be viewed and analyzed for trends—actual vs. scheduled. If the number of estimated use cases rises dramatically, it is an indication of either an initial lack of understanding of the requirements or requirements creep.
During the coding phase, the software lines of code (SLOC) count can be used to determine progress. And finally, during integration and test, the execution of test cases can be monitored. This process has consistently demonstrated that if objective measures are planned early in the development life cycle, are well understood at the outset of the development phase, and then regularly monitored for progress over time, product completion can be estimated at any stage of the development.
During the integration and test phase, the other piece of the status puzzle may be found in trouble reports. The IEEE Standard of Anomaly Priorities was used so that across the enterprise, there is uniform agreement to the quality requirements of each software component.
Many papers have been written on the estimation of the number of problems expected during the software life cycle. By analyzing the number of test cases successfully executed, the number of SLOC available for test, and the number of open and closed error reports, the health of the product can be monitored.
The number of open and closed error reports should start to converge when the test cases are starting to complete. At the beginning of testing, a large number of errors can be a good sign that thorough testing is being executed. If enough planning is not defined, a product can end up with a status of 99 percent complete yet with very little insight into the work necessary to satisfy all requirements.
Other product metrics that are collected include software complexity and productivity numbers. Software complexity demonstrates the testability of the code, and is an important criterion for software certification. These criteria will be discussed further in subsequent articles.
The examples in the accompanying charts demonstrate the valuable information consistently available through the use of this incremental, objectively measured approach to software development. Notice the concurrency of design, code, and integration activity and progress.
In conclusion, the visibility and ability to anticipate and identify problems early, to make informed decisions and strategies, and to develop contingency plans and mitigate risks during the product development cycle more than compensates for the extra effort invested in defining the objective measures, planning progress over time and monitoring progress over time against the plan. ♦