1. Share This:
2.  
3.  
4.  
5.  

The National Aeronautics and Space Administration (NASA) plans to invest billions of dollars in the coming years to explore space, understand Earth’s environment, and conduct aeronautics research. We designated NASA’s acquisition management as high risk in 1990 in view of NASA’s history of persistent cost growth and schedule slippage in the majority of its major projects. Our work has identified a number of causal factors, including antiquated financial management systems, poor cost estimating, and underestimating risks associated with the development of its major systems.

NASA’s senior leadership is committed to improving overall acquisition outcomes and has implemented key components of the agency’s action plan. Specifically, the agency has instituted new tools aimed at providing increased insight into project performance, such as the collection of earned value management (EVM) data and the Joint Cost and Schedule Confidence Level (JCL) process. However, capacity remains an issue as the guidance for these tools has not been finalized and training to address identified skill gaps has only recently begun. NASA has established metrics to monitor progress in improving acquisition management, and in recent years we have found improvement in the cost and schedule performance of the agency’s portfolio of major projects. However, more recently, a few of NASA’s major projects are rebaselining their cost, schedule, or both in light of management and technical issues, which is tempering the progress of the whole portfolio. In addition, several of NASA’s largest and most complex projects, such as NASA’s human spaceflight projects, are at critical points in implementation. We have reported on several challenges that may further impact NASA’s ability to demonstrate progress in improving acquisition management.

NASA continues to take steps to reduce acquisition risk. Over the past several years, there has been improvement in the cost and schedule performance of the agency’s portfolio of major projects, which include highly complex and sophisticated space transportation vehicles, robotic probes, and satellites equipped with advanced sensors to study space and earth. For example, we found in 2011 that—for projects in implementation—the portfolio average development cost growth was more than 14 percent and the average launch delay was 8 months. In 2014, when excluding the James Webb Space Telescope (JWST) project, we found those averages had fallen to 3 percent cost growth and launch delays were less than 3 months on average.^[1] In addition, NASA reports that it is meeting all of the metrics it established to monitor progress in improving its acquisition management. Some noteworthy steps the agency has taken include the following:

• In 2014, NASA drafted revisions to its acquisition planning regulations, which it anticipates will be sent for notice and comment in February of 2015. If these revisions are issued as final rules, they would require acquisition plans to include a detailed independent government cost estimate as well as lessons learned from prior or predecessor contracts to improve follow-on acquisitions.
• As of 2014, all NASA projects required to develop a Joint Cost and Schedule Confidence Level (JCL)—a tool which assigns a confidence level, or likelihood, of a project meeting its cost and schedule estimates—have done so.
• NASA has also taken steps to improve the agency’s use of earned value management (EVM)—a tool designed to help project managers monitor risks. In fiscal year 2013, the agency began a phased rollout of the EVM Capability process on the Space Launch System (SLS) project at the Marshall Space Flight Center and the Ice, Cloud, and Land Elevation Satellite (ICESat)-2 project at Goddard Space Flight Center. NASA started the EVM Capability process for the Ground Systems Development Office at the Kennedy Space Center in fiscal year 2014. In addition, the agency plans to implement the process for the Orion Multi-Purpose Crew Vehicle (Orion) program at Johnson Space Center in fiscal year 2015. NASA plans indicate that during the rollout phase, the agency will provide support to the respective centers to develop the institutional capability to support future projects.
• In 2012, the agency established metrics to more consistently measure a project’s design progress and, in 2014, we found that most major projects in the portfolio were tracking and reporting those metrics. In addition, experts that we met with confirmed that NASA’s metrics are valid measures to assess design maturity in space systems.

These actions have helped NASA to launch more projects on time and within cost and, ultimately, to maximize science and exploration objectives. We have also found, however, that NASA still faces significant challenges in managing and overseeing its most expensive and complex projects. More specifically, we have identified instances where the agency has either underestimated the risks and potential impacts; not reacted quickly enough to risks when they worsen; or resisted independent assessments of risk in light of changing conditions, as the following examples illustrate:

• Over the past 2 years, the JWST project’s cryocooler—which is necessary to cool one of the telescope’s instruments—has had a series of design and manufacturing challenges that have used a disproportionate amount of the project’s reserves and caused significant schedule delays. While the project has recently taken steps to increase oversight of the cryocooler development and manufacturing it remains to be seen if these efforts will keep the cryocooler effort on its current schedule. The cryocooler is now the main driver of schedule risk facing the $8.8 billion project.
• In 2014, we found that neither the JWST project nor the prime contractor planned to update their cost risk analysis even though the analysis was based on risk data that was 3 years old and will be 7 years old by the time the telescope is launched and that the project’s JCL remained outdated. We recommended that, consistent with best practice, NASA should update its cost estimate and risk analysis for the JWST prime contractor’s remaining work. NASA disagreed, but then subsequently agreed to conduct an updated cost risk analysis for the prime contractor’s remaining work.  If done correctly, this type of information will provide increased insight into the project at a time when it is facing technical challenges that are increasing risk to the project.
• NASA officials told us that the project underestimated the technical complexity of the ICESat-2 project’s design and the project began underperforming against cost and schedule just 1 month after its baselines were set at confirmation. The project was subsequently rebaselined more than 30 percent higher than its original baseline. The laser contractor has encountered multiple technical issues while building the prototype laser for the project’s only instrument. Recently, work to resolve an unexpected power issue encountered in testing has delayed qualification testing of the laser by at least six weeks.
• The Space Network Ground Segment Sustainment project’s contractor provided unrealistic estimates that, despite project officials being aware of this issue prior to project confirmation, led to the need for a rebaselining shortly after the project was confirmed. Specifically, the contractor underestimated costs for information technology infrastructure and overestimated its ability to meet its staffing requirements. As a result, the project’s costs are expected to exceed the agency’s committed baseline cost by 30 percent and the project’s committed final acceptance review date is expected to occur 23 months past the originally scheduled date, slipping from June 2017 to May 2019.

In addition, while NASA has implemented tools in recent years to provide better insight into and oversight of its acquisition projects, the agency has not consistently applied training for and implementation of these tools. In some instances, the reason may be a lack of agency expertise or guidance in how to apply the tools, but in others adherence to tools has not been consistent. For example:

• The implementation of EVM across NASA’s major projects has been critical to better understanding project development needs and reducing cost and schedule growth. We have found, however, that the amount of expertise available at NASA centers and the training provided to staff on how to both construct and apply the collected information is lacking. For example, in 2012 we found that the agency lacks adequate numbers of staff with the skill set needed to analyze EVM data and this was confirmed through NASA’s skills gap analysis. To address these gaps, the agency released an EVM training plan in August 2014, and the training is currently underway.
• NASA has used the JCL process since 2009 but has yet to release agency-wide guidance, which may have been a contributing factor to the inconsistent use of the factors that constitute the joint cost and schedule confidence level that we found for several projects and weaknesses found in the JWST project’s JCL. For example, in 2012 we found that, in some instances, when projects provided data for JCL calculations they excluded or did not fully consider relevant cost inputs and risks, such as launch vehicle costs or risks associated with challenges faced by development partners. In 2012, we reported that the JWST project’s cost estimate did not meet criteria for being accurate, in part because the summary schedule used to derive the JCL did not include the necessary level of detail nor did the project provide evidence that it planned to update the cost estimate. Despite weaknesses identified by us, the JWST project—rebaselined in 2011—had not updated its JCL to be in line with cost estimating best practices as of 2014.
• NASA’s measurement of its progress for reducing acquisition risk has been inconsistent. Specifically, when NASA reports on cost and schedule performance for individual projects or across its portfolio, the agency uses rebaseline data rather than original project baseline data for measuring outcomes. In other words, cost and schedule growth that occurred prior to the replan of a troubled project would be excluded from tracking of overall progress.

Finally, funding instability has been a leading cause of cost growth for NASA’s largest projects as unstable funding combines with cost and schedule baselines to produce incentives for projects to stretch out, delay, or cut expensive but necessary risk reduction activities. Yet addressing this challenge is difficult in the context of constrained budgets and sometimes competing priorities. Having a complete picture of costs can enable both the Congress and the administration to set priorities for both the short and long term. However, insight into the costs of two of NASA’s primary human spaceflight projects is limited. Specifically, in 2014, we found that NASA’s cost estimates for the SLS and Orion projects do not extend beyond the first flight for the combined system. These estimates do not include production costs for the second test flight scheduled for 2021; development costs for advanced booster and upper-stage development for future SLS variants; or production, operations, and sustainment costs for Orion beyond the first test flight. The limited scope that NASA has chosen for constructing cost estimates for these projects means that the estimates are unlikely to serve as a way to measure progress and track cost growth over the life of the projects. In addition, NASA has continued to request funding that does not meet requirements for some of its most expensive programs. For example, in 2014 we reported the SLS project has carried a high risk that it would not meet project cost and schedule requirements with the funding available which put at risk the agency’s planned launch readiness date in 2017.^[2]

There will always be inherent technical, design, and integration risks associated with NASA’s major acquisitions as these projects are highly complex and specialized and often push the state of the art in space technology. But there does not have to be a significant degree of management risk, particularly when it comes to estimating what resources are needed to complete a project, assessing whether projects are ready to move forward, and enabling sound management and oversight. NASA has already made strides in reducing acquisition risk from both a technical and management standpoint. More can be done, however, particularly with respect to anticipating and mitigating risks, implementing management tools, and forecasting costs for its largest projects. Areas that will be critical to improving NASA’s acquisition outcomes include the following:

• Ensuring that adequate and ongoing assessments of risks for larger projects—JWST, SLS, and Orion—are conducted, especially since each of these projects are at different critical points in development and implementation, as the impacts of any potential miscalculations will be felt across the portfolio.
• Ensuring that improvements to its acquisition policies are implemented consistently agency-wide. Specifically, providing completed guidance and training for both EVM and the JCL process would ensure that NASA’s workforce has the skills necessary to make best use of these tools and would provide additional insight into project performance and allow for more consistent JCL data across the portfolio.
• Ensuring that projects’ JCLs are updated regularly and consistently across the portfolio. As a project reaches the later stages of development, especially integration and testing, its risk posture may change. An updated project JCL would provide both project and agency management with data on relevant risks and impacts that can guide project decisions.
• Ensuring that projects that have been rebaselined report cost and schedule growth from original baselines in order to provide stakeholders with a more accurate view of project performance and to enhance accountability.
• Ensuring that long-term human spaceflight projects’ costs are understood. Specifically, long-term cost estimates for the SLS and Orion projects, which currently have estimates through only the first test flight, would provide decision makers with an informed understanding of the agency’s plans going forward.

Our ongoing work assessing the SLS project, the development of JWST, and the performance of the portfolio as a whole will provide insight into how well NASA is performing over the next several years.

Recommended Links

• National Defense and Space System Acquisitions