Fueled by the premise that improving products is predicated on improving processes used to develop and deploy them, early process improvement efforts based on the Capability Maturity Model® (CMM®) were focused on software systems. Documented cases of software systems fraught with problems underscored the need for scrutinizing software engineering processes against an industrystandard model compiled from proven best practices.

A plethora of benefits have been attributed to using the CMM for Software (SW-CMM)¹ across a broad spectrum of areas directly related to business growth and success. Such benefits include improvements in quality (measured in terms of defect reduction or earlier detection), productivity, cost, and schedule. Due to the scope of the model used, these benefits focused on the software aspects of system development, since improvements were typically constrained to software engineering processes and those directly supporting them.

Successful software process improvement spawned the development of models focused on other disciplines, including systems engineering and work force management. Each new model was earmarked for use across a subset of an organization, e.g., the systems engineering or software engineering organizational elements. The resulting stovepiped approach to process improvement resulted in inefficiencies caused by a different model for each discipline, and often inattention to integrated process improvement. However, as capability maturity model use extended across an enterprise, the benefits also accrued in those areas where process improvement ensued.

In 1998, an industry and government need surfaced for an integrated maturity model to achieve efficiency and effectiveness of processes and process improvement in a multidisciplinary environment. Earlier work by the Federal Aviation Administration (FAA), as described in this article, demonstrated a proof of concept for an integrated model. The 1998 industry/government effort led to creation of the CMM Integration^SM (CMMI®); CMMI for Systems Engineering, Software Engineering, Integrated Product and Process Development, and Supplier Sourcing (CMMI-SE/SW/IPPD/SS) V1.1¹, and several variations with more limited scope.

Source models used to create CMMI were the SW-CMM, Systems Engineering CMM (SE-CMM) Electronic Industries Alliance/Interim Standard (EIA/IS) 731², and the Integrated Product Development CMM¹. The CMMI model, with its focus on systems engineering, software engineering, integrated process and product development, and supplier sourcing had a broader scope than its predecessor's single-discipline models. However, it still lacked comprehensive coverage of broader enterprise processes.

Complex enterprises like the FAA and Lockheed Martin engage in engineering activities; operations; acquisition; supply; strategic and portfolio management; financial management; human resource management; and a host of technical, management, and support functions to operate their business. A number of these enterprise processes are not currently covered by the CMMI. However, there are multiple industry standards and models that can provide additional process improvement guidance to bridge the gap.

Both Lockheed Martin and the FAA have developed approaches to address process improvement needs that extend beyond CMMI and have incorporated practices and guidance from additional sources into their process improvement programs. The FAA and Lockheed Martin approaches differ, however, as described below.

The FAA's approach is to integrate existing models and standards into a single process improvement framework that can be used by any organization to guide process improvement within the scope of the model. The framework was developed with government and industry participation and is not FAA-specific. The framework has been designed to be flexible, with process areas used selectively according to the business needs of the implementing organization. The framework's scope continues to expand as explained below.

First Integrate Capability Maturity Models

The first problem the FAA faced was the concurrent use of multiple, single-discipline capability maturity models. Prior to 1997, the FAA was using the SW-CMM, the SE-CMM¹, and the Software Acquisition CMM (SA-CMM)¹. Each model provided guidance for different aspects of the FAA's work, which was very useful. In some instances, however, all three models provided guidance for the same work, performed by the same integrated team, which was confusing since the three models have different architectures and approaches, and use different, sometimes inconsistent, terminology. It was inefficient and ineffective to use the three models concurrently.

To solve this multiple-capability maturity model problem, the FAA integrated the SW-CMM, SE-CMM, and SA-CMM into a single integrated capability maturity model known as FAA integrated Capability Maturity Model (FAA-iCMM, or simply iCMM). The iCMM v1.0 was released in 1997 and as the first major integrated capability maturity model, it demonstrated that it was possible to integrate capability maturity models of different structures and scopes into a single model capturing all the principles and practices of the sources, using a single continuous with staging representation that includes both capability and maturity levels [1].

Next Integrate Beyond Capability Maturity Models

The iCMM rapidly became the predominant framework for capability maturity model-based improvement in the FAA, with programs and organizations making major strides in integrated process improvement. Yet there remained other critical software-related processes as well as broad enterprise processes that were not included in iCMM v1.0. Furthermore, there were non-capability maturity model standards that were of interest to stakeholders such as ISO 9001:2000 Quality Management Systems³ and Malcolm Baldrige National Quality Award criteria.

Since the concepts and approach of the iCMM were becoming institutionalized, the following question emerged: Can other standards and models besides capability maturity models be incorporated into the same framework? The FAA rose to this challenge and the iCMM was revised and expanded to update software and systems engineering guidance to the latest standards, to expand iCMM scope to address the full software/systems life cycle, and to address enterprise management. A total of 10 standards and models⁴ were integrated into iCMM v2.0, which was released in 2001 [2].

Each source integrated into iCMM v2.0 provided valuable insights and contributed to the content, comprehensiveness, and cohesiveness of the model. The following are some examples:

• Life-Cycle Coverage. Two source standards specifically intended to establish common frameworks for the life cycle were integrated: the ISO/ International Electrotechnical Commission (IEC) 12207 Standard for Information Technology — Software Life-Cycle Processes3, and the ISO/IEC 15288 System Engineering — System Life Cycle Processes3. These contributed to new process areas extending iCMM life-cycle coverage beyond development and maintenance to include deployment, transition, disposal, operation, and operational support.
• Acquisition and Supply. Three source standards (ISO/IEC 15288, ISO/IEC 12207, and ISO/IEC 15504 Information Technology — Software Process Assessment3) provide guidance for both acquisition and supply activities performed in an enterprise.
• Strategic Management, Business Results, and Performance Measurement. Strategic management guidance is provided in Baldrige, ISO/IEC 15288, ISO/IEC 15504, and ISO 9001, contributing to a new process area for enterprise management. The iCMM reflects a strong emphasis on performance management and business results that are fundamental in Baldrige, and the importance of measurement is strongly reinforced in both Baldrige and ISO 9001.
• Quality Management. ISO 9001 influenced the inclusion of prevention and root cause analysis as a natural part of quality assurance in the iCMM, as well as the need to determine customer satisfaction.
• Maturity and Capability Levels. The CMM and capability maturity model-type sources for iCMM (CMMI, EIA/IS 731, SA-CMM, SWCMM and SE-CMM) served to consolidate definitions of maturity levels (across the staged models) and capability levels (across the continuous models, with ISO/IEC 15504 also providing input regarding capability levels and generic practices).

For details showing how each source contributed to iCMM v2.0 at the practice level, see [3].

The iCMM continues to evolve with recent government/industry projects focusing on synthesizing and harmonizing standards-based best practices in safety and security assurance for use with both iCMM and CMMI [4], and in developing guidance for use of a common process improvement framework in the context of developing and using an enterprise architecture. Accompanying the iCMM are public training courses and a variety of appraisal methods as described in this article in the section Appraising Beyond Capability Maturity Models.

Lockheed Martin has had a long history of involvement in model-based process improvement and demonstration of high maturity using the SW-CMM, the EIA/IS 731, the CMMI, and other models. In 1998, an internal corporate study, "The Elements of Success," focused on program performance and reaffirmed process performance as a critical success factor in program performance. Subsequent to the study came the realization that integration of single-discipline processes across the organization was not guaranteed without a mechanism for measuring enterprise process integration. As a result, Lockheed Martin created the Integrated Engineering Process (LMIEP) Standard project to establish an integrated engineering process standard, a supporting infrastructure, and a measurement framework that enables collaborative, integrated, engineering and enterprise environments.

The LM-IEP Standard [5] provides a set of process integration requirements to be satisfied by each business unit's organizational standard process and related command media. The purpose is to create a concise, non-overlapping set of normative requirements applicable across a broader segment of the enterprise than covered by any individual industry standard or model already in use across the corporation. The integrated standard also allows for more efficient standards compliance, given the degree of overlap of several of the source documents. A Lockheed Martin corporate policy requires each business unit to conform to the standard, with application guidelines and timetables.

The LM-IEP Standard Revision 2.0 synthesizes requirements from CMMISE/ SW/IPPD/SS V1.1, American National Standards Institute/EIA 632 Processes for Engineering a System², ISO 9001:2000, ISO/IEC 12207, ISO/IEC 15288, Institute of Electrical and Electronic Engineers 1220 Standard for the Application and Management of the Systems Engineering Process [6], and an internal Lockheed Martin standard for hardware engineering. In 2004, AS9100 (Quality Systems — Aerospace — Model for Quality Assurance in Design, Development, Production, Installation, and Servicing) is being added, along with additional details on process architecture conformance.

Complementing the LM-IEP Standard is a comprehensive product suite, including training, a corporate-wide Process Asset Library, integrated measurement and risk management guides, and an appraisal method as described in the section below, Appraising Beyond Capability Maturity Models.

Benefits from implementing the SWCMM had been previously demonstrated by Lockheed Martin business units; there was high expectation that these benefits could be multiplied by deploying process requirements across a broader segment of the enterprise than required by stovepiped capability models already in use. Additional benefits accrue from using integrated processes and teams, founded on IPPD principles. Conformance to the LM-IEP Standard also provides a shared vision for integrated processes across the corporation and facilitates sharing work across business units.

Traditionally each maturity model has had its own appraisal methods issued as part of the product suite. For the SW-CMM, the methods include the CMM-Based Appraisal for Internal Process Improvement (CBA-IPI)¹ and the Software Capability Evaluation (SCE)¹; for the CMMI, the method is the Standard CMMI Appraisal Method for Process Improvement (SCAMPI)¹. Each of these methods complies with a defined set of appraisal requirements: the CMM Appraisal Framework1 for the CBA-IPI and the SCE, and the Appraisal Requirements for CMMI (ARC)¹ for the SCAMPI.

It is important to distinguish between appraisal methods and the reference models against which they appraise. Appraisal methods should be generic and applicable to any reference models that align with basic architectural structures used during appraisal such as goals (outcomes) and practices (activities) expected to be performed to achieve goals. The specific content of the reference model is not relevant as far as applicability of an appraisal method is concerned. Both Lockheed Martin and the FAA have developed appraisal methods that can be used to appraise processes in areas that extend beyond the SW-CMM and CMMI.

Lockheed Martin initially developed the Continuous Appraisal Method (CAM) [7] for use with EIA/IS 731, but the method is equally applicable to CMMI as well as other models with analogous architectures. To date, the CAM has been deployed extensively across the corporation with CMMI, and the method has been shown via a pilot to be well suited for appraisal against the extended process requirements in the LM-IEP Standard.

Having extensive experience with CMM appraisals using the CBA-IPI, Lockheed Martin developed the CAM with a vision of a new paradigm for process appraisal and improvement. The CAM differs from a traditional formal appraisal approach in its focus on appraising incrementally, over a period of nine to 12 months, with an opportunity to correct weaknesses documented during the appraisal, and have improved processes reappraised.

After identified weaknesses have been addressed, typically incrementally during the course of the appraisal, the CAM Maintenance Review is scheduled. This Review acts as a checkpoint to assure model compliance and process fidelity, i.e., that no backsliding occurred during the course of the appraisal. Limiting the overall appraisal period to a maximum of one year provides a boundary on the timeframe within which the organization must address weaknesses related to its target profile in order to achieve the desired rating.

The CAM was designed as a rigorous appraisal method, intended to satisfy all of the ARC Class A requirements. Additional design drivers for CAM included reducing appraisal cost; interleaving appraisal with process improvement in an open, penalty-free environment; minimizing appraisal disruption; and facilitating institutionalization.

The CAM reduces appraisal cost by minimizing appraisal preparation efforts, beginning with eliminating the need for preparing an extensive hardcopy objective evidence library. The ability to address weaknesses during the course of the appraisal also eliminates the need for multiple informal assessments to ensure that all of the practices/goals in the appraisal scope are in compliance before CAM begins. Extensive preparation of appraisal participants is unnecessary since CAM allows for explanation and/or clarification of practice interpretation during the course of the interviews, and there is no risk of failing the appraisal if a weakness is uncovered during an interview.

The CAM's interleaving of process appraisal with improvement (fixing weaknesses) allows for a timely feedback loop where practitioners get confirmation from the appraisal team that improvements resulted in model compliance. This approach also promotes shorter cycles of continuous process improvement as opposed to longer periods of process definition and rollout followed by extended periods of appraisal preparation and appraisal. Furthermore, there is no fear that failure to comply with a single goal/practice could result in missing achievement of the appraisal objective (e.g., a process maturity/capability level goal). As a result, CAM participants are more readily inclined to volunteer areas where improvement is warranted.

Appraisal disruption is minimized using CAM since the extended appraisal duration provides ample opportunity for scheduling around project and organizational milestones. In the case of a traditional two- or three-week formal appraisal, the impact on projects, as well as the organization, can be significant.

Although CAM initially focuses on a set of representative programs, the method promotes institutionalization across the organization by providing a mechanism for appraising additional projects following the initial appraisal. After the maintenance review, additional cycles of project appraisals can continue until all programs in the organization have been appraised. During each project appraisal cycle, CAM requires indicators of at least three months of process implementation as evidence that the process has been institutionalized.

Just as the FAA chose an integration approach for development of the iCMM reference model, it similarly integrated various appraisal methods for use in a variety of process improvement contexts. The evolution of the FAA-iCMM Appraisal Method (FAM) [8] has mirrored the evolution of the model.

Integrate Various Capability Maturity Model-Based Appraisal Methods

The FAM integrates a variety of appraisal approaches, offering six methods and variations: Full Internal, Full External, Questionnaire-Based, Interview-Based, Document-Intensive, and Facilitated Discussion. These methods draw upon various capability maturity model-based appraisal methods including the CBA-IPI, the SE-CMM Appraisal Method¹, the SCE, and the Interim Profile¹. In addition, the FAM formally describes methods based on document review and facilitated discussion self-appraisal. It is also possible to use the SCAMPI with the iCMM since the iCMM and the CMMI architectures are compatible. Similarly, the FAM variations are being used in safety and security assurance pilot appraisals that appraise organizational processes against both the iCMM and the CMMI.

Provide Multiple Results With a Single Appraisal

Improvements realized when using the iCMM simultaneously yield improvements against all its source standards and models. For example, achieving maturity Level 2 on the iCMM aligns with achieving maturity Level 2 on all its staged sources, including the CMMI, the SA-CMM, and the SWCMM. But what happens when going beyond capability maturity models?

For example, organizations pursuing iCMM-based process improvement might also have a business objective to achieve ISO 9001 certification; organizations that are already ISO 9001 certified might have additional business goals that iCMM can support. Such simultaneous improvements can be accomplished efficiently with an integrated model; it is important to provide explicit guidance regarding these needs [9]. To implement and demonstrate this concept, the FAA is piloting the Single Appraisal, Multiple Certification idea in collaboration with ISO 9001 auditors, whereby a single appraisal-audit process can result in both ISO 9001 certification and the iCMM appraisal results.

Appraise More Than Capability

The iCMM also has an appraisal method designed to measure the usefulness and cost effectiveness of process performance results [10]. This method builds on generic attribute concepts introduced in the EIA/IS 731, and encompasses similar ideas found in ISO 9000 and ISO/IEC 15504. It focuses on performance results rather than capability.

Both the FAA and Lockheed Martin have been implementing process improvement — beyond CMMI — for several years with resulting lessons learned.

For Lockheed Martin, the boldness of undertaking a corporate-wide LM-IEP Engineering Excellence Program whose scope was broader than any single model or standard is attributed to enlightened executive leadership that recognizes the business value of integrated process improvement across an enterprise. However, defining the process improvement agenda for the corporation based on the LM-IEP standard was no small feat. Specifically, synthesizing requirements from a diverse set of standards and models, many overlapping and written at varying levels of detail, was a difficult task that required expert knowledge of the source documents being synthesized.

A particular challenge was the objective of reducing the number of requirements in the LM-IEP standard to be significantly less than the composite number in the source documents. Furthermore, meticulous traceability of each requirement in the LM-IEP standard to its source was required to provide implementers with insight and informative references to facilitate understanding and interpretation.

For the FAA, using a single, flexible enterprise process improvement framework has paid off. Integrated iCMM-based process improvement has fostered shared improvement goals, a common improvement approach, and vertical and horizontal collaboration across disciplines, organizational lines, and the complete product or service life cycle. It enables organizations to focus improvement efforts on those parts of the iCMM that align with their business needs, and the model scope incorporates the business needs across a broad segment of the enterprise. The FAA's variety of appraisal methods has also facilitated improvement efforts.

A critical success factor in developing an enterprise improvement model is to recognize, incorporate, and integrate the principles and practices of international and national standards and performance-excellence criteria, while providing robust traceability to those sources.

Based on these experiences, the FAA and Lockheed Martin recognize the value of an integrated enterprise improvement framework to guide process improvement. Such a framework should be designed for flexible use across an enterprise, and should draw together widely recognized standards and approaches.

In future releases, the scope of the CMMI framework could be extended beyond engineering development and maintenance to address broader enterprise needs. For example, future extensions could include the following:

• Broader life-cycle coverage, e.g., deployment, transition, disposal, and operations.
• Broader enterprise coverage, e.g., acquisition, hardware engineering, finance, strategic management, work force management, information management, and the work environment.
• Mechanisms for adding specialty areas, e.g., safety and security.

In expanding the CMMI model scope, practices from international and national standards as well as other recognized best practices should be incorporated, as appropriate, with full traceability to sources. In addition to the model, the SCAMPI needs to be broadened to address incremental, delta, and multiple-certificate appraisals to meet user needs for efficient, effective appraisals in a variety of user modes and circumstances. A variety of successful methods should be considered for synthesis in developing new appraisal approaches.

Future releases of the CMMI Product Suite afford the opportunity to address broader, enterprise-level needs of organizations interested in realizing process improvement benefits across additional segments of their business.

1. Ibrahim, Linda, et al. The Federal Aviation Administration Integrated Capability Maturity Model (FAAiCMM), v1.0. Washington, D.C.: Federal Aviation Administration, Nov. 1997 www.faa.gov/ipg.
2. Ibrahim, Linda. et al. The Federal Aviation Administration Integrated Capability Maturity Model (FAAiCMM), v2.0. Washington, D.C.: Federal Aviation Administration, Sept. 2001 www.faa.gov/ipg.
3. Ibrahim, Linda, et al. Mapping Table Supplement to the FAA-iCMM v2.0. Washington, D.C.: Federal Aviation Administration, Oct. 2001 www.faa.gov/ipg.
4. Ibrahim, Linda, and Joe Jarzombek. "Safety and Security Extensions to Integrated CMMs." Software Engineering Process Group Conference 2004, Orlando, FL, Mar. 2004 www.faa.gov/ipg.
5. Lockheed Martin. Lockheed Martin Integrated Engineering Process (LM-IEP) Standard, EPI 280-01 Revision 2.0. 3 Jan. 2003.
6. Institute of Electrical and Electronics Engineers, Inc. IEEE Std. 1220-1998, Standard for the Application and Management of the Systems Engineering Process. New York: 1998 www.ieee.org.
7. Carr, Marvin, and W. Neil Crowder. "Continuous Appraisal Method (CAM) ... A New Paradigm for Benchmarking Process Maturity." Proc. of the Tenth Annual International Symposium of the International Council on Systems Engineering. Minneapolis, MN, July 2000.
8. Ibrahim, Linda, et al. The Federal Aviation Administration Integrated Capability Maturity Model (FAAiCMM) Appraisal Method (FAM), v1.0. Washington, D.C.: Federal Aviation Administration, Apr. 1999 www.faa.gov/ipg.
9. Ibrahim, Linda, and Curt Wells. Guidelines for Using FAA-iCMM v2.0 and ISO 9001:2000 in Process Improvement. Washington, D.C.: Federal Aviation Administration, 2004 www.faa.gov/ipg.
10. Wells, Curt, Linda Ibrahim, and Larry LaBruyere. "A New Approach to Generic Attributes." Systems Engineering 6.4 (2003): 301-308.

1. See www.sei.cmu.edu for information on the CMM and CMMI models and their appraisal-related products.
2. See www.eia.org or www.geia.org for information on Electronics Industries Alliance (EIA) standards, including Systems Engineering Capability Model (EIA 731-1), Systems Engineering Capability Model Appraisal Method (EIA 731-2), and Processes for Engineering a System (ANSI/EIA-632).
3. See www.iso.ch for information on standards from ISO and IEC.
4. Ten sources integrated into iCMM v2.0: ISO 9001:2000, EIA/IS 731, Malcolm Baldrige National Quality Award/Presidents Quality Award, CMMI, ISO/IEC TR 15504, ISO/IEC 12207, ISO/IEC CD 15288, and iCMM v1.0, containing SW-CMM, SA-CMM, and SE-CMM.

Linda Ibrahim, Ph.D., is the Federal Aviation Administration's (FAA) chief engineer for Process Improvement where she led development on and is lead author and architect of the FAA-integrated Capability Maturity Model v1.0 and v2.0, and its appraisal method. Ibrahim has been working in software engineering for more than 30 years in the United States, Europe, and the Middle East. She previously worked for the Software Engineering Institute, and is a member of the Capability Maturity Model® Integration Steering Group. Ibrahim has a Bachelor of Arts in mathematics, a Master of Science in information science, and a doctorate degree in electrical engineering.

Federal Aviation Administration
800 Independence AVE SW
Washington, D.C. 20591
Phone: (202) 267-7443
Fax: (202) 267-5069
E-mail: linda.ibrahim@faa.gov

Joan Weszka is the manager of Process and Program Performance at Lockheed Martin's Systems & Software Resource Center, providing consulting and training in areas including process improvement, program management, and engineering. Weszka has more than 25 years of experience in software and systems engineering and program management, and is a member of the Capability Maturity Model® Integration Steering Group. She has a Bachelor of Science in mathematics and a Master of Science in computer science from the University of Maryland.

Lockheed Martin Corporation
Systems & Software Resource Center
700 North Frederick AVE
Gaithersburg, MD 20879-3328
Phone: (301) 240-7013
Fax: (301) 240-7009
E-mail: joan.weszka@lmco.com