Quality of CPARS

CPAR Quality Checklist.

CPARS:  Building Quality into the Process

Air Force, Navy and Contractor perspectives

A. CONTRACTOR PERSPECTIVE

Mr. David M. Bowman
General Manager, AV-8B Program
The Boeing Company

Requirements/Recommendations:

• FAR and DFAR rules require that contractors have the opportunity to comment on adverse PPI on report cards as well as other PPI gathered under less formal collection methods.
• The form and content of this assessment continuum should be consistent throughout the contract performance period to ensure successful performance.
• The narrative is the most critical aspect of PPI assessments.
• Performance assessments are the responsibility of the program/project/contracting team, considering the customer’s input; no one office or organization should independently determine a performance assessment.
• Performance assessments should be developed throughout the period of contract performance, and not held to the end of the performance period.

Characteristics of a Win-Win CPAR Scenario:

1. Accurate
2. Fair
3. Relevant
4. Comprehensive
5. Repeatable Process
6. Timely
7. Joint Agreement
8. Consistent

Services and Program offices use/enforce CPAR process in same manner.

No surprises. Mutually agreed-upon expectations established early.

No hidden agendas. Not used as a "hammer" on the contractor.

Well trained subject matter experts.

Win-Win means Interim Assessments, Interim Assessments, Interim Assessments

 

Joint Win-Win Process

1. Clear Expectations
2. Interim Assessment 1
3. Interim Assessment 2
4. Interim Assessment… n
5. CPAR Report Card

 

B. NAVY PERSPECTIVE

Mr. Wallace Coggins
U.S. Navy, Naval Air Systems Command
PRAG Chairman, Joint Strike Fighter Program Office

Many CPARS are excellent:

• Current
• Ratings are justified and supported by narrative
• Narrative appears to tell the "whole" story

Lessons Learned:

• Keep CPARS current - avoid calls from the PRAG
• Primary purpose of CPARS is for use in source selections - not to manage contractors
• Accurate CPAR information allows source selection team to assess level of risk based on offeror's performance
• Contractors with strong past performance will be more competitive
• Contractors have an incentive to maintain good performance and address poor performance

Better CPARS:

• CPAR grades are consistent with CPAR definitions
• Writer backs up ratings with narrative, i.e., not just, "Contractor was exceptional" - but what was accomplished that exceeds requirements to Govt’s benefit
• Ratings and narrative are consistent
• Ratings and other program metrics are consistent (e.g., award fee, CPI, SPI)
• Tie the story together
• Make sure large rating shifts are well supported
• Document resolution of problems previously identified
• Involve DCMA personnel in CPAR process

C. AIR FORCE PERSPECTIVE

Ms. Margaret Gillam
U. S. Air Force, Air Force Space Command
Lead CPARS Focal Point

A quality CPAR:

• Allows the reader (who may not have source selection experience or personal knowledge of the Contractor’s performance) to gain a complete understanding of the Contractor’s performance on your contract
• Fully addresses the Contractor’s performance with respect to: RECENCY, RELEVANCY and QUALITY

Recency:

• Period of performance evaluated is a mandatory fill-in on the CPAR
• Is it within the past three years?

Relevancy:

• Need to provide a thorough CPAR so future readers can easily determine how the CPAR relates to the acquisition the reader is evaluating
• Consider scope of work, dollar amount, magnitude of the project, area of work
• Information contained in Block 17 (Contract Effort Description) is CRITICAL

CPARS form guidance:

• Block 17 - Contract Effort Description. Provide a complete description of the contract effort that identifies key technologies, components, subsystems, and requirements. This section is of critical importance to future performance risk assessment groups (PRAGs) and source selection authorities. The description should be detailed enough to assist a future PRAG in determining the relevancy of this program to their source selection. Also, keep in mind that users of this information may not understand program jargon. It is important to address the complexity of the contract effort and the overall technical risk associated with accomplishing the effort. For intermediate CPARs, a brief description of key milestone events that occurred in the review period may be beneficial (e.g., critical design review (CDR), functional configuration audit (FCA)), as well as, major contract modifications during the period. For task/delivery order contracts, state the number of tasks issued during the period, tasks completed during the period, and tasks which remain active. For contracts which include multiple functional disciplines or activities, categories should be designated to: (1) reflect the full scope of the contract, and (2) allow grouping similar work efforts within the categories to avoid unnecessary segregation of essentially similar specialties or activities. Each category or area should be separately numbered, titled and described within Block 17 to facilitate cross-referencing with the evaluation of the contractor's performance within each category in Blocks 18 and 19. If necessary, the description within this block may be extended to one additional typewritten page. MANDATORY.

Quality:

• Describe the Contractor’s performance for the time period covered by this CPAR
• Provide comprehensive and thorough details on the quality of services rendered by the Contractor
• A quality narrative should be based on objective data, program reviews, etc.

The NARRATIVE:

• Insufficient narrative is a common problem on CPARs
• Evaluators rely on the narrative, not the ratings, to support their source selection decision
• CPARS Guide requires that the Assessing Official provide narrative for each element that they rate, EVEN IF THE RATING IS SATISFACTORY

Who does past performance evaluations and the quality check of those evaluations?

• Evaluations will be done by the Business Resource Advisory Group (BRAG), if the contract were written under AFI 63-124, Performance Based Services Acquisitions
• Evaluations will be done by functional, technical, contracting, quality assurance and other specialties (JAG and finance) if contract were not written under AFI 63-124
• Past performance evaluation is a standard function of contract administration (FAR 42.1503)

Evaluation team must:

• Assign input duties and order of input (especially if assigning multiple Program Manager Reps)
• CO MUST review each CPAR
• Ensure evaluations/input timelines adhere to the 120 day goal for CPAR completion (including mandatory 30 days for Contractor review)
• Advise unit focal point of access authorization changes
• On an annual basis, schedule an evaluation meeting right after the end of the annual contract performance period (e.g., first week of October)
• BUT, don’t wait until the end of the evaluation period to provide the Contractors with feedback; good communication is a continuous process
• As a reminder, not ALL contracts begin in October so need to adjust according to your contract’s period of performance

Checklist for a "quality" CPARS:

• Establish up-front with the Contractor the areas which will be evaluated
• Contractor and all evaluators shall have a copy of the AF CPARS Guide and the definitions of the ratings
• Provide at post-award conference if new contract award
• Provide prior to annual evaluation meeting if an existing contract
• Areas to be evaluated may change based on additional work within scope changes or different mission emphasis
• Communicate, communicate, communicate…
• Don’t wait until the annual evaluation to make the Contractor aware of their performance
• Continuous communication gives the Contractor the opportunity to correct any deficiencies and should ultimately ensure better performance for the Government
• Contractor performance should NOT be a mystery to the Contractor!

Document, document, document…

• CPAR is done annually BUT need to document performance regularly (e.g., monthly Certificate of Service, semi-annual award fee, etc.) to ensure accurate recollection at the end of the evaluation period
• Also assists follow-on evaluators in the event the original evaluator’s areas of responsibility change during the life of the contract

Out-of-Cycle CPAR:

• Prior to an assessing official departing (or contract being transferred to another organizational element), the assessing official should complete an informational CPAR if at least four months have elapsed since the last CPAR was completed
• Informational form need not be processed through the Contractor and CPAR reviewing official; it should be passed to the succeeding assessing official for background information for completing the next CPAR

Team Member Responsibilities:

• All evaluating team members should provide input and thoroughly review the CPARS
• Review the definitions for the evaluation ratings contained in the AF CPARS Guide - compare them to the ratings/narrative given on the Contractor’s evaluation
• Ensure consistency between the rating given and the narrative for the rating
• An EXCEPTIONAL rating should have exceptional narrative to back it up
  

Last Updated: September 19, 2002