THIS OPINION WAS INITIALLY ISSUED UNDER PROTECTIVE ORDER AND IS BEING RELEASED TO THE PUBLIC IN REDACTED FORM ON JANUARY 30, 1995 ___________________________ DENIED: November 15, 1994 ___________________________ GSBCA 12959-P ATLIS FEDERAL SERVICES, INC., Protester, and RMS TECHNOLOGIES, INC., Intervenor, v. DEPARTMENT OF HEALTH AND HUMAN SERVICES, Respondent, and FORENSIC MEDICAL ADVISORY SERVICE, INC., Intervenor. Richard A. Dean of Arter & Hadden, Washington, DC, counsel for Protester. Pamela J. Mazza and Andrew P. Hallowell of Piliero, Mazza & Pargament, Washington, DC, counsel for Intervenor RMS Technologies, Inc. Lloyd M. Weinerman, Rodney L. Benson, and Jonathan A. Baker, Office of the General Counsel, Department of Health and Human Services, Baltimore, MD, counsel for Respondent. William A. Bradford, Jr., Robert J. Kenney, Jr., and Timothy L. Schroer of Hogan & Hartson L.L.P., Washington, DC, counsel for Intervenor Forensic Medical Advisory Service, Inc. Before Board Judges HYATT, WILLIAMS, and DeGRAFF. WILLIAMS, Board Judge. ATLIS Federal Services, Inc. (ATLIS), joined by intervenor RMS Technologies, Inc. (RMS), has challenged the award of a contract for Clinical Data Abstraction Center Services (CDACs) by the Department of Health and Human Services, Health Care Financing Administration (HCFA) to Forensic Medical Advisory Service, Inc. (Forensic).[foot #] 1 Pursuant to Rule 11, the parties elected to forego a hearing and present their cases on the written record. In the instant protest, ATLIS and RMS claim that HCFA failed to perform a proper cost/technical tradeoff. They argue that the solicitation required that the agency weigh cost and technical factors equally and that the agency failed to do so. We find that the agency did conduct a proper cost/technical tradeoff at the outset and affirm the agency's conclusions based upon our de novo review of the full record. Findings of Fact The Program This procurement involves HCFA's administration of the Medicare program, a Federal health insurance program for people sixty-five or older and certain disabled people. Protest File, Exhibit 7 at 137. Historically, HCFA has used peer review organizations (PROs) to assess the quality of care provided to Medicare patients. Id. PROs are groups of practicing physicians and other health care professionals who are funded by HCFA to review the care given to Medicare patients. Under contracts with HCFA, private PROs in each state review medical records "to determine the necessity, appropriateness, and quality of primarily inpatient hospital services and the validity of diagnoses and procedure . . . codes." Id. at 138. Reviews of PRO professionals may lead to remedial education, peer counseling, payment denial, and as a last resort, the exclusion of a provider or practitioner from Medicare. Id. The instant procurement is designed to move the PRO program from its emphasis on individual clinical errors to helping providers improve the mainstream of medical care through the use of the Uniform Clinical Data Set System (UCDSS). Protest File, Exhibit 7 at 140-42. The UCDSS is a Government-furnished software system developed by HCFA which includes the various data elements necessary for PRO case review. Id. at 140-41. The ----------- FOOTNOTE BEGINS --------- [foot #] 1 A protest challenging this award was originally filed by Aspen Systems Corporation (Aspen) on September 2, 1994. However, Aspen subsequently withdrew its protest, and the Board ruled that ATLIS' intervention survived as a protest. ATLIS Federal Services, Inc. v. Department of Health _____________________________________________________ and Human Services, GSBCA 12959-P (Oct. 13, 1994). __________________ ----------- FOOTNOTE ENDS ----------- purpose of this procurement is to have the CDACs, under a prime contract with HCFA and under subcontracts with the PROs, perform data entry/abstraction using the UCDSS software. Under the HCFA/CDAC contract, HCFA will work directly with and make payments to the CDACs on all administrative and management activities associated with the CDAC operation. Id. at 142. Under the subcontract, PROs will work directly with and make payments to the CDACs on data abstraction and hospital medical record management services. Id. The CDACs are a key component of HCFA's Health Care Quality Improvement Initiative (HCQII), its overall plan to improve the efficiency and effectiveness of the delivery of health care in the United States. Protest File, Exhibit 7 at 137. This acquisition represents the first time that the abstraction of clinical data from medical records is being done on such a broad scale and in such a large volume. Id., Exhibit 1 at 10. Initially, the CDACs will collect information in support of the Cooperative Cardiovascular Project based upon a sample of Medicare beneficiaries experiencing certain cardiovascular conditions and surgical procedures. Id., Exhibit 7 at 141. When fully operational, the CDACs will abstract clinical data from medical records for approximately an 8% sample of Medicare inpatient hospital discharges nationwide, which is expected to equate to 800,000 records each year. Id. For a typical inpatient medical record, the UCDSS will capture from 200 to 350 data elements including patient demographics, the case history, and medical intervention. Id. The Solicitation On July 19, 1993, HCFA issued request for proposals (RFP) number HCFA-RFP-92-034/dk, for CDACs to perform data entry and abstraction using standard data entry software provided by HCFA. Protest File, Exhibit 7 at 7. The offerors were to supply all necessary personnel, equipment, and materials that were required to perform the CDAC services. Id. at 15. The RFP advised offerors that HCFA intended to award a total of two CDAC contracts and that the workload for each such contract was determined by dividing the country into east and west zones. Protest File, Exhibit 7 at 146. The solicitation advised offerors that HCFA reserved the right to make a single award, multiple awards, or no award at all. Id. This protest is concerned with the east zone contract. The RFP provided that the east zone contract would be for an initial period of two years, with three one-year option periods. Protest File, Exhibit 7 at 7. The solicitation contemplated award of a cost-plus-fixed-fee type contract for the first year, and a cost-plus-award-fee type contract for each of the remaining four years. Protest File, Exhibit 7 at 7. HCFA's five-year cost estimate of the contract was $300,000,000. Id., Exhibit 5 at 3. The solicitation required that a CDAC provide five key personnel as well as skilled abstractors. Protest File, Exhibit 7 at 147-48. The CDAC was required to train and maintain a cadre of clinical data abstraction staff proficient in the use and operation of UCDSS. Id. at 183. The Statement of Work (SOW) required that the CDAC ensure that abstractors achieve an overall 85% reliability (accuracy) level in year one, a 90% reliability level in year two, and a 95% reliability level in years three, four, and five. Id. The SOW further required that for proposal purposes the CDAC use the average abstraction time of forty minutes per medical record for each year of the contract. Id. at 184. The SOW summarized the tasks as follows: On a monthly basis, CDACs shall request, on behalf of PROs [Peer Review Organizations] in their areas, and under authority of the subcontract, selected medical records (list to be provided by HCFA) from hospitals. Upon receipt of requested medical records from hospitals, the CDAC abstractors shall identify (abstract) specific data elements (as defined by HCFA) from the records and enter these elements into the UCDSS. The UCDSS PCAS [patient care algorithm system] expert system will screen the data elements from the selected medical records to identify those cases that should be subjected to physician review (hereinafter referred to as "flagged" cases). The CDACs shall print a case summary report upon completion of the abstraction process and the screening of the abstracted material through the PCAS. The CDACs shall forward the flagged cases along with the case summary to the appropriate PRO for physician review. The medical records and case summaries for cases not flagged by the UCDSS/PCAS hereinafter referred to as "clean" cases) will be maintained by the CDAC .... The CDACs will electronically transmit administrative workload reporting and tracking data (indicates the status of the medical record at all times) to HCFA and PROs on a weekly basis. The CDACs will electronically transmit the UCDSS clinical data (record of all abstracted cases) to HCFA (and by HCFA to PROs) on a monthly basis. Protest File, Exhibit 7 at 142-43. The CDAC contract requirements were identified and described in ten chapters of the SOW. Protest File, Exhibit 7 at 126-28. The solicitation apprised offerors of the weights assigned to each of ten chapters as follows: Chapter Summary Weight 1 The offeror shall be evaluated .5 on [its] understanding of HCFA/PRO roles, project background, general overview of CDAC operations, and the contract objective. 2 The offeror shall be evaluated 11.5 on the location of the CDAC facilities, the hours of operation, and the personnel that will work on this contract. 3 The offeror shall be evaluated 11 on the methods, policies, and procedures that will be used for accomplishing a timely and effective CDAC phase-in and workload phase-in (specifically address CCP [cooperative cardiovascular project] abstraction, random sample abstraction, and full implementation). 4 The offeror shall be evaluated 10 on an administrative workload reporting and tracking system. 5 The offeror shall be evaluated 8 on all hardware, software, and communications systems which will be used during the course of the contract. 6 The offeror shall be evaluated on 9.5 the methods, equipment, and techniques that will be used to exchange data with the PRO and with HCFA in an efficient and effective manner. The offeror shall also be evaluated on how data on CDAC operations will be reported to the PRO and to HCFA. 7 The offeror shall be evaluated 13 on how [it] will meet the workload management requirements, and on [its] contingency and workload back-up plan. 8 The offeror shall be evaluated 13 on [its] understanding of CQI [ c o n t i n u o u s q u a l i t y improvement],[[foot #] 2] [its] ability to participate in an EQC [external quality control program, and an innovative IQC [internal quality control] program that meets the requirements in the statement of work. The offeror shall also be evaluated on a training plan and [its] ability to implement the plan. 9 The offeror shall be evaluated on 15 [its] ability to abstract medical records in an efficient and effective manner. The offeror shall also be evaluated on all techniques that will be used to ensure accurate and reliable abstraction. 10 The offeror shall be evaluated on 8.5 [its] compliance plan that demonstrates an understanding of all of the security and confidentiality concerns associated with this contract. The compliance plan shall also be evaluated on how the specific requirements in the statement of work and the legal documents cited [therein] will be met. ----------- FOOTNOTE BEGINS --------- [foot #] 2 The solicitation required each offeror to have a continuous quality improvement (CQI) plan in place. Protest File, Exhibit 7 at 178. CQI is a management tool through which "[o]pportunities for improvement are identified through the focus on systems and processes and the use of statistical techniques to identify system/process improvements." Id. at 177. ___ ----------- FOOTNOTE ENDS ----------- Protest File, Exhibits 7 at 126-28, 8 at 126-28. Section C.8.3(3) of the RFP, "Continuous Quality Improvement," included: "The CDAC shall ensure that the IQC [internal quality control] program will adequately identify, through reabstraction of a sample of cases and other methods, specific and general sources of reliability (dependable, reproducible) and unreliability . . . ." Protest File, Exhibit 7 at 178. Additionally, the CDAC is required to "design an IQC program which allows the CDAC to determine overall abstractor reliability" and to conduct the minimum number of IQC reabstractions to ensure "reliable valid abstraction." Id. at 179. The RFP provided that the ten chapters would be evaluated based upon five technical evaluation criteria weighted as follows: Criteria Weights Technical Understanding (TU) 15% Technical Approach (TA) 25% Management Approach (MA) 25% Experience (E) 20% Risk Assessment (RA) 15% Protest File, Exhibit 7 at 123-24. The evaluation criteria were further described as: Technical Understanding (TU) -- The offeror shall be evaluated on [its] understanding of the purpose of the contract, the specific tasks to be performed, the roles and responsibilities of each of the organizations involved in the abstraction process, the workflow between the offeror, the PROs and HCFA, and the contracting arrangements between the offeror, the PROs and HCFA. Technical Approach (TA) -- The offeror shall be evaluated on [its] technical approach to performing the specific tasks in the statement of work -- the overall technical strategy, how that strategy will be implemented, what tools will be used, why those tools will be used, and how those tools and techniques will be acquired or developed. Management Approach (MA) -- The offeror shall be evaluated on [its] management and organizational approach as it relates to CDAC functions -- offerors will be evaluated on their overall organizational structure, specific organizational structures for each CDAC function where applicable, their overall corporate personnel structure, specific personnel structures for each CDAC function where applicable, specific cost containment techniques, and the proposed relationships and lines of authority between the offeror, the offeror's subcontractors, the PROs, and HCFA. Experience (E) -- The offeror shall be evaluated on [its] General experience, Organizational experience, Performance history, Pertinent contracts, Pertinent grants and Personnel experience. General experience is the overall experience and qualifications of the offeror. Organizational experience is the accomplishment of work, either past or present, which is comparable to the work described in this RFP. Performance history is the meeting of contract objectives within delivery and cost schedules for either past or present contracts in which the work is comparable to the work described in this RFP. Pertinent contracts are contracts that were similar in scope to the work described in this RFP and the work was completed in the last 3 years or is currently in progress. Pertinent grants are grants that involved work similar to that described in this RFP. Personnel experience is the experience of all the personnel that will be working on . . . this contract. Risk Assessment (RA) -- The offeror shall be evaluated on the risk involved with the offeror's ability to perform the specific tasks in the statement of work. Protest File, Exhibit 7 at 124-25. The RFP Paragraph L.1.6, Evaluation of Proposals, General Procedures, provided that equal consideration would be given to the evaluation of technical and cost proposals and stated: You are advised that equal consideration shall be given to the evaluation of technical and business (cost) proposals. Following evaluation of the best and final offers, the contracting officer (or other designated source selection authority) shall select that source whose best and final offer is most advantageous to the Government. Protest File, Exhibit 7 at 93. Paragraph M.1.0, Evaluation Factors for Award, provides in pertinent part: GENERAL PROCEDURES a. . . . . The evaluation process for this acquisition will be two-phased. Initially, the offeror's technical proposal will be evaluated for technical merit using the evaluation categories and relative weights . . . . Secondly, the offeror's proposed price and/or costs will be considered independently of the technical proposal criteria. Any contract(s) resulting from this solicitation will be awarded to that responsible offeror whose offer, conforming to the solicitation, will be most advantageous to the Government in terms of technical merit, cost or price and other factors. . . . . b. You are advised that equal consideration will be given to technical and cost proposals. Protest File, Exhibit 7 at 122. Paragraph M.4.0, Cost Evaluation, provided that the evaluated cost for each offeror would be determined by adding the total estimated cost and award fee for all periods of performance (base and option periods). This paragraph further advised offerors that the realism of the proposed cost would be evaluated and that the Government would perform a probable cost analysis if necessary. Protest File, Exhibit 7 at 129. The Source Selection Plan On July 26, 1993, HCFA approved the source selection plan for the CDAC procurement. Protest File, Exhibit 5. The source selection plan indicated that the procurement would be conducted via formal source selection procedures using three tiers: (1) separate cost and technical evaluation panels, (2) a source evaluation board (SEB), and (3) a source selection official (SSO). Id. Specifically, the technical evaluation panel (TEP) was responsible for evaluation of all technical aspects of the proposals and was to work independently and report its findings, ratings, and rankings to the SEB. Id. at 9-10. The TEP was also to prepare a final ranking of the proposals in a final evaluation report. Id. at 10. The business evaluation panel (BEP) was responsible for evaluating business considerations or factors relating to cost/price analysis and determining the contractor's responsibility. Id. at 10-11. This panel was also to work independently and to prepare its final cost evaluation report. Id. The SEB was to be comprised of senior level individuals having responsibility for the various functional areas involved in the acquisition. Protest File, Exhibit 5 at 7. The SEB was also charged with preparing a final report for the SSO supporting the recommendation for the award of a contract to that offeror whose proposal was deemed to be most advantageous to the Government. Id. at 8. Finally, the source selection plan established that the SSO would make the competitive range determination and the final source selection decision following an in-depth review and consideration of all information and data available from the SEB. Id. at 6. According to the source selection plan, the business management and price proposals would not be point-scored during evaluation. Instead, they would be analyzed and evaluated in order to determine price reasonableness. Protest File, Exhibit 5 at 5. The Technical Evaluation Guide In September 1993, HCFA issued the technical evaluation guide for the CDAC procurement which essentially reiterated the source selection procedures in the source selection plan. Protest File, Exhibit 6. This guide was designed to assist the TEP. Id. at 4. The technical evaluation guide assigned weights, for purposes of the evaluation of proposals, to each of the ten chapters or sections of the CDAC statement of work, and to each of the five evaluation criteria (TU, TA, MA, E, & RA) as follows: CHAPTER TU TA MA E RA TOTALS 1 .5 2 11.5 3 11 4 10 5 8 6 9.5 7 13 8 13 9 15 10 8.5 TOTALS 15 25 25 20 15 100 Id. at 9. Each of the forty-three evaluation subcriteria containing a numerical weight in the above grid contained one or more subrequirements and/or questions. Id. at 9-10. As reflected in the chapter summary chart, in chapters 3, 4, 6, 7, 8, 9, and 10, offerors were required to address each of the five technical criteria, i.e., technical understanding, technical approach, management approach, experience, and risk assessment. Protest File, Exhibit 6 at 9. For chapter 1, offerors were only to address technical understanding. For chapter 2, offerors were to address management approach, risk assessment, and experience, for chapter 5, everything except management approach. Each subcriterion was to be evaluated on a scale of from one to ten. Protest File, Exhibit 6 at 27. In scoring each subrequirement, the evaluator was to use the following as a guideline: DESCRIPTION CREDIT AWARDED EXAMPLE--10 POINTS Exceptionally meets Extra credit 9-10 requirements Meets requirement Full credit 8 Mostly meets Almost full 6-7 requirement credit Partially meets Some credit 1-5 requirement Does not meet No credit 0 requirement Id. at 10. Thus, any score of less than 8 meant that an offeror had not fully met a requirement. The technical evaluation guide provided that the score for a subrequirement, on the scale of one to ten, was to be multiplied by the weight assigned to the subrequirement to determine a score for the subrequirement. Protest File, Exhibit 6 at 9-10, 27. The sum of scores for the subrequirements for each subcriterion determined the total score for the subcriterion. Id. After the technical evaluators had finished scoring a proposal individually, the TEP was to determine a consensus score for the proposal. Protest File, Exhibit 6 at 10. The Submission of Initial Proposals By the September 24, 1993, closing date for the receipt of proposals, HCFA received fifteen proposals for the east zone and twelve proposals for the west zone. Protest File, Exhibit 20 at 4-6. The offerors submitting proposals for the east zone included ATLIS, RMS, and Forensic. Protest File, Exhibits 12, 13, 14, 15, 18, 19. Protest File, Exhibit 20 at 5-6. ATLIS submitted proposals for both the east and west zones. Id. at 5-6. On March 23, 1994, the BEP and the TEP submitted a report on the evaluation of initial proposals to the SSO and the SEB. Protest File, Exhibit 20. ATLIS' Proposal ATLIS proposed imaging technology. Protest File, Exhibit 12, at 4-5. This means that the entirety of the record would be placed in computer memory and could be retrieved. Id. According to ATLIS' proposal, it would be using "high speed optical scanner[s] to create an image of each medical record and store the image on magneto/optical disk for subsequent recall, abstraction, distribution and storage." Id. ATLIS' abstractors would work from the image of the records on the optical disk, not from the hardcopy records. Id. at 4-6, 4-27. ATLIS' approach to the reporting and tracking systems was described in its proposal to be: Prepared to respond to HCFA and PROs at any time regarding the location, processing stage, photocopying and mailing costs of medical records; Capable of providing automatic updates for each function performed; and, Capable of tracking medical records from initial HCFA notification of the sample selection through release of flagged medical records to the PRO or destruction of clean case medical records. Protest File, Exhibit 12 at 1-8. ATLIS described its CQI plan as including: Total Quality Management/Continuous Quality Improvement program awareness training, a CQI steering committee, baseline assessment which will provide the CDAC with the data that it needs to assess accurately its strengths and priorities for improvement, strategic planning, and quality council meetings in which councils will develop missions and operating guidelines. Protest File, Exhibit 12 at 8-3 through 8-6. RMS' Proposal RMS' proposal indicated that the contract would be performed by RHS, a newly formed entity which would only exist in the event of this award. RHS was comprised of RMS and its joint venture partner West Virginia Medical Institute (WVMI).[foot #] 3 Protest File, Exhibit 14. ----------- FOOTNOTE BEGINS --------- [foot #] 3 We refer to this offeror (RMS and WVMI) as RMS, as has been done in this intervenor's submissions in this protest. ----------- FOOTNOTE ENDS ----------- Forensic's Proposal Forensic related its experience in similar abstracting of large amounts of medical record data. Protest File, Exhibit 18 at 2-30 through 2-33, 3-20 through 3-22, 7-15 through 7-16. In 1986, Forensic was the data collection subcontractor for the Department of Defense (DoD) Civilian External Peer Review Program (CEPRP) and provided on-site medical record abstracting at the more than 160 military medical treatment facilities around the world. Id. at 3-20. Its proposed CDAC project director was also the project director for CEPRP. This work was accomplished on time, and to the satisfaction of DoD. Id. at 3-21. In 1987, Forensic was awarded the prime contract to administer DoD's CEPRP. Id. at 2-31, 3-21. In 1992, Forensic won the recompetition of that contract for performance through 1997. Id. at 2-31. The proposed Forensic CDAC project director who is also a vice president at Forensic had directly managed the CEPRP project through all seven years of its contracts, as well as the DHHS medical review program through two different contract periods. Id. at 9-24. Forensic's DHHS contract was a series of task order contracts with the Office of Inspector General of the Department of Health and Human Services which Forensic has had since 1987. Id. at 3-22. Under those task orders, Forensic abstracted medical records and performed peer review on cases involving surgical care, psychiatric care, and medical care for ambulatory and inpatient facilities. Id. In another contract, Forensic operated a similar system for automobile insurers in the state of Pennsylvania involving claim processing for personal injury claims. Id. at 3-21 through 3-22. With regard to recruitment, Forensic recognized that staff recruited for the Forensic CDAC might not be available at the time of contract award and, therefore, obtained written commitments from more than twice the number of personnel that would be needed during the initial four months of the contract. Protest File, Exhibit 18 at 3-23 through 3-24. Forensic stated its CQI program is designed to allow the CDAC employees to: Forensic also identified measures by which to assess quality control. Protest File, Exhibit 18, Figures 8.3-8.4. As part of its internal quality control program, Forensic identified factors which affect the quality of abstracts, and controls to implement quality assurance at the three stages of production as well as methodology for measuring the quality of collected data. Competitive Range Determination On March 31, 1994, HCFA made its competitive range determination. Protest File, Exhibit 21. On April 6, 1994, HCFA informed ATLIS, Forensic, and RMS that their proposals were in the competitive range. Protest File, Exhibit 22. Between June 15, 1994, and June 30, 1994, oral discussions/negotiations were conducted telephonically with the offerors in the competitive range. Protest File, Exhibit 53 at 6-7. The Submission and Evaluation of BAFOs By letters dated June 30, 1994, HCFA requested best and final offers (BAFOs) from all offerors in the competitive range. Protest File, Exhibit 41. On or before July 14, 1994, BAFOs were received from each of the offerors in the competitive range. Protest File, Exhibits 44-47, 49-50. On August 16, 1994, the BEP and the TEP submitted to the SSO and the SEB a report on their respective evaluations of the offerors' BAFOs. Protest File, Exhibit 51. This report was provided to the SSO and SEB for use at an August 17, 1994, meeting at which time the CDAC contract award decisions were made. Id. This memorandum contained a comparison of the technical and business proposals of all BAFOs in the competitive range in both zones. Protest File, Exhibit 51, Attachment A at 18-70. The memorandum also included the results of the TEP evaluation after BAFOs. Id., Attachment B. The final technical ranking of the offerors for the east zone, and the offerors' proposed costs, were as follows: Contractor Score Cost Forensic Medical $33,676,519 Atlis Federal Systems 87.10 86.40 Dyncorp/KePRO (DynCorp) 85.80 85.10 Birch & Davis 85.00 Aspen Systems Corp. 79.60 73.90 Protest File, Exhibit 51 at 9. The final technical ranking of offerors for the west zone, and offerors' proposed costs, were as follows: Contractor Score Cost Protest File, Exhibit 51 at 9. In the east zone evaluation, the TEP concluded that the proposals of Forensic, ATLIS, DynCorp/Kepro, and three other offerors exceeded the scope of work requirements, offered tremendous enhancements, and based upon the proposal demonstration, presented no major risks to the Government. Protest File, Exhibit 51 at 73. RMS was not included in this grouping. In the final technical evaluation, RMS received the following scores: Protest File, Exhibit 51 at 100. The TEP concluded that although RMS slightly exceeded the scope of work requirements in some areas, it "continues to be weak in critical areas, and based on the technical proposal poses a major risk to the Government." Protest File, Exhibit 51 at 74. Specifically, the TEP concluded: Id. With respect to RMS, the TEP identified "current proposal weaknesses" as follows: [[foot #] 4] Protest File, Exhibit 51 at 99. The TEP identified three "[p]otential [m]ajor [r]isks" associated with RMS' proposal -- The TEP scored ATLIS' technical BAFO as follows: ATLIS TU(15) TA(25) MA(25) E(20) RA(15) TOTAL 1 90% 90% 2 89% 83% 82% 86% 3 87% 82% 82% 92% 76% 84% 4 87% 92% 93% 96% 85% 91% 5 84% 98% 92% 88% 93% 6 90% 90% 83% 85% 85% 87% 7 82% 88% 86% 90% 80% 86% 8 84% 78% 87% 83% 75% 82% 9 97% 95% 90% 89% 84% 91% 10 84% 89% 83% 98% 82% 86% TOTAL 87% 89% 87% 88% 82% 87% Protest File, Exhibit 51 at 85. ----------- FOOTNOTE BEGINS --------- [foot #] 4 "Section" refers to a numbered chapter of the proposal. ----------- FOOTNOTE ENDS ----------- In its final evaluation report, the TEP stated with regard to ATLIS: Atlis achieved enhanced scoring in all 5 evaluation criteria. However, looking at the 43 breakdown scores, they continue to have a weakness in their risk assessment for Section 3 (CDAC and Workload Phase-In), and technical approach and risk assessment for Section 8 (Continuous Quality Improvement). The offeror did not adequately demonstrate the risk and the mitigation factors for phase-in and did not demonstrate the exact statistical measures they intend to employ to operate their CQI program. Protest File, Exhibit 51 at 84. The TEP scored Forensic's technical BAFO as follows: FORENSIC TU(15) TA(25) MA(25) E(20) RA(15) TOTAL Protest File, Exhibit 51 at 82. On August 17, 1994, the SEB and the SSO, the director of HCFA's Health Standards and Quality Bureau, met to make a selection of the CDAC contractors. Protest File, Exhibit 51 at 4, Exhibit 52. They first analyzed the technical and business results of the west zone offerors, and eliminated those proposals which presented an unacceptable technical risk. Protest File, Exhibit 52 at 4. After considering all remaining proposals in the west zone, they concluded that because DynCorp proposed enhancements directly related to the primary objective of the contract, i.e., to retain reliable, efficient data abstraction, had extensive continuous quality improvement experience, was a low technical risk, and proposed lower costs than any other remaining offeror, award should be made to DynCorp. Protest File, Exhibit 52 at 7-8. Following their decision to award the west zone contract to DynCorp, the SEB and the SSO considered the viable technical and business results of the east zone competition. After analyzing all nine proposals, they concluded that four proposals presented high risk to the Government -- including that of RMS. Protest File, Exhibit 52 at 6. Thus, RMS and three other offerors were eliminated from further consideration. Id. Id. In a clarification to the source selection memorandum, the chairperson of the TEP stated that the weaknesses of RMS presented a "substantial and unacceptable risk to the Government," despite the lower cost of the proposal. Id., Clarification Memorandum, at 2-3. Because the technical differences among the remaining five proposals were not numerically significant, the SEB and the SSO decided to start with the lowest cost proposal and analyze the characteristics of each of the offerors. Id. at 10. The lowest cost proposal was that of DynCorp whose costs were Id. Because DynCorp submitted proposals of such value in both the west and east zones, the SEB and SSO reconsidered the intention to award the contracts to two different organizations. Id. However, they ultimately concluded that, to insure a different backup in case of failure, they would turn to another offeror. Id. In doing so they recognized that "[l]arge scale abstraction of medical records into a standard data set is virtually a new industry." Id. The SEB and the SSO next evaluated each of the remaining offerors, examining the lower cost proposals first. Protest File, Exhibit 52 at 11. They concluded that the proposal of and that of ATLIS were "essentially equal." They reasoned: Both ATLIS and proposed innovative operating, medical records scanning and imaging. As a result of this new technology, many cost containment measures were proposed by both offerors. Use of this technology also enhances the confidentiality and security measures proposed by the offerors since images of the hard copy medical records can be produced. There is no longer any need to maintain the hard copy medical records which reduces the offeror's storage requirements. Id. However, they also noted with regard to ATLIS' proposal: However, the primary objective of the CDAC contract is reliable, efficient data abstraction/entry services and ATLIS failed to achieve a met [or fully acceptable] score (80%) in 2 of the 5 subcriteria related to continuous quality improvement which is the means by which reliability of data abstraction is measured. Id. Moving on to the next low cost proposal, i.e., that of Forensic, the chairman of the TEP noted that unlike ATLIS and Forensic achieved a met score (80%) in all sections of its proposal. Protest File, Exhibit 52 at 12. In assessing the Forensic proposal, the SEB concluded: Many of the enhancements proposed by Forensic are directly related to the primary objective of the contract; i.e., reliable, efficient data abstraction/entry services. Forensic demonstrated that they have an enhanced understanding of the HCQIP program objectives and the administrative requirements of the contract. The offeror proposes strong key management staff with outstanding qualifications. They have implemented an advanced hiring program. Forensic has interviewed and screened personnel and provided a complete list of skilled abstractors with enhanced qualifications. These abstractors can report to the CDAC facility immediately. This advanced hiring reduces the workload phase-in risk which is critical to this contract. Forensic demonstrated an excellent working knowledge of many of the aspects of the Scope of Work. They were able to outline extensive risk factors and how those factors would be handled. Throughout their proposal, Forensic was able to note enhancements and cost containment measures, explaining to HCFA how to improve their Scope of Work requirements. Forensic has had extensive medical record data abstraction experience similar to that required within the CDAC contract. Of all the offerors in the east and west zones, Forensic achieved the highest score (90%) in the experience evaluation criteria. Currently, Forensic has a DoD contract for External Peer Review which involves abstraction of clinical data. Under this contract, Forensic maintains a program of inter- rater reliability assessment and reabstraction. Their results have yielded a accuracy rate in data collection which is well above the rate required under the CDAC contract. Forensic also possesses an excellent understanding of and approach to the CDAC operating environment. Id. at 12-13. The SEB and the SSO concluded that both and ATLIS posed more of a technical risk to the Government than did Forensic. Protest File, Exhibit 52 at 13. When ATLIS' proposal was compared with Forensic's, the SEB and SSO determined: Atlis is proposing to handle a huge volume of clinical data abstractions using a new technology that has never been tried and proven on as large a scale as contemplated in the CDAC acquisition. . . . HCFA cannot afford increased risk of failure or work slow-downs under this contract since this would have a serious impact on 53 PRO programs and the Health Care Quality Improvement Program. In addition, Atlis demonstrated weaknesses in 2 of the 5 subcriteria related to continuous quality improvement (CQI). CQI improves reliability, a quality emphasized in the solicitation. Id. at 13. The SEB and the SSO determined that posed increased technical risk because it did not have experience using the imaging method on a large scale basis, and because it demonstrated a weakness in its ability to recruit and maintain abstractors. Id. After highlighting the benefits of Forensic's proposal, the SEB and the SSO concluded that: Forensic's slightly higher cost spread out over a 5 year period was well worth the technical advantages and diminished risk associated with the Forensic proposal. Based upon this analysis, the SEB and SSO determined that for the East Zone Forensic's proposal was most advantageous to the government. Protest File, Exhibit 52 at 14. Award On August 26, 1994, HCFA awarded the west zone contract to DynCorp/KePRO and the east zone contract to Forensic. Protest File, Exhibit 52 at 9, 14. Discussion ATLIS, joined by RMS, contends that the agency failed to conduct a proper cost/technical tradeoff in selecting Forensic for the east zone contract award. ATLIS does not challenge the technical scoring or evaluation of any proposals. Rather, ATLIS contends that the cost/technical tradeoff analysis was contained "in one sentence" and was wholly inadequate. ATLIS quotes that one sentence from the source selection memorandum as follows: Forensic's slightly higher cost spread out over a 5 year period was well worth the technical advantages and diminished risk associated with the Forensic proposal. ATLIS' Record Submission at 6 (quoting Protest File, Exhibit 52 at 14). ATLIS contends that no reasons for this summary finding were advanced. We disagree with ATLIS' characterization of the record in this case. As detailed in our findings of fact, the TEP, the SEB, and the SSO articulated reasoned conclusions reflected in the source selection documents about the strengths and weaknesses of the technical proposals in the competitive range. The conclusions in the written memoranda support the SSO's determination that Forensic's proposal was most advantageous to the Government. The record demonstrates that the technical enhancements in Forensic's proposal justified award to it despite the slightly higher cost. Forensic had extensive experience in comparable projects -- including one particularly similar contract with the Department of Defense -- it fully met every requirement of the RFP, its abstractors had enhanced abstracting skills, and it had already recruited abstractors. Throughout its proposal Forensic identified risks and explained how it would address them. ATLIS' proposal in contrast did not receive a fully acceptable score in all forty-three subcriteria. ATLIS was rated less than acceptable in certain areas of continuous quality improvement -- a criterion directly related to reliability, which is critical to this program. In addition, the SEB and SSO reasonably considered ATLIS' imaging technology to present a performance risk given the newness of this type of technology being used on such a large scale. Because this was a cost reimbursement type contract and proposed costs were necessarily uncertain, the decision to award to the higher-scored technical and higher-cost proposal was reasonable. Federal Acquisition Regulation 15.605(d), 48 CFR 15.605(d) (1994), instructs that in a cost reimbursement type procurement, cost should not be the paramount factor in the selection decision. (In evaluating offers in cost reimbursement type contracts, agencies should not accord controlling weight to cost proposals because evaluated costs may not prove to be valid indicators of final actual costs.) Thus, the uncertainty of the actual costs to be incurred coupled with the agency's exposition of the technical superiority of the Forensic proposal over those of ATLIS and RMS persuades us to deny the protest. Moreover, our de novo review of the record causes us to affirm the conclusions reached by the SSO. ATLIS further argues that the SEB's conclusion that ATLIS' proposal presented unacceptable risks was inconsistent with the TEP's statement that ATLIS' proposal presented no major risks. ATLIS' Record Submission at 8. ATLIS' argument must fail. Even if there were inconsistencies, the SEB was not bound to adopt each and every finding of the TEP. As the Board recognized in Arthur Andersen and Co., GSBCA 8870-P, 87-2 BCA 19,922, at 100,815, 1987 BPD 94, at 18, a source selection advisory council is entitled to reject the evaluation of the source evaluation board because "discretion . . . must be exercised by each level in the evaluation and selection process." See also Burnside-Ott Aviation Training Center, Inc. et al., B-233113, 89-1 CPD 158, at 9 (Feb. 15, 1989); Barron Builders and Management Co., B-225803, 87-1 CPD 645, at 5 (June 30, 1987) ("[G]iven their wide discretion, selection officials are bound neither by those technical scores nor by the source selection recommendations of the technical evaluators."). ATLIS further contends that risk was subsumed in the technical evaluation of the proposals and was accounted for in its technical score, and that HCFA effectively "count[ed] risk twice." ATLIS' Record Submission at 9. We reject this contention. The SEB's and SSO's conclusions about the risk associated with ATLIS' proposal were rational and consistent with the solicitation. The SEB noted that ATLIS proposed to handle a huge volume of clinical data abstractions using a new imaging technology that has never been tried and proven on as large a scale as contemplated in the CDAC acquisition. The source selection memorandum further noted that HCFA could ill afford an enhanced risk of failure which could compromise the program. Given the volume of records to be abstracted, the SEB and SSO were justified in concluding that the newness of ATLIS' methodology posed risk, and in using risk as a discriminator. See Oak Creek Funding Corp., GSBCA 11244-P, 91-3 BCA 24,200, at 121,041, 1991 BPD 156, at 9-10. (Risk is valid consideration in cost/technical tradeoff). Moreover, the agency's decision to award the east zone and west zone contracts to two different vendors, so as to have a backup in the event one contractor failed, underscores the agency's concern with risk. ATLIS also claims that because its technical score was only .1% lower than Forensic's and its costs were .96% lower, it is entitled to a directed award. We disagree. The SSO is permitted to look behind the numerical scores given to the proposals by the evaluators and assess on a larger scale and on a comparative basis the relative strengths and weaknesses of the proposals. As this Board has noted: The legal standard against which we review best value determinations is not one of perfection or even accuracy but rather one of reasonableness. Indeed, we have upheld the best value determination even when 'some elements of the best value analysis lack a rational basis.' There is no formulaic methodology for conducting a best value determination; what matters is that award is consistent with the terms of the solicitation and that any price premium is justified by specific technical enhancements. Grumman Data Systems Corp. v. Department of the Air Force, GSBCA 11939-P, 94-2 BCA 26,822, at 133,435, 1994 BPD 62, at 56, aff'd sub nom Grumman Data Systems Corp. v. Widnall, 15 F.3d 1044 (Fed. Cir. 1994). Here, ATLIS would have us derive the costs associated with each technical point scored by Forensic and ATLIS and, since ATLIS' per-point score was lower, direct award to ATLIS. ATLIS' Record Submission at 10-11. We reject forthwith such a formulaic, mechanistic approach to a cost/technical tradeoff. ATLIS' approach over-emphasizes the equal weight to be given cost and technical and ignores the solicitation's mandate to award to the offer most advantageous to the Government. In the context of cost/technical tradeoffs, the United States Court of Appeals for the Federal Circuit has recognized that "agencies 'are entrusted with a good deal of discretion in determining which bid is the most advantageous to the Government.'" Lockheed Missiles & Space Co. v. Bentsen, 4 F.3d 955, 958 (Fed. Cir. 1993) (quoting Tidewater Management Servs., 573 F.2d 65, 73 (Ct. Cl. 1978)). Further, this Board and GAO have held that the SSO retains the discretion to determine whether the technical advantage associated with a given proposal is worth a higher price notwithstanding the fact that price is to be given equal consideration under the evaluation scheme of an RFP. This Board has upheld cost technical tradeoffs where the agency has awarded to the higher-priced, technically superior offeror under solicitations where technical and cost factors were of equal value. E.g., Systems & Computer Technology Corp., GSBCA 8817-P, 87-2 BCA 19,703, at 99,761, 1987 BPD 34, at 15. There, the Board recognized: "[cost/technical] tradeoffs may be made and the extent to which one may be sacrificed for the other is governed only by the tests of rationality and consistency with the established evaluation factors." 87-2 BCA at 99,761, 1987 BPD 34, at 15 (citations omitted), cf. Syscon Corp. v. Department of the Army, GSBCA 12803-P, et al. 94-3 BCA 27,007, at 134,545, 134,559-60, 1994 BPD 132, at 14, 41-42 (award to higher-priced vastly technically superior offeror upheld where cost and technical factors were "comparatively equal" in weight); University Systems, Inc., GSBCA 11145-P, 91-2 BCA 23,997, at 120,105, 120,108, 1991 BPD 108, at 2, 8 (award to higher- priced technically superior offeror upheld where technical merit and cost were of equal value; protester's lower price "was not sufficiently low to offset the commanding lead [the awardee] enjoyed in technical merit"). GAO has also upheld awards to technically superior, higher- priced proposals where cost and technical were weighted equally. High-Point Schaer, 70 Comp. Gen. 524, 530, 91-1 CPD 509, at 9 (citations omitted); Babcock and Wilcox Co., B-235502, 89-2 CPD 237, at 4-5 (Sept. 17, 1989) ("Agencies may award to higher rated offerors with higher proposed prices or costs where the agency reasonably determines that the cost premium involved is justified considering the technical superiority of the selected offeror's proposal, even where cost is equal in weight to the technical factors."). In High-Point, GAO stated: "In a negotiated procurement, the Government is not required to make award to the firm offering the lowest price unless the RFP specifies that price will be the determining factor." 70 Comp. Gen. at 528 n.7 (citing Sach Sinha and Assoc., Inc., B-241056.3, 91-1 CPD 15 (Jan. 7, 1991). Under this clear precedent, HCFA was justified in awarding the east zone contract to a technically-superior higher-priced offer even though cost and technical were to be weighed equally. Nor has RMS demonstrated that it is entitled to any relief. In CompuAdd Corp. v. Department of the Air Force, GSBCA 12301-P, et al., 93-3 BCA 26,123, at 129,857, 1993 BPD 144, at 27-28, the Board approved a winnowing process or a process of elimination similar to that used by the SEB and SSO here with regard to RMS' proposal. The record clearly supports the SSO's determination that award to RMS would present an unacceptable technical risk. Protest File, Exhibit 52 at 6. Indeed, the TEP concluded that although RMS' proposal slightly exceeded the scope of work requirements in some areas, it continued to be weak in critical areas and posed a major risk to the Government. Id., Exhibit 51 at 74. In conclusion, the record clearly demonstrates that the technical enhancements of Forensic's proposal over that of RMS, in particular with respect to Forensic's experience, reliability, and approach, justify the additional cost. Decision The protest is DENIED. _____________________________ MARY ELLEN COSTER WILLIAMS Board Judge We concur: ____________________________ ____________________________ CATHERINE B. HYATT MARTHA H. DeGRAFF Board Judge Board Judge