THIS OPINION WAS INITIALLY ISSUED UNDER PROTECTIVE ORDER AND IS BEING RELEASED TO THE PUBLIC IN REDACTED FORM ON JUNE 23, 1995 DENIED: May 22, 1995 GSBCA 11643-P-REM BIRCH & DAVIS INTERNATIONAL, INC., Protester, v. AGENCY FOR INTERNATIONAL DEVELOPMENT, Respondent. Shelton H. Skolnick, Judy D. Leishman, Amy M. Hall, Wayne Finegar, and Bruce Trimble of Skolnick & Leishman, P.C., Derwood, MD; and Brian P. Hochheimer, General Counsel and Chief Financial Officer of Birch & Davis International, Inc., Silver Spring, MD, counsel for Protester. Jeffrey Marburg-Goodman, Office of the General Counsel, Agency for International Development, Washington, DC, counsel for Respondent. Before Board Judges PARKER[foot #] 1, NEILL, and HYATT. NEILL, Board Judge. This protest was filed by Birch & Davis International, Inc. (BDI) on December 20, 1991. The protest concerned the rejection of three offers BDI had submitted on a procurement conducted by the Department of State's Agency for International Development (AID or USAID). In accordance with Rule 11, the parties elected to have this case decided on the record without benefit of a hearing. By decision dated February 27, 1992, the Board denied the protest. ----------- FOOTNOTE BEGINS --------- [foot #] 1 Board Judge Vincent A. LaBella, a member of the original three-judge panel assigned to this protest, died on April 11, 1994. A random selection from the remaining judges resulted in his replacement by Board Judge Parker. ----------- FOOTNOTE ENDS ----------- Birch & Davis International, Inc. v. Agency for International Development, GSBCA 11643-P, 92-2 BCA 24,881, 1992 BPD 63. BDI appealed the decision of the Board to the United States Court of Appeals for the Federal Circuit. By decision dated September 13, 1993, the Court of Appeals wrote: "On the record before us there is an absence of substantial evidence to support the conclusion that Birch has no 'reasonable chance' of success. We thus are unable to affirm the Board." The Court, therefore, vacated our decision and remanded the case to us for such proceedings as may be necessary consistent with its opinion. Birch & Davis International, Inc. v. Christopher, 4 F.3d 970 (Fed. Cir. 1993). Upon receipt of the case on remand, the Board convened a conference with counsel. At that conference counsel for respondent explained that award of the contract had been made earlier in the year and that performance was underway. The Board advised counsel that it did not intend to reopen the evidentiary record for this case, but that it would welcome any briefs counsel might wish to submit in view of the decision of the Court of Appeals. Counsel accepted the invitation and a briefing schedule was established. Shortly before its brief was due, protester moved the Board to reopen the record to introduce evidence that approximately eleven months of negotiations were required for AID and the awardee to correct deficiencies in the awardee's proposal. Protester argued that this evidence is relevant to the issue of whether protester's proposals in this procurement had a reasonable chance of award. The Board invited counsel to brief the issue. Upon review of the arguments raised, the Board granted protester's motion to reopen the record. In granting the motion, the Board observed: For reasons of fairness and efficiency, we see no need to reopen the record regarding events which occurred prior to the date of our original decision. It does seem appropriate, however, since our review is de novo, that before deciding this case anew, we should reopen the record to permit the parties to update it with any relevant evidence regarding events which have occurred since our earlier review. Birch & Davis International, Inc. v. Agency for International Development, GSBCA 11643-REM, order at 2 (Feb. 4, 1994). The Board thereupon authorized discovery regarding events which had transpired in AID's procurement since the initial closing of the evidentiary record. At the conclusion of this discovery, the parties were given the opportunity to supplement the evidentiary record and to submit a second set of briefs on the merits.[foot #] 2 In their most recent briefs, the parties have appropriately focused almost exclusively on the reasonableness of the contracting officer's decision to eliminate protester's proposals from the competitive range. Nevertheless, since our original decision was vacated in its entirety, we address in this decision all counts of BDI's original protest. In its first count, protester has alleged that the contracting officer violated section 15.609(a) of the Federal Acquisition Regulation (FAR) by eliminating from the competitive range proposals of BDI which had a reasonable chance of being selected for award. In Counts II and III, BDI has accused AID of using hidden evaluation factors in violation of FAR 15.605(e) and 15.608(a), which require that all significant factors and subfactors be identified in the solicitation. BDI's fourth and final count is that AID has failed to seek full and open competition and thus violated the Competition in Contracting Act when it eliminated BDI's proposals for minor matters that were not evaluation factors. Complaint 17-25. For the reasons set out below, we deny this protest. Findings of Fact The Solicitation 1. On April 1, 1991, AID issued Request for Proposals (RFP) Egypt-91-008 for the design and implementation of a computerized management information system (MIS) for the Egyptian Health Insurance Organization (HIO). The solicitation envisioned award of a cost-plus-fixed-fee, level of effort, term contract for non-personal technical services. Protest File, Exhibit 1 at B-1, C-16. 2. The solicitation directs offerors to explain in detail in their technical proposals how they will complete each of thirteen tasks, namely: Task 1: Project Orientation and Planning Task 2: Project Monitoring and Reporting ----------- FOOTNOTE BEGINS --------- [foot #] 2 In submitting its second brief on the merits of this protest, BDI has attached several exhibits containing documentation available at the time the record was initially closed. Since this material belongs to the period of time for which the evidentiary record was not reopened, the Board, by separate order bearing the same date as this decision, rejects this material for inclusion in the evidentiary record. ----------- FOOTNOTE ENDS ----------- Task 3: Physical Site Preparation Task 4: Finalize Functional Design Considering NWDB[foot #] 3 Task 5: Review NWDB Detail Design, Enhance and Finalize Task 6: Arrange for Ordering and Delivery of System Hardware and Software Including Telecommunications Task 7: Arrange, Coordinate, and Supervise Installation of System Hardware and Software Including Tele- communications Task 8: Develop/Acquire Application Software Task 9: Complete System and User Documentation Task 10: Plan and Conduct Training Task 11: Conversion Task 12: Implementation Task 13: Post-Implementation Audit Protest File, Exhibit 1 at C-21 to C-22. The solicitation requires that offerors describe in detail: (1) how they intend to accomplish each task; (2) the time frame and level of effort for each (use of Gantt charts, Pert diagrams, or some other task mapping method is required); (3) the necessary staff resources by position type; (4) deliverables for each; and (5) where the activity will be conducted. Id. at C-21. 3. The solicitation's evaluation factors for award are assigned weights as follows: 5% Understanding of the proposal requirements 15% Technical approach task plan 10% Hardware configuration 15% Database management system, operating system, and other software 10% Management plan 25% Project staffing 10% Corporate qualifications 10% Cost evaluation criteria of reasonableness, realism, and completion Protest File, Exhibit 1 at M-3 to M-6. Overall evaluation scoring, therefore, is based on a total of 100 possible points of which technical considerations were worth a total of ninety points and cost was worth ten points. Id., Exhibit 11. 4. A requirement of this RFP of particular importance for this protest concerns portability. The RFP states: ----------- FOOTNOTE BEGINS --------- [foot #] 3 NWDB is the Northwest Delta Branch Automated Information System, a pilot system setup in 1987 by HIO with USAID funding to assist in the control of health program costs. Protest File, Exhibit 1 at C-6. ----------- FOOTNOTE ENDS ----------- HIO desires software capable of operating with no modification on a number of larger or smaller hardware configurations of the same manufacturer. It is equally desirable for this software to operate with little or no modification across hardware of different manufacturers. In its proposal, the offeror must describe the portability of the software products proposed. Protest File, Exhibit 1 at J-6. A subfactor under the evaluation factor for "Database management system, operating system, and other software" (see Finding 3) reflects the importance of this requirement. It reads: Portability and Flexibility -- This criterion will be used to project the ease with which changes to hardware components and/or system software can be accommodated. Specifically, HIO will evaluate technical resource requirements and required down time (in addition to funding requirements for new equipment or software), in the event of a major hardware upgrade (i.e., moving to a larger or smaller cpu of the same vendor as the proposed system, or changing hardware vendors completely). Id. at M-5. 5. The solicitation also contains the following provision regarding the acquisition of application software: The contractor should consider which application software, if any, should be purchased commercially instead of being custom-built by the contractor. In the proposal, the offeror must present its approach to deciding the make-vs-buy issue, and selecting a package when the decision is to buy it. Whenever a commercial package is recommended by the offeror, its specifications shall be attached to the proposal and an explanation shall be provided showing how the package meets HIO's requirements and fits the functional design. Protest File, Exhibit 1 at C-34. A subfactor under the evaluation factor for "Database management system, operating system, and other software" (Finding 3) reflects the importance of this requirement. It reads: The Appropriateness of System Software to HIO's Applications -- This criterion will be used to determine how effectively and efficiently the software will facilitate the design and development of the applications and then process the projected workload. The offeror must specify the number of copies installed and operating in Egypt for each product proposed. The offeror must demonstrate his/her knowledge of the required application/module functionality through a detailed explanation of how the proposed software products will satisfy the functional requirements. Id. at M-5. 6. The solicitation is specific about the level of effort which offerors are to describe in their proposals. The technical proposal is to contain a technical approach task plan which is to include, among other things: A chart showing the estimated level of effort that each staff member (prime and subcontractor staff) will devote to the project (by task) and distinguish between time spent on-site (i.e. working in or near HIO offices in Egypt) and time spent elsewhere. Protest File, Exhibit 1 at L-5. A subfactor for the evaluation factor "Technical approach task plan" (Finding 3) refers expressly to the requirement that offerors state: "Estimated level of effort for each project member by task, and time spent in Egypt versus elsewhere." Id. at M-3. 7. With regard to cost proposals, the solicitation requires that they "be prepared in sufficient detail to permit cost analysis." Offerors are directed to submit information reasonably required to explain their estimating process." Protest File, Exhibit 1 at L-11 to L-12. 8. With regard to staffing, the solicitation calls for a project contractor team consisting of the following: 1. Project Manager/Chief of Party 2. Senior Systems Analyst (2) 3. Senior Analyst/Programmer (2) 4. Database Specialist 5. Training Specialist 6. Documentation Specialist 7. Administrative Assistant 8. Procurement Specialist 9. Telecommunication Specialist 10. On-call System Analyst/Programmers (to be specified in number and duration by offeror). Protest File, Exhibit 1 at C-45. 9. The offeror's staffing plan is to include prescribed bio-data forms for each proposed team member outlining his/her accomplishments, and academic and professional experience relevant to the project. The solicitation outlines the qualifications and duties of each of the team members. The positions of Documentation Specialist, Procurement Specialist, Telecommunication Specialist and the On-call System Analyst/Programmers are not full-time positions. Protest File, Exhibit 1 at C-46 to C-55. Finally, the solicitation provides that project staffing will be evaluated based on how well the proposed personnel meet the qualifications as detailed in the RFP. Id. at M-6. 10. The solicitation provides as part of accomplishing Task 5 that the contractor present a comprehensive review of all documentation, plans, strategies, methodologies, prototypes and other relevant aspects of the detailed design to selected AID and HIO personnel. This is to be structured in the form of a "detail system design walk-through," integrating all detail planning and design work. Protest File, Exhibit 1 at C-29. 11. The solicitation expressly advises offerors that award of the contract may be made on the basis of initial offers received, without discussions. The offerors are advised, therefore, to submit initial offers which contain the offeror's best terms from a cost or price and technical standpoint. Protest File, Exhibit 1 at M-1. 12. Offerors are also advised that a demonstration of operational capabilities (DOC) shall be required of either the offerors invited to participate in a BAFO round or from the top ranked offeror if a BAFO is not called for. The DOC may be at a site of the offeror's choosing within two weeks' notice from the contracting officer. Protest File, Exhibit 1 at M-2. Technical Evaluation of Offers 13. The technical evaluation addressed all the evaluation factors (except the cost criteria) which accounted for 90% of the total score. See Finding 3. A technical evaluation panel (TEP) of four persons was appointed for the technical evaluation of proposals. Each member of the TEP was to evaluate and score individually each proposal according to an agreed methodology. The methodology included use of 335 various criteria spanning the seven technical evaluation factors. Each criterion was scored on the basis of five possible ratings, i.e., 0=unacceptable, 1=inadequate, 2=adequate, 3=good, 4=excellent. Protest File, Exhibits 6, 19 at 82-83, Exhibits 21-24. 14. Raw scores from the individual evaluations were fed into a special computer program which calculated an average score on each technical evaluation factor for each proposal. These scores were then matched by the program against the maximum possible score available for that factor. The actual percentage of achievement was then weighted by the relative importance assigned by the RFP to that particular evaluation factor. When this figure was added to similar calculations made by the program for the remaining six technical evaluation factors, one then had an overall score from one to ninety points for the proposal in question. For example, on a given proposal, evaluators assigned scores of 143, 147, 168 and 99 for the evaluation factor relating to understanding the proposal requirements. This yielded an average score of 139.25, which represented .7567 of the maximum possible score for that factor. The solicitation assigned a weight of 5% to that factor. For the proposal in question, therefore, a weighted score of 3.78 was calculated for that factor. Protest File, Exhibits 2 (Evaluation of Technical Proposals/Consolidated Summary Worksheets), 6 at 1-2. 15. Between two members of the TEP, there was a definite disparity in the technical evaluation scores they assigned to the proposals found to be technically acceptable. This disparity, however, is seen in the evaluation of all proposals. The one member gave to each evaluated proposal a score which was consistently lower than that assigned to the same proposals by the other. In one instance, the difference was as great as 32.09 points (out of a possible ninety points). In another instance, the difference was no more than 7.27 points (out of a possible ninety points). Protest File, Exhibit 2 (Technical Proposal Scoring Summaries for ****** and ******). With regard to one particular proposal, protester calls our attention to the fact that on at least 23 of the 335 individual criteria involved in the technical evaluation of that proposal, one of the same two TEP members assigned a maximum score of 4 points while the other allowed no points. Compare Protest File, Exhibit 22 with Exhibit 24; see also Protester's Brief 37. The TEP members did not discuss among themselves disparities in scoring on individual criterion levels. There was, however, some discussion of section scores representing each specific evaluation factor. Protest File, Exhibit 19 at 81-82. 16. By memorandum dated November 18, 1991, the TEP chairman informed the contracting officer of the panel's findings and recommendations relative to the proposals evaluated. Of the seven proposals evaluated, the TEP ranked three as technically unacceptable and four as technically acceptable. Of the four technically acceptable proposals, one was submitted by Maximus Inc. (Maximus), another was submitted by Seacor Services Inc. (Seacor). The remaining two were submitted by BDI. BDI's first proposal provided for the utilization of equipment produced by the International Business Machines Corporation (hereinafter referred to as the "BDI/IBM proposal"). The other BDI proposal found to be technically acceptable involved use of equipment produced by the NCR Corporation (the "BDI/NCR proposal"). The TEP report contains a detailed analysis of each proposal. Before turning to the individual proposals, however, the report discusses some general problems common to all or at least a number of the proposals. These include the distributed processing architecture offered by several vendors, proposed training in the United States versus Egypt, and a need for additional information regarding the reliability of proposed hardware in a climate similar to that of Egypt. Protest File, Exhibit 6 at 4-6. 17. Before discussing individual proposals, the TEP report singles out the proposals submitted by BDI for the following general comment: Finally, the Panel recommends that the Contracting Officer communicate to BDI that specifics must be emphasized in any BAFO process. As we get to the end of this complex procurement, it does little good to suggest that AID or HIO should "trust us" simply by invoking their past reputation, and yet fail to indicate precisely how something is to be done on this project. Protest File, Exhibit 6 at 7. 18. The TEP report does not recommend that the contracting officer negotiate with any specific vendor among those found to be technically acceptable. The TEP does, however, highly recommend "the BAFO process in this procurement" rather than award without further negotiations. The possibility of a competitive range determination in which some offerors would be eliminated from consideration nevertheless appears to have been recognized by the TEP as a distinct possibility. The report states: We understand that cost, totaling 10 additional points, will be factored into these technical rankings, once the Contracting Officer completes her cost analysis. From these separate examinations, a final competitive range will be determined, upon which the Best and Final Offers will be determined. Protest File, Exhibit 6 at 7. 19. The TEP report contains a detailed individual analysis of all four technically acceptable proposals. The discussion of each proposal mentions numerous deficiencies. Protest File, Exhibit 6, Notes From the Evaluation of Technical Proposals. Evaluation of Cost Proposals 20. Based on her initial review of the cost proposals received, the contracting officer concluded that there was an obvious problem with the Government's own cost estimate. She, therefore, suspended her evaluation and conferred with the individual responsible for preparing the Government's original cost estimate. This individual was an outside consultant working with ********* ********** Corporation *****. Protest File, Exhibit 18 at 9-10. 21. The contracting officer explains that she sought out the consultant because of the "vast difference" she found between most of the proposed costs and the Government's own estimate. On conferring with the consultant, the contracting officer found that the consultant had not included such items as indirect costs, fringe benefits, travel allowances, and costs for expatriates living in Cairo. Because she considered that such costs are normally included in the Government's estimates, she directed this individual to revise the cost estimate. Protest File, Exhibit 18 at 9-10, 14, 95, 101-2. This was done. Although this consultant was familiar with the technical proposals received for this procurement (he was a member of the TEP), he has stated under oath at his deposition taken on January 26, 1992, that he had not otherwise been involved with costs in this procurement. Id., Exhibit 19 at 56. 22. As a preliminary step in the cost evaluation, the contracting officer also sought the assistance of another *** consultant. This consultant was asked to undertake a preliminary technical review of the cost proposals. The purpose of this review was: (1) to identify for the contracting officer anything which the consultant, based on his own experience, considered to be out of line, and (2) to make some comparative analyses of different types of costs, such as hardware, software, personnel, and site preparation. Protest File, Exhibit 20 at 8. 23. Once the contracting officer received the revised Government cost estimate she resumed her evaluation of the cost proposals. Protest File, Exhibit 18 at 6, 36, 104. By then she had received the report of the TEP. She, therefore, limited her evaluation of cost proposals to those of the four offerors which the TEP had found to be technically acceptable. Id. at 33-34. 24. The contracting officer has stated that, in conducting her evaluation of cost proposals, she made limited use of the *** consultant's report. Protest File, Exhibit 18 at 46-49. The notes she made during her detailed evaluation of the cost proposals are contained in the protest file. Compare id., Exhibit 18 at 33 with Exhibit 2. In evaluating the cost proposals, she divided the ten evaluation points available for scoring an offeror's cost proposal as follows: four points for price reasonableness, three points for price realism, and three points for completeness (i.e. following the RFP instructions for preparation of cost proposals). See id., Exhibit 2. We have examined the contracting officer's notes on cost evaluation and, in particular, her stated rationale in support of the various scores given. We find her comments to be in accordance with applicable guidelines. As a result of her evaluation, the contracting officer scored the price proposals as follows: Maximus 9.55 Seacor 5.76 BDI/IBM 5.61 BDI/NCR 5.34. Id. at 1. Evaluation of Protester's Offers 25. Protester's response to the RFP was threefold. It proposed, in the alternative, the use of three separate systems which would utilize equipment provided by three different suppliers. The contracting officer determined that BDI's submission should be treated and scored as three separate proposals. As already noted, the BDI/IBM proposal was based on the use of IBM equipment. The BDI/NCR proposal was based on the use of NCR equipment. A third proposal, the "BDI/Unisys proposal," was based on the use of equipment produced by the Unisys Corporation. Protest File, Exhibits 4 at 3-6, 6 at 1, 11 at 1. 26. The TEP ranked the BDI/Unisys proposal number six among the total seven proposals. This proposal received a score of 46.3 technical points out of a possible 90 points. It was found to contain numerous statements indicating that Unisys did not currently have an effective support capability in Egypt; the number of field engineers proposed was, therefore, found to be grossly inadequate. The proposal recommended a commercial hospital package to meet all HIO application requirements. BDI failed to explain, however, how the package would meet the requirements or the extent of modifications and level of effort necessary to have it meet requirements. The TEP also found that the proposal failed to address numerous RFP requirements, such as training, data entry configurations to be used in both branches and polyclinics, PCs for use by HIO's application development staff, and data communication equipment. The TEP also stated that the BDI/Unisys proposal could not be effectively reviewed with the main BDI proposal because the Unisys proposal volume was completely different in structure from the main BDI proposal. The two volumes, taken together, simply did not mesh. Protest File, Exhibit 6 at 2-3. 27. The TEP did find the BDI/IBM and BDI/NCR proposals technically acceptable. Of the four proposals found to be technically acceptable, however, the TEP ranked the BDI/IBM proposal as fourth with a technical score of 55.5 out of a possible 90. The BDI/NCR proposal was ranked as third with a technical score of 56.7. out of 90. The Seacor proposal was ranked second with a technical score of 59.1. The Maximus proposal was ranked first by the TEP with a technical score of 67.6 out of 90 points. Protest File, Exhibit 6 at 8. 28. The contracting officer then ranked the four technically acceptable offers based on their combined technical and cost scores. The ranking was as follows: Maximus 77.05 Seacor 64.86 BDI/NCR 62.04 BDI/IBM 61.11 Protest File, Exhibit 11. The Competitive Range Determination 29. Once the combined scores for the four technically acceptable offerors were available, the contracting officer made a competitive range determination in which she excluded all offers except that submitted by Maximus. Protest File, Exhibit 11. 30. In her deposition, the contracting officer has readily agreed that, in excluding offerors from the competitive range, she was required to determine whether they had any reasonable chance to receive an award. Protest File, Exhibit 18 at 106. In this regard, she has testified that in making such a determination she certainly did consider the possibility that the excluded companies might have increased their scores had they submitted a BAFO. Id. at 62-63, 106. She states that she recognized that this was a possibility -- as it was likewise possible that Maximus could improve at least on its technical score. Id. at 59-60, 106. Nevertheless, she concluded that the remaining three offers did not have a reasonable chance of being awarded the contract. Id. at 63, 107. 31. When asked to explain the basis for her determination that BDI did not have a reasonable chance of being selected for award, she replied: It is based on the technical evaluation that was conducted and reflected in the notes which you have s e e n , a n d i n f o r m a t i o n i n t h e letters[foot #] 4, as well as the cost evaluation. Protest File, Exhibit 18 at 112. 32. Before making her final determination, the contracting officer did confer with several individuals. She has stated that she discussed with the "technical people" whether it was reasonable to expect that these companies could receive an award after a BAFO. Protest File, Exhibit 18 at 62. She also has stated that she met with her supervisor and with counsel to discuss her intended competitive range determination. Id. at 65. It is her testimony, however, that no one ever suggested or advised that all four technically acceptable offers be retained in the competitive range. Id. at 52-55, 112. ----------- FOOTNOTE BEGINS --------- [foot #] 4 Although not expressly identified in this quotation, it is clear from the context that the reference is to letters the contracting officer wrote to BDI and Seacor, dated December 10, 1991, informing them that their offers had been eliminated from the competitive range. See Findings 37-40. ___ ----------- FOOTNOTE ENDS ----------- 33. From the record we cannot determine the exact date on which the contracting officer made her competitive range determination for this procurement. By December 10, 1991, it obviously was already made. See Protest File, Exhibits 7-9. The contracting officer's written determination is, however, undated. Id., Exhibit 11. In her deposition she stated that she does not recall the exact date, but believes that the competitive range determination had been made by November 24. Id., Exhibit 18 at 73. The TEP chairman, who met with the contracting officer on three or four occasions before the competitive range determination was made, was unable to testify with any certainty regarding when those meetings occurred. Id., Exhibit 17 at 47. 34. On November 24, 1991, the contracting officer telephoned a representative of Maximus to confirm that statements in Maximus' proposal regarding compliance with the authorized geographic codes did in fact cover all the equipment proposed. Protest File, Exhibit 18 at 69-70, 84-85. The company representative contacted confirmed that the statements did apply to all equipment. The representative further assured the contracting officer that the company was "familiar with [this] AID requirement" and was "very conscious" of the requirement while working on the proposal. The contracting officer asked for written confirmation. Id., Exhibit 13. Written confirmation was provided by facsimile transmission received on November 25. It stated: "Thank you for your phone inquiry . . . . To reaffirm, [our] proposal as written complies with the Authorized Geographic Code as stated in the RFP." Id., Exhibit 18 at 69-70, Exhibit 25. 35. It was the contracting officer's plan to notify all offerors at the same time regarding the status of the offers they had made on this procurement. Protest File, Exhibit 18 at 110. On December 10, 1991, therefore, by separate letters relating to each proposal received, the contracting officer notified each offeror, with the exception of Maximus, that its offer had either been rejected as technically unacceptable or had been eliminated from the competitive range. The contracting officer has also stated that because of the level of detail in each of the letters, it "took quite a while" after the competitive range determination was made to prepare these letters for concurrent release on December 10. Id. at 83. 36. Each of the contracting officer's letters of December 10 regarding the elimination of BDI's three offers and Seacor's one offer is based almost entirely on the specific observations provided to her by the TEP and on her own notes taken during her evaluation of cost proposals. Compare Protest File, Exhibits 2, 6 with Exhibits 7-9, 34. The letters to BDI demonstrate in considerable detail the underlying bases for the contracting officer's conclusion that the proposals in question did not have a reasonable expectation of being selected for award. Because the propriety of the contracting officer's decision to eliminate the BDI/IBM and BDI/NCR proposals from the competitive range is at issue in this protest, we set out here the verbatim text of her letters on each of those proposals. 37. The contracting officer's letter of December 10, 1991, to protester regarding the BDI/IBM proposal reads as follows: Thank you for submitting your proposal in response to the Request for proposal Egypt-91-008. The evaluation of proposals has now been completed and we regret to inform you that your proposal which contains IBM as the computer vendor, is not within the competitive range of offerors with whom the Best and Final Offers (BAFO) process continues. The following comments are those of the Technical Panel and are provided for your edification. While generally well written, your proposal does not provide sufficient supporting explanations, is unclear and/or contradictory in places, fails to provide a firm configuration, and contains some unacceptable staffing. Some specific deficiencies and weaknesses of your proposal are commented upon below. Portability * ***** ******* ** **** *** *********** ******** ** *** ****** *** *** ********** ******** **** *** ******* *** **** ****** *********** ** **** ********* ** ****** ************* ************ ***** *** **** ***** ******* **** *** ** **** ***** ********* ** ****** ** **** ********* **** ******** ******** ** ********** ****** *** ********* ****** ** *** ****** ********** ** ***** *** ******** ****** *** ******** **** ** *** ******* *********** ******** ********* ** *** ****** *** *** ****** ** **** ** ******* ** *** ***** ******** ******* ***** *** *********** ****** ** **** ********* *** ******** ****** **** ******** * ******** *********** ** *** ************* ***** *** ********* ** *** ******* ***** ** ****** ** ******* ** ***-****** ******** *** *** ********* * *********** ** *********** ** *** ******** ********* *** **** ******** **** *** ******* * ********** ************ Compatibility **** ******** ********* ************* ** ******** *** ******** ****** *** **** *** **** ***** ******** ** ** *** ***** ******* *** ******** *** ******** ********** *** ***** ********** ***** ******* *** ********* ****** **** **** *** **** ****** *** *** ******** * ******** *********** ** *** ************* ***** ******* *** ********* ** **** ******** ******** ********** ******** *** ********* ***** ** **** ******** ** ******* ** ** ******* **** ********* *** ****** ******** **** ******** ********* ** ***** ********** ** ***** ** ***** ************* **** ***** **** ** ***** ***** ** ******** **** *** ** ** ****** ****** *** **** ******** ******* **** ***** *** ** **** ******** **** *** ****** ******** *** ** ***** *********** ** ********* Data Entry in the Branch *** *** ************* ****** ******** *** *** ********** *** ****** **** ***** ************ ******* ***** ** *** *** ********* *** *********** *** * ****** **** ***** ****** ************ ** ***** **** ********* **** **** ***** ********** *********** **** *** ** **** ******** ********* ** ****** ********* ****** **** *** **** ***** ************* ** ** ** ********** ******* * ****** *** ***** ****** ** ********** **** *** **** ********* **** **** ***** ***** ********** ***** *** ********* **-******* **** ************ *** ****** ** ***** *** **** **** **** ********** ********** *** ******* ** ** **** *** ***** -- ****** *** *** ******* ***** *** ********** -- ** ***** **** **** ******** ***** ** ********** ******** **** *** ** **** ******** ********* ** ****** ********* *********** *** ******** ******** ** ****** **** **** **** ***** ** ******** ******* **** *** ******** *** ***** ** ******* ******** ** *** ************* ********* ** ***** ** - ** ** ***** ******** ***** **** ***** *** ********* *** ********** *** **** ***** **** ** *********** ********* Data Entry in the Polyclinics ***** *** *** *** ** **** ******** ******* ** ******** ***** ******* *** ********** ********* *** **** *** *** **** ** *** ******** ************* ** ******** * ** ***** ********* ** ** ********** **** **** ******* ******** ****** *** *** **** ******** *** ** ***** *** ****** ** * ***** ** **** * *********** ***** *** ** *********** *** ********* ** * **** ***** **** ** * *********** **** ********** ************ *** *********** ** ** ******* **** ***** ** ********** ** ******** *** ****** ** * ********** ** ******** **** * **** ****** ***** **** ****** ** ***** **** *** ********** ** *** ****** ********* ** ******** ** *** ******** **** ******** *** **** **** ******** ***** *** ****** ******** ** ** *** ********* *** ***** ***** *** ******** ** ** **** -- ***** **** *** **** **** *** **** ***** ****-*** ***** ******** *** **** ****** *** **** ** *** ********** ** *********** *** **** ***** ******** *** ********* ** *** **** ** * *********** RFP Requirements Omitted in Proposal **** ******** ** *** ********** ** *** ********* ********* ******** ************* ********* *** ******** ******** ********** ******** ** *** ****** ************** ** ** ************ ** ***** **** ***** ******** *** *** ******** ** **** ** ******** ** ******* *********** Software **** ******** ** *** ***** ** ** ***** ************ ** *********** *** ********* ******** ************ *** ******* ********* ** ********* ****** ** *** ********* ** ** ****** **** **** *** ***** *** *** *** ***** ********** ******** ***** ** *** *********** *** **** ********* ********* ******** * ** **** ******** ********* ***** ************** ******** ********** *********** ******* ********** **** ******* ** ******** ** **** *** *** *********** ************ ********* ** *** **** *** ******** *** **** **** ******* ***** ********* *** **** ************ ************* *** ** ******** ** *** ******* ** ******* ***** ***-******** *********** ********** **** **** ********* ** ******* **** ******* ** * *** ******* ***** ** *** ***** *********** *** *** ***** **** ** *** ***** ****** ********* ************* *** *** *** ****** ** ****** *** **** ***** *********** * *** ******** ******* ***** ***** ************* ******** *** *********** ********* ******* ** * ******** ******* -- *** ********* *** *** *** ******* ********** *** ********** ****** **** ******** ** **** *** ********* ** ******* ******* *** ** ****** *********** **** *** ** * ******** ** *** ** * *********** **** ******** **** *** ********** ******* *** ******* *** *** ******* ******* ***** *** *** *********** ************* *** **** **** ******** ******* *** ***** ** ****** ** ************* ********* **** ******** **** *** ******* *** **** ***** ******** ** ** **** ** *** *** ** *** ****** *********** ****** **** *********** ******* ********* ******* ******* *********** ****** *** ********** ** ****** ** ******* ******* **** *** ** ***** *** ** **** ******* ** *** ******* **** ******* *** ******** ******* ** ****** ******** ********* ** *** ************ Data Communication Equipment **** ******** ******** **** *********** *********** ********** *** ********* ************** *** *** **** ************* ********** **** *** ********* **** *** ** *********** *********** *** **** ************* ********** ******** *** ************** ** **** *** ********* *** **** ********* ****** ** ******* *********** *** ******* ****** ** ** *** ***** **** **** ************* ********* *** ******** ****** ** *** ************** ** **** *** *** ******* ****** **** *** ********** ** *********** *** ********* *** ********* **** ************* ********** Training in the United States **** ******** ********** ******** *** ********* ** *** **** *** ***** ** ********** ******* *** ** *** ******* ** **** ************** ** ** ****** **** **** ********** ** **** * *** *********** ****** ** ***** **** *** * ****** ****** ** *** ********* ** ****** ** *** **** Level of Effort While your proposal does not contain level of effort data per se,[[foot #] 5] there is discussion related to the level of effort which is of concern. Page 38 of your proposal states that there is no intention to repeat the analysis already developed by HIO, but the proposal goes on to describe subtasks which appear to repeat the work previously performed. Considerable studies have already been performed and documented, and are not to be repeated. The responsibility of Task Four (4) is to finalize the functional design, not develop it from scratch. Proposal's Response to RFP's Requirements Is Not Firm Your proposal contains comments indicating that you may change your mind about certain hardware and software products once a contract is signed and the project initiated. Below are two examples: "Key sizing factors are not presented in sufficient detail in the RFP to enable us to propose a final and guaranteed architecture and configuration for the MIS at this time." (page 16) "After this workload projections review, we will subsequently re-examine our proposed hardware and software configurations for the various types of HIO facilities and fine tune them where necessary." (page 56)[[foot #] 6] ----------- FOOTNOTE BEGINS --------- [foot #] 5 The TEP reported that BDI did not provide any indication of the level of effort. Because BDI failed to indicate how many people would be assigned to each task and subtask, the TEP found it impossible to know the total estimated effort. Protest File, Exhibit 6, Notes from the Evaluation of the Technical Proposal BDI with IBM at 8. [foot #] 6 These are but two of five examples provided to the contracting officer by the TEP regarding the lack of firmness (continued...) ----------- FOOTNOTE ENDS ----------- The RFP contains considerable workload data which was sufficient to allow offerors to propose specific and firm configurations. Staffing Based on the staffing information contained in your proposal, a number of concerns are identified: The two (2) people proposed as Senior Computer Operators have never worked as Computer Operators, The proposed Senior Systems Analysts and Senior Analyst Programmers have no documented experience in the proposed hardware and software, and No Training Specialist is proposed. It is unacceptable to state that one will be named later. It was expected that offerors would propose personnel who met the position qualifications specified in the RFP. Another concern about staffing in your proposal was your intent to utilize the services of Mr. Don Pugliese, as a resource to the project, when he is already employed full-time on another contract in Egypt. This approach is not acceptable. The Business Management Plan was evaluated separately in the Office of Contract Services. Comments relative to the cost evaluation follow. ----------- FOOTNOTE BEGINS --------- [foot #] 6 (...continued) for the products proposed in the BDI/IBM proposal and the associated budget for them. The TEP was also concerned by certain statements made in this proposal such as "[I]t would be poor judgment to lock into a large computer system before such certain questions are resolved." The "questions," however, were not identified. Elsewhere in its proposal BDI stated: "As a result of this process, we anticipate negotiating (at some future time, as specified in the technical proposal) a list of fixed ordering prices for hardware and software, for each year of the effective contract period." This led the TEP to conclude that BDI was not at all firm on the products that it had proposed. The TEP also was concerned that BDI had proposed little regarding subtask 11.1, the development of a conversion plan. BDI had argued instead that many questions had to be answered before the conversion plan could be developed. The TEP disagreed with this contention. Protest File, Exhibit 6, Notes from the Evaluation of the Technical Proposal BDI with IBM at 5. ----------- FOOTNOTE ENDS ----------- General The referred budget notes are missing from BDI's cost proposal. This made evaluation extremely difficult, as explanations/justifications which were alluded to were missing. Third country national costs are proposed in dollars. This would require waivers, which were not anticipated by USAID. ************ ** ******* **** *** ******* **** *** ******** *** **** **** ** **** Statement as to acceptability of the terms and conditions of the RFP was missing. There were a number of items omitted from the technical proposal, which thus has a direct bearing on the cost of the proposal and its realism and completeness. These have been mentioned in the comments of the technical panel. The omitted items and statements made in the technical proposal as to BDI's inability to guarantee the price of its system equipment due to a need for more information led to uncertainty in the cost evaluation as to the realism and completeness of the proposed price. BDI was also not firm on its estimates for renovation, and remodeling site prep[aration] costs were missing. Fixed Fee *** *** ******** *** *** *** *** ************** *** **** **** ** *** ******* ********* **** *** ** ********* *** ***** *** ****-************* ********** ***** ***** ****** ** ** **** *** *** *********** ***** *** ******* **** **** **************** *** ************ *** ***** *** ************* ** *** *** *** ********* Personnel and Associated Costs Eight biodata sheets were missing, some biodatas were missing complete salary history. This made verification of salary reasonableness impossible. Personnel policies regarding differentials and travel were not provided, personnel policies were not provided at all for the subcontractors, some of whom were providing significant amounts of staffing. Nine of the required letters form [sic] proposed personnel stating their intention to serve were missing. The job description on the letter was not always the same as on the biodata. Pugliese is proposed, however he is allocated full-time to another AID-funded contract. This is not an accepted practice. In addition, his proposed rate is in excess of the FS-l rate, while BDI had stated that no one was proposed over the cap. Cost proposed for shipment of vehicles is extremely high - 2 POVs for $26,224? ******* ********* **** **** **** *** * ** *** ******** ********* - **-**** **** *** ********** ********* **** ******** *** ****- **** ************ ***** **** ********* ***** ************* ** **** ***** ******** **** *** ********* ** *** ***** ***** ** ***** *** *** ********** **** **** ****** ********** ** ****** ******** **** *** ************** ** **** ******** ********* **** ****** ** *** ********* ******* ********* Indirect Costs Letter from cognizant audit agency confirming indirect cost rates and bases was not provided as required, for BDI or for subcontractors. Statement as to impact that an award would have, if any, on indirect cost rates was not provided as required, either for BDI or subcontractors. Subcontracts The amount appearing in BDI's budget table for computer equipment does seem to be the same as that given by the hardware vendor, no explanation of discrepancy given. ***** ******* ** ** *********** ** ****** ***** ** **** ** ********* ** ******** **** **** ******** ** ******* **** ***** *********** ** ******** ** ***** ****** BDI's proposal and its hardware vendor's (IBM) separate volume do not always agree. BDI states that all equipment meets with AID's requirements with regard to geographic code, IBM states that they do not. We recognize that negotiations might have resulted in an improved proposal. However, it was concluded that the needed improvements were so great that your proposal, even if improved through negotiation and submission of a best and final offer, did not have a reasonable expectation of being selected for award. A revised proposal is neither requested nor will it be accepted. We sincerely appreciate your considerable effort in preparing and submitting and [sic] offer on this extremely technical (and voluminous) requirement. We regret that it was not sufficiently strong to be considered in the competitive range. We hope this will not discourage you from responding to future USAID requests for proposals. Indeed, we would welcome your participation, and look forward to future opportunities to work together in USAID's development programs. Protest File, Exhibit 7. 38. The contracting officer's letter of December 10, 1991, to protester regarding the BDI/NCR proposal reads as follows: The following comments are those of the Technical Panel and are provided for your edification. While generally well written, your proposal does not provide sufficient supporting explanations, is unclear and/or contradictory in places, fails to provide a firm configuration, and contains some unacceptable staffing. Also, the main proposal volume, which prefers a computer supplier other than NCR, does not easily fit with the NCR proposal. Specific deficiencies and weaknesses of this proposal are commented upon below. Integration Problems Between Main and NCR Proposal Volumes Because the main volume of the technical proposal is written around a specific computer supplier (not NCR), the evaluation of the NCR proposal is more difficult. The main proposal contains sections on the Technical Approach and Understanding of the RFP's Requirements and so does NCR's proposal. Performing the evaluation required moving back and forth between the various volumes. The table of contents in NCR's proposal does not break down the hardware and software into useful groupings, and there are no page numbers which makes it extremely difficult to locate information. Fully Distributed Architecture Your proposal recommends a fully distributed architecture and identifies the advantage of limiting reliance on any single system. However, there are also disadvantages to a fully distributed architecture. A fully distributed architecture implies at least ninety- eight (six branches, sixty-seven polyclinics, and twenty-five hospitals) computer systems which would increase by one with the implementation of each polyclinic or hospital to exceed 200 by 1997. The following list contains some of the considerations which should have been addressed in thoroughly exploring the feasibility of a fully distributed architecture for HIO: The more computer systems that are installed, the greater the cost (acquisition and maintenance), The more equipment installed, the greater the likelihood of a failure which would require maintenance, The more sites installed, the greater coverage to be provided by the maintenance organization (i.e., a higher ratio of sites and equipment per Field Engineer), The approach to ensure data integrity between local data bases which may be updated and the branch database which is the master, and Local area networks (LANs) require constant technical care to monitor and tune for performance. The major concern is that to implement and support a fully distributed architecture on a national scope in Egypt, of the size envisaged by HIO, could be a problem. ************* *** ******* ** ******* ********** ********* ********* ** ********* ** ******* ******** ** ********* ********** ****** * **** *** ******** ******** ** *** ******** ************ ******** ********** *** ***** ***** ** ****** *** ********** *** **** ******** ***** ******** ********* *** ****** *** ************** ******** **** ******** **** *** ********** ******* *** ******* *** ******** ************ ** *********** *** ********* ************** Proposed Equipment ** ******* ***** *** ** **** ** **** ******** **** *** *** **** ******* *** **** **** *** **** ******* ********* ** ****** *** ******* **** ** **** *** ****** ** ******* ****** *** ***** ********** *** *** ******** **** **** ******** ******* *** ****** ** ******* *** ********* *********** ** **** *********** ******* *** **** *********** ** *** ******** ** **** ********* **** ******** ** *** **** *** *** ******* ** *** ******** ** ******** *********** *** **** ******** ******** ***** ******** ** *** ***** ** ** *** ************* ** **** ******** ******* ****** *** ****** **** ****** ***** ***** ***** ******** ******** * ******* ********* ************* ** **** ****** ************* ** ****** ***** *** ****** ** *********** ****** ****** *** **** ******** **** *** ** ************ ** **** ******* *** *** ******** * ******** *********** ** *** ******* **** ********* * ********** ** ************* ***** ******* *** ********* ** *** ******** ******** *********** **** *********** ** *** ***** ** **** ********* ************* *** *** ******** ********* ******** *************** ********** ** ** ******** ** *** ******** *** **** ************** ***** ******** ******** **** *********** *** *********** ************ *** ***** ******** *************** *** *** *** ******* **** ***** *** ***** ******** *************** ***** ******** ********** *** ** **** *** ******* ** ** **** ** **** ********** ** **** ** *** ******** ** ******* ******* ***** *** **** ********* ***** ** ********* ** *** **** *** ******* **** ********** ********* **** ** **** ** ********* *** ******* ** *** **** ***** ******* *** ***** ******** ******* ** * ****** ** **** ******** ** * ****** *** ********** ************ *** *********** *** *********** ********* ** **** ******** ********** *** **** *** ** *** ********** ** ***** **** ** ***** **** ************ ******** *** ********** ** *** *** ** ******** *** *********** ** ********* ******* ** **** ********** ******* ** *** ******* ********* *** ********** *** ********** ********* ** *********** ******* *** ** ***** ******** ******** * ** *** ******* *** **** ******** ******** *** **** *** ******* **** **** ********* ** *** ********* ** *** **** ***** *** *** **** ******** *** ******* ** ******* *** ********* ***** **** ******* ** **** ****** ** **** ** *** ****** *** **** ************ **** ** ********* ** *********** *** **** ********* ** ******* ** **** ********* ******* ***** ** ***** ******** ********** **** *********** *********** ******** ******* * **** ******* ***** * ******** **** ****** ** ** ********* ********* ********* *** ******** ***** ** ******** *** ********* ******** ********** **** **** ********* * ******** ******** ****** **** ******** **** ********** ** *** ******* ** *** ******** ***** ***** ********* ** ******* **** *** *** ********* ** *** ********* ** *********** ********** *** ***** ** ****** *** ***** ***** *** **** ** ** ********* *** ********* ** *** ******** ** *** ******** *** ****** ** *** **** **** *** ********* ******** ****** ********* *** ********* **** *** **** ********** ** ***** ********** Data Communication ***** ******** ******** ******** ******* ***** ** *** **** ******** **** ****** *** *** **** ** *** ********** ** ** *** ***** ** **** ** * ****** **** ******* *** ******** ** ********* ** ** ** ** * ********* **** ** **** ***** ** ******** ***** ***** ***** **** ******* ********* ******** *** ******** ******** **-*** ****** **** **** ********* *** ****** ****** ***** *** *********** ** ************ **** ***** ** *** ******* ***** **** ******** ** ** *** ********* ** * ****** ******* ****** ** ** ******* **** ************* ******** *** ***** ******** ** **** *** ********* ***** *** ******* ************* ************ ** *** ***** ****** **** ******** ** *** ******* *** **** ********* **** **** ** ****** ****** **** ** *** ** *** **** *** ************* ************ *********** *** *** ** ** ***** ******** -- *** ** ** ********* *** ********* ** * ****** ***** ** ****** *********** Training in the United States **** *** **** *** *** ********* ********* ******** *** ********* ** *** **** *** ******* ******** ******** ******** ************** ***** ******** ********* ******** ** **** *** ********* **** ******* *** ******** *** *** ** * ***** *** ** ***** *** ******* ** *** ******* ** *** ************** **** ** ******** ** ************* Level of Effort While your proposal does not contain level of effort data per se,[[foot #] 7] there is discussion related to the level of effort which is of concern. Page 38 of your proposal states that there is no intention to repeat the analysis already developed by HIO, but the proposal goes on to describe subtasks which appear to repeat the work previously performed. Considerable studies have already been performed and documented, and are not to be repeated. The responsibility of Task Four (4) is to finalize the functional design, not develop it from scratch. Proposal's Response to RFP's Requirements Is Not Firm Your proposal contains comments indicating that you may change your mind about certain hardware and software products once a contract is signed and the project initiated. Below are two examples: "Key sizing factors are not presented in sufficient detail in the RFP to enable us to propose a final and guaranteed architecture and configuration for the MIS at this time." (page 16) "After this workload projections review, we will subsequently re-examine our proposed hardware and software configurations for the various types of HIO facilities and fine tune ----------- FOOTNOTE BEGINS --------- [foot #] 7 The TEP found that the BDI/NCR proposal, like the BDI/IBM proposal, contained estimated start and end dates but no indication of the level of effort. The proposal provided no indication of how many people would be assigned to each task and subtask. It was impossible, therefore, to know the total estimated effort. Protest File, Exhibit 6, Notes from the Evaluation of the Technical Proposal BDI with NCR at 3. ----------- FOOTNOTE ENDS ----------- them where necessary." (page 56)[[foot #] 8] The RFP contained considerable workload data which was sufficient to allow offerors to propose specific configurations. Staffing Based on the staffing information contained in the proposal, a number of concerns are identified: The two (2) people proposed as Senior Computer Operators have never worked as Computer Operators, The proposed Senior Systems Analysts and Senior Analyst Programmers have no documented experience in the proposed hardware and software, and No Training Specialist is proposed. It is unacceptable to state that one will be named later. It was expected that offerors would propose personnel who met the position qualifications specified in the RFP. Another concern about staffing in your proposal was your intent to utilize the services of Mr. Don Pugliese, as a resource to the project, when he is already employed full-time on another contract in Egypt. This approach is not acceptable. The Business Management Plan was evaluated separately in the Office of Contract Services. Comments relative to the cost evaluation follow. ----------- FOOTNOTE BEGINS --------- [foot #] 8 As in the case of the BDI/IBM proposal, these are but two of five examples provided to the contracting officer by the TEP regarding the lack of firmness for the products proposed by BDI and the associated budget for them. The TEP's additional concerns with the firmness of the BDI/NCR proposal are the same as those outlined above in footnote 5 regarding the BDI/IBM proposal. Protest File, Exhibit 6, Notes from the Evaluation of the Technical Proposal BDI with NCR at 8. ----------- FOOTNOTE ENDS ----------- General The referred budget notes are missing from BDI's cost proposal. This made evaluation extremely difficult, as explanations/justifications which were alluded to were missing. Third country national costs are proposed in dollars. This would require waivers, which were not anticipated by USAID. ************ ** ******* **** *** ******* **** *** ******** *** *** ** **** Reps and Certs not provided for NCR. Statement as to acceptability of of [sic] the terms and conditions of the RFP was missing. There were a number of items omitted from the technical proposal, which thus has a direct bearing on the cost of the proposal and its realism and completeness. These have been mentioned in the comments of the technical panel. The omitted items and statements made in the technical proposal as to BDI's inability to guarantee the price of its system equipment due to a need for more information led to uncertainty in the cost evaluation as to the realism and completeness of the proposed price. BDI was also not firm on its estimates for renovation, and remodeling site prep costs were missing. Fixed Fee *** *** ******** *** *** *** *** ************** *** **** **** ** *** ******* ********* **** *** ** ********* *** ***** *** ****************** ********** ***** ***** ****** ** ** **** *** *** *********** ***** *** ******* **** **** **************** *** ************ *** ***** *** ************* ** *** *** *** ********* Personnel and Associated Costs Eight biodata sheets were missing, some biodatas were missing complete salary history. This made verification of salary reasonableness impossible. Personnel policies regarding differentials and travel were not provided, personnel policies were not provided at all for the subcontractors, some of whom were providing significant amounts of staffing. Nine of the required letters form [sic] proposed personnel stating their intention to serve were missing. The job description on the letter was not always the same as on the biodata. Pugliese is proposed, however he is allocated full-time to another AID-funded contract. This is not an accepted practice. In addition, his proposed rate is in excess of the FS-1 rate, while BDI had stated that no one was proposed over the cap. Cost proposed for shipment of vehicles is extremely high - 2 POVs for $26,224? ******* ********* **** **** **** *** * ** *** ******** ********* - **-**** Very few expatriate personnel were proposed for long- term assignments. While this decreases costs considerably, it does raise concerns over the abilities of the local staff to carry out the assignment with very little expatriate TA input. Concerns over the qualifications of some proposed personnel were raised in the technical panel's comments. Indirect Costs Letter from cognizant audit agency confirming indirect cost rates and bases was not provided as required, for BDI or for subcontractors. Statement as to impact that an award would have, if any, on indirect cost rates was not provided as required, either for BDI or subcontractors. Subcontracts ***** ******* ** ** *********** ** ****** ***** ** **** ** ********* ** ******** **** **** ******** ** ******* **** ***** *********** ** ******** ** ***** ****** We recognize that negotiations might have resulted in an improved proposal. However, it was concluded that the needed improvements were so great that your proposal, even if improved through negotiation and submission of a best and final offer, did not have a reasonable expectation of being selected for award. A revised proposal is neither requested nor will it be accepted. We sincerely appreciate your considerable effort in preparing and submitting an offer on this extremely technical (and voluminous) requirement. We regret that it was not sufficiently strong to be considered in the competitive range. We hope this will not discourage you from responding to future USAID requests for proposals. Indeed, we would welcome your participation, and look forward to future opportunities to work together in USAID's development programs. Protest File, Exhibit 8. 39. A third letter dated December 10, 1991, was sent by the contracting officer to BDI regarding its BDI/Unisys proposal. This letter sets out the principal reasons reported to the contracting officer by the TEP (Finding 26) for not finding this proposal technically acceptable. Protest File, Exhibit 9. 40. A letter dated December 10, 1991, was also sent to Seacor by the contracting officer. It explains in detail, similar to that provided to BDI, why Seacor's proposal was not included in the competitive range. Protest File, Exhibit 34. Seacor has not protested this determination. Negotiations With Maximus 41. By letter also dated December 10, 1991, the contracting officer advised Maximus that its proposal was within the competitive range but that there were a number of items in the proposal which needed to be addressed in the BAFO. Maximus was advised that its BAFO should be submitted on January 16, 1992. A separate enclosure with this letter identifies in considerable detail various items which the contracting officer asked Maximus to consider before submitting its BAFO. Protest File, Exhibit 12. 42. Among the areas and items Maximus was asked to consider in preparing its technical proposal for BAFO were the following: Source/Origin requirements; Maintenance and warranty provisions; Additional justification of proposed distributed architecture; Justification of stateside training; Clarification of proposed level of effort; Certain technical information regarding equipment; Assignment of primary responsibility for the SOW tasks; More detail regarding possible recurring costs for HIO; Additional detail regarding site preparation; Future changes in proposed hardware and software; Additional detail regarding certain proposed hardware and software; Additional detail regarding planned data communication; Adjustment and changes in some proposed staffing; and Leadtime for equipment delivery and payment terms upon shipment. Protest File, Exhibit 12 at 2-8. 43. Among the areas and items Maximus was asked to consider in preparing its cost proposal for BAFO were the following: More detail for subcontractors' budgets; More detail regarding staffing levels, staffing commitments, use of expatriate personnel, salaries and salary increases; Information regarding offeror's negotiated indirect cost rate agreement and possible impact of award on this agreement; Accuracy of rates used for allowances, benefits, and travel; Certificates for cost and pricing data and other representations and certifications from some subcontractors; Additional information from some subcontractors regarding local holidays and personnel policies; Clarifications regarding the possible inclusion of tax and/or duty and vehicle operational costs in some estimates; Information regarding method of estimating site preparation costs; and More detail regarding calculation of fee. Protest File, Exhibit 12 at 9-13. 44. In her deposition, the contracting officer has stated that she did not consider the various deficiencies in Maximus' proposal to be major or to impede her from determining that the offer was realistic and reasonable. Protest File, Exhibit 18 at 45-46. 45. On December 24, 1991, the contracting officer responsible for the competitive range determination left the USAID mission in Cairo in anticipation of a new overseas assignment. Protest File, Exhibit 18 at 6. 46. Maximus submitted its first BAFO on January 16, 1992. Protester's Second Protest File Supplement, Exhibit 2.[foot #] 9 ----------- FOOTNOTE BEGINS --------- [foot #] 9 In response to the Board's order of February 4, 1994, which afforded the parties an opportunity to update the record with any relevant evidence regarding events which had occurred since the Board's first decision on the merits, BDI (continued...) ----------- FOOTNOTE ENDS ----------- 47. In February 1992, a new contracting officer assumed responsibility for the procurement which is the subject of this protest. He served in this capacity until January 1993. Declaration of Braden W. Enroth (Enroth Declaration) (Dec. 20, 1994) at 1. 48. The technical proposal submitted by Maximus in its first BAFO was evaluated and commented on by the TEP in a report submitted in early April 1992. Protester's Second Protest File Supplement, Exhibit 7. 49. The same *** consultant who had assisted the contracting officer in the early phase of the first cost evaluation (Finding 22) reviewed the cost proposal submitted with Maximus' first BAFO. He was particularly concerned with the accuracy of the figures provided. For any upcoming negotiation session with Maximus, he recommended: Maximus should recheck all calculations and correct mistakes. Also, they should bring all details to support how the cost figures were derived and be ready to modify their costs again, based on changes made during negotiations. Protester's Second Protest File Supplement, Exhibits 4 at 1, 6 at 1. 50. One technical issue which received some attention after the submission of Maximus' first BAFO concerned the requirement to provide a demonstration of operational capability (DOC) for the system proposed. In a letter dated March 18, 1992, to the contracting officer, the *** consultant involved in the technical evaluation of proposals, noted that the local sites identified by Maximus in its demonstration plan did not have the architectural configuration or applications which Maximus was bidding in its proposal. Instead, the consultant proposed a stateside demonstration since more of the equipment proposed was in use in the United States than in Egypt. Protester's Second Protest File Supplement, Exhibit 3. 51. In a letter dated March 23, 1992, to Maximus, the new contracting officer passed on the concerns voiced by *** regarding the proposed demonstration of operational capability. He explained: We are looking for a[n] integrated, turnkey hardware/software system demonstration that closely ----------- FOOTNOTE BEGINS --------- [foot #] 9 (...continued) filed this Second Protest File Supplement which contains a total of sixty exhibits. ----------- FOOTNOTE ENDS ----------- replicates the proposed system in terms of system load (use), configuration (capacities, types and numbers of stations), external network communications, and functional system features (health/information applications). Protester's Second Protest File Supplement, Exhibit 5. 52. Documentation in the record indicates that two demonstrations did take place in Egypt in March 1992 but, because they did not completely satisfy the Government's requirements, it was decided to continue the DOC in the United States. In mid-May of 1992, a representative of Maximus and a *** consultant resumed the DOC with a visit to Shaw Supermarkets in Boston. The purpose of this visit was to review Shaw's computing environment to determine how similar it was to HIO and the computer system proposed by Maximus. Protester's Second Protest File Supplement, Exhibit 20. 53. Shaw's operation was found to offer some but not all of the characteristics of the ideal site. The *** consultant reported that Shaw did not regret its decision to utilize NCR equipment, that it was pleased with the NCR equipment, and that it found the NCR hardware very reliable. At this point, however, the *** consultant recommended to the contracting officer that he not require Maximus to continue to provide a complete DOC in view of the time and effort already expended. He noted that the X.25 network proposed has been widely used in the United States for at least fifteen years, that the configuration proposed by Maximus appears to be adequate to meet the needs of the RFP, and that the open systems architecture provided via UNIX does not lock HIO into using only NCR equipment forever. Instead, he recommended that Maximus be required to certify that the proposed architecture and system configuration will meet the HIO workload and all requirements stated in the RFP. Protester's Second Protest File Supplement, Exhibit 20. 54. There does not appear to have been a formal negotiation session with Maximus before submission of its second BAFO. Instead, by letter dated April 23, the contracting officer provided Maximus with extensive comments on its first BAFO and requested that it submit a second BAFO by May 17. Protester's Second Protest File Supplement, Exhibit 9. 55. In calling for a second BAFO, the contracting officer pointed out to Maximus that the TEP was of the opinion that Maximus' first BAFO had successfully addressed only twenty-one of the sixty specific points or questions listed by the previous contracting officer with her letter of December 10, 1991, to Maximus. See Findings 41-43. This, however, was interpreted by the contracting officer not as a failure of Maximus to be responsive but simply an indication of the complexity of the remaining thirty-nine items. Protester's Second Protest File Supplement, Exhibit 9. 56. In addition to requesting changes in the technical proposal, the successor contracting officer directed Maximus to revise its cost proposal completely. He passed on to Maximus the specific comments provided by the *** consultant regarding the cost proposal submitted with the first BAFO and insisted: "Every figure and calculation must be explained. Spreadsheets should be built from the bottom up starting at the lowest level of cost." Protester's Second Protest File Supplement, Exhibit 9 at 1. The demand for a complete revision of the cost proposal was modified somewhat by the contracting officer in a subsequent letter dated May 6 in which he advised Maximus that there would be no need to complete the additional cost tables included as Volume Two of the cost proposal submitted in the original BAFO. Id., Exhibit 13. 57. Although the record is somewhat unclear on this matter, it appears that Maximus submitted its second BAFO in advance of the May 17 deadline. Protester's Second Protest File Supplement, Exhibit 14 at 1. Both the technical and cost proposals were reviewed by the agency and its consultants in anticipation of formal negotiations. Id., Exhibits 15-17, 21. By letter dated May 21, the contracting officer provided Maximus with written comments regarding certain aspects of the revised cost proposal. Id., Exhibit 17. By letter dated the following day, May 22, Maximus provided extensive cost notes and replacement pages in response to the concerns and errors pointed out by the contracting officer in his letter of May 21. Id., Exhibits 18- 19. 58. Face-to-face discussions opened with Maximus during the last week of May. During these discussions, various questions arose regarding subcontractor costs and fees. On June 1, 1992, the parties met for negotiations. An unsigned memorandum, presumably written by the contracting officer himself, regarding this session, states: The opening comments were provided by the C.O. Buzz Enroth. Maximus was informed, as stated in the RFP, that the USAID was contemplating a CPFF [cost plus fixed fee] contract and because of this type of contract, it would necessitate that all activities and approaches be agreed to prior to any contract award. The Fixed Fee would be based on the negotiated total estimated cost. I informed Maximus that a field pricing support audit would be requested from DCAA [Defense Contract Audit Agency] and this activity [would] need to be completed and analyzed prior to award of a contract. Protester's Second Protest File Supplement, Exhibit 26 at 1. 59. The negotiation session of June 1 ended abruptly when it became clear that agreement could not be reached on matters relating to Maximus' fee. Protester's Second Protest File Supplement, Exhibit 26 at 2. However, in a letter to the contracting officer dated June 2, Maximus reaffirmed its second BAFO on its own behalf and on behalf of its subcontractors, but suggested a week for the parties to take some time "to think through the issues and possibilities away from the crucible of the negotiations." Id., Exhibit 27. 60. By letter dated June 10, Maximus submitted for the contracting officer's consideration, a number of counterproposals or suggestions "in the spirit of moving the negotiations to a swift conclusion." Protester's Second Protest File Supplement, Exhibits 29-30. 61. A lengthy USAID status report[foot #] 10 on contract negotiations, dated June 11, sums up the progress made during negotiations with Maximus and also briefly describes the impasse reached regarding Maximus' fee. The report anticipates resuming negotiations in mid-July provided the audit work on Maximus' own proposal as well as on the proposals of its local subcontractors can be completed by that time. Protester's Second Protest File Supplement, Exhibit 31. Maximus was provided a copy of this status report. Id., Exhibit 32. 62. In mid-June, 1992, the contracting officer went ahead with his plan to seek field pricing support in the form of audit assistance from DCAA pursuant to FAR 15.805-5 and FAR 15.806-3. Protester's Second Protest File Supplement, Exhibit 33; Enroth Declaration at 1. 63. In late July 1992, USAID's request for audit assistance from the DCAA was renewed. The letter was written by the contracting officer's supervisor and, in accordance with USAID procedures, was sent to the audit office of the agency's inspector general. The letter indicates that DCAA was apparently reluctant to provide the audit assistance previously requested. Protester's Second Protest File Supplement, Exhibit 35; Enroth Declaration at 2. This same letter observes that if any certified cost or pricing data is found to be erroneous, "we may have a fraud case for you to look into." Supplemental Protest File, Exhibit 35 at 1. The contracting officer has declared that with the exception of this one reference to fraud, he is aware of no other document which he wrote or reviewed in connection with the HIO procurement which raises concern over any possible fraud by Maximus or its subcontractors. Enroth Declaration at 2. 64. The DCAA audit process took considerably longer than originally anticipated. The final audit was not received by USAID in Egypt until September 21, 1992. Protester's Second ----------- FOOTNOTE BEGINS --------- [foot #] 10 This report is not signed but is acknowledged in other documentation in the record as being the report of the contracting officer, Braden Enroth. E.g. Protester's Second ____ Protest File Supplement, Exhibits 32, 47. ----------- FOOTNOTE ENDS ----------- Protest File Supplement, Exhibit 42. Maximus provided its own comments regarding this final audit report by letter dated October 20. Id., Exhibit 44. Documentation in the record indicates that once the audit process was complete, negotiations became considerably more intense. Id., Exhibits 45-59. 65. A handwritten document prepared by the contracting officer dated "1/93" summarizes the major areas of "savings" or "cuts" as a result of negotiations with Maximus. Protester's Second Protest File Supplement, Exhibit 53. The contracting officer explains that this memorandum represents notes prepared as a review sheet prior to a meeting with USAID/Cairo's top management and project personnel to discuss the HIO procurement. He further explains that the negotiations with Maximus during 1992 were dominated by cost issues. He states: It is my opinion that as a result of BDI's protest and the GSBCA's decision denying that protest in 1992, Maximus became aware that it was alone in the competitive range of the HIO Procurement and essentially in a sole-source negotiating position. It is also my opinion that Maximus showed intransigence regarding any lowering of its fee in 1992, and in fact raised its fee by almost $2 million between its BAFO I and BAFO II, due to its awareness that it was alone in the competitive range of the HIO Procurement. Enroth Declaration at 3. 66. The contracting officer's notes of "1/93" contain the following comment in the upper right-hand corner of page one: General Issues BDI - Cost going down. Sust. Protest. Protester's Second Protest File Supplement, Exhibit 53 at 1. The contracting officer explains that this note refers: to the fact that the cost negotiations with Maximus were at that time finally starting to break, and refers as well to my belief that at least one reason Maximus was finally revising its fee downwards was because it had learned that BDI was not dropping its protest and had, in fact, appealed to the Federal Court of Appeals. It is my opinion that Maximus finally wanted to bring negotiations to closure and commence contract performance, just in case BDI's protest was ultimately sustained. Enroth Declaration at 3. Protester disputes the contracting officer's reading of this note and claims instead that the phrase "Sust. Protest." actually reads "Just. Protest." Protester's Second Brief on the Merits at 64. Having examined the writing in question as well as the sworn statement of its author regarding the phrase, we find, as fact, that the term reads: "Sust. Protest." 67. By memorandum dated January 18, 1993, the contracting officer reported to his supervisor regarding the outcome of final negotiations with Maximus. He states that negotiations with Maximus should result in a contract of $21 million -- as contrasted with the Government's cost estimate of $27 million; $1.4 million of this reduction is said to be the result of Maximus having obtained current pricing from NCR on the original equipment list. Protester's Second Protest File Supplement, Exhibit 59. 68. On January 21, 1993, contract award was made to Maximus. Protester's Second Protest File Supplement, Exhibit 60. Discussion We turn now to a discussion of protester's specific allegations. Count I In its first count, protester alleges: AID violated FAR 15.609(a) by disqualifying Protester's proposal based on minor matters that were not evaluation factors in the Solicitation -- and which could be easily corrected by clarifications or negotiations. Complaint 18. The specific FAR provision which protester relies on in this count reads as follows: The competitive range shall be determined on the basis of cost or price and other factors that were stated in the solicitation and shall include all proposals that have a reasonable chance of being selected for award. When there is doubt as to whether a proposal is in the competitive range, the proposal should be included. 48 CFR 15.609(a) (1994) (FAR 15.609(a)). Protester's arguments in support of Count I fall into two basic categories. One line of argument is simply that the contracting officer's determination that the BDI proposals did not have a reasonable chance of being selected for award was flawed because it was based on erroneous technical and cost evaluations. The other line of argument is that the contracting officer's conclusion in this regard was based on unsupported facts. We turn first to the argument that the technical and cost evaluations were erroneous. Was the Contracting Officer's Conclusion that the BDI Proposals Did Not Have a Reasonable Chance of Being Selected for Award Based on Erroneous Technical and Cost Evaluations? In support of Count I, BDI argues that the contracting officer's competitive range determination was essentially flawed because it was based on an erroneous technical evaluation and an erroneous cost evaluation. In challenging the reliability of the technical evaluation, protester stresses the disparity between the overall technical scores given to proposals by two members of the TEP. Protester also points to the wide disparity in the scores these two individuals gave on a limited number of individual criteria. Protester's First Brief on the Merits at 33- 39.[foot #] 11 The presence of disparity among the scores of the TEP members is certainly not unprecedented. It is not unusual for individual evaluators to reach different conclusions and assign different scores when evaluating proposals since both objective and subjective judgments are involved. Novel Pharmaceutical, Inc., B-255374, 94-1 CPD 149, at 6 (Feb. 24, 1994) (citing Cybernated Automation Corp., B-242511.3, 91-2 CPD 293 (Sept. 26, 1991)). The procedure provided for in this procurement, Findings 14-15, of averaging the technical scores given on the various evaluation factors is a common measure taken to safeguard against the possibility of an inadvertent, unfair result where such disparity may exist. We recognize that a quality review of scores given on the 335 individual criteria for each proposal might have improved the accuracy of the evaluation. Nevertheless, failure to make such a review certainly does not constitute a fatal flaw in the technical evaluation process. Given the large number of individual criteria involved, the consistency in higher or lower scoring among evaluators for all proposals, the averaging technique employed, and the occasional comparison of results on at least a section-by-section basis, we conclude that the technical evaluation was sound and reliable under the circumstances. ----------- FOOTNOTE BEGINS --------- [foot #] 11 In referring here to protester's "first brief on the merits" and elsewhere to respondent's "first brief on the merits," we are referring to the briefs submitted by counsel prior to the Board's original decision on the merits which was subsequently vacated by the Court of Appeals. ----------- FOOTNOTE ENDS ----------- Protester's principal objection to the accuracy of the contracting officer's cost evaluation is her requesting and using a revised Government estimate shortly after she began her evaluation of cost proposals. Protester's First Brief on the Merits at 40-44. Protester apparently believes this was a deliberate ploy on the part of respondent to favor one offeror over others. The record does not support this contention. The original error was readily understandable and the need for correction obvious. See Findings 20-21. Protester has provided us with no credible reason to question the motives of the contracting officer in calling for a revised estimate. Protester also contends that the contracting officer's cost evaluation was flawed because her "subjective evaluation" of realism and completeness did not reflect all of the comments provided by the *** consultant who made a preliminary review of cost proposals. Protester's First Brief on the Merits at 40. We find no merit in this argument. The contracting officer has made it clear that she herself performed the evaluation of cost proposals and that, in doing so, she made selective use of the information provided by the *** consultant. Finding 24. We find nothing wrong with this. In the final analysis we would expect the evaluation to be based on her judgment and not on that of her consultant. Protester has not demonstrated to us that any of the scores given by the contracting officer in her evaluation of cost proposals lacked a rational basis or represented an abuse of discretion. Furthermore, in view of the extraordinarily high weight given in this procurement to the technical merit of proposals (90%), even if a limited number of flaws did exist in the evaluation of cost proposals, they would have to be nigh on egregious for the overall evaluation of proposals to be fatally flawed. In this regard, protester does allege the existence of a blatant error made in the evaluation of Maximus' cost proposal. See Protester's First Brief on the Merits at 43. We are not convinced, however, that the error was of the magnitude alleged by protester. Protester's contention is based on deposition testimony of the *** consultant who, at the contracting officer's request, made an initial review of the various cost proposals. It was his understanding that Maximus' supplier proposed to add 120% to the sales price of certain equipment to cover taxes, duties, and cost of money. Protest File, Exhibit 20 at 34-39. Upon closer review, however, it appears that the adjustment in question related to a 20% markup calculated by multiplication of the net selling "by 1.2 (for customs duties, sales tax, and cost of money)." See Respondent's Brief at 32. In summary, we find none of protester's arguments questioning the fundamental soundness of the technical or cost evaluation of offers to be convincing. We turn, therefore, to protester's other line of argument. Was the Contracting Officer's Conclusion that the BDI Proposals Did Not Have a Reasonable Chance Of Being Selected for Award Based on Unsupported Facts? In its first brief on the merits of this case, protester contends that the contracting officer's decision to eliminate BDI from the competitive range was unsupported because the combined evaluation scores for the four technically acceptable offers showed "there was ample opportunity for the BDI and Seacor proposals to exceed the Maximus score after a BAFO for the top four proposals." Protester's First Brief on the Merits at 26. This argument, based as it is on the raw point-spread between evaluation scores, provides no real support for Count I. As already noted, that count states that AID violated FAR 15.609(a) by disqualifying BDI's proposals based on minor matters which could be easily corrected through clarifications. The "point-spread argument" essentially ignores the comparative review conducted by the contracting officer in making her competitive range determination. Any probability of BDI improving its standing vis a vis Maximus in the BAFO process obviously depends on the qualitative merit of the proposals and not simply on the quantitative relationship of their evaluation scores. For this reason, an argument based solely on the point- spread without reference to reasons behind it is of no real value. Protester may well be correct in suggesting that the point-spread between scores in this procurement is far from conclusive, on its face, as to the prospects of the lower-scored offers for award. The analysis, however, must go deeper and examine the facts specific to this procurement. In rearguing its case on remand, protester has finally done this, turning to the specific deficiencies listed by the contracting officer in her letters of December 10 regarding the BDI/IBM and the BDI/NCR proposals. In discussing the contracting officer's letter regarding the BDI/IBM proposal, protester has singled out forty-one items for specific analysis. In discussing the letter regarding the BDI/NCR proposal, protester identifies and discusses thirty-nine items. The conclusion reached after this lengthy treatment is that a limited number of the items touched upon in the letters are simply based on incorrect assumptions.[foot #] 12 The numerous concerns remaining in the contracting officer's letters are described as merely informational deficiencies which could have been easily remedied. Assuming this to be the case, protester argues that any determination that BDI's offers did not ----------- FOOTNOTE BEGINS --------- [foot #] 12 We count eleven such items said to be present in the letter regarding the BDI/IBM proposal and seven in the letter regarding the BDI/NCR proposal. ----------- FOOTNOTE ENDS ----------- have a reasonable chance of award was clearly without basis. See Protester's Second Brief on the Merits at 5-41, 44-60. We are unpersuaded by this new argument. We turn first to the argument that some of the contracting officer's concerns are plainly wrong because they are based on incorrect assumptions. In support of this contention, protester has submitted, as exhibits with its second brief on the merits, excerpts from its original proposals. These materials, although available at the time, were not offered for inclusion in the record prior to its being closed in advance of the Board's first decision on the merits.[foot #] 13 Respondent has questioned whether BDI at this late date should be allowed a second bite at the apple by setting forth in its second brief on the merits detailed arguments contesting the substantive elements of the contracting officer's competitive range determination when it passed on the opportunity to do so in its first brief on the merits. See Respondent's Second Brief on the Merits at 21. In deciding this case on remand and affording the parties the opportunity to submit a second round of briefs on the merits of this case, we have permitted them to supplement the argumentation previously provided in the first round of briefs. However, in making any new argument, the parties must comply with the rules established regarding the evidentiary record in this case. Those rules are simple. For reasons of efficiency and fairness, we have declined to reopen the evidentiary record as to matters which occurred prior to our first decision on the merits. We have, however, in response to protester's requests, agreed to reopen the record to permit the parties to update it with evidence regarding events which occurred since our earlier review of the case. We, therefore, will not admit to the record at this late juncture those exhibits which protester now offers in support of its allegations that several of the statements made in the contracting officer's letters of December 10 were incorrect.[foot #] 14 We likewise decline to accept any argument based upon this documentation. Protester had ample ----------- FOOTNOTE BEGINS --------- [foot #] 13 We note that the record does not contain copies of BDI's proposals. At the start of this case, the parties were invited to propose for inclusion in the record those portions of BDI's and other vendors' proposals considered relevant to this protest. Conference Memorandum (Dec. 24, 1991). This was done by both parties but on a very limited basis. [foot #] 14 In a separate order issued this same date, we have identified the specific exhibits proffered with protester's second brief on the merits, which are being rejected for the reason set out herein. ----------- FOOTNOTE ENDS ----------- time prior to our first decision on the merits to include in the evidentiary record proof of such allegations. This litigation cannot continue ad infinitum. Although the TEP found two of the BDI proposals technically acceptable, protester, in developing the evidentiary record to support its original case-in-chief, chose not to include any materials which might specifically disprove the accuracy of the many discrepancies and weaknesses listed by the contracting officer in her letters of December 10 to BDI. We consider it only fair that protester now accept the consequences of that decision. The deficiencies which protester would now eliminate as plainly erroneous, however, are relatively few when compared to those which remain in the two letters of December 10. It is the contention of protester that all of the remaining cost deficiencies and technical deficiencies, with minor exceptions, are merely informational.[foot #] 15 Protester's Second Brief on the Merits at 5-24, 24-41. There are undoubtedly several genuine informational deficiencies in the contracting officer's letters to BDI regarding its proposals. This is obvious from a review of each letter. Nevertheless, we remain in radical disagreement with the protester that virtually all of the concerns listed by the contracting officer in her letters fall into this category. To describe the concerns raised by the contracting officer in her letters, BDI has adopted a definition of "informational deficiency" which exceeds the limits traditionally established for the term. In defining an informational deficiency, protester states that it is: one based on lack of information, lack of clarity, lack of explanation, or insufficient explanation. Protester's Second Brief on the Merits at 41. Our understanding of the term "informational deficiency" is similar to that of the Comptroller General, namely, "deficiencies that could readily be remedied by additional information during discussions, such as the omission of letters of commitment . . . ." Apt Corp., GSBCA 9237-P, 88-1 BCA 20,411, at 103,249, 1987 BPD 284, at 12 (citing Associated Corp., B-225562, 87-1 CPD 436, at 8 (Apr. 24, 1987)). Key to the traditional notion of informational deficiency is the fact that this type of deficiency can be "readily remedied." The missing information in question ----------- FOOTNOTE BEGINS --------- [foot #] 15 The only substantive technical deficiencies which BDI is prepared to acknowledge are its failure to offer an embossing machine and metaphor graphic capability software and its omission of a proposed training specialist as called for in the "staffing" requirements of the RFP. Protester's Second Brief on the Merits at 42. ----------- FOOTNOTE ENDS ----------- need not be developed by any extensive dialogue and often can be provided by a mere unilateral submission by the offeror -- as in the case of a missing letter of commitment. It goes without saying that unless truly authentic informational deficiencies in a proposal are so numerous as to require major revisions, a proposal with nothing more than this type of deficiency should obviously not be dismissed as lacking any reasonable chance of award. This is particularly true in situations where the elimination of such proposals would leave only one offeror in the competitive range. Falcon Systems Corp. B-213661, 84-1 CPD 658 (June 22, 1984). This is not the situation confronting us in this protest however. Looking at the quantity and quality of the specific concerns listed in the contracting officer's letters, we are far from convinced that they would have been readily resolved. Many are simply not informational deficiencies as that term is normally understood. Rather they reflect a substantial lack of understanding of requirements or a similar lack of diligence in preparing the proposal. Correction of these deficiencies would have undoubtedly required considerably more discussion and substantive revision of the proposals. Furthermore, given the initial quality of protester's proposals, we consider it highly unrealistic to assume, as protester does, that in each and every case, additional discussion would have led to a satisfactory conclusion. The term "informational deficiency," as normally used, simply does not aptly describe all the deficiencies in question. For example, in the BDI/IBM proposal, we find the concerns regarding portability, compatibility, data entry, software, and the proposal's general lack of firmness to be far more than informational deficiencies. See Finding 37. Similarly, in the BDI/NCR proposal we find the concerns regarding proposed equipment, data communication and the proposal's lack of firmness to be examples of far more than informational deficiencies. See Finding 38. In essence, protester has adopted an expansive definition of "informational deficiency" and then proceeded to avail itself of the benefit of precedents based on the traditional meaning of the expression. We conclude this part of our discussion with one final observation regarding protester's efforts to categorize the majority of the concerns raised by the contracting officer in her letters of December 10 to BDI as "informational deficiencies." Protester compares these concerns with those mentioned by the contracting officer in the attachment she sent to Maximus by letter of the same date. According to protester, many of the items mentioned to Maximus were relatively "minor issues specifically implicated in the BAFO process" and would most likely not have been raised "if Maximus had not been allowed to enter the negotiation phase." Protester's Second Brief on the Merits at 60. The obvious implication is that the deficiencies listed for Maximus were purely informational. The record, however, says otherwise. Of the various "minor issues" in the Maximus proposal, only approximately one third were successfully addressed in the first BAFO. Finding 55. This hardly supports BDI's contention that the items mentioned in the letters it received, which were of more substantial import, were merely informational deficiencies.[foot #] 16 In its second brief on the merits, protester has fashioned an additional argument in support of Count I. This argument is based upon an examination of events occurring in this procurement after the competitive range determination. The duration and complexity of these negotiations with Maximus allegedly demonstrate that Maximus' offer was not sufficiently superior as to justify the conclusion that BDI's proposals had no reasonable chance of being selected for award. Protester's Second Brief on the Merits at 61. In presenting this argument, BDI calls our attention to a number of facts relating to negotiations with Maximus. In the area of costs, protester argues that several events support the contention that Maximus' price was not as good as originally believed. Protester describes the difficulties encountered with the costs provided by Maximus' subcontractors, the resultant need for an "emergency audit," the problems and delay encountered in negotiating Maximus' fee, the contracting officer's demand for a complete revision of Maximus' cost proposal, a reduction of 1.4 million dollars from Maximus' original price only after respondent was allegedly "forced to audit Maximus and its subcontractors." Finally, protester contends that a marginal comment written into the notes of the successor contracting officer allegedly indicates that USAID's contract estimate was excessive and that concessions wrung from Maximus during negotiations brought its price closer to that of BDI and thus justify BDI's protest. Protester's Second Brief on the Merits at 61-64. In the area of technical discussions, protester refers to a "litany of technical problems" throughout negotiations and in particular to USAID being forced to waive the demonstration of ----------- FOOTNOTE BEGINS --------- [foot #] 16 We note in passing that BDI's admission that its deficiencies were "fundamentally different" from the "minor issues" identified by the contracting officer for Maximus is somewhat surprising. Nevertheless, BDI presses the argument that because its deficiencies were more than mere "minor issues," the improvement in its score after negotiations would have been considerably greater than that achieved by Maximus. Protester's Second Brief on the Merits at 60. We find this argument conflicts with and severely undermines BDI's basic contention that its own deficiencies were merely informational. In this regard, we readily understand why respondent aptly complains: "BDI cannot have it both ways . . . ." Respondent's Second Brief on the Merits at 50. ----------- FOOTNOTE ENDS ----------- operational capability (DOC) requirement because the competitive range was reduced to only one offeror. Id. at 64-67. We have examined the record, as supplemented by protester, regarding events which occurred during negotiations with Maximus. Where necessary, we have made various findings of fact. Having done so, we find protester's characterization of events which occurred during negotiations to be exaggerated and, at times, in direct conflict with the evidence offered in support of protester's contentions. We would expect USAID to examine Maximus' cost proposal with extreme care once the competitive range was reduced to one offeror. The record confirms that respondent did so. The cost proposal submitted with Maximus' BAFO was closely scrutinized by the *** consultant who was advising the successor contracting officer. Finding 49. Based on the consultant's recommendations, this contracting officer insisted on substantial revisions of the cost proposal which was submitted as part of Maximus' BAFO. Finding 56. As to the contracting officer's decision to audit Maximus' cost proposal, we do not look upon this as an "emergency measure." Rather, it appears to us as a logical, precautionary measure appropriate for a situation such as this where the competitive range was reduced to one offeror. Field pricing support was available to the contracting officer, and he wisely chose to make use of it. Documentation from the procurement files indicates that his plan to have the cost proposal audited was made known to Maximus even before the first round of discussions reached a snag on the issue of fee. Findings 58-59. As to the disagreement regarding fee, it does not appear to have significantly stalled negotiations. The dialogue between respondent and AID did not cease after face-to-face negotiations temporarily concluded on June 1, 1992. Indeed, by letter dated June 2, Maximus was careful to reaffirm its second BAFO on its behalf and on behalf of its subcontractors. Finding 59. In addition, by letter dated June 10, Maximus submitted a number of counter-proposals or suggestions to move negotiations "to a swift conclusion." Finding 60. As to the time consumed in auditing the costs of Maximus and its subcontractors, we are not convinced that the time ultimately required for the audits necessarily demonstrated that the contracting officer's initial assessment of Maximus' proposal was substantially flawed. There were clearly some administrative problems associated with the start of the process. This was an overseas procurement not without some complexity of its own. Furthermore, the audits were requested during the summer season when vacation leave often results in additional delay. Findings 61-64. The $1.4 million reduction in Maximus' price is readily explained in terms of a change in current pricing data rather than the result of any "forced" or "emergency audit." See Finding 67. As for the significance of the contracting officer's marginal note regarding Maximus' lowering of costs in the face of a continuing BDI protest, we find the contracting officer's explanation considerably more credible than the protester's suggested interpretation. Finding 66. It is indeed true that the original price bid by Maximus finally did go down significantly -- considerably more than $1.4 million. See Finding 67. Protester has failed to convince us, however, that this ultimate price reduction proves that the contracting officer's original assessment of Maximus' initial proposal was somehow flawed. Rather, the facts strongly suggest that this reduction was attributable to the persistence of the AID negotiating team and, ironically, to recognition on the part of Maximus that continued delay might lead to the return of BDI to the procurement in the event it should prevail in this protest after a successful appeal. Findings 65-67. We likewise are unconvinced that the judgment of the original contracting officer in assessing BDI's possibility of award is put into question by the alleged emergence of a "litany of technical problems" with Maximus' technical proposal. The principal problems identified by the successor contracting officer in writing to Maximus after BAFO are basically those which were identified by the previous contracting officer in the attachment which she sent to Maximus under cover of her letter of December 10. Findings 41-43, 55. Certainly, therefore, the first contracting officer was aware of these technical deficiencies when she made her competitive range determination. As to AID being forced to waive the DOC requirement, we find nothing in the record suggesting that the DOC could not be completed. Instead, the facts indicate that AID's consultant did not believe the expenditure of time and resources justified in view of the apparent reliability of the solution actually proposed. He, therefore, suggested a certification as an alternative -- which suggestion the contracting officer apparently chose to accept. Findings 50-53. In short, the events occurring during the negotiations with Maximus, which protester would have us focus upon, do not convince us that the contracting officer's original assessment of Maximus' proposal and of BDI's inability to compete seriously with it was incorrect or unrealistic. The record, as updated, does not indicate the emergence of any new and substantial problems with Maximus' technical proposal during negotiations which might have surprised the original contracting officer or prompted her to change her assessment of Maximus' and BDI's proposals. As to the close scrutiny given to Maximus' cost proposal, this is not particularly surprising given the fact that only one vendor remained in the competitive range. Indeed, protester has not convinced us that the original contracting officer would not have been as exacting on subcontractor costs as was her successor -- had she remained to conduct negotiations with this sole vendor. Protester also calls our attention to the fact that the contracting officer conferred with several individuals regarding her competitive range determination before actually making it. This, according to protester, indicates that there was doubt in her mind and, therefore, pursuant to FAR 15.609(a), BDI should have been included in the range. Protester's First Brief on the Merits at 32. Protester would have us infer solely from the fact that the contracting officer conferred with others that she was in doubt as to her proposed determination. This is not an inference which we readily draw. Given the facts, as we understand them, it is equally as probable -- if not more so -- that the purpose of these meetings was simply to test and/or confirm a determination already relatively firm -- particularly since no suggestion was made during these consultations that she include all four offers in the competitive range. See Finding 32.[foot #] 17 Conclusion Regarding Count 1 In vacating our first decision on the merits, the Court of Appeals for the Federal Circuit observed: To promote the legislative and regulatory policy favoring competition while allowing contracting officers the full range of their discretion, the Board of Contract Appeals and the Comptroller General have given "close scrutiny" to contracting officer evaluations that result in a competitive range of one. . . . ----------- FOOTNOTE BEGINS --------- [foot #] 17 Protester contends that the contracting officer disregarded the TEP's recommendation that all four technically acceptable proposals be subject to BAFOs. Protester's First Brief on the Merits at 51. This assertion is at odds with the contracting officer's statement that no one suggested the inclusion of all four offers in the competitive range. Protester's contention appears to be based on a distorted interpretation of the TEP's recommendation that the contracting officer use the BAFO process. The RFP provided for award without further discussion. Finding 11. It was obviously this alternative which the TEP was attempting to dissuade the contracting officer from using when it urged her to make use of the BAFO process. As to the possibility of a more restrictive competitive range, this was expressly recognized by the TEP as a distinct possibility -- once the cost evaluation was complete. See Finding 18. ___ ----------- FOOTNOTE ENDS ----------- We agree with this approach. If the contracting officer has determined an initial competitive range of one, there must be a clear showing that the excluded bids have "no reasonable chance" of being selected. . . . Birch & Davis International, Inc. v. Christopher, 4 F.3d at 974 (citations omitted). The prolonged litigation of this protest has afforded us ample opportunity to scrutinize closely the competitive range determination which is the subject of this protest. Upon review of the entire record, as updated, it is our conclusion that a convincing body of evidence has been assembled in support of the affirmative determination that the excluded BDI bids had no reasonable chance of being selected. The reasons stated by the contracting officer in her letters of December 10 are highly persuasive. This is especially true when they are taken in the aggregate. It is no less true, however, when one focuses instead on substantive deficiencies relating to portability, compatibility, software, data entry, data communication or lack of firmness in the proposals. See Findings 37-38. We find the comments of the TEP, as they confirm and supplement the contracting officer's comments, persuasive as well. The qualitative difference between the proposal of Maximus and those of BDI is reflected in far more than the point-spread between scores. As a matter of fact, evidence in the record convinces us that the TEP was somewhat generous in its scoring of the two BDI proposals. BDI was alone among offerors singled out for criticism in the introduction of the TEP report to the contracting officer. The incompleteness of BDI's proposals was a matter of special concern. The report observed: [I]t does little good to suggest that AID or HIO should "trust us" simply by invoking their past reputation, and yet fail to indicate precisely how something is to be done on this project. Finding 17. This general comment on incompleteness was confirmed more specifically in the TEP's specific comments on each of the three BDI proposals. See Finding 26, notes 6, 8. Similarly, the TEP voiced its concern over the total absence of any level of effort data in the BDI proposals. See notes 5, 7. This is a remarkable oversight in a vendor's response to a solicitation which indicated that a cost-plus-fixed-fee, level of effort, term contract was contemplated and which expressly called for level of effort data. See Findings 1, 2, 6. Finally, there is the fact that, despite the technical scores actually given to two of BDI's proposals, when the contracting officer discussed BDI's prospects for award with her technical people, none suggested that BDI's proposals be kept in the competitive range. Finding 32. In addition, the genuine informational deficiencies actually present in BDI's proposals and listed by the contracting officer, although undeniably not in and of themselves sufficient to reject the proposals, certainly do not evince a promising potential on the part of BDI to improve significantly on the quality of the proposals -- particularly when the RFP itself initially advised offerors that award might be made without discussions. In contrast to BDI's proposals, Maximus obtained a relatively superior score but one which was far enough away from the total points available as to leave a significant potential for improvement. Findings 3, 28. Admittedly, apart from the general deficiencies found by the TEP to be common to nearly all proposals (Finding 16), there are some deficiencies and characteristics in the BDI and Maximus proposals which at first blush appear somewhat similar. We note, however, that even in these instances there is a notable qualitative difference. Perhaps the most obvious similarity is that the BDI/NCR and Maximus proposal both indicate an intention to use NCR equipment. Nevertheless, the BDI/NCR proposal is said to suffer from a lack of integration between BDI's main proposal and that portion dealing with NCR. Finding 38. Given the absence of any similar comments regarding Maximus, it is apparent that the NCR elements were considerably better integrated into the Maximus proposal.[foot #] 18 Two examples of the qualitative difference between seemingly similar deficiencies in the BDI and Maximus proposals concern the RFP requirement to provide level of effort data and the requirement to provide site preparation costs. Notwithstanding an express requirement that vendors indicate how many people would be assigned to each task and subtask, BDI failed to provide this data. Findings 37-38. Maximus, on the other hand, did provide the data. The contracting officer did, however, seek some clarifications regarding the data which was provided. Finding 42. Similarly, the contracting officer observed that BDI's proposals did not contain remodeling site-preparation costs. Findings 37-38. Maximus, in contrast, did provide this data but was asked to explain how it developed its estimates. Finding 43. ----------- FOOTNOTE BEGINS --------- [foot #] 18 The fact that both BDI and Maximus envisioned use of NCR equipment also undermines BDI's contention that any concerns regarding its technical solution could have been readily resolved. Given the superior technical score given initially to Maximus, there was clearly potential for technical leveling which would undoubtedly have further complicated any discussions with BDI. ----------- FOOTNOTE ENDS ----------- In making her competitive range determination, the contracting officer relied, in part, on the evaluation scores assigned to the four technically acceptable offers. Protester has not convinced us that the process leading to these scores was substantially flawed. The contracting officer, however, took into account considerably more than the raw numerical scores. Her decision to exclude BDI was based on a careful comparison of proposals and on the conduct of an in-depth review of the weaknesses and gaps in the four proposals which were evaluated. She looked to the detailed analysis of technical proposals by the TEP. She considered the potential of each offeror to improve on its evaluation score during the course of negotiations. She consulted with her technical staff regarding the prospects of offerors receiving an award after BAFO. She likewise looked to her own analysis of the cost proposals submitted by those offerors. Findings 30-31. Nevertheless, it remained her considered opinion that the BDI offers had no reasonable chance of being selected. Given the record before us, we agree with the contracting officer that the disparity between the overall scores of the BDI proposals and that of Maximus would not have decreased significantly during the BAFO process and that BDI, therefore, had no reasonable chance of being selected for award. Accordingly, further discussions with BDI would have been meaningless. The elimination of the BDI proposals, notwithstanding the fact that this resulted in a competitive range of one was, consequently, reasonable under the circumstances. American Systems Corp., B-247923.3, 92-2 CPD 158 (Sept. 8, 1992). For this reason, we deny Count I of the protest. Counts II and III In the second and third counts of its protest, BDI has alleged that AID violated FAR 15.605(e) and 15.608(a) by not identifying significant factors and subfactors in the solicitation. The two FAR sections cited by protester read as follows: The solicitation shall clearly state the evaluation factors, including price or cost and any significant subfactors, that will be considered in making the source selection and their relative importance (see 15.406-5(c)). Numerical weights, which may be employed in the evaluation of proposals, need not be disclosed in solicitations. The solicitation shall inform offerors of minimum requirements that apply to particular evaluation factors and significant subfactors. FAR 15.605(e). Proposal evaluation is an assessment of both the proposal and the offeror's ability (as conveyed by the proposal) to successfully accomplish the prospective contract. An agency shall evaluate competitive proposals solely on the factors specified in the solicitation. FAR 15.608(a). In its complaint, protester offers three examples of the allegedly hidden subfactors. They are listed as follows: (a) The requirement for significant levels of expatriate personnel participation; (b) The specific requirement of hardware and software compatibility; (c) The requirement of local training as compared to training in the United States. Complaint 24-25. During the course of discovery, protester, in response to an interrogatory posed by respondent, provided a more complete listing of the alleged hidden factors and subfactors said to be contained in the contracting officer's letters of December 10 regarding the BDI/IBM, the BDI/NCR, and the BDI/Unisys offers: The BDI/IBM offer: "Compatibility" paragraph; "Data Entry in the Branch" paragraph; "Data Entry in the Polyclinics" paragraph; The software requirements in the "RFP Requirements Omitted in the Proposal" paragraph; The "Data Communication Equipment" paragraph; The Third Country National costs statement in the "General paragraph;" The "Fixed Fee" paragraph; The "Indirect Costs" paragraph. In the BDI/NCR offer: The "Fully Distributed Architecture" paragraph; The "Data Communication" paragraph; The Third Country National costs statement in the "General" paragraph; The "Fixed Fee" paragraph; The "Indirect Costs" paragraph. In the BDI/Unisys offer: Paragraph 3 on page 1 discussing data entry configurations, data communication equipment and PC's for use by HIO application development staff. Protest File, Exhibit 15. In briefing these counts of its protest, BDI has offered no argument or proof. Instead, it simply repeats the allegations contained in its complaint and further elaborated on in its answer to respondent's discovery request. See Protester's First Brief on the Merits at 47-50. We have examined the contracting officer's letters of December 10 regarding the BDI/IBM, BDI/NCR, and the BDI/Unisys proposals. See Findings 26, 37, 38. In particular, we have examined the specific sections in the letters dealing with the BDI/IBM and BDI/NCR proposal which are said by protester to involve hidden evaluation factors and subfactors. We find all of them to be reasonably related to the technical and cost evaluation factors in the RFP. See Finding 3. In the absence of any specific showing by protester to the contrary, we deny both Counts II and III. Count IV In the fourth count of its protest, BDI contends that AID has failed to seek full and open competition and thus violated the Competition in Contracting Act when it eliminated BDI's proposals for minor matters that were not evaluation factors. Protester has failed to convince us that its proposals were eliminated for minor matters that were not evaluation factors. We, therefore, deny this count as well since the premise on which it rests has not been proven. An Additional Count In its first brief on the merits, protester also argues that, in violation of FAR 15.610(b), respondent began discussions with Maximus prior to establishing the competitive range for this procurement. This allegation is not contained in BDI's complaint. It is said to be based on protester's learning during discovery that on November 24, 1991, the contracting officer called a Maximus representative to confirm that statements made in the proposal regarding compliance with geographic code requirements of the RFP applied to all proposed equipment. See Finding 34. We reject protester's allegation for two reasons. First, this contention of protester is based upon the assumption that at the time the inquiry was made, the competitive range had not yet been determined. The record remains uncertain on that point. See Finding 33. Second, given the nature and purpose of the inquiry, we do not view it as anything more than a request for clarification as that term is understood in FAR 15.601. Decision This protest is DENIED. ___________________________ EDWIN B. NEILL Board Judge We concur: _____________________________ ROBERT W. PARKER Board Judge _____________________________ CATHERINE B. HYATT Board Judge