THIS OPINION WAS INITIALLY ISSUED UNDER PROTECTIVE ORDER AND IS BEING RELEASED TO THE PUBLIC IN ITS REDACTED FORM ON MAY 20, 1994 ___________________________________________________ GRANTED IN PART AND DISMISSED IN PART: March 15, 1994 ___________________________________________________ GSBCA 12692-P SPRINT COMMUNICATIONS COMPANY, Protester, and GTE GOVERNMENT SYSTEMS CORPORATION, Intervenor, v. DEFENSE INFORMATION SYSTEMS AGENCY, Respondent, and GOVERNMENT SYSTEMS, INC., Intervenor. Carl J. Peckinpaugh, Scott M. Heimberg, Janet Z. Barsy, and Sheila C. Stark of Akin, Gump, Strauss, Hauer & Feld, Washington, DC; and Julie Lunceford Witcher, George J. Affe, and Anthony L. Cogswell of Sprint Communications Company, Herndon, VA, counsel for Protester. George W. Stiffler, Charles M. Reifel, and Nick R. Hoogstraten of Bastianelli, Brown & Touhey, Washington, DC; and Sidney G. Masri of GTE Government Systems Corporation, counsel for Intervenor. Douglas G. White and H. Jack Shearer, Office of the General Counsel, Defense Information Systems Agency, Scott Air Force Base, IL; and John R. McCaw, Gregory C. Carter, and Robert Zuckerman, Office of the Chief Counsel, Federal Aviation Administration, Washington, DC, counsel for Respondent. Jeffrey H. Smith, J. Robert Humphries, Ronald A. Schechter, and Susan B. Cassidy of Arnold & Porter, Washington, DC, counsel for Intervenor Government Systems, Inc. Before Board Judges PARKER, NEILL, and HYATT. NEILL, Board Judge. This protest, filed by Sprint Communications Co. (Sprint), challenges the award of a contract by the Defense Commercial Communications Office (DECCO) of the Defense Information Systems Agency (DISA). The award was made to Government Systems, Inc. (GSI). The contract was awarded by DECCO on behalf of the Federal Aviation Administration (FAA). It calls for the furnishing of an agency data telecommunications network for FAA. GSI and GTE Government Systems Corporation (GTE), another participant in this procurement, have both intervened in this protest as intervenors of right. Sprint's protest, as amended, includes seven counts. Count I alleges that the Government failed to evaluate properly Sprint's technical proposal. Count II alleges that the Government misapplied the award criteria of the solicitation by selecting a higher cost offer that did not provide material advantages to the Government. Count IV alleges that the Government failed to award to the most advantageous offeror. Count V alleges that the Government failed to conduct meaningful discussions. Count VI alleges that the Government failed to properly evaluate Sprint's management proposal. Count VII alleges that the GSI proposal fails to meet mandatory requirements of the solicitation, and Count VIII alleges that GSI and the Government engaged in post-best and final offer (BAFO) discussions.[foot #] 1 For reasons explained in the discussion portion of this opinion, we grant Counts I, II, IV, V and VI of Sprint's protest. We dismiss Count VII as untimely filed and we dismiss Count VIII for failure to state a valid basis of protest. ----------- FOOTNOTE BEGINS --------- [foot #] 1 Sprint's original protest included six counts. Sprint withdrew one of those counts (Count III) and added two others (Counts VII and VIII) by amendment. ----------- FOOTNOTE ENDS ----------- Findings of Fact The Solicitation 1. On July 27, 1992, DECCO issued a solicitation, request for proposals (RFP) no. DCA200-92-R-0029, to provide telecommunications services for the FAA under what was then called the Administrative Data Transmissions Network 2000. DECCO subsequently issued twelve amedments to this RFP. Amendment 1 (issued on September 21, 1992) changed the name of the procurement to Agency Data Telecommunications Network 2000 (ADTN2000). Protest File, Exhibit 5, Appendices A, A-2. 2. The current FAA Administrative Data Transmission Network (ADTN) is a nationwide, private, packet switched network which provides connectivity between users and information system host computer resources. The DECCO's solicitation for the ADTN2000 calls for a new end-to-end network service which will use FTS2000 [Federal Telecommunications System 2000] for all long haul data transport and connectivity and replace the current ADTN. Protest File, Exhibit 5, Appendix A at C-2. The ADTN2000 is to (i) provide data network services among geographically dispersed FAA users and computer resources; (ii) be internetworked with local and wide-area national and international networks; and (iii) provide video conference capabilities. Id., Appendix A, Section J-2, Functional Specification at 1. This new network service is to be provided by an integration contractor having total responsibility to meet approved user requirements for intersite administrative data communications. A five year requirements contract with five option years is envisioned. Id., Appendix A at C-2. Instructions on Preparation of Proposals 3. Solicitation clause L.18 provided offerors with the instructions for the preparation of offers. Protest File, Exhibit 5, Appendix A at L-9. Pursuant to these instructions, offerors were directed to submit their proposals in four parts: Part I -- Technical, Part II -- RFP and associated information, Part III -- Management, and Part IV -- Price Proposal. Id. at L- 10. Detailed formats were provided for the technical proposal, id. at L-11 to L-15; the management proposal, id. at L-15 to L- 18; and the price proposal, id. at L-18 to L-23. 4. With respect to the technical proposal, the instructions directed: The technical proposal shall respond to each Task and SubTask in the Statement of Work (SOW) and Functional Specification (FS). The technical proposal shall be in sufficient detail and include a written description which is complete enough for the Government to validate the degree of compliance to the stated requirements. The proposal shall outline the offerors' understanding of the operational environment, technical solutions, implementation, and any risks inherent with each task/subtask recognized by the offeror with the proposed solutions thereto. The technical proposal will use the same paragraph numbering system as the SOW and/or FS, and shall be self-sustaining without reference to any other response or literature for support. . . . . 2. Completeness. a. Technical proposals shall identify and explain any exceptions or deviations taken or conditional assumptions made with respect to the offeror's proposal versus the RFP requirements. Any exceptions taken must contain sufficient amplification and justification to permit evaluation. . . . . . . . c. Technical proposals should be straightforward and free of extraneous information. Proposals should be sufficiently detailed to clearly and fully demonstrate that the offeror has a thorough understanding of all the RFP requirements. Statements that the offeror "has read, understands, can or will comply" with the specifications, and statements paraphrasing the specification or parts thereof, are not satisfactory, and phrases such as "standard procedures will be employed" or "well known techniques will be used," will not be acceptable and may result in rejection of the proposal. The technical proposal must set forth enough of the technical approach, including technical methods and engineering approaches to sufficiently show how it is proposed to meet RFP requirements. A full explanation of techniques and procedures proposed to be followed shall be included. Block diagrams, drawings or charts should be included, as appropriate, to present a full, complete and clear explanation. . . . d. In addition to responding to all RFP requirements, the following information must be included in the appropriate section of the technical proposal: . . . . (ii) The manufacturer's name, model number, nomenclature and operating characteristics for each type of equipment proposed for use in the network must be provided. A description of the technology used in the proposed equipment must be included, as well as any modifications, necessary to meet FAA requirements. Protest File, Exhibit 5, Appendix A at L-12 to L-13. Technical Requirements 5. Four technical requirements in the RFP are of particular importance in this protest. The first concerns multilink procedures. The basic requirement is described as follows: 3.4.2.4 Link Control - Where a packet network solution is employed, ADTN2000 link control shall comply with the following mandatory requirements: . . . . b. Multilink procedures, as specified in CCITT X.25- 1988, shall be implemented where multiple physical circuits are required between the user DTEs [Data Terminating Equipments] and network DCEs [Data Circuit Terminating Equipments]. Protest File, Exhibit 5, Appendix A, Section J-2, Functional Specification at 17. The second technical requirement of importance in this protest concerns dial-up access service. It is described as follows: 3.5.3 Dial-up access service - Users needing switched, or alternate connection to the ADTN2000 shall be provided access via dial-up connections. Users outside the local calling area of an ADTN2000 node or user end point shall access the ADTN2000 via FTS2000 services. Users within the local calling area of a node or user end point shall be provided access via ADTN2000 via access ports which meet the following mandatory requirements: a. Dial access ports shall be installed at appropriate ADTN2000 nodes and user end points as required to meet user dial-in and dial-out requirements in the local calling area of those sites. . . . . e. X.25 dial-up in accordance with CCITT recommendation X.32 shall be provided initially at all major concentrations of users for synchronous circuits. f. All synchronous circuits shall use X.32 facilities for authentication, billing, and optionally, encryption. The intent of this requirement is to support evolution to a more sophisticated dial-up access capability. Id. at 22. The third technical requirement relevant to this protest concerns inactivity timers. The RFP provides: 3.5.4.1 h. The ADTN2000 shall terminate the logical connection when failure of the user DTE occurs or when there is no activity on the call for a selectable time from 1 to 30 minutes. The time value shall be under configuration control of the NCC [Network Control Center]. Id. at 23. The fourth and final requirement relates to closed user groups (CUGs) and is described as follows: 3.5.4.3 User Groups - The contractor shall develop a system of user groups with the following parameters/capabilities: a. A user may be a member of one or more user groups (no limit). The extended format shall be supported. Id. at 24. The Solicitation's Commerciality Requirement 6. Amendment 1 of the solicitation contains the following provision regarding the commerciality of the equipment offered: H.25 COMMERCIALITY - DECCO/RPFF (JUL 1992) The contractor shall provide equipment that meets the requirement for commercial or commercial-type products as defined in FAR [Federal Acquisition Regulation] Part 1 1 . The contractor shall substantiate the claim that the equipment offered are [sic] commercial or commercial- type products by completing the certification in Section K. The commerciality requirements shall apply at the time proposals are submitted. Protest File, Exhibit 5, Appendix A-2, Tab 1 at H-8. With Amendment 4 to the RFP, dated October 29, 1992, the Government added Section K.26 (CERTIFICATION COMMERCIAL ITEMS). This clause requires offerors to certify that equipment offered in response to the RFP qualified as one of four types of commercial items. One of the four types is a commercial item that requires "only minor modification in order to meet the requirements of the procuring agency." Id., Tab 4 at K-14. Section K.26 defines minor modification as "a modification to a commercial item that does not alter the commercial items'[sic] function or essential physical characteristics." Id. 7. On August 11, 1993, DECCO issued Amendment 11 to the RFP. This amendment added the following provision to the solicitation's statement of work: C.3.1.2 Commerciality. Offerors shall use commercial products for the ADTN2000 technical solution. A commercial product is an item, material component, subsystem, or system sold or traded to the general public in the course of normal business operations at prices based on established catalog or market prices. Commercial products shall be available to the general public no later than proposal submission to meet commerciality criteria. Protest File, Exhibit 5, Appendix A-2, Tab 11 at 13 (unnumbered). 8. On September 14, 1993, DECCO issued Amendment 12 to the RFP. A cover letter explained that the amendment "eliminates inconsistencies among three proposal areas regarding commerciality." These sections are said to be clauses H.25, C.3.1.2, and K.26. The letter goes on to advise offerors to review their commerciality certifications called for in Section K and recertify as appropriate. Protest File, Exhibit 5, Appendix A-2, Tab 12 at 1 (unnumbered). 9. Amendment 12 of the RFP revises clause H.25 to read as follows: H.25 COMMERCIALITY - DECCO/RPFF (SEP 1993) The contractor shall provide equipment that meets the requirement for commercial or commercial-type products/items as defined in the Statement of Work (SOW), paragraph C.3.1.2. The contractor shall substantiate the claim that the equipment offered are commercial or commercial-type products/items by completing the certification in Section K. The commerciality requirements shall apply at the time proposals (or amendments thereto) are submitted. Protest File, Exhibit 5, Appendix A-2, Tab 12 at 4 (unnumbered). Amendment 12 also revises and expands clause C.3.1.2 to read as follows: C.3.1.2 Commerciality. Offerors shall use commercial or commercial-type products/items for the ADTN2000 technical solution. C.3.1.2.1 Commercial Products/Items. A commercial product is an item, material, component, subsystem, or system sold or traded to the general public in the course of normal business operations at prices based on established catalog or market prices. Commercial products/items shall be available to the general public no later than proposal submission (or amendments thereto) to meet commerciality criteria. C.3.1.2.2 Commercial-type Product/Items. A commercial-type product/item is a commercial product/item (a) modified to meet some Government peculiar physical requirement or addition, or (b) otherwise identified differently from its normal commercial counterparts. Any commercial-type product/items shall require only minor modifications from its commercial counterpart in order to meet the Government's requirements, where a minor modification means a modification that does not alter the commercial product's/item's function or essential physical characteristics. Commercial-type products/items shall be available for delivery prior to the earliest time established for delivery of the required service. Id. at 5 (unnumbered). The Solicitation's Evaluation Scheme 10. Section M of the RFP addressed "Evaluation Factors for Award." Section M.3 listed three evaluation factors for award: (1) technical, (2) management, and (3) price. Section M.3.a. described the relationship among the three factors as follows: The technical factor is significantly more important than management, and technical/management combined are more important than price. Cost may become more important as the difference between combined technical/management scores decrease[s]. Protest File, Exhibit 5, Appendix A at M-1. 11. Section M.4 addresses eligibility for award. To be eligible for award: (1) the proposal must comply in all material respects with the requirements of the law, regulation and conditions set forth in the solicitation; (2) the proposal must meet all mandatory solicitation requirements; and (3) the offeror must be determined responsible. RFP Section M.5 governed the evaluation of the technical, management and price factors. Protest File, Exhibit 5, Appendix A at M-3. Evaluation of Technical Proposals 12. Section M.5.a. outlines the procedure for evaluating an offeror's technical proposal. It requires the Government to consider specific information including a proposal's "feasibility, soundness and any risks." Protest File, Exhibit 5, Appendix A at M-3. Section M.5.a. also provides that all of the technical requirements in the functional specification and the statement of work are mandatory requirements, with the exception of specification paragraph 3.6.6 which relates to the operating environment. This provision of the RFP expressly provides: Failure to fully meet or address a mandatory requirement will be cause for rejection of the proposal. Id. 13. Section M.5.a. requires the technical evaluation to address eight technical subfactors, namely, (1) network design, (2) network functional requirements, (3) network implementation, (4) network performance, (5) network management, (6) network equipment requirements, (7) training, and (8) documentation. Each technical subfactor contains a multitude of specified elements and subelements within the individual subfactors. Offerors are advised that a technical score will be developed for each subfactor by computing a weighted average of the numerical scores assigned to every requirement applicable to that subfactor. Protest File, Exhibit 5, Appendix A at M-4 to M-6. 14. Section M.5.a. provides that each requirement of the specification and statement of work will be evaluated separately and assigned a score of zero to 130 points as follows: 0............ fails to meet/address a mandatory requirement 100.......... fully meets the stated requirement 101-130...... exceeds the stated requirement, and provides additional value/advantage to the Government. Protest File, Exhibit 5, Appendix A at M-4. In the technical evaluation, offerors were scored in this fashion on a total of 413 items. Id., Exhibit 130. Evaluation of Management Proposals 15. RFP Section M.5.b. governs the evaluation of management proposals. It provides that the scoring shall "be based on the evaluator's ability to obtain, from the proposal, an in-depth understanding of the offeror's comprehensive corporate capabilities to perform the ADTN2000 contract." Protest File, Exhibit 5, Appendix A at M-7. Section M.5.b does not state that the management requirements are mandatory but only that "[m]ajor deficiencies in any one of the factors may be cause for rejection of the proposal." Id. at M-10. 16. Section M.5.b. establishes the following five management subfactors: (1) contract management, (2) previous performance, (3) key personnel, (4) financial capacity, and (5) proposed exceptions. Protest File, Exhibit 5, Appendix A at M-8 to M-10. The first three of these subfactors are the most important and of equal weight. Id., Exhibit 5, Appendix A at M-8 to M-10. Offerors are advised that a management score will be developed for each subfactor by computing a weighted average of the numerical scores assigned to every requirement applicable to that subfactor. Id. at M-10. 17. Section M.5.b. provides that each requirement of the statement of work and the RFP related to management will be evaluated separately and assigned a score from zero to 130 points as follows: 0............. inadequate or fails to address a stated requirement 70-99......... less than fully meets the stated requirement, but is adequate 100........... fully meets the stated requirement 101-130....... exceeds the stated requirement, and provides additional value/advantage to the Government. Protest File, Exhibit 5, Appendix A at M-8. In the evaluation of management proposals, offerors were scored in this fashion on a total of 58 items. Id., Exhibit 130. 18. Despite the solicitation's characterization of a management score of 70-99 as "adequate," a typical score sheet nevertheless describes the score of 70-99 as "Less Than Meets Requirements." E.g., Protest File, Exhibits 111, 113. Evaluation of Price Proposals 19. Section M.5.c. provides that price proposals are to be evaluated on the basis of the prices offered in response to section B of the solicitation and on the quantities listed on site worksheets for each location of the model system solution, and the network quantity worksheet for non-site specific services. Offerors are also expected to provide an estimate of FTS service costs based on the mandatory use of the AT&T Service Analysis Tool pricing model. Protest File, Exhibit 5, Appendix A at M-11. The Technical/Management Evaluation Plan 20. The Government prepared for its own internal use a Technical/Management Evaluation Plan (TMEP), dated December 1992, to provide the organization and procedure for evaluating technical and management proposals. The TMEP: (1) established a technical/management evaluation team (T/MET) to evaluate the proposals, and (2) provided evaluator/scoring guidelines for those of its members who served on its Technical Evaluation Team (TET) and for those who served on its Management Evaluation Team (MET). Protest File, Exhibit 4. 21. Four members of the T/MET served as scoring members of the TET and three T/MET members served as scoring members of the MET. Protest File, Exhibit 4, Table 2. The work of the TET and MET extended over a nine-month period. Transcript at 817, 1029- 30. 22. Section 3.1 of the TMEP, purporting to use a definition found in the FAR, defines "deficiency" as "any part of a proposal that fails to satisfy (fully meet) the Government's requirements."[foot #] 2 Protest File, Exhibit 4 at 3- 1. Elsewhere in the TMEP, section 3.4.2 provides: A discussion item (DI) should be prepared whenever an evaluator, after Group discussion, concludes that a proposal, as it relates to a specific sub-factor is either (1) unclear, confusing, or otherwise insufficient ----------- FOOTNOTE BEGINS --------- [foot #] 2 We note that the definition found in the FAR 15.601 does not include the terms "fully meet" which appears in the TMEP definition. See 48 CFR 15.601 (1993) (FAR 15.601). ___ ----------- FOOTNOTE ENDS ----------- to enable a firm scoring decision, or (2), deficient (i.e., less than fully meets the stated sub-factor requirement). All DI's should be forwarded to the Contracting Officer, who will request a written response from the Offeror. Id. at 3-5. 23. The MET team leader has testified that in evaluating and issuing discussion items with regard to the management proposals, the evaluators did not consider an "adequate" score between 70-99 to be deficient. Transcript 984-85, 1033-34. 24. Appendix F of the TMEP contains the weighing and scoring database procedures. It provides that this information is not to be distributed to evaluators. Protest File, Exhibit 4, Appendix F. Initial Offers 25. Offerors were directed to submit initial technical and management proposals on December 7, 1992. Protest File, Exhibit 5, Appendix A (Amendment 5). Seven offerors, including Sprint and GSI, submitted timely technical and management proposals. Id., Exhibit 14 at 1 (unnumbered). The TET and MET commenced the proposal evaluation process in early January. Id., Exhibit 10, Parts B at 5, C at 4. Negotiations with Sprint 26. According to Sprint's opportunity manager for ADTN2000, Sprint understood that the Government wanted each offeror to propose the most cost-effective and optimum solution that it could with the items that it was proposing. Sprint understood that the combined technical and management aspects of the proposals were to be accorded more importance than price, but that price could become relatively more important as the difference between combined technical and management scores decreased. Given the requirements of the solicitation, and the fact that the Government had "fairly narrowly defined what it was looking for in ADTN2000," Transcript at 36-38. 27. The initial evaluation of Sprint's management and technical proposals led to the issuing of over DIs to Sprint. These various DIs were identified either as "deficiencies," "ambiguities," or, in a few cases, "requested exceptions." Protest File, Exhibit 161 at 75-94, 133-256. Negotiations with Sprint on its Technical Proposal 28. In their review of the initial Sprint proposal, the evaluators noted that Sprint did not propose multilink procedures, as required by Functional Specification 3.4.2.4.b. Accordingly, in DI E101-D20T (issued April 28, 1993), Sprint was asked to explain how it intended to comply with the requirement for implementation of CCITT Multilink procedures. Protest File, Exhibit 161 at 347. In its response to the DI on May 12, 1993, Sprint answered: Id. 29. In their initial review of the Sprint technical proposal, the evaluators also noted that Sprint did not specify how it would meet X.32 dial-up access, as required by numerous RFP specifications. As a result, DI E205-055T was issued. It asked how Sprint intended to meet that requirement. Protest File, Exhibit 161 at 411. Sprint responded: Id. 30. In responding to other DIs sent out initially by DECCO, Sprint also indicated that it planned to add enhancements to its switches to comply with two other functional requirements -- closed user group (CUG) formats and inactivity timers. Protest File, Exhibit 161 at 271, 350. 31. In a second round of DIs issued on August 11, 1993, DECCO questioned how Sprint's plans to engage in development to meet the multilink, X.32 dial-up access, CUG, and inactivity timer requirements squared with the commerciality provision of the RFP. DECCO was pointed out that clause H.25 requires the contractor to provide equipment which meets commerciality requirements at the time proposals are submitted. Protest File, Exhibit 161 at 15, 22, 26, 43. 32. This second round of DIs, questioning Sprint's plans for development to meet certain technical requirements of the RFP, was issued at the same time as Amendment 11 which added to the SOW clause C.3.1.2 dealing with commerciality. See Finding 7. Upon receipt of Amendment 11, Sprint's opportunity manager concluded that the new provision regarding commerciality, which had been added to the SOW, was inconsistent with related provisions already in sections H and K. He has testified that this apparent conflict caused him to question whether the commerciality requirement was being changed. Transcript at 68. 33. On August 17, 1993, Sprint's opportunity manager and some other Sprint representatives placed a conference call to the DECCO contracting officer. In that conversation, the Sprint representatives sought to determine whether a commercial-type item would be compliant with the Government's requirement, where the item is defined as a commercial item requiring only minor modification to meet the requirements of the procuring agency as described in paragraph K.26 of the solicitation. See Protest File, Exhibit 19, Contracting Officer's Memorandum for Record (Aug. 17, 1993). The issue was revisited in another conversation with the contracting officer the following day. See id., Contracting Officer's Memorandum for Record (Subject: Teleconference with Sprint) (Aug. 18, 1993). By the end of these conversations, DECCO had agreed with Sprint that the terms of the solicitation should be read to allow an offeror to propose commercial-type items or products, which would include items to which minor modifications were required in order to meet the requirements of the solicitation. Transcript at 72-73. What would constitute an acceptable minor modification was defined by reference to solicitation paragraph K.26. Id. at 73. 34. DECCO confirmed the two telephone conversations in a letter to Sprint, dated August 19, 1993. The letter noted that the commerciality requirements were to be met as of December 7, 1992, the proposal submission date; it pointed out that Sprint's original certification of commerciality under paragraph K.26 of the solicitation had not indicated that Sprint was proposing minor modifications. See Protest File, Exhibit 19, Contracting Officer's Letter to Sprint (Aug. 19, 1993). The letter, therefore, asked Sprint to address five specific points regarding the proposed modifications as part of its responses to the second round of technical DIs. The five points were identified in the letter as follows: a. Specify what enhancements will be made to the equipment; b. Specify how the enhancements will be achieved; c. Certify that the enhancements will not alter the functionality or physical characteristics of the item; d. Specify the associated risk; and e. Specify when testing and validation of the enhancement's functionality will be achieved. Id. The letter closes with a request that Sprint confirm in writing by August 23, 1993, its understanding of the commerciality requirement for ADTN2000. Id. 35. By letter dated August 20, 1993, Sprint replied to DECCO. Sprint's letter explained that the items it was offering were commercial items as of December 7, 1992, although the minor modifications would be completed subsequently. See Protest File, Exhibit 19, Letter from Sprint to the Contracting Officer (Aug. 20, 1993). Accordingly, Sprint asserted that its products met the commerciality requirement as of the proposal submission date. Sprint's letter closes with a request that the Contracting Officer confirm in writing that she concurs with Sprint's position as stated in the letter. Id. 36. In a telephone conversation on August 23, 1993, the DECCO contracting officer informed a senior contract administrator at Sprint that she was in agreement with Sprint's position regarding commerciality but that she did not plan to send a letter confirming this fact. She also pointed out that the Government is more concerned with Sprint's ability to address the risk issue. See Protest File, Exhibit 343 (handwritten note at bottom of second page); Transcript at 1120. The contracting officer's reference in that conversation to "risk" was understood by Sprint's opportunity manager as referring to the issue of risk raised as the fourth point of concern in DECCO's letter of August 19 (see Finding 34). Transcript at 77-80. 37. On August 26, 1993, Sprint submitted its responses to the second round of technical DIs. Protest File, Exhibit 161 at 12. As requested, Sprint addressed DECCO's five points under each of the four DIs. Transcript at 80-89; see Protest File, Exhibit 161 at 15-18 (Sprint's response to DI E007a070T), 22-25 (Sprint's response to DI E101aD20T), 26-29 (Sprint's response to DI E104aD77T), 42-49 (Sprint's response to DI E205a055T). For each DI, Sprint specified what enhancements would be made to the equipment, specified how the enhancements would be achieved, certified that the enhancements would not alter the functionality or physical characteristics of the item at issue, specified the associated risk, and specified when testing and validation of the enhancement's functionality would be achieved. With respect to the issue of associated risk, Id. 38. Also, on or about August 26, Sprint provided the contracting officer with a revised commerciality certification pursuant to solicitation paragraph K.26. This revised certification indicated that Sprint was offering some equipment that would include minor modification in order to meet the Government's requirements and some that would not. Protest File, Exhibit 104. 39. On September 2, 1993, the FAA sent the DECCO contracting officer a draft letter which it wanted DECCO to send to Sprint. The draft letter would have provided Sprint with additional DIs and advised Sprint that the Government had found its proposal to be unacceptable. The basis for this position was FAA's belief that no equipment modifications after the original proposal due date were allowed under the solicitation as amended (by Amendment 11). Protest File, Exhibit 105. 40. On September 7, 1993, in preparation for an Source Evaluation Board (SEB) teleconference scheduled for September 8, the contracting officer sent the DECCO SEB members a memorandum regarding the ADTN2000 commerciality requirements. In that memorandum, the contracting officer noted: Per teleconference Fri., the FAA does NOT want modifications, minor or otherwise, to any equipment proposed for ADTN2000. However, this is allowable per the K.26 definition for a commercial item. The FAA cited bad experiences with allowing previous equipment modifications on other contracts. The FAA thought that the SOW paragraph superseded the Section H and Section K requirements. DECCO advised that this is not so. According to the Section I clause, Order of Precedence, Section H and Section K take precedence over the SOW and FS. FAA wants the Section K certification deleted or changed. There is no FAR certification - only DFARS [Defense Federal Acquisition Supplement]. Protest File, Exhibit 19, Contracting Officer's Memorandum (Sept. 7, 1993). 41. A memorandum from counsel, attached to the contracting officer's memorandum, also noted: FAA seems to insist on the SOW provisions [regarding commerciality]. It is unwilling to risk what it considers the loophole of a "minor modification." Protest File, Exhibit 107. The legal memorandum goes on to recommend that the Government amend the solicitation to eliminate the offering of minor modifications as an acceptable approach to meeting the requirements of ADTN2000. Id. 42. The contracting officer's memorandum of September 7 describes two options: 1. Issue Amendment 0012 changing the Section K "commercial item" certification and definition. (See attached Legal recommendation. Note Legal subsequently advised that the evaluators interpret the offerors' proposed modifications as a change to the equipment's functionality.) 2. Issue D/I to the two offerors not in compliance with the commerciality requirement stating the Government's interpretation (we need to discuss and reach an agreement on our interpretation) of what constitutes a commercial item, and specifically state that the proposed modification(s) changes the functionality of the commercial item and does not meet the commerciality requirement, as defined in K.26, for ADTN2000. The D/I should request current data regarding the proposed enhancements. The offeror has to provide hard data, such as the equipment component/feature/model identification codes, industry standard requirements, and GOSIP conformance, to substantiate the proposed modification meets the ADTN2000 technical requirements. Protest File, Exhibit 19, Contracting Officer's Memorandum (Sept. 7, 1993). 43. The contracting officer has no recollection of the discussion regarding the RFP commerciality requirement which presumably took place at the SEB teleconference on September 8. However, she agrees that she wrote at the bottom of the file copy of her September 7 memorandum to the DECCO SEB members: SEB interpretation - minor modifications are allowed. Inconsistencies between Secs H, K, and SOW will be resolved in Amendment 0012. Offerors will need to recertify as appropriate. Protest File, Exhibit 19, Contracting Officer's Memorandum (Sept. 7, 1993); Transcript at 1114-15. 44. The record contains nothing which indicates that Sprint was ever informed of this internal Government debate regarding the commerciality requirement of the RFP and its application to the Sprint proposal. Rather, on September 3, 1993, Sprint was sent a third round of technical DIs, which included one related to the use of modems for the X.32 dial-up feature. Protest File, Exhibit 161 at 7. Neither it nor any other DIs indicates any lingering concern with respect to the modifications described by Sprint in its reply to the earlier round of DIs. 45. On receiving this third round of DIs, Sprint's opportunity manager concluded that Sprint and the Government now clearly understood each other. Transcript at 89-90. This was further confirmed in his mind when, on September 14, 1993, DECCO issued Amendment 12 (Findings 8-9), which expressly corrected the conflicts in the RFP provisions which had prompted Sprint to seek clarification from the contracting officer on August 17 and 18. Transcript at 90-92. Sprint responded to Amendment 12 by noting that it had already recertified as required. Protest File, Exhibit 344. 46. In order to be certain that it could meet the requirements of the solicitation, Sprint held discussions with its primary equipment supplier, [foot #] 3 to secure commitments would do what was required to meet the Government's schedule. Transcript at 93-97. The record contains documentation indicating that Sprint very much aware that development would be required to meet the requirements of the solicitation using equipment Protest File, Exhibits 349, 351-353. Id., Exhibits 354, 364-365. There is no indication that any of these concerns or internal estimates of the time and effort required were shared with DECCO or FAA by Sprint during negotiations. 47. A representative testified that development can be accomplished much faster than apparent from the estimates only. Transcript at 250.[foot #] 4 For example, Transcript at 250-51. 48. Sprint's opportunity manager for this procurement has testified under oath that he still believes Sprint's proposal fully ----------- FOOTNOTE BEGINS --------- [foot #] 3 Id. at 247. ___ [foot #] 4 Transcript at 250. ----------- FOOTNOTE ENDS ----------- meets the stated requirements of the solicitation. Transcript at 92. The TET Report Regarding Sprint's Proposal 49. In describing the evaluation of Sprint's technical proposal, the TET report states: There were numerous deficiencies and ambiguities identified in the initial Sprint proposal, many of which persisted until the third round of DIs. As a result of many poorly written and incomplete changes made to the proposal, a DI was issued stating that the proposal had become nearly unintelligible. Sprint responded by submitting an updated proposal which cleared up much of the confusion but some minor errors remained. Protest File, Exhibit 10, Part B at 50. 50. Despite the remaining "minor errors," the TET concluded that Sprint's proposal did not meet certain technical requirements of the RFP. The TET report reads: Despite DIs on deficiencies related to mandatory requirements for X.25/X.32 dial-up services, providing extended CUG format, inactivity timers on native X.25 circuits and implementing X.25 multilink procedures, the Technical Evaluation Team assessed the Sprint proposal (as amended by response to these DIs) as not adequate to meet requirements of the RFP. As a result, the failure to meet these mandatory requirements significantly lowers their overall technical evaluation results. Protest File, Exhibit 10, Part B at 50. 51. Contrary to the TET report, the TET evaluators did not find Sprint noncompliant with all four of the mandatory technical requirements mentioned in the report. Sprint was given a score of points (the requirement fully met) on the requirement dealing with inactivity timers and the extended CUG format. Sprint's perceived failure to meet the X.32 dial-up requirements and the X.25 multilink procedures, however, resulted in a score of for eight related items. Protest File, Exhibits 109, 130. 52. The TET's reason for concluding that Sprint could not meet the remaining two requirements stemmed from its conviction that, in the aggregate, all of the modifications required to comply with the RFP's requirements constituted a single, major integrated development effort and, as such, did not satisfy the commerciality requirement of the RFP. Protest File, Exhibit 10, Part B at 48. 53. The individual score sheets covering the eight items relating to multilink and X.25/X.32 dial-up services on which Sprint was given a score of confirm that the individual evaluators considered more than minor modifications were required to meet the requirements in question. For example, one evaluator wrote: for X.32 dial-out and authentication and multilink procedures for the , together with the other planned modifications proposed for this equipment, constitutes a major modification which is not permissible under Amend[ment] 12 for commercial type equipment. Protest File, Exhibit 109 (Cossa Evaluation Item Score Sheet). 54. In scoring the same eight items relating to multilink and X.25/X.32 dial-up services, a second member of the TET wrote that the multilink functionality is no trivial implement and the addition of the required X.32 functions to the switches would alter the function of these commercial products. Protest File, Exhibit 109 (Ellis Evaluation Item Score Sheet). The third evaluator put it even more succinctly. He wrote: In view of the other changes being planned for the , the "minor changes" being proposed are, in the aggregate, major. Id. (Kaiser Evaluation Item Score Sheet). This same evaluator, however, testified that in distinguishing between a major and minor modification, he was not using the term "minor modification" to mean one which does not change the functionality or central physical characteristics of the device. He agreed that the addition of a function such as multilink to the switch does not change the functionality of the switch. Transcript at 852. 55. In a similar vein, the fourth evaluator on the TET wrote on his score sheets relating to the eight items covering multilink and X.25/X.32 dial-up services the following: Vendor states that the four developments that he explicitly proposes are independently minor and that, therefore, he satisfies the Commerciality requirement. However, this effort must be viewed as a single, major, integrated development effort . . . . Protest File, Exhibit 109 (Newcomb Evaluation Item Score Sheet). Negotiations with Sprint on its Management Proposal 56. In evaluating Sprint's management proposal, the MET ultimately scored Sprint above points on all 58 scored items. Twelve of these scores, however, fell between and . Protest File, Exhibit 130. In this protest, Sprint challenges the correctness of those twelve scores. For purposes of this opinion, we make the following findings (namely 57 to 84) regarding the twelve scores in controversy. 57. The manager of the MET has testified that during the course of the first round of evaluation of management proposals, DIs were sent to offerors whenever an evaluator rated an offeror at less than 100 points on a management evaluation item. She has confirmed, however, that if the reply received did not raise the offeror's score to 100 points, no additional DI was sent. Transcript 981-82. When asked why this was not done, she replied: Because we felt we discussed the information that came in and if we thought that it would be leading them to another conclusion of telling them how to do it, we didn't send another DI. Id. at 982. 58. [Organization, Item E-02] The solicitation requires each offeror to submit a contract management plan. One of the requirements of this plan, on which management proposals were scored, concerned the requirement for organization. The RFP reads: Organization: The Plan shall describe the Contractor's organizational structure and flows of authority for administrative and business matters arising under this contract . . . . Protest File, Exhibit 5, Appendix A, Section J-4, CDRL/DID AOO1. Upon review of Sprint's proposal, the three MET members all scored Sprint points on this item. One evaluator wrote: Your proposal has the NCC [Network Coordination Center] Id., Exhibit 127. A DI was issued asking what the specific Id., Exhibit 128. Sprint replied that Id. 59. Upon review of Sprint's reply to the DI regarding organization, two evaluators revised their score on this item to . The third, left her score of unchanged. She wrote: Response does not fully demonstrate affirmative action on [sic] part. has tech[nical] expertise and what extent can Protest File, Exhibit 127. 60. [Key Personnel, Item E-03] Another requirement of the contract management plan called for in the solicitation concerned key personnel. The solicitation provided: The plan shall identify key management personnel, their organizational relationships, their authority with respect to contract and program administration; [s]taffing and resumes of key technical and management personnel . . . . Protest File, Exhibit 5, Appendix A, Section J-4, CDRL/DID AOO1. This was a specific item on which offerors were scored in the evaluation of their management proposals. Initially, one evaluator concluded that the proposal fully satisfied this requirement. Two other evaluators, however, rated Sprint because there had a question whether there would be a change in key personnel and positions following transition to the new ADTN. Id., Exhibit 123; Transcript at 1058. 61. In response to a DI on these key management personnel, Sprint assured the Government Protest File, Exhibit 124. As a result of this explanation, one evaluator, who had scored Sprint at , revised his score to 100. The remaining evaluator noted the clarification but raised his original score from . The MET team leader has testified that this evaluator did not raise the score on this item any higher because, although Sprint had clarified the issue, it had not gone back into the proposal to change the part which had given rise to the ambiguity in the first place. Transcript at 1058. 62. [Project Tracking, Item E-04] The contract management plan called for in the solicitation was also expected to include a section on: Project Tracking: Procedures for automated project tracking and progress reporting including how the FAA can access the system to obtain status information . . . . Protest File, Exhibit 5, Appendix A, Section J-4, CDRL/DID AOO1. Related to this requirement was a provision in the RFP's functional specification requiring the Network Control Center of the ADTN2000 to be equipped: with an automated project scheduling and tracking system that can be accessed by the FAA (read only) to determine, on a near real-time basis, the status of service orders for new installs, moves, changes or upgrades. Id., Functional Specification at 33. 63. In response to the project tracking requirement, Sprint proposed All three members of the MET scored Sprint on the project tracking item as originally proposed. See Protest File, Exhibit 117. One evaluator scored Sprint at ; the other two scored it at . Id. Two DIs were issued. They sought clarification on the mentioned by Sprint and on how the FAA would have access on a near real-time basis to check on the status of service orders. Id. 64. Sprint answered the DIs on project tracking. Sprint explained that Protest File, Exhibit 161 at 98, 118. 65. The MET evaluator who originally scored Sprint at on the project tracking item raised that score to upon review of Sprint's answer to the two DIs which were issued. Although recognizing the FAA would have access to the he remained convinced that the proposal did not meet requirements since the Protest File, Exhibit 117. The second evaluator left her score unchanged at . She wrote: The offeror has neglect[e]d to address the procedures, definition of process/procedure. Less than meets requirements of RFP because of this omission. Id. The third evaluator likewise left her original score of unchanged after reviewing Sprint's reply to the two DIs. She wrote: The response states that FAA will Id. 66. [Automated Project Scheduling, Item E-30] In addition to an evaluation item for project tracking, the MET also scored offerors on the related item of automated project scheduling. The item, like the project tracking item, was based on the contract requirement that the Network Control Center of the ADTN2000 be equipped: with an automated project scheduling and tracking system that can be accessed by the FAA (read only) to determine, on a near real-time basis, the status of service orders for new installs, moves, changes or upgrades. Protest File, Exhibit 5, Appendix A, Section J-2, Functional Specification at 33. 67. Initially, all three of the MET evaluators scored Sprint at on the automated project scheduling item. Protest File, Exhibit 125. However, in responding to the DI regarding project tracking, Id., Exhibit 161 at 118. This prompted one evaluator to revise her score on the project scheduling item . She wrote: Offeror's resolution to DI introduces will require learning curve of FAA user. Id., Exhibit 125. This evaluator has testified that avoidance of is not a stated requirement of the solicitation. Transcript at 1013. 68. [Sprint Coordination, Item E-05] As already noted, the solicitation required offerors to submit a contract management plan. Among other items, the plan was to include a section on: Sprint Coordination: Procedures for the coordination with U.S. Sprint during implementation and cutover . . . . Protest File, Exhibit 5, Appendix A, Section J-4, CDRL/DID AOO1. This was an item on which all offers were scored in the evaluation of management proposals. 69. Relying on its incumbency, Sprint dealt with the Sprint Coordination requirement in a cursory fashion. Sprint's management proposal simply stated: SPRINT COORDINATION Protest File, Exhibit 8, Appendix G at III-Appendix C-36. All three members of the MET initially assigned a score of to Sprint on this item. Id., Exhibit 119. A discussion item was issued. It advised Sprint of an "ambiguity" in that "procedures for facilitating the transition within the Sprint organization have not been documented." Id., Exhibit 120. Sprint's reply to the DI, Two of the MET evaluators declined to increase their original scoring to . The third evaluator did increase his score to but with the explanation that a technical explanation was deemed sufficient since the "original question was aimed at non-Sprint bidders." Id., Exhibit 119. 70. [Risk Management, Item E-22] Within the management evaluation, section M of the RFP provided that the Government would evaluate the soundness of the corporate approach to risk management. The solicitation, however, included no requirement that risk management be addressed in any particular way or in any particular part of the proposal. See Protest File, Exhibit 5, Appendix A at M-8. The MET found that Sprint's proposal did not fully meet this requirement. A DI was prepared but was never issued. Transcript at 986. The average raw score eventually given by the MET to Sprint on that item was . Protest File, Exhibit 130. In determining this score, the evaluators relied on information provided by Sprint in response to two other DIs. The one dealt with the experience of the project manager, and the other dealt with a management tracking system. Protest File, Exhibit 111. 71. [Quality of Proposal, Item E-24] As noted above (Finding 16), RFP Section M.5.b. lists the subfactors to be used for evaluation of management proposals. Under the first subfactor, Contract Management, the solicitation advises offerors that they will be evaluated in this section on five of nine items listed above on the same page of the solicitation. One of the items included in this list of nine items, but not among the five to be used in this part of the evaluation, is: "clarity, sufficiency, completeness, and substantiation of the management proposal." Protest File, Exhibit 5, Appendix A at M-8. A review of the evaluation items corresponding to this portion of the subfactor on contract management clearly reveals that an item entitled "Quality of Proposal" was included to cover this sixth item dealing with the "clarity, sufficiency, etc." of an offeror's proposal. Id., Exhibit 130. 72. Two of the three MET evaluators assigned a score of to Sprint on the "quality of proposal" item. The third evaluator, however, scored Sprint at on this item, pointing out that "[t]here are some important topics which are addressed but not elaborated very well." Protest File, Exhibit 121. No express DI appears to have been generated on this item. Instead, the third evaluator's score sheet refers to a DI which was issued to Sprint regarding the solicitation requirement of delivery of ADTN2000 services 180 days after issuance of the initial delivery order. Id., Exhibit 122. This third evaluator found the response to that DI to be ambiguous and nothing more than another example of the poor quality of the proposal. He, therefore, did not make any upward revision of his original score of on the "quality of proposal" item. Id., Exhibit 121. 73. [Proposed Personnel/Previous Work, Item E-45] The solicitation also required offerors to include in their management proposal information regarding experience in similar areas of endeavor. Section L of the RFP provides: Experience. Offerors should submit individual project summaries of recent experience in performing work for industry and Government similar to that described in the RFP. Protest File, Exhibit 5, Appendix A at L-16. This section of the RFP then goes on to provide a detailed format to be followed in providing the information sought. Id. 74. The initial scoring of Sprint on the previous work item was rather . One evaluator gave a score of . Another listed a score of . The third scored Sprint at . The perceived problem common to all these scores was that Sprint had failed to include data elements called for in the specified format. Protest File, Exhibit 113. A DI was issued pointing out to Sprint that its proposal did not address all elements required for each of the cited experiences. The DI spoke to missing data, not conflicting data. It offered Sprint the opportunity to amend the proposal. Id., Exhibit 114. Sprint did so. As a result, two evaluators revised their original scores to . The third evaluator left her score of unchanged. She wrote: Response to DI presents doubts in tally of the percentage of time individuals were on projects. Offeror shows some individuals, e.g., using 153% of his time. How can one individual do that? Id., Exhibit 113. This evaluator has testified regarding other inconsistencies in time claimed by Sprint in its reply to the DI for certain individuals working on various Sprint projects. Transcript at 989-97. Notwithstanding these inconsistencies, however, there is no indication in the record that a follow-up DI was issued on this item or that Sprint was advised of any existing inconsistencies in the initial DI. 75. [Competing Projects, Item E-51] Another item on which offerors were scored in the evaluation of management proposals concerned other projects that are competing for essential personnel. The solicitation required offerors to include in their management proposals: [i]dentification of personnel considered essential to the work described herein and to be designated as key personnel. The resume's [sic] of key personnel and a description of how they will be employed in the performance of this contract, including the extent of their commitment to the effort necessary to satisfy the requirements of the specified tasks in accordance with the work schedule and staff assignments proposed by the offeror. Protest File, Exhibit 5, Appendix A at L-17. 76. Two of the MET evaluators assigned a score of to Sprint on the competing projects item. The third evaluator assigned a score of . Protest File, Exhibit 115. This gave rise to a DI which noted that Id., Exhibit 116. Sprint replied that Id. The MET evaluator who previously scored Sprint at on this item did not change that score after review of Sprint's reply to the DI. He noted: DI response indicates that they less than meet the requirement in that they propose to simply share some of the key people with other projects; score unchanged. Id., Exhibit 115. 77. [Resumes - Personnel Commitment, Item E-53] Based on the same contract provision supporting the evaluation item regarding competing projects is another item on which offerors were scored in the evaluation of management proposals. It concerns the degree of commitment of identified key personnel to the ADTN program. Two of the evaluators assigned a score of to Sprint on this item. The third did not raise his original score of on this item -- even after review of Sprint's reply to the DI generated for the competing projects item. He apparently remained convinced, as he was on the competing projects item, that the 80% commitment level did not fully meet the requirements of the RFP. Protest File, Exhibit 115. 78. [Project Manager, Item E-55] The solicitation directed offerors to identify a proposed project manager (PM): who should have successfully managed projects of this type, magnitude, and complexity within the past three years. A concise description of his experience, duties, and responsibilities should be included. Protest File, Exhibit 5, Appendix A at L-17. The solicitation did not require that the project manager have any minimum number of months managing telecommunications projects. In fact, the evaluators did not believe that the project manager had to have any minimum number of months managing telecommunications contracts. Transcript at 1016-17. As explained by the team leader for the MET, the appropriate minimum number of months which might be sufficient would vary, depending upon how directly related the experience was. Id. Thus, a proposed project manager with fewer months of very directly relevant telecommunications experience might be better than another with more months of less directly relevant experience. Id. at 1017. 79. Sprint proposed as its ADTN2000 project manager See Protest File, Exhibit 8, Appendix G at III-6.0-4, III-7.0-1 to 0- 4, III-Appendix C-5 to C-6; see also Transcript at 125. Transcript at 126. 80. The Government sent Sprint two DIs related to this proposed project manager. The narrative description of each DI is that of "proposal deficiency." The DIs each noted that Sprint had not provided sufficient information on and asked whether Sprint wished to amend its proposal to provide additional information. Protest File, Exhibit 161 at 65-67. Based on the reply received from Sprint, the MET assigned Sprint a raw average score of on this item. Id., Exhibit 130. 81. In scoring Sprint on the proposed project manager, all three evaluators gave Sprint a score of less than . One evaluator was of the opinion that Sprint had provided evidence of telecommunications experience relevant to managing a program such as ADTN but remained less than satisfied. The second was of the opinion that simply lacked telecommunications experience. The third concluded that Sprint was not entitled to a score of on this item because it had not provided information sufficient to demonstrate that the requirements for this assignment could be met. Protest File, Exhibit 155. 82. [Capacity of Workforce, Item E-57] The solicitation directs offerors to address the following requirement in their management proposals: Capacity. Data on the offeror's total capacity for work of the type covered by this RFP, including relevant sales or production figures for the past year. The offeror's existing commercial and government business commitments, and backlog of work which would utilize the same facilities and resources as proposed for use under this requirement should be presented with an explanation of how the work required will be accomplished in view of those other commitments. Protest File, Exhibit 5, Appendix A at L-16. The MET scored offerors on this requirement under item E-57, "Capacity of Workforce." 83. Two of the MET evaluators scored Sprint below on the capacity of workforce item. Protest File, Exhibit 156. This gave rise to a DI already mentioned with regard to the scoring of the competing projects item (E-51) above (see Finding 76). As noted, that DI pointed out that some key personnel proposed Sprint was asked to explain if these people Id., Exhibit 166. 84. Sprint's reply to the DI regarding capacity of the workforce lead to revised scores below from all three members of the MET. All three evaluators rated Sprint below points because the key personnel Protest File, Exhibit 156. GSI's Technical Proposal 85. In response to the requirement for multilink procedures (Finding 5), GSI's proposal stated with respect to the requirements in paragraph 3.4.2.4.b of the solicitation's functional specification: Protest File, Exhibit 6, Appendix B at I-B-1-14 to 1-15. 86. A considerable amount of time was spent during the hearing and subsequent briefing of this case on the issue of whether the See Protest File, Exhibit 159. Protester contends that in stating that This, according to protester, is consistent with a number of other provisions of its technical proposal such as paragraphs 1.B.2.d.1, 2.B.1.1, 2.B.1.3.a, 2.C.4.e, and 2.C.4.f. Sprint's Posthearing Brief at 15. 87. A member of GSI's technical team who actually wrote the initial draft[foot #] 5 of this section of GSI's technical proposal dealing with multilink procedures testified Transcript at 460. GSI's business development official responsible for the ADTN2000 procurement, when asked about this provision in GSI's proposal, testified that he did not Id. at 314-15. However, he also testified that, at the time GSI submitted its proposal, Id. at 316-17. Not until after this protest was filed did he discover, Id. at 317. 88. One member of the TET has testified that references in GSI's proposal to the provision of functions through the use of were "high level" statements devoid of "a lot of detail in terms of diagrams or anything." Transcript at 647. In the absence of more detailed information, statements such as these did not permit the evaluators to conclude with certainty what would be involved. Id. at 642-43, 646-47, 649-50. This evaluator appeared untroubled by the absence of such detail since, in his words, "We were looking for a level of commitment and we found it in that statement." Id. at 649. With regard to the GSI's specific reference to this evaluator testified that he was initially confused by the terms because he was looking specifically for what was being proposed. He was not prepared at that point, however, to assume GSI meant He states that it "would have been nice" to know GSI intended to use but that information was not contained in the statement. Id. at 655. This evaluator further explained that what the evaluators were looking for with this provision regarding people have the capability to offer the service and did what they propose sound reasonable." In his opinion it did. Id. at 656. When asked what level of detail he was looking for in this procurement, he stated: The reality is in this procurement, it's not a hardware procurement. The Government is simply purchasing services and one of the services is multilink. Did we ----------- FOOTNOTE BEGINS --------- [foot #] 5 This individual left the employ of GSI shortly before the initial proposal submission. Transcript at 456. ----------- FOOTNOTE ENDS ----------- really have to Not necessarily, as long as the service worked. Id. at 659. 89. The team leader for the TET, when asked at the hearing whether the evaluators understood GSI to state that the multilink function would be provided by testified that he did not recall. He explained that it was not an issue at the time. He further explained: At the point we saw what they said, they said they would support it, and it looked like a good solution to us. And so we didn't -- we didn't pursue it any further. Transcript at 814. Elsewhere he explained: What we were after was a commitment and we were buying a service and they had committed to provide it at contract award and it turns out they could do exactly that. Id. at 847. This same evaluator, however, also testified that he assumed the evaluators "probably" thought GSI intended Id. at 812. It was not until after this protest was filed that this evaluator learned that Id. at 848. 90. GSI's technical proposal was the first to be evaluated by the TET. Particular items were not reviewed again unless they were subject to a DI. Transcript at 646, 653. The record contains no indication that a DI was sent out regarding GSI's statement that it would provide multilink functions Protest File, Exhibit 9, Appendix H. GSI's Management Proposal 91. In the evaluation of GSI's management proposal, the MET assigned a score of of the twelve items for which Sprint was given a score of On the remaining items, GSI was given an average raw score of Protest File, Exhibit 424. The TET Report Regarding GSI's Proposal 92. The TET report regarding GSI's proposal concludes: Throughout the technical proposal evaluation process, GSI was consistently compliant with the RFP requirements and as a result, very few technical DIs were issued. The large number of strengths discussed above reflects a highly credible solution for implementing the ADTN2000. The fact that the Technical Evaluation Team felt no need to identify a single issue or other consideration to highlight in this report speaks for itself. Protest File, Exhibit 10, Part B at 43. 93. In its report regarding GSI's management proposal, the MET listed several strengths and noted the following weaknesses: Protest File, Exhibit 10, Part C at 23. Evaluation Scores and Prices 94. The ranking of all offerors on this procurement based on their combined weighted is as follows: Protest File, Exhibit 10, Part A, Attachment 2. 95. If the scores for all of Sprint's technical/management evaluation items in which Sprint was deemed to not fully meet requirements were raised to Sprint's total technical/management score would increase to or to within points of the GSI score. Protest File, Exhibit R-7 (Version 3). 96. The Government as follows: Protest File, Exhibit 14. Competitive Range Determination 97. The Source Selection Authority (SSA) for this procurement has testified that at an SEB meeting held prior to his being briefed by the SEB, there was a discussion of whether Sprint should be eliminated from the competitive range since it had been determined that Sprint did not meet certain mandatory requirements of the RFP. Transcript at 546-51. The SSA attended that meeting. He explained that a collective decision was arrived at to "give Sprint the benefit of the doubt" and not eliminate it. Id. at 549. At that meeting, the SEB directed the DECCO contracting officer to issue a call for best and final offers (BAFOs). Minutes of the SEB meeting reflect, however, that the BAFO submissions were to be for price only. Protest File, Exhibit 12 (Meeting of Oct. 19, 1993); Transcript at 550, 912. Source Selection 98. Following the submission of BAFOs, the SEB met to review the final offers and prepare a briefing for the SSA. Transcript at 914. The SEB developed a series of briefing charts, including a number of bar charts reflecting differences in the offerors' scores, and a "Significant Discriminators" slide for each factor-level bar chart. The "Significant Discriminators" slide was intended to describe some of the reasons for the score(s) of the one or two offerors whose point scores on a given factor were, in the opinion of the SEB, significantly different from those of the other offerors. See id. at 925. 99. On November 18, 1993, the SEB co-chairman, together with the SEB members, briefed the SSA. Transcript at 514-15. A set of summary briefing slides or charts was used for the briefing. Id. at 515; see Protest File, Exhibit 158 (the SSA's copy of the briefing charts). During the briefing, the SSA made various notations on his copy of the charts. Transcript at 516. 100. The SSA's briefing took approximately two hours. Transcript at 516-17. Other than the briefing slides, and a three-page decision memorandum which he signed on November 19, 1993, the SSA reviewed no other documents regarding the evaluations. Id. at 516. The SSA never read the solicitation for this procurement. Id. at 518-19. 101. One bar chart shown to the SSA displayed the weighted technical evaluation scores of all offerors. Protest File, Exhibit 158 at 11. The SSA has testified that for him what stood out in that chart was that "Sprint was the only one of seven offerors that failed to meet the mandatory requirement . . . ." Transcript at 526-28. In the same vein, the SSA testified that the critical difference, as seen on this same chart, between GSI and Sprint was that Sprint had failed to meet the mandatory requirements of the solicitation. Id. at 538. The SSA also admitted under examination that the spread of points between GSI and Sprint as shown on that same chart is close but nonetheless does represent a differentiation and a value to the Government. Id. at 539-40. 102. Another briefing slide contained a bar chart that showed the final combined weighted technical and management scores of all offerors. Protest File, Exhibit 158 at 14. When asked whether a difference of between GSI and Sprint, as shown on this bar chart, was "pretty close," the SSA answered "[p]erhaps, but there was clearly a difference." Transcript at 542. 103. As already noted (Finding 98), in addition to the technical, management, and combined-level bar charts, the briefing slides for the SSA included a series of bar charts corresponding to the individual technical and management factors, and a slide labelled "Significant Discriminators" for each. See Protest File, Exhibit 158 at 16-41. On the first such bar chart, for technical factor number one, the SSA noted on his copy of the slide that Sprint was "unable to provide [services without] significant develop[ment;] unacceptable because of [unintelligible]." Protest File, Exhibit 158 at 16; Transcript at 544. The SSA has testified that this note was in reference to Sprint and meant it was unacceptable because it could not meet the mandatory requirements of the solicitation since additional modification was required. Transcript at 544-45, 555. 104. With regard to Sprint's plans for development, a briefing slide identifying a "significant discriminator" for the comparison of the scores of offerors on technical factor one states: "Sprint unable to provide X.25 multilink procedures capability without development effort." Protest File, Exhibit 158 at 17. The SSA, however, was not briefed on the details of Sprint's proposal in this regard, or even whether or how the facilities to be developed by Sprint might be needed in contract performance. Transcript at 555-58. 105. Similarly, a briefing slide identifying a "significant discriminator" for the comparison of the scores of offerors on technical factor two states: "Sprint unable to implement X.25/X.32 packet mode terminal dial-up access without development effort." Protest File, Exhibit 158 at 18-19; transcript at 559- 60. 106. Another briefing slide identifying a "significant discriminator" for the comparison of the scores of offerors on technical factor four states: "Sprint deficient in ADTN2000 concept due to lack of capability to support dial-up X.25/X.32 service and other technical design deficiencies pending development efforts." Protest File, Exhibit 158 at 22-23; Transcript at 565. 107. On the first slide among those dealing with individual management factors, the SSA added the note that there were "[n]o mandatory requirements in management." Protest File, Exhibit 158 at 32. The SSA has testified that this was to reflect the fact that in the management area, the evaluation was not on a pass/fail basis. Transcript at 566-67. The only "significant discriminator" slide in the management area that contains any mention of Sprint is on a slide referring to a bar chart showing the scores of all offerors under factor number three of the management evaluation. The statement regarding Sprint reads: Protest File, Exhibit 158 at 37. The SSA does not recall any details regarding these comments. Transcript at 567-70. 108. The briefing slides shown to the SSA also included two pages that listed a "Summary of Concerns." Protest File, Exhibit 158 at 49-50. These included four "concerns" with respect to the Sprint proposal, namely: 8. Pricing note suggests Id. at 50. 109. The SSA states the he did not consider the last two of these comments in his deliberations, and that they were not factors in his decision. Transcript at 573- 75.[foot #] 6 The SSA understood the first two comments to reflect the same information that was conveyed on the technical factor slides regarding multilink and X.32. Id. at 572-73. 110. The briefing charts included a "Recommended Source Selection Methodology" which reads as follows: (1) Rank offers by combined technical/management scores. (2) Examine list and eliminate any offers where there is a clearly better alternative. (higher technical/management, lower cost, and no "concerns") (3) Compare highest technical/management offer vs. lowest cost offer -- eliminate the offer which is not the better value. (4a) Compare highest remaining technical/management offer vs. lowest remaining cost offer -- eliminate the offer which is not the better value. (4b-c)Continue step 4 until a "winner" emerges. (5) Double-check by comparing the "winner" vs. all offers. Protest File, Exhibit 158 at 52. 111. In deciding to award the contract to GSI, the SSA followed this methodology recommended by the SEB. Transcript at 577. Under the methodology, various combinations of proposals were ----------- FOOTNOTE BEGINS --------- [foot #] 6 Protester challenges the accuracy of these two statements. Sprint's Posthearing Brief at 66 n.39. Given the relief we are providing protester in this decision and the SSA's testimony that he did not rely on these statements in making his decision, we see no need to reach the issue of whether or not these two statements were, in fact, correct. ----------- FOOTNOTE ENDS ----------- compared, culminating in a head-to-head comparison of each of the other proposals against the GSI proposal. See Protest File, Exhibit 158 at 53-62. 112. In step five of the selection procedure, a comparison is made of the GSI and Sprint proposals. A briefing slide reflects the following comments: (see also "step 4(a)"). Protest File, Exhibit 158 at 61. 113. In testifying regarding the briefing slide containing comments comparing GSI and Sprint's proposal, the SSA stated that he understood GSI's proposal to be dramatically superior to Sprint's because Sprint had failed to meet the mandatory requirements and GSI had a higher score. Transcript at 578-79. The SSA testified that he was told that Sprint's technical proposal was bare bones, but he admits that he has no understanding of what that comment meant except that Sprint had not scored as highly as others. Transcript at 611-13. 114. The reference to step 4(a) in the final comment on comparison of the GSI and Sprint proposals does not involve a comparison of those two proposals but rather a comparison of the Sprint proposal to that of . This slide dealing with step 4(a) contains the following comments regarding Sprint's proposal: Protest File, Exhibit 158 at 56-57. 115. The SSA testified regarding the comments on Sprint's proposal which are contained in the slide dealing with step 4(a). He stated that he understood the concerns of the SEB to include Sprint's failure to meet the mandatory requirements of the RFP and its submission of what has been characterized as a "bare bones proposal." Transcript at 611-12. As to the comment regarding the need to continue the existing contract if Sprint failed to perform, the SSA recognized at the hearing that it would apply equally to any offeror.[foot #] 7 Id. at 612-14. 116. The SSA also testified that he was not told of the strengths found to exist in the Sprint proposal. As a matter of fact, he explained that he was not told about the strengths of any of the proposals other than the few significant discriminators. Transcript at 588-91. Similarly, the SSA testified that he was not advised of any of the weaknesses found in the GSI proposal. Id. at 591-94. 117. During his testimony, the SSA was asked to comment on what he considered to be nothing more than the hypothetical possibility that Sprint's proposed development efforts were, in fact, minor modifications allowed by the solicitation. His response was: Well, obviously that would be significant and then it would depend upon the relative scores and the competition. Transcript at 586-87. The Protest 118. This protest was initially filed on December 3, 1993. On December 16, 1993, Sprint filed a first amended complaint. This amendment did not contain any new counts. 119. On January 11, 1994, Sprint filed a second amended protest complaint. With this amendment, Sprint added a seventh count alleging that GSI failed to meet mandatory requirements of the RFP, and an eighth count alleging that the Government and GSI had engaged in post-BAFO discussions. Second Amended Complaint at 21-22. 120. In a third amended complaint, filed on January 14, 1994, Sprint further specified the allegation that GSI's proposal was noncompliant by alleging that mandatory requirements. Third Amended Complaint at 18-19. ----------- FOOTNOTE BEGINS --------- [foot #] 7 As to the comment on the slide regarding possible pricing irregularities, we do not discuss it here since the SSA made clear in his testimony that he disregarded the supposed "concerns" with Sprint's price proposal. See Finding ___ 109. ----------- FOOTNOTE ENDS ----------- 121. In its initial discovery requests to GSI, Sprint served several interrogatories which concerned GSI's compliance with the requirements of the RFP. At the same time, Sprint requested GSI to identify and produce all documents that relate to how commerciality requirements of the ADTN2000 solicitation. Protest File, Exhibit 133. Answers were provided. Id. 122. Sprint received the GSI proposal for this procurement as part of the Rule 4 file on December 17, 1993. Sprint's Opposition to Joint Motion to Dismiss Counts VII and VIII at 5. The proposal contained explanations of how GSI intended to meet the solicitation's requirements. Protest File, Exhibit 6, Appendices B, C. 123. On December 20, 1993, GSI provided Sprint with answers to Sprint's interrogatories regarding GSI's compliance with the requirements of the RFP. Protest File, Exhibit 133; Transcript at 368-69. 124. Sprint readily admits that in response to its request to GSI for production of documents, it received on December 20 a large box of documents several of which[foot #] 8 showed that certain requirements of the RFP. Sprint's Opposition to Joint Motion to Dismiss Counts VII and VIII at 6-7, 13. 125. As a result of the Sprint protest and discovery requests received from Sprint regarding the ability of representatives of GSI Transcript at 327-28. 126. At this meeting of the GSI and representatives on or around December 16, Transcript at 332, 341-42. 127. Following the mid-December meeting and GSI representatives, GSI contacted the FAA project officer for this procurement to schedule a technical meeting with government representatives. Transcript at 875, 894. 128. The meeting of GSI representatives and government personnel took place on December 28, 1993. At that meeting, the GSI official responsible for the company's proposal briefed the ----------- FOOTNOTE BEGINS --------- [foot #] 8 E.g., Protest File, Exhibits 134-38. ___ ----------- FOOTNOTE ENDS ----------- Government representatives. The meeting took place at the law offices of counsel for GSI. It was attended by a representative as well. Also in attendance were attorneys representing GSI, the Government, During the meeting, it was explained how GSI intended to It was explained that Transcript at 332-38, 426-29. 129. Counsel for Sprint contends that after responding to written discovery requests, "GSI admitted in the depositions that its solution is noncompliant" -- "to the extent that GSI's proposal was to meet the Sprint's Opposition to Joint Motion to Dismiss Counts VII and VIII at 8. Similarly, Sprint contends: [D]uring the depositions, Sprint learned that the GSI proposal to the Government does not provide support for Id. at 13-14. Discussion We turn now to a discussion of the individual counts in this protest. Count I - Improper Technical Evaluation In this count of its protest, Sprint alleges the SEB was required to evaluate Sprint's proposal solely on the basis of the criteria set forth in the solicitation. Protester contends that this was not done and that its proposal was accorded disparate treatment during the technical evaluation. We agree. We find the evaluation scores of on the eight technical evaluation items relating to multilink and X.32 dial-up requirements are in fundamental conflict with the solicitation's commerciality provisions. Since the evaluation of these portions of Sprint's proposal has not been carried out in accordance with the terms of the solicitation, it must be rejected as contrary to statute and applicable regulation. See 10 U.S.C. 2305 (1988); 41 U.S.C. 253b (1988); FAR 15.608(a). Pursuant to the commerciality provisions in clauses H.25 and C.3.1.2, vendors are permitted to offer commercial equipment subject to minor modification. Finding 9. "Minor modification" is defined as a "modification that does not alter the commercial product's/item's function or essential physical characteristics." Findings 6, 9. During negotiations, Sprint made known its intent to rely on the "minor modification provision" in the RFP. On receipt of Amendment 11 and a second round of DIs, Sprint became seriously concerned regarding the continued availability of this provision. This prompted an extensive exchange between Sprint and the contracting officer on the subject. Findings 32-36. The final outcome of their discussions was an acknowledgement on the part of the contracting officer that she was in agreement with Sprint's position regarding commerciality. Finding 36. Having reached what it considered to be an understanding with the contracting officer on this matter, Sprint recertified its compliance with the RFP's commerciality requirement. Finding 38. As a result of this protest, it has come to light that during this procurement, the FAA felt extremely uncomfortable with the "minor modification loophole" and attempted to close it by proposing DIs and a letter to Sprint advising that Sprint's plans for extensive development were unacceptable. Finding 39. The contracting officer rightly advised that without a change in the language of the solicitation, this could not be done. Findings 40-42. The contracting officer's position appears to have carried the day -- at least initially. Her notes confirm that the SEB agreed with her interpretation and contract modification 12 was issued to clarify matters. Finding 43. The score sheets of the technical evaluators, however, tell a different story. The evaluators seized on the concept of minor modifications becoming major modifications when taken "in the aggregate." Findings 52-55. This notion of major modification "in the aggregate" is not found in the commerciality provision. Indeed, we do not consider that it is even implicit in that provision. The wording of the clause is clear. The factor distinguishing major from minor modification is said to rest in the degree to which the modification changes the function or essential physical characteristics of the product or item in question. See Finding 9. It is this and not the aggregate complexity of the proposed alterations which distinguishes major from minor modification as that term is used in the commerciality provisions of the RFP. During the hearing in this case, one of the members of the TET testified that the enhancements of multilink or X.25/X.32 dial-up services do not change the functionality of Sprint's switches. We agree. We likewise agree with this evaluator's candid admission that, in determining whether Sprint's proposed developments constituted "minor modifications," he used the term "minor modification" in a sense different from that given to it in the clause itself. See Finding 54. Indeed, we find that this error was made by all three technical evaluators. Compare Finding 9 with Findings 53-55. We find the scores given Sprint by the evaluators on the eight technical evaluation items in question to be clearly unjustified since they obviously conflict with language in the solicitation's commerciality provisions. We agree with protester that the risk recognized by the evaluators as inherent in Sprint's development plans was within the RFP's specified acceptable level of risk. Consequently, any conclusion that Sprint failed to meet the mandatory requirements supporting these eight items is unjustified and should be set aside. We, therefore, grant Count I of the protest. Count II - Misapplication of Award Criteria Count IV - Failure to Award To Most Advantageous Offeror Because Count II and Count IV both relate to the selection process, we treat them together. Under applicable regulation, following the evaluation of BAFOs, the SSA is to select the source whose offer is most advantageous to the Government, considering only price and other factors included in the solicitation. See FAR 15.611(d). In this case, the solicitation provided that the technical factors were significantly more important than management, and that technical/management combined were more important than cost. The RFP also noted, however, that cost may become more important as the difference between combined technical/management scores decrease. Finding 10. We have examined in detail the facts relating to the source selection made in this case. It is our conclusion that the selection is substantially defective, based as it is on extremely limited information most of which is fundamentally flawed. Given these facts, we cannot conclude that the selection made by the SSA was the most advantageous to the Government or that it was made in accordance with the RFP's criteria for award. The SSA's decision making process was a simple one. Following the methodology recommended by the SEB, he looked for a low priced proposal with a relatively high technical/management score. He eventually chose GSI. Then, in accordance with step five of the selection methodology, he made one last head-to-head comparison of GSI's proposal to all other proposals. Findings 110-11. GSI's price was higher than that of Sprint which was the of all offerors. Finding 96. As to the difference between the combined, weighted, technical/management evaluation scores of GSI and Sprint, the difference was less than out of a total of over 270 points. Findings 94, 102. Given the prices and combined scores of Sprint and GSI, the fact that Sprint was presumably still in the competitive range (Finding 97), and the solicitation's provision regarding the greater importance of technical/management factors over cost, with the prospect of cost becoming more important in the event of close technical/management scores, one would expect the SSA to have engaged in some form of cost/technical tradeoff analysis to support the selection of the higher priced GSI proposal. CRC Systems, Inc., GSBCA 9720-P, 89-1 BCA 21,411, 1988 BPD 302. The closest we come to such an analysis is the head-to-head comparison of GSI's proposal to that of Sprint. This analysis, however, is entirely unsatisfactory given the findings made and the conclusions reached elsewhere in this decision regarding the defective technical and management scores for Sprint and the inadequacy of discussions. The SEB briefing slide outlining the head-to-head comparison of the GSI and Sprint proposals notes that GSI's technical management proposal is dramatically superior to Sprint's. Finding 112. The SSA has explained that he understood this superiority to be based on the fact that Sprint failed to meet mandatories and had a lower technical/management score. Finding 113. As we have noted, however, the conclusion regarding Sprint's failure to meet mandatory requirements is incorrect. As to the difference in scores, it is difficult to understand how one could consider this significant in and of itself in view of the total number of points involved. However, this ceases even to be an issue after Sprint's technical score and management scores are corrected (Counts I and VI), thus reducing the point spread to Finding 95. In comparing the GSI and Sprint proposals, the SEB briefing slide also points out to the SSA that the discounted life cycle cost of the GSI proposal is higher than Sprint's. Finding 112. Under the solicitation provision regarding the role of cost in the evaluation process, this price differential argues against the selection of GSI when one takes into consideration the diminished differential in point scores resulting from corrections made herein to the original technical and management scores. The briefing slide comparing the GSI and Sprint proposals also notes that the SEB had serious concerns about Sprint's proposal but none about GSI's. Id. Such an observation conveyed nothing to the SSA beyond that which he had already been told regarding the alleged failure of Sprint to meet mandatory requirements because of the need for software development. The SSA has testified that he was told nothing of the strengths of Sprint's proposal or of the weaknesses found in GSI's. Finding 116. He admits to having been told that Sprint's proposal was "bare bones" but does not claim to understand what that entailed beyond Sprint's score being lower than that of other offerors. Finding 113. Particularly telling is the testimony of the SSA when asked to comment on a hypothetical possibility that Sprint's planned development efforts were in accordance with the solicitation requirements. Under such a scenario, the assumption would be that Sprint met the mandatory requirements. The SSA replied that this would be "significant" and that the selection would then depend upon the relative scores and competition. Finding 117. Based on the evidence reviewed in our discussion above regarding Count I, we conclude that the hypothetical situation offered to the SSA for comment is that which actually existed. We agree -- this is "significant." Clearly with the correction of Sprint's combined, weighted technical/management score and the consequent reduction in the difference between the scores of GSI and Sprint, the earlier selection must be revisited. Counts II and IV of the protest are, therefore, granted. Count V - Lack of Meaningful Discussions The FAR provides that in conducting discussions, the contracting officer shall: Advise the offeror of deficiencies in its proposal so that the offeror is given an opportunity to satisfy the Government's requirements; Attempt to resolve any uncertainties concerning the technical proposal and other terms and conditions of the proposal . . . . FAR 15.610(c)(2), (3). Unfortunately this has not occurred in the discussions between Sprint and the Government in this procurement. It is clear from the scores given by the TET members to Sprint on the items relating to multilink and X.32 dial-up access that the members considered Sprint's proposal deficient with regard to these mandatory requirements. It is equally clear, however, that the Government failed to inform Sprint of this fact. During negotiations, Sprint readily revealed its intention to meet the requirements for multilink, closed user group, X-32 dial-up access and inactivity timers through development. Findings 28-30, 37. The issue of how this would square with the solicitation provisions regarding commerciality was expressly discussed. Findings 31-38. The record does show that during the exchanges between the contracting officer and Sprint on these matters, she shared with Sprint officials her concerns regarding the risk attendant to Sprint's development plans. Findings 34, 36. The record likewise confirms that Sprint, in responding to various DIs, attempted to address this concern. Finding 37. At no time, however, was Sprint informed that its approach was unacceptable. On the contrary, receipt of a follow-up DI regarding the use of modems for the X.32 dial-up feature was reasonably interpreted by Sprint as an indication that the approach it planned to use was satisfactory. Findings 44-45. Similarly, the issuance of Amendment 12 was reasonably interpreted by Sprint as confirmation of the understanding that the development planned for Sprint's equipment was allowable under the solicitation's commerciality provision. See Finding 45. During the course of this protest, respondent and intervenor have dwelt on the fact that internal documentation produced by Sprint in response to discovery indicates that if award had been made to Sprint, See Findings 46- 47. Respondent and intervenor apparently believe that the information reflected in this internal documentation should have been furnished by Sprint during negotiations. We do not fault Sprint for not doing so. The risk which Sprint was apparently willing to take was within the level of risk permitted by the solicitation. The Government, for all practical purposes, confirmed this fact for Sprint by agreeing with Sprint's interpretation of the RFP's commerciality provisions while knowing the degree of development Sprint planned to take on. Under circumstances such as these, where presumably both parties were aware of the risks involved, it would have made poor business sense for Sprint to share spontaneously its concerns with the Government. Had the Government chosen to press discussions further, some disclosure might have been required. This, however, did not occur. In short, if the Government believed Sprint's proposal to be deficient with regard to the multilink and the X.32 dial-up access requirements, it was required to advise Sprint of this fact. It failed to do so. Indeed, it led Sprint to conclude reasonably that its proposal was compliant with regard to these requirements. For this reason we grant Count V. In reviewing the facts relating to the negotiations with Sprint on the multilink and X.32 dial-up access requirement, we find an additional basis for granting Count V. We have no problem with the DIs issued by DECCO on these requirements. See Findings 28-29. The solicitation clearly calls for a considerable degree of technical detail. Finding 4. Our objection is to what we consider disparate treatment of GSI and Sprint regarding that portion of their proposals relating to the multilink procedures. See Finding 85. The technical evaluators who have testified on this issue have attempted to explain their failure to issue a DI on this portion of GSI's proposal in various ways. We are told that the statement in the proposal was not intended to serve as a technical explanation. Rather it is a "high level statement" indicating a "commitment" to provide the service. This, we are told, is sufficient since the Government here is purchasing services and not equipment. Findings 88-89. The solicitation states: The technical proposal must set forth enough of the technical approach, including technical methods and engineering approaches to sufficiently show how it is proposed to meet RFP requirements. A full explanation of techniques and procedures proposed to be followed shall be included. Block diagrams, drawings or charts should be included, as appropriate, to present a full, complete and clear explanation . . . . Finding 4. In view of this and related provisions, we are unpersuaded by the justifications offered by the evaluators for not scrutinizing GSI's proposal more closely and issuing a DI on the multilink procedures. The TET report points to the of technical DIs issued for GSI's proposal as a sign of its superior quality. Finding 92. We have no intention of passing judgment on the validity of this observation with regard to the whole of GSI's technical proposal. Nevertheless, with regard to GSI's multilink statement, we are more inclined to conclude that the TET's failure to issue a DI on this item was attributable rather to the fact that GSI was the first proposal reviewed and the TET had not yet warmed to its task. See Finding 90. The TET's failure to issue a DI to GSI on multilink contrasts strikingly with the more aggressive tack taken with Sprint on the same issue. We find it reflects a disparate treatment of offerors during discussions which ultimately led to distorted technical evaluation scores for GSI and Sprint. It is a fundamental principle of competitive negotiation that offerors must be treated equally by a procuring activity. CPT Corp., GSBCA 8134-P-R, 86-1 BCA 18,727 at 94,240, 1986 BPD 12, at 68 (citing Computek Inc., 54 Comp. Gen. 1080 (1975), 75-1 CPD 384); accord, Wisconsin Physicians Service Insurance Corp., GSBCA 9674-P, 89-1 BCA 21,366, 1988 BPD 268. We conclude that the disparate treatment found to exist in this case regarding the proposal of multilink function by GSI and Sprint constitutes additional support for our conclusion that discussions with Sprint were inadequate and that Count V should, therefore, be granted. Finally, we find that discussions with Sprint regarding its management proposal were likewise marred by a failure to advise the offeror of perceived deficiencies in its proposal. For reasons discussed in detail under Count VI, we have found a discrete category of management evaluation scores which must be rejected because of inadequate discussions. These defective scores likewise support our conclusion that Count V of the protest should be granted. Count VI - Improper Management Evaluation Protester has challenged all twelve management evaluation scores which were given to Sprint and which were under points. The judgments of evaluators will normally not be disturbed unless it can be demonstrated that the conclusions reached were unreasonable ones. See Computer Sciences Corp., GSBCA 11497-P, 92-1 BCA 24,703, at 123,297, 1992 BPD 6, at 32 (and cases cited therein). In the instant case, after review of the applicable solicitation provisions, the score sheets of the evaluators and applicable provisions of the technical/management evaluation plan, we hold that the conclusions reached were unreasonable. Protester's principal contention in challenging the technical evaluation scores is that its proposal deserved at least a score of on each of the twelve items because the particular requirement in question was, in fact, met. We agree that in nearly all cases the requirement in question was met. As for the limited number of scores where this may be in doubt, we conclude that the lower score is unjustified and should be raised at least to in view of the Government's failure to alert Sprint through discussions that its management proposal was deemed deficient regarding these few items. The major difficulty with the MET scores in controversy stems from the scoring scheme used for the management evaluations. In describing the scoring, the RFP employs a perplexing distinction between a score which indicates that a specific requirement is adequately but not fully met (70 to 99) and a score which indicates that the requirement is fully met (100 to 130). See Finding 17. This gives rise to the question of whether an "adequate" score signifies that the requirement has or has not been met. Our conclusion is that a score of adequate indicates that the proposal is at least in some respect noncompliant with regard to the item on which it is being evaluated. We say this first of all because of the wording of the RFP itself. Given the plain meaning of the terms "fully meets requirement," we consider it reasonable to conclude that if a proposal falls short of that level of scoring, it obviously has failed to meet that requirement. Our interpretation is supported by language in the Technical/Management Evaluation Plan as well. The plan contains a definition of "deficiency" which is said to be based on the definition of that term found in the FAR. The definition in the plan specifically explains that the term "satisfy" as used in the FAR phrase of "fails to satisfy the Government's requirement" should be understood as equivalent to "fully meet." Finding 22. The plan also explains that a proposal is "deficient" if it "less than fully meets the stated sub-factor requirement." Id. In a similar vein, the narrative description found in all MET score sheets describes the adequate score between 70-99 as: "Less Than Meets Requirements." See Finding 18. Finally, there is the language written in some technical evaluation score sheets by the evaluators themselves. On occasion, they have assigned a score between 70 and 99 specifically because they found that the offeror did not meet the requirement in question. Findings 65, 76, 81. Despite the actual language of the solicitation, the guidance contained in the evaluation plan, the description of the "adequate" score on the evaluation score sheets and the entries made by some evaluators on their scoring sheets, the MET team leader has testified that the evaluators did not consider an adequate score to be deficient. Finding 23. Nevertheless, DIs were sent out when an offeror was rated by an evaluator at less than 100 points but no subsequent DIs were sent. The reason for not sending additional DIs appears to have been a concern that this might lead to some form of leveling. Finding 57. On balance, we find the scoring of Sprint's management proposal to be both inconsistent and unreasonable. If Sprint actually met a particular management requirement, then it was entitled to a score of 100. If Sprint did not meet the requirement, then, in accordance with instructions set out in the TMEP, it was entitled to a deficiency DI. See Finding 22. Our analysis of the facts regarding the scoring of the twelve contested items convinces us that in each case, the evaluation score given to Sprint lacks a reasonable basis. In addressing the individual scoring of the twelve contested items in this discussion, we need not discuss them in the same detail found in our findings of fact. Several of the items fall into general categories which can be dealt with in a summary fashion. One large group of contested evaluation items is that in which final adequate scores were given when, in reality, Sprint was entitled to a score of at least 100 points because it clearly met the requirement in question. The objections registered by the evaluators on these items may have justified a score no higher than 100 points but they did not justify a score below that level. We include in this category the following items: E-02, Organization: The contract requirement was for an organization plan showing structure and flows of authority. See Findings 58-59. E-30, Automated Project Scheduling: The contract requirement was for an automated scheduling system that can be accessed by FAA (read only) to determine the status of service orders. See Findings 66-67. E-51, Competing Projects; E-53, Resumes - Personnel Commitment; E-57, Capacity of Workforce: Sprint received a final evaluation score of We find nothing in the contract requirements supporting these three items which precludes this. See Findings 75-77, 82-84. E-55, Project Manager: All three evaluators assigned a score of less See Findings 78-81. The second category of contested management evaluation scores involves those items on which Sprint received a score of less than 100 but was not advised of deficiencies on these items or, where initial DIs were issued, of continued deficiencies. We recognize that respondent now argues that deficiency notices were not required for items scored between 70 and 99 points. Nevertheless, since we find that position in conflict with the language of the RFP, the evaluators' score sheets, and the guidance in the evaluation plan, we reject it and conclude that additional discussion was required on the following items. E-04, Project Tracking: E-05, Sprint Coordination: A DI was issued on this item but noted an ambiguity not a deficiency. Finding 69. The inadequacy of the reply was never communicated to Sprint. E-22, Risk Management: The MET recognized the need to issue a DI on this item but failed to send it. Finding 70. E-45, Proposed Personnel/Previous Work: The DI issued for this item spoke to missing data, not conflicting data. No follow-up DI was provided on this deficiency. Findings 73-74. Having examined the individual items relating to this second category of scores, we can find no basis to support respondent's reluctance to pursue discussion of deficiencies out of fear of leveling. For the instances outlined above, any additional DI or, in the case of Risk Management, a make-up DI should have served rather to remedy defects associated with the original DIs. The eleventh contested evaluation score concerns item E-03, Key Personnel. A DI issued to Sprint regarding this item produced a reply which prompted two of the three evaluators to revise their scores on this item to The third evaluator declined to revise her score to that level. Finding 61. We have examined the contract requirement which is the subject of this evaluation item and are convinced that Sprint's proposal, as clarified by the response to the DI, did meet it. See Findings 60-61. The last contested evaluation score concerns item E-24, Quality of Proposal. We find that scoring on this item was in conflict with the language of the solicitation. The Quality of Proposal item relates to an area expressly excluded under the terms of the RFP from the factor and subfactor to which it is said to relate. Findings 71-72. For this reason the item itself should not have been included among the management evaluation scoring items and any score based on it should be ignored.[foot #] 9 In sum, we find the management evaluation substantively defective in all twelve areas claimed by protester. For the reasons set out above, we grant Count VI of the protest. Count VII - GSI Fails To Meet Mandatory Requirements Of The Solicitation This count was added to Sprint's protest by its second amended complaint, filed on January 11. Respondent and GSI both argue that it is untimely and urge us to dismiss it. ----------- FOOTNOTE BEGINS --------- [foot #] 9 In calculating a corrected score for Sprint and comparing it to that of GSI, we have left this item in our calculations and simply raised it to the minimum score of 100. See Finding 95. ___ ----------- FOOTNOTE ENDS ----------- Our Rules of Procedure provide that protests: shall be filed no more than 10 working days after the basis for the ground of protest is known or should have been known, whichever is earlier. 58 Fed. Reg. 69,246, 69,254 (1993) (to be codified at 48 CFR 6101.5(b)(3)(ii)). In its initial discovery requests served on GSI, Sprint made several inquiries regarding GSI's compliance with requirements of the RFP. Sprint also asked GSI to identify and produce all documents relating Finding 121. This information was provided on December 20. Findings 121, 123-24. On December 17, Sprint received a copy of GSI's proposal. The proposal contained explanations of how GSI intended to meet the Finding 122. Sprint readily admits that the documentation provided by GSI in response to Sprint's request for production of documentation Finding 124. The question of GSI's compliance with RFP requirements has, therefore, been an obvious matter of interest to Sprint from the start of this protest. Sprint's original discovery requests confirm this. Indeed, Sprint's initial probing Finding 125. When GSI failed to produce We can find no justification for Sprint's waiting until well past the tenth working day from receipt of GSI's document production to amend its protest. Subsequent discovery may have clarified issues for Sprint and flushed out the closely related issue of GSI's decision to rely on Nevertheless, this does not excuse Sprint from putting the basic question of GSI's compliance in issue upon receipt of GSI's reply to its earlier discovery request. We, therefore, agree with the contention of respondent and GSI and dismiss Count VII as untimely filed. Count VIII - Post-BAFO Discussions This count is based in part on GSI's noncompliance, which protester has attempted to allege in Count VII. In this count, Sprint contends that although GSI's proposal failed to meet mandatory requirements of the solicitation at the time of award, the Government and GSI have engaged in post-BAFO discussions in an effort to cure the alleged deficiencies in GSI's proposal. GSI and respondent have filed a joint motion to dismiss this count for failure to state a valid basis for protest. They point out that FAR 15.611(c), on which Sprint relies, is applicable to preaward discussions and not to events occurring after contract award. Sprint in reply cites decisions of this Board as well as the General Accounting Office where protests alleging actual changes in proposals subsequent to award have been sustained. Berkshire Computer Products v. Department of Commerce, GSBCA 12172-P, 93-2 BCA 25,724, 1993 BPD 36; Federal Data Corp., 69 Comp. Gen. 196 (1990), 90-1 CPD 104; International Transcription Services, Inc., B-240488, 90-2 CPD 437 (Nov. 28, 1990). Technically, Sprint is correct. Nevertheless, we still dismiss this count for failure to state a valid basis for protest. Our reason for doing so, however, differs from that offered by respondent and GSI in their joint motion. Sprint alleges that GSI's proposal was modified after award. We are not at all certain that this did occur. There is ample evidence of confusion -- even on the part of GSI's own personnel -- regarding the original intent of GSI's proposal. However, the thrust of GSI's defense on this issue appears to be that no actual change or substitution was required, See Findings 87, 89, 125-28. For purposes of this count, we find no merit in pursuing the issue of whether there was or was not an actual modification of GSI's proposal after award. We have already dismissed as untimely filed Sprint's Count VII which alleges that GSI's proposal is noncompliant. For purposes of Count VIII, therefore, we assume that GSI's proposal is compliant. If the parties have, in fact, agreed to modify GSI's proposal after award, this is certainly not prohibited and, therefore, does not constitute a valid basis for protest. For this reason, the Board, sua sponte, dismisses Count VIII. Decision Counts I, II, IV, V, and VI of this protest are GRANTED. Count VII is DISMISSED as untimely filed and Count VIII is DISMISSED as failing to state a valid basis for protest. Given the facts in this protest and the conclusions we have reached regarding protester's allegations, it is clear that the Government is ambivalent regarding the allowable degree of development for modification of the equipment to be used in performing the ADTN2000 contract. This has led to inconclusive discussions, incorrect technical evaluations, and a defective selection decision. Problems encountered in the management evaluation scheme and the Government's interpretation and application of it have likewise contributed to the defective selection. Finally, during the course of this protest, issues have arisen regarding GSI's proposal which also indicate the need for additional discussions. We, therefore, direct respondent to terminate its award to GSI and reopen discussions with offerors. Before doing so, respondent should amend the solicitation to eliminate those ambiguities which have given rise to this protest. The amendment may also address any refinement or change respondent believes to be appropriate with regard to its actual requirements. We expressly amend the applicable delegation of procurement authority to permit this. Our oral order of December 7 (confirmed in writing on December 8) suspending that delegation of procurement authority expires in accordance with its terms. ___________________________ EDWIN B. NEILL Board Judge We concur: _____________________________ ___________________________ ROBERT W. PARKER CATHERINE B. HYATT Board Judge Board Judge