THIS OPINION WAS INITIALLY ISSUED UNDER PROTECTIVE ORDER AND IS BEING RELEASED TO THE PUBLIC IN ITS ENTIRETY ON FEBRUARY 17, 1994 September 25, 1991 Protests of : International Business : GSBCA No. 11359-P Machines Corporation : 800 North Frederick Avenue : Gaithersburg, MD 20879 : : Lockheed Missiles & Space : GSBCA No. 11362-P Co., Inc. : 1111 Lockheed Way : Sunnyvale, CA 94089 : : Solicitation No. IRS-88-079 : Granted Contract No. TIR-91-0072 : : Appearances for Protester/ : William A. Bradford, Esq. Intervenor, : Robert J. Kenney, Jr., Esq. International Business : Wendy T. Kirby, Esq. Machines Corporation : David W. Burgett, Esq. : Janet P. Holt, Esq. : L. Anthony Sutin, Esq. : Cindy K. Domer, Esq. : Lisa R. Bonanno, Esq. : Kevin P. Gallagher, Esq. : Thomas L. McGovern, III, Esq. : George W. Miller, Esq. : Hogan & Hartson : Columbia Square : 555 13th Street, N.W. : Washington, DC 20004 : Appearances for Protester/ : Stuart B. Nibley, Esq. Intervenor, : Laura K. Kennedy, Esq. Lockheed Missiles & Space : Timothy F. Haley, Esq. Co., Inc. : Michael B. Hubbard, Esq. : Glenn S. Nozick, Esq. : Daniel Marino, Esq. : Marc Gold, Esq. : Daniel McNulty, Esq. : Seyfarth, Shaw, Fairweather : & Geraldson : 815 Connecticut Avenue, N.W. : Suite 500 : Washington, DC 20006-4004 : William M. Weisberg, Esq. : Barton, Mountain & Tolle : 1320 Old Chain Bridge Road : Suite 440 : McLean, VA 22101 : Appearances for Respondent, : Francis C. Inserra, Esq. Department of the Treasury : Harold G. Mosher, Esq. : Corlyss Drinkard, Esq. : Terrence P. Dermott, Esq. : Donald M. Suica, Esq. : Mark A. Allen, Esq. : Internal Revenue Service : Office of Chief Counsel : General Legal Services : Aerospace Center : Box 69, Room 208 : 901 D Street S.W. : Washington, DC 20024 : Appearances for Intervenor, : Joel R. Feidelman, Esq. AT&T Federal Systems : James J. McCullough, Esq. : Daniel I. Gordon, Esq. : Deneen J. Melander, Esq. : William A. Davis, Esq. : Fried, Frank, Harris, Shriver : & Jacobson : Suite 800 : 1001 Pennsylvania Avenue, N.W. : Washington, DC 20004 Order by Administrative Judge LaBella Pursuant to Rule 18, a hearing examiner was appointed to conduct proceedings in these protests and prepare a recommended decision for submission to the panel. The panel adopts the opinion recommended by the hearing examiner and grants the protests. Respondent's delegation of procurement authority is amended to require respondent to proceed in accordance with the Board's opinion. ______________________________ VINCENT A. LaBELLA Administrative Judge We concur: ______________________________ ______________________________ CATHERINE B. HYATT EDWIN B. NEILL Administrative Judge Administrative Judge Opinion Recommended by Hearing Examiner Miller Protesters in these consolidated protests present the question, How much is too much? The procurement at issue, the Treasury Multi-User Acquisition Contract, commonly known as TMAC, was conducted by the Department of the Treasury, Internal Revenue Service (IRS) in order to obtain office automation systems and software, as well as maintenance and support services. IRS awarded the contract to intervenor AT&T Federal Systems (AT&T) at an evaluated cost of $1.4 billion. Protesters International Business Machines Corporation (IBM) and Lockheed Missiles and Space Company (Lockheed) allege, in part, that this amount is too much. Both protesters assert that AT&T's technical proposal does not justify the non-selection of their proposals, with evaluated costs of $700 million (IBM) and $900 million (Lockheed). Although stated somewhat differently, both protesters make essentially the same three allegations: (1) the IRS conducted an improper price evaluation or made an irrational price/technical tradeoff; (2) the IRS did not follow the evaluation scheme stated in the request for proposals (RFP), and (3) the IRS misevaluated protesters' proposals.[foot #] 1 We conclude that, in the context of this RFP, a price difference of $500-700 million was indeed too much. The IRS' evaluation of the offers shows that price was a minimal factor in the agency's award decision. The RFP, however, did not permit the agency to discount the significance of cost as a factor to the degree that the IRS actually did in making its award decision. As a result, the degree of competition was lessened because, in essence, protesters came to play rugby and AT&T came to play chess. A competition conducted at such cross-purposes is not likely to produce the best possible value for the Government. We therefore grant the protests. ----------- FOOTNOTE BEGINS --------- [foot #] 1 At the hearing, IBM dropped its allegation that AT&T's proposal failed to comply with mandatory requirements of the RFP. ----------- FOOTNOTE ENDS ----------- Findings of Fact 1. The TMAC RFP contemplated the procurement of "products rang[ing] from a family of multiuser computer systems and networked workstations to a complete line of expansion components and peripherals as well as a full line of compatible and portable software products and capabilities." It also called for various support services. The resulting contract was to have a one-year duration, with four, one-year options for products and six for support services. The contract was to be a mandatory, requirements-type contract for the IRS and optional for all other bureaus within the Department of the Treasury. Protest File, Vol. 25, at 9555-56.2 2. For TMAC, the Department of the Treasury received from the Administrator of the General Services Administration a delegation of procurement authority (DPA) for $1.8 billion. Protest File, Vol. 1, Exhibit 3. Minimum Mandatory Requirements 3. With respect to hardware, the RFP stated as follows: The primary requirement is to provide a flexibility in system configuration, topology, and communications transmission media that can meet the diverse applications for which the equipment shall be used and with which the equipment shall be interconnected. The multiuser systems will expand in both workload and processing power to support an ever changing user workload. . . . The Offeror shall understand that the multiuser computer systems shall be installed in a variety of office environments. Protest File, Vol. 25, at 9561. 4. The RFP called for a variety of hardware items, including various storage devices, terminals and workstations, monitors, local area network (LAN), and various printers. It also called for three models of multiuser computer systems (MUSs), as follows: ____________________ 2The original protest file in these protests comprised some twenty-four volumes, to which the parties have made numerous additions. The IRS eventually provided, as volume 25, an integrated RFP, i.e., a version of the RFP incorporating the numerous amendments made during the course of the procurement. In most cases, protest file exhibits will be identified by volume and exhibit number. The integrated RFP, however, does not have an exhibit number. a. Model 1 -- A minimum of 8 ports, which are physical connections, and support a maximum of 32 devices through physical or logical connections. b. Model 2 -- A minimum of 16 ports, which are physical connections, and support a minimum of 32 to a maximum of 128 devices through physical or logical connections. c. Model 3 -- A minimum of 24 ports, which are physical connections, and support a minimum of 128 to a maximum of 256 devices through physical or logical connections. Id. at 9562. The IRS estimated that Treasury would purchase 2,175 model 1 systems, 695 model 2 and 350 model 3. Id. at 9516. 5. The RFP required a variety of "off-the-shelf" software packages and stated, "The degree of integration of software provided in response to this proposal shall be evaluated." Id. at 9594. 6. The RFP stated: Current Production. All equipment, software, and related peripherals must be state-of-the-art technology, in current production, and commercially available for the date set for the beginning of the initial Live Test Demonstration (LTD). Id. at 9558. The RFP also included definitions of these terms. Id. at 9558-60. 7. The software requirements included one for security: At a minimum, [the security] policy shall include all the controlled access protection requirements applicable to Class "C2" level systems plus additional requirements stated in this specification. The policy shall include a discretionary policy for protecting the information being processed based on the authorizations of individuals, users, or groups of users. . . . . The enforcement mechanism . . . shall allow users to specify and control sharing of those objects [e.g., files and programs] by named individuals or defined groups of individuals or both, and shall provide controls to limit propagation of access rights. The discretionary access control (DAC) mechanism shall, either by explicit user action or by default, provide that objects are protected from unauthorized access. These access controls shall be capable of including or excluding access to the granularity of a single user. Id. at 9601-02. 8. Each offeror was to propose three database management systems (DBMSs). The RFP required each to propose systems known as Informix and Oracle; the third DBMS was to be of the offeror's choosing. Id. at 9609. 9. The RFP contained a requirement for office automation software. It stated generally as follows: The Offeror shall propose a utility that integrates the word processing, calendar, card file/directory, spreadsheet and electronic mail into an office automation software package. The "look and feel" integration of this software shall be the same across all the utilities within the Office Automation software. The degree of integration of these utilities shall be evaluated. . . . The degree of ease of use by novice users shall be evaluated. Id. at 9628. Offerors were required to provide word processing, electronic mail, spreadsheet, calendar/scheduler, and card file/directory software. Id. at 9629-41. 10. The RFP required offerors to provide support, which it divided into the following categories: program management support, configuration management, installation/site planning support, warranty support, training, maintenance, hardware/software support, personnel support/integration services, and manuals and publications. Id. at 9662. 11. The training requirement covered the following areas: 1. Hardware courses to include courses on both the workstations and multiuser systems. 2. Software courses to include operating system and application software. 3. New system orientation. 4. Systems administration courses to include security administration. Id. at 9690. 12. The hardware/software support requirement included hot- line support, as follows: The Contractor shall provide a technical support staff to man a hot-line service for the contract's Treasury customers. . . . The hot-line technical support shall include assistance with hardware, communication, and software problems and their resolution. Id. at 9714. 13. Each offeror had to perform a live test demonstration (LTD), which consisted of five components: an operating system demonstration, the Neal Nelson Business Benchmark (NNBB), the Neal Nelson UNIX LAN Demonstration, a functional workload demonstration (FWD), and other functional demonstrations (OFDs). Id. at 9735-36. Evaluation Criteria 14. The "Evaluation Criteria/Method of Award," contained in Section M of the RFP, were stated as follows: Offerors whose proposal(s) are submitted in accordance with the instructions given in Section L and which meet the mandatory requirements in Section C shall be evaluated in accordance with the evaluation criteria presented below. Offerors whose proposals are not submitted in accordance with the instructions given in this RFP or which do not meet the mandatory requirements shall not be considered for award. Award will be made to that offeror whose proposal, containing the combination of technical, management and support, and cost features, offers the best overall value to the Government. The Government is more concerned with obtaining superior technical (technical and management and support) features than making award at the lowest price. Award will be determined by comparing differences in the value of technical features with difference in overall cost to the Government (overall cost to the Government includes the offeror's prices for equipment, software, maintenance, support and any other solicited items[)]. However, the Government will not make an award at a significantly higher overall cost to the Government to achieve slightly superior technical features. Evaluation Factors. The evaluation factors to be utilized in the evaluation process are listed below in their descending order of importance: Technical Factor 200 points The subfactors for the technical evaluation which will be evaluated are shown as follows: Technical Features (100 points) Management and Support Features (100 points) Cost Factor (No points) Technical will be a scored factor with all subfactors weighted equally within their respective feature group. Cost will not be a scored factor, but will be evaluated as described in Section M.4.3. Id. at 10141-42. The precise point breakdown--i.e., the number of points available for each of the individual items that the IRS planned to evaluate in the technical proposals--was not made available to the offerors. 15. The evaluation scheme broke the technical subfactor down into five major components: operating system, system architecture, software functionality and user interface, system integrity and security, and the LTD. Id. at 10143-50. 16. With respect to software functionality and user interface, the evaluation scheme stated as follows: The Government will evaluate: A. The third Database Management System (DBMS) for ease of use, on how easy it is to learn, to update, and to troubleshoot; and the extent of integration with a spreadsheet provided under the contract; B. System design features which reduce requirements for user training; C. Commercial software packages with regard to ease of use, capability, and function; and, D. System noise, system heat, footprint, weight, power consumption and cabling as they affect system use in all applicable environments, including the ergonomics of an office environment. Id. at 10145-46 . 17. With respect to the LTD, the evaluation scheme provided that the results of the performance test on each of the three (3) multiuser systems during the Functional Workload Demonstration (FWD) and the Neal Nelson Local Area Network (LAN) Benchmark will be the elements scored. They will be equally weighted. Id. at 10147. 18. The evaluation scheme stated that the purpose of the LTD was to A. Validate that required functions, features, capabilities, and capacities are operational and available at the time of the LTD with regard to: . . . . B. Confirm that the elements are as described in the Offeror's technical proposal with regard to: 1. The results of the Neal Nelson Business Benchmark. 2. Acceptable performance in additional areas via Other Functional Demonstrations. C. Observe the following: 1. The performance received during the Functional Workload Demonstrations (FWDs). 2. The performance received during the Neal Nelson LAN Demonstration. 3. The results of the CDN Conformance Test. Id. 19. The NNBB required each offeror's MUSs and workstations to conduct certain tests within specified time frames. Id. at 9740-45. It was pass/fail and thus was not scored. Id. at 10148-49. 20. In the FWDs, each offeror was required to test each of its three MUSs at approximately seventy percent of the system's maximum user level with varying workloads. Transcript at 816. The evaluation scheme informed offerors that the IRS would give the fastest offeror for each test a value of one, the slowest a value of zero, and any others a fractional value for that test. It further stated: The Government will total the results for each Offeror for each configuration. These scores will then be pro- rated against the allowable value to determine the final score for each multiuser system. Protest File, Vol. 25, at 10149. 21. With respect to the OFDs, the evaluation scheme provided: Other functional demonstrations shall be performed to complete the verification of the Offeror's compliance with the mandatory requirements and shall be used to subjectively evaluate such factors as: A. The ease of use of the system interface between the hardware and software; B. The availability of help menus at the system or application levels; C. The degree of conformance with interface standards; D. The portability of application software from the multiuser systems to the intelligent workstations; E. The ease with which workstations can use the electronic mail facilities. Id. at 10150. Competitive Strategies 22. Both protesters concluded from the RFP that price would be critical in determining which vendor would prevail. They based this conclusion on their view that, because the RFP called for off-the-shelf technology, there could be little room for difference among the technical proposals. Transcript at 64-66, 543-52, 630-33. 23. AT&T also believed that price would likely be critical in determining the awardee. Protest File Supplement (IBM), Exhibit 1, at 509. AT&T's internal competitive analysis predicted that IBM's total cost would be nearly $1.5 billion. Id. at 652. 24. All three offerors expected to receive some points in the technical scoring for complying with the mandatory requirements. Transcript at 135, 582; Deposition of C.T. Nixon at 184. The Evaluation 25. Ultimately, the three companies that are parties here made it through the entire process to the award decision. See Protest File, Exhibit 648.3 26. A subjective evaluation of the three technical proposals was conducted by the Technical Evaluation Panel (TEP). The TEP used a rating scale of 0-5 at the criteria level. A score of 0 meant the proposal met the mandatory requirements and provided no strengths related to value or quality for that criterion. To receive a score of 5 the proposed implementation of the mandatory requirements must provide a superior solution with strengths related to significant value or quality. For each subfactor, the offeror's score was derived by summing the ratings (as percentages of the total possible scores) for the subfactor's evaluation criteria. Id. at Appendix A-2. 27. The TEP scored the technical proposals of the three offerors as follows: Maximum Score AT&T IBM Lockheed TECHNICAL Operating System 20 8 8 6.67 POSIX Conformance at 6.66 1.33 1.33 1.33 proposal POSIX Conformance at LTD 6.66 4 4 2.66 Conformance with J.12 6.66 2.66 2.66 2.66 ____________________ 3Exhibit 648 is the Source Evaluation Board's report. It was distributed to the parties and the Board under separate cover prior to distribution of the protest file, and so was not included in any of the protest file volumes. System Architecture 20 6.4 10.4 6.4 Modularity, expandability, 4 3.2 3.2 2.4 ease of use MUS & WKS Modularity & reliability 4 0 1.6 1.6 LAN MUS administration 4 0.8 1.6 0.8 Local & remote LAN admin. 4 1.6 2.4 0 Interprocess commun. 4 0.8 1.6 1.6 implementation, quality & integration Software Funct. & Interf. 20 14 5 8 3d DBMS ease, learn, 5 3 1 3 troubleshooting Sys. design reduce train'g 5 4 1 1 Comm. s/w ease, capability 5 4 1 3 & function. System noise, etc. 5 3 2 1 System Integrity & Security 20 10 10 8 System Security 10 6 6 4 System Integrity 10 4 4 4 TMAC LTD 20 13.89 11.17 7.57 Model One 5 2.28 1.94 3.51 Model Two 5 4.93 4.04 0.36 Model Three 5 4.83 2.2 1.64 LAN 5 1.85 3 2.06 _____ _____ _____ _____ TOTAL TECHNICAL 100 52.29 44.57 36.64 MANAGEMENT & SUPPORT Project Management 50 37.5 22.5 32.5 K mgmt. plan & corporate 12.5 10 5 7.5 capabilities K admin. & proj. mgmt. 12.5 10 5 7.5 Past performance 12.5 10 5 10 Prop. subcontracting 12.5 7.5 7.5 7.5 Support 50 27.5 20 17.5 Integration & engineering 12.5 0 2.5 0 support Tech. supp. for TMAC cust. 12.5 10 5 7.5 Maintenance supp. 12.5 7.5 5 10 Training 12.5 10 7.5 0 ______ _____ _____ _____ TOTAL MGMT. & SUPPORT 100 65 42.5 50 TOTAL SCORE 200 117.29 87.07 86.64 Protest File, Vol. 15, Exhibit 584. 28. As the above scoring shows, the principal areas accounting for AT&T's advantage over IBM were software functionality, project management, and technical support. AT&T also had an edge with its model 3 MUS. AT&T's primary advantages over Lockheed were in the performance of the models 2 and 3 MUSs and in training, and to a lesser degree in software functionality. 29. The offerors' business proposals were evaluated by the Business Management Evaluation Panel (BMEP) to determine responsibility and to determine price reasonableness. Areas evaluated include corporate management (subcontracting plans), financial data (price proposals, Dun & Bradstreet reports, and Defense Contract Administration Management Area Operations (DCMAO) reports) and references. Protest File, Exhibit 648, at Appendix A-2. 30. The IRS evaluated the total, out-of-pocket costs of each offer as follows: AT&T $1,431,366,424 IBM 708,480,438 Lockheed 914,802,546 Id. at A-4. AT&T's cost fell below the Government's estimated contract price. Id. at 3. 31. Of AT&T's evaluated cost, approximately $137,000,000 was attributable to anticipated purchases of the model 2 MUS and $117,000,000 to those of the model 3. Id., Appendix at J-5 to J-6. Approximately $300,000,000 was attributable to anticipated software purchases.4 Protest File, Vol. 21, Exhibit 639, at 8680. Only a fraction of this total would have been attributable to office automation software. See Protest File, Vol. 25, at 9521-27. AT&T's evaluated software cost was roughly triple that of IBM and double that of Lockheed. Protest File, Vol. 21, Exhibit 639, at 8680. 32. IBM's prices for MUS models 2 and 3, as well as main- tenance, were as high as or higher than those of AT&T in the first two years of the contract, then decreased much faster than those of AT&T over the remaining years. Protest File, Vol. 25, at 79. ____________________ 4AT&T's software prices were higher at least in part because the producers of the software charged more for versions to be used with more powerful hardware. Protest File, Exhibit 648, at 3. 33. The Source Evaluation Board (SEB) was tasked with making an award recommendation based on the RFP's evaluation criteria. In doing so, the SEB examined the reports of the TEP and the BMEP, and drafted its own report discussing at length the relative merits of the three offers. The SEB concluded that AT&T's offer presented the best value to the Government based on its superior technical provisions and recommended award to AT&T. Protest File, Exhibit 648. 34. The SEB reached the following conclusions, among others, with respect to the relative merits of the technical proposals: a. Software functionality and user interface. The SEB found AT&T's solution for office automation to be considerably superior to that of IBM. For the three basic elements--word processing, spreadsheet, and office functions such as calendar and e-mail--AT&T and Lockheed each proposed an integrated package, i.e., a set of programs from one company meant to be used together. IBM, on the other hand, proposed software from three different companies, with another program needed to integrate them. The SEB, and the TEP as well, concluded that this solution would require increased training and be more difficult to use. Id. at 27- 29, 31-32. The SEB also found Lockheed's office automation solution to be weaker than AT&T's because of two problems. The first was that a document printed during the OFD came out scrambled. The second was that a portion of Lockheed's proposal relied on operating system commands, which essentially means commands that must be done from memory rather than selected from a menu. Id. at 29. b. TMAC LTDs. The SEB noted that, during the FWDs, the timings of AT&T's MUS models 2 and 3 were superior to those of IBM and dramatically superior to those of Lockheed. The timings are as follows, expressed in minutes and seconds: Test # 1 2 3 4 5 6 7 8 9 Model 2 Lockheed 6.35 8.00 3.58 6.06 8.54 2.07 1.33 7.49 0.21 AT&T 0.40 1.51 0.16 0.20 1.53 0.11 0.13 1.24 0.09 IBM 1.33 1.37 0.34 1.24 1.30 0.28 4.08 1.12 0.11 Model 3 Lockheed 4.24 7.25 2.07 0.10 5.44 0.07 AT&T 0.11 0.46 0.10 0.07 1.47 0.08 IBM 2.40 2.40 1.01 1.36 1.56 0.12 These tests represented three (model 2) or two (model 3) tests run on each of the three DBMSs proposed by each offeror. Id. at 41, 44 (best time for each test in bold). c. Maintenance and technical support. The SEB criti- cized IBM's support scheme in comparison to the other two on two basic grounds. One was that IBM provided an unclear escalation path, i.e., an unclear plan for determining where a problem would go within the corporate structure if it could not be resolved. The other was that IBM placed some of the burden on Government users of system administrators in determining the nature of a problem. See Findings 51, 52 infra. The SEB concluded that this would add to the Government's staffing expenses. Protest File, Exhibit 648, at 66-67. d. Training. The SEB criticized Lockheed's training solution heavily. It expressed the belief that the courses were not designed well because they lacked adequate hands-on exercises and face-to-face contact. Lockheed's training solution relied often on teletraining, a technique with which the SEB stated Treasury had little experience. The SEB also noted that Lockheed offered few vendor training sites. Id. at 70-72. 35. The SEB concluded that IBM's offer presented price risks such that IBM's price advantage might be at least partially illusory. Specifically, the SEB found that IBM's MUS models 2 and 3 employed four-year old technology with respect to their microchips. It noted that industry literature indicated that the microchip would soon be upgraded and that IBM's proposal in fact stated that IBM planned to upgrade its processors soon after award to employ the newer technology. In light of the fact that IBM's prices declined rapidly in the later years of the contract, the SEB concluded that the IRS would not likely receive the advantage of those lower prices because it would have to negotiate a contract modification to include the newer hardware. Id. at 79-80. 36. In this regard, the RFP contained standard contract language allowing value engineering changes that could be utilized to upgrade technology. Such a change could be made only with the IRS's agreement. Protest File, Vol. 25, at 9794-95. 37. The SEB engaged in only one attempt to quantify the value of AT&T's technical advantages so as to determine whether those advantages justified paying the price premium that AT&T's offer represented. This consisted of two charts or spreadsheets, denoted Method A and Method B, which proved to be the centerpiece of the hearing in these protests. The SEB did not consider these spreadsheets to be the primary justification for its recommenda- tion. Rather, it used them simply as one factor in its analysis. Transcript at 199, 209-10, 902-05. a. In Method A, "the system that provide[d] the greatest expanded level of performance, i.e., the one with the greatest system power, [was] treated as a base." The SEB then added to the evaluated costs of the other two offerors the cost of purchasing the number of systems needed to raise their performance level to that of the most powerful system. For instance, because the SEB concluded that AT&T's model 2 MUS was six times as powerful as Lockheed's, it added the cost of buying six times as many model 2 systems to Lockheed's overall cost. Method A resulted in adjusted evaluated costs as follows: AT&T $1,549,103,900 IBM 1,147,583,162 Lockheed 1,906,903,577 Protest File, Exhibit 648, at 84-85. b. In Method B, "the system closest to the Govern- ment's required basic level of performance, i.e., the one with the least system power, is treated as a base." The SEB then discounted the costs of the other offerors to, essen- tially, lower their performance to that of the least powerful solution. For instance, to use the same example as above, AT&T received a discount of five-sixths the cost of its model 2 MUSs to account for the fact that that model was six times more powerful than Lockheed's. Method B resulted in adjusted evaluated costs as follows: AT&T $1,337,441,800 IBM 820,336,471 Lockheed 1,116,496,451 Id. at 84, 92. 38. The SEB emphasized the power, integration, and user- friendliness of AT&T's solution in recommending award to AT&T. Id., Executive Summary. Outside of Methods A and B, the SEB report contains no meaningful discussion of the price differences between the three offers. In fact, the three evaluated prices do not even appear in the report itself; to find them one has to look in the appendix. Protest File, Exhibit 648, Appendix at A- 4. The SEB discusses the technical advantages of AT&T's offer at length and concludes that it offers the best value to the Government, but at no point, other than Methods A and B, does the SEB address the size of the price differential, thus leaving the impression that the SEB would have recommended award to AT&T at any price differential. See generally id. 39. The source selection official concurred in the SEB's recommendation and selected AT&T for award. Protest File, Exhibit 648, first page. Disputes Over IRS Evaluation Methods A and B 40. Protesters maintain that Methods A and B contained a number of flaws undermining their validity. Some of the criticisms which we credit are as follows: a. Reliance on Methods A and B as a measure of benefits to be received is illogical and obviously misplaced. For instance, starting from the proposition that an AT&T model 2 MUS is six times more powerful than a Lockheed model 2, it does not necessarily follow that the former is worth six times as much as the latter to the Government. Nothing in the record establishes that the extra speed will translate into sixfold productivity once IRS employees sit down in front of a workstation. Methods A and B do nothing more than state the relative technical merits of the systems in different terms. They do not measure the benefits the Government will receive from the different offers and, thus, do not measure whether any offer is worth its cost in relation to the others. b. The method of adding whole systems5 distorts reality. To increase the speed of the Lockheed model 2 sixfold, the IRS would not have to buy six entire MUSs.6 It would be able to decrease the response time by isolating the bottleneck amongst the system's components and purchasing only the necessary component(s).7 Transcript at ____________________ 5Or subtracting them: giving AT&T, for example, a five- sixths discount on model 2s in Method B was the equivalent of stating that the Government would need to buy only one-sixth as many. 6There is also no indication in the record that the IRS would always need the extra processing power. As stated above, the tests were run in a manner representing seventy percent of each MUSs maximum number of users. A given ordering office might only have half as many users as the maximum. 7The IRS' response to this argument--that it would have to purchase whole systems because that is what the offerors proposed--is unfair. The offerors could hardly be expected to anticipate the methodology employed in Methods A and B, and therefore could not anticipate the need to make proposals that would allow the IRS to increase the speed in the manner suggested 409, 412-13, 698-99, 1308. The IRS could also simply purchase a model 3. c. The dollar totals added and subtracted in Methods A and B distorted the evaluation criteria. For instance, under Method A, the SEB added nearly $800,000,000 to Lock- heed's evaluated price--roughly 85% of Lockheed's total price and over half of AT&T's--to account for the performance of models 2 and 3, yet such performance accounted for only ten of the two hundred available points in the technical scoring. Protest File, Exhibit 648, at 85. 41. We find that, as tools in aiding to determine which offer represented the best value to the Government, Methods A and B had no value.8 The FWDs 42. Both protesters assert that the FWDs did not produce sufficient data to support the IRS' conclusion regarding the relative power of the various MUS models 2 and 3. Their contentions center around the fact that the FWDs employed a constant number of users. In other words, each workload that was tested was tested with the same number of users. Protest File, Exhibit 648, at 39. 43. Testimony at the hearing established that computers tend to behave in an essentially similar manner as the number of users is increased with a constant workload.9 The response time will increase only marginally until a certain point is reached, after which the response time will deteriorate exponentially as the number of users continues to increase. This results in a curve generally (and roughly) resembling the following: ____________________ in those methods. 8Along with everything else, we cannot help but question the value of these spreadsheets given that, under both, IBM came out as representing the best value to the Government by considerable margins, yet the SEB recommended award to AT&T. 9By workload we mean a given task that the system was required to perform. | * | | | | * Response | | Time | * | Longer | * | | * | * | * | * * * * ----------- FOOTNOTE BEGINS --------- # of Users More Transcript at 686-87; IBM Exhibit 88. The curve may differ significantly from one system to another. Transcript at 1354. 44. Protesters' contention is fairly straightforward. According to protesters, because the FWD tested each workload with only one number of users, there is only one data point to place on the graph. Thus, it is impossible to plot the curve. Protesters assert that the IRS cannot tell how the MUS would perform with a different number of users, and furthermore cannot tell how two different MUSs would compare with different numbers of users because, theoretically speaking, it is possible that the curves may cross. For example, one MUS might perform better with fewer users, but might begin to deteriorate in performance sooner. Transcript at 417-18, 1289, 1357-65. Thus, protesters assert, the FWDs provided inadequate data to support the conclusion that one MUS is more powerful than another. 45. We find that the IRS reasonably concluded that AT&T's MUS models 2 and 3 offered more power than IBM's and far more than Lockheed's. We base this conclusion primarily on the following factors: a. While it is possible that Lockheed's and IBM's systems may perform better than AT&T's with fewer users, the difference would be minimal, given that on most tests AT&T's times were twenty seconds or less. We are convinced, however, that the IRS could reasonably conclude that the "single data point," to use protesters' phrase, was suffi- cient to support the conclusion that AT&T's performance with larger numbers of users would be substantially superior. We draw support from the fact that most of IBM's times were in the one- to two-minute range and Lockheed's in the two- to eight-minute range. In addition, there was testimony, which we credit, that the high response times indicated that IBM's ----------- FOOTNOTE ENDS ----------- and Lockheed's systems had passed the "knee" of the curve,10 and that there is little possibility of the curves crossing once one of them has passed that point. Transcript at 984-85. b. There was also testimony, which we credit, that each offeror used similar architecture in all of its MUSs, and that the reactions to stress of the different models of each offeror should be similar to one another. In addition, each model tended to react similarly as the workload changed. In other words, there was only a little variance in the test results of any of the models.11 Protest File, Exhibit 648, at 40-41, 44. Thus, it was reasonable to conclude generally that, as the size of the machines and the user levels grew, the IBM architecture would react negatively, and the Lockheed architecture would react very negatively, considerably sooner than the AT&T architecture would. Transcript at 975-85. c. It might well have been impractical for the IRS to conduct tests that would satisfy protesters' demands. The LTDs as a whole took nearly two weeks for each offeror, id. at 816-17, and even then the IRS received complaints about the burdens they placed upon the companies. Id. at 815. To "plot the curve" for each MUS would require running several times the number of FWDs that were actually run, at the least, in order to vary the number of users. Even this would only have given the IRS data with respect to one workload. Software Functionality and User Interface (Of Mice and Men) 46. IBM disputes the IRS' conclusion that its office automation software was not user friendly. Its two primary contentions are as follows: a. The process of transferring documents between software packages--e.g., from the spreadsheet to the word processor--does not require operating system commands. ____________________ 10This conclusion is buttressed by the fact that the model 1 MUSs of all three offerors had very short response times, indicating that all were to the left of the "knee." Protest File, Exhibit 648, at 40. 11What variance there was consistently occurred on tests involving one of the three DBMSs, thus indicating that the results may have been attributable more to the software than the MUS. Transcript at 972-73. b. The IRS drew unjustified conclusions with respect to the impact of IBM's use of software packages from three different companies. 47. With respect to the first of IBM's arguments, IBM's expert witness testified that documents can be transferred without operating system commands. The procedure he outlined, however, required the use of a mouse. Transcript at 447-49. Given that the IRS is essentially a number-crunching agency, it is unlikely that its employees will make significant use of mice (mouses?). This is evidenced by the fact that the estimated quantity for mouses (?) in the RFP is 22,958, Protest File, Vol. 25, at 15, while the number of potential users of TMAC within the IRS is well over 100,000. Transcript at 891. 48. We also are convinced that the TEP reached reasonable conclusions with respect to the impact of IBM's office automation solution on the IRS' users. The use of three different software packages would result in a user having to employ three different templates, and confront programs with different appearances and sets of common commands. Id. at 1153-58; Protest File, Exhibit 648, at 28-29. In addition, the IRS anticipated that most of the TMAC users would be novices or, at best, moderately experienced in computer use. This, plus the agency's high turnover rate, Transcript at 891-92, makes it clear that IBM's solution would entail significantly increased training needs and lower productivity.12 Id. at 1160.13 Price Risks 49. IBM also challenges the IRS' conclusion that IBM's use of relatively old technology rendered its evaluated price ____________________ 12IBM also contends that its choice of WordPerfect as a word processing package presented advantages to the IRS that the latter failed to take adequately into account. The record contains evidence that WordPerfect is the most common word processing program within the IRS. Transcript at 132. Several considerations, however, dilute this advantage. First, many IRS employees use word processing programs other than WordPerfect. Id. at 890. Second, there is no way to predict what word ___ processing experience, if any, new IRS employees will have. Third, IBM offered an outdated version of WordPerfect. Protest File, Exhibit 648, at 32. 13IBM contends at length that the dollar values assigned to lost productivity due to its software solution in Methods A and B are irrational. Given our conclusion that reliance on Methods A and B was illogical and misplaced, we see no need to address this point. It is enough for our purposes that the conclusions reflected in the TEP's scoring are reasonable. advantage potentially illusory. IBM makes the following assertions: a. AT&T also intended to substitute newer technology at some point. Deposition of C.T. Nixon at 82-83. b. There is no proof that AT&T's technology was more recent. C. Because no substitution can be made without the IRS' agreement, the agency will not have to accept substantially higher prices, but instead can insist on performance at IBM's bid prices. 50. All three of the above contentions are somewhat bound together. In fact, there is evidence that AT&T's technology was state-of-the-art. Transcript at 1218-19. In addition, the most reliable testimony at the hearing indicated that IBM's technology was in fact six years old, id. at 1028, rather than four, as the SEB believed. Thus, IBM's equipment would become outdated much sooner than AT&T's, leading to a much greater motivation on the agency's part to agree to a substitution. This would put the agency in a weaker bargaining position, so that it might be forced to agree to higher prices to get the improved performance that advances in technology might offer. The SEB's conclusions in this regard were therefore reasonable. Help! 51. IBM disputes the IRS' criticism of its help solution. While all three offerors proposed a hot-line, IBM alone also provided a help desk dedicated exclusively to TMAC. The latter appears to have been intended for use in more critical situa- tions.14 The user would not contact either directly. Instead, the user would contact a system administrator, who would be an on-site, Government employee. The latter would then have to determine whether to contact the help desk or the hot-line. Transcript at 304-08. 52. The IRS reasonably concluded that AT&T's help solution was superior. The use of a system administrator would entail increased staffing and training burdens for the Government.15 In addition, the IBM solution might well prove significantly more ____________________ 14IBM's expert gave the example of smoke coming out of the computer; the help desk presumably would dispatch a technician promptly. Transcript at 306. 15IBM again disputes the dollar figures attributable to these increased burdens as reflected in Methods A and B. We again decline to get drawn into that quagmire. cumbersome in practice due to days when the system administrator was sick or on vacation, problems in coordinating the system administrator's regular workload with TMAC functions,16 problems in finding a capable employee to act as system administrator, and the necessity of having to get help from two people rather than one to address a problem. The Bottom Line 53. We have considered all of the challenges that protesters have raised to the IRS' technical evaluation.17 If perfection was possible, minor changes to the scores might be necessary. We find, however, that the scores at which the TEP arrived and the analysis contained in the SEB's report accurately reflect the relative technical merits of the three offers. Discussion Standing AT&T contends that both protesters lack standing. With respect to IBM, it asserts that IBM's offered workstation was not in current production as required by the RFP. With respect to Lockheed, AT&T argues that Lockheed's proposal failed to comply with the RFP's requirement that a user be able to grant or deny access to a file to any other users or groups of users. AT&T also in its reply brief18 adds the argument that Lockheed lacks standing because it is not next in line for award. Because the IRS did not reject either protester's proposal as non-compliant, dismissal at this stage is inappropriate. Each offer is capable of being made acceptable. We faced a similar situation in Sysorex Information Systems, Inc., GSBCA No. 10642-P, 90-3 BCA 23,036, 1990 BPD 153. In that protest we ____________________ 16The system administrator would often be a secretary or other clerical worker who would be trained for that function. Aside from serving as a liaison between the system users and IBM, the system administrator also would be trained to deal with minor problems such as a workstation not being plugged in. Transcript at 377. 17We also note that protesters have not challenged some of the TEP's conclusions. For instance, IBM has not challenged the criticisms of its project administration capability. 18We hardly favor the practice of adding arguments, juris- dictional or not, in a reply brief. There also appears to be no reason for AT&T not to have raised this one earlier. Because we conclude that Lockheed has standing, however, we need not address the appropriateness of this tactic. pointed out that, in a negotiated procurement such as this one, an agency may amend the solicitation even after receipt of best and final offers. Therefore, where the agency has not insisted on compliance with a mandatory requirement, the possibility remains that the agency could remove the requirement.19 In other words, "the procurement remains a living thing." Id. at 115,664, 1990 BPD 153, at 7-8. In situations such as the present one, it is preferable--assuming the protest is granted-- to allow the agency to decide what to do about the alleged non- compliance. With respect to Lockheed's ranking vis-a-vis IBM, AT&T is factually incorrect. The IRS never determined that Lockheed was third in line and nothing in the RFP requires it to do so, particularly given the agency's conclusion about the price risks inherent in IBM's proposal. We would be usurping the agency's function to determine that it necessarily preferred IBM to Lockheed.20 In any event, given our disposition of these protests as well as our reasoning above, the IRS' evaluation factors remain subject to change, making it inappropriate to eliminate Lockheed at this stage. Lockheed, as well as IBM, has standing. The Merits The legal principles germane to this case are considerably more straightforward than their application. The Federal Acquisition Regulation (FAR), 48 CFR 15.605(e) (1990), provides: The solicitation shall clearly state the evaluation factors, including price or cost and any significant subfactors, that will be considered in making the ____________________ 19For this reason, AT&T also is misguided in urging us to eliminate protesters ourselves because we now have a complete record on the issue of standing, something AT&T asserts the IRS did not have. By insisting on compliance before the agency has made a determination that it wishes to do so, we would effectively be determining the agency's needs, a role that we leave to the IRS. See Sysorex, 90-3 BCA at 115,665, 1990 BPD ___ _______ 153, at 7. 20For this reason, AT&T's reliance upon Unit Data Service __________________ Corp., GSBCA No. 10775-P, 91-1 BCA 23,501, 1990 BPD 374, is _____ misplaced. In that case, although the procurement was negotiated, the agency had scored the technical and cost ___ proposals, and ranked protester last of four offerors. We rejected on the merits protester's attempts to move up in the rankings, therefore conclusively establishing that it could not win the award. In this case, by contrast, we are in no position to determine the relative ranking of IBM and Lockheed. source selection and their relative importance. . . . Numerical weights, which may be employed in the evaluation of proposals, need not be disclosed in solicitations. The solicitation shall inform offerors of minimum requirements that apply to particular evaluation factors and significant subfactors. See 41 U.S.C. 253a(b)(1). The selection of evaluation factors and their relative importance lies "within the broad discretion of agency acquisition officials." 48 CFR 15.605(b). In evaluating proposals, the Government must adhere to the stated evaluation factors: Proposal evaluation is an assessment of both the proposal and the offeror's ability (as conveyed by the proposal) to successfully accomplish the prospective contract. An agency shall evaluate competitive proposals solely on the factors specified in the solicitation. 48 CFR 15.608(a). The FAR reemphasizes this requirement in 48 CFR 15.612(d), stating, "The source selection authority shall use the factors established in the solicitation . . . to make the source selection decision." If the proposal that the agency considers most advantageous involves a departure from the stated criteria, the vendors must be allowed to submit new best and final offers based on the revised requirements. 48 CFR 15.606. The goal of these principles is, in part, to "[m]aximize competition." 48 CFR 15.603(a). In keeping with the above standards, we have held that an agency must adhere to the evaluation factors in making its award decision. Hughes Advanced Systems Co., GSBCA No. 9601-P, 89-1 BCA 21,276, at 107,323, 1988 BPD at 253, at 49. For instance, in Contel Federal Systems, Inc., GSBCA No. 9743-P, 89-1 BCA 21,458, 1988 BPD 311, we held that it was an abuse of discretion for the source selection official to reject an otherwise-superior offer because of its word processing solution where word processing was only a minor item in the RFP. See Government Technology Services, Inc., GSBCA No. 10389-P (Jan. 31, 1990), 1990 BPD 37, at 15 (violation for agency effectively to redistribute weightings of factors in making award decision).21 ____________________ 21We are puzzled by the IRS' response to protesters' argument that the SEB distorted the selection criteria by placing too much emphasis on the FWDs. The IRS asserts that the SEB was not required to place the same emphasis on the FWDs as was the TEP. It points to language in Oakcreek Funding Corp., GSBCA No. ______________________ 11244-P (July 15, 1991), 1991 BPD 156, where we stated that the contracting officer, in making a price/technical tradeoff, was The substantive issues in these protests--other than the contention that the IRS misevaluated protesters' technical proposals, which we have rejected, Finding 53--are inextricably bound together. At the root of all of protesters' allegations is the complaint that, in some way or another, the agency did not make a selection in accordance with the RFP's terms and conditions. Two basic inquiries are important to the resolution of this allegation: How did the IRS conduct its price/technical tradeoff, and Did the IRS adequately inform vendors of the relative importance of the pertinent factors? Protesters more or less start with the technical scoring scheme itself. They contend that it was improper for the IRS to assign no points for meeting mandatory requirements. This, according to protesters, misled them about the degree to which offering technology that exceeded the mandatory requirements would be important. Attacking the notion of "zero points for meeting mandatory requirements" makes a good sound byte, but it fails as a protest contention. Point scores are inevitably arbitrary, at least to some degree. Thus, we have upheld scoring schemes that awarded one point, on a one-to-five scale, for meeting mandatory requirements, Tymnet-McDonnell Douglas Network Systems Co., GSBCA No. 10108-P, 89-3 BCA 22,171, 1989 BPD 243, and one that awarded negative points for merely meeting the mandatory requirements, Honeywell Federal Systems, Inc., GSBCA No. 9807-P, 89-1 BCA 21,444, 1989 BPD 23. A scheme awarding some set amount for meeting the mandatories--even if it were a thousand points--likely would still have left both protesters thirty points behind AT&T. Of course, the differences could be expressed as a percentage, but where, as here, cost proposals were unscored, the ultimate question for the source selection official to determine would have been the question whether those thirty points were worth $500-700 million. Offerors were informed that offers meeting the mandatory requirements of Section C would be evaluated according to how they met criteria listed in Section M. These criteria are not identical to the mandatory requirements, so it should have been clear that the IRS was looking for something more than the minimum. Findings 14-21. ____________________ not "required to reapply the evaluation criteria in the same fashion in assessing independently whether the higher cost of HFSI's proposal was justified." We were not faced in that case with a situation involving an award decision that had skewed the relative priority of the evaluation factors. In that case, technical merit constituted eighty percent of the points available in the award determination, thus alerting offerors to the precise degree to which technical outweighed cost. We simply held that the agency had conducted an appropriate tradeoff and that its decision did not constitute an abuse of discretion. Protesters contend more broadly that the RFP misled them into believing that price would be the critical factor in the award decision. They argue that, because the RFP sought off-the- shelf products, the possible deviations in technical quality were minimal, so that price had to be critical. We cannot agree with this contention, either. Numerous factors should have tipped protesters off to the fact that the IRS regarded technical as more important. Not the least of these is the fact that Section M expressly said so. Protesters should not have assumed that the section's drafter had his fingers crossed when he drafted that language; the procurement system is unlikely to work effectively if solicitation-writers are not taken at their word. In addition to the RFP's express language, half the technical points were assigned to administration and support functions, which is hardly consistent with protesters' "off-the-shelf" theory. Furthermore, given the sheer size of this procurement, as well as the near-legendary decrepitude of the IRS' data processing capabilities, see, e.g., Soria v. Commissioner, 55 T.C.M. (P-H) 86,206, at 861 (1986), it should have been readily apparent that even small technical differences could have a significant impact on the Government. Although the RFP made it clear that technical was to be more important than cost in the award decision, however, we find that it did not permit the IRS to make the price/technical tradeoff that it actually made. Although the RFP made technical clearly more important than price, it did not allow the IRS to discount price as a factor almost entirely. Our conclusion that the IRS in fact did so is based on a number of factors. First, other than Methods A and B, which we have rejected, the SEB report contains essentially no discussion of the issue of price/technical tradeoff.22 The report's approach is simply to state that AT&T's technical offer is stronger, so it is worth the price. Certainly there is no analysis showing that the Government would receive benefits commensurate with the particular price premium involved here.23 See DALFI, Inc., ____________________ 22The IRS points to the TEP and BMEP report as being part of the basis for its decision. Those reports, however, contain no discussion of the tradeoff issue, nor could they, because the TEP did not examine cost issues and the BMEP did not examine technical issues. The tradeoff issue was left solely to the SEB and the source selection official. 23The IRS points to DynaLantic Corp., GSBCA No. 10956-P, ________________ 91-1 BCA 23,665, 1991 BPD 10, rev'd, DynaLantic Corp. v. _____ ____________________ United States, No 91-1162 (Fed. Cir. Sept. 10, 1991), where we _____________ upheld an award at roughly two and one-half to three times the protester's price. Apart from the fact that DynaLantic was __________ GSBCA No. 8975-P, 87-3 BCA 20,018, 1987 BPD 131 (tradeoff analysis must be sufficiently comprehensive to explain why awardee offers best value to Government). Furthermore, the SEB did not even seem especially troubled at the fact that IBM came out best under Methods A and B.24 Another reason for concluding that the IRS discounted price beyond the limits of the RFP is the fact that AT&T was not superior to either offeror across the board, as one might reasonably expect based solely on the price premium the IRS was willing to pay. There was only a fifteen percent difference in technical scores between AT&T and protesters, as compared to a fifty-five to one hundred percent price difference. Furthermore, the individual items that accounted for the bulk of AT&T's advantage in the technical scoring were relatively isolated. In Lockheed's case, for example, most of its technical deficit, eighteen of the thirty points, occurred in three areas--the models 2 and 3 FWDs and training--that made up a total of only 22.5 of the 200 possible points, barely over ten percent.25 The fact that the agency was willing to pay half a billion dollars primarily on account of these items leads to two possible conclusions: (1) the IRS' price/technical tradeoff skewed the relative importance of the scored items, and/or (2) the IRS was willing to pay virtually any price for any noticeable technical advantage.26 ____________________ reversed on appeal, the price difference there was "only" several million dollars, as opposed to several hundred million. Sums on the scale at issue in this protest represent vastly more buying power, even on a relative basis, than was the case in DynaLantic, __________ and simply are not comparable. In any event, regardless of the reasonableness of the cost/technical tradeoff, the agency still is obliged to inform the vendors how an awardee will be selected. 24Protesters both try to hoist the IRS on its own petard by adjusting the dollar figures in Methods A and B to reflect their view of the correct technical analysis, and then asserting the results as evidence that the IRS made the wrong award decision. Our conclusion remains, however, that Methods A and B are best put out of their misery. 25As another example, MUS models 2 and 3, and office automation software, accounted, along with project administration, primarily for AT&T's technical edge over IBM. Anticipated purchases of such models and software, however, accounted for only about one quarter of AT&T's evaluated price. 26We make no decision here, however, whether a price premium of fifty-five to one hundred percent fails to give effect to the legal requirement that price be a factor in a selection decision. 41 U.S.C. 253a(b)(1). We likewise find the parties' own understanding of this RFP adds support to our conclusion that nothing in the RFP justified the overwhelming priority that the IRS accorded technical factors over price. The mere statement that technical was more important than cost in no way communicated the degree to which the IRS emphasized the former.27 We attach no small significance to the fact that all three parties to these protests concluded that price would be determinant. Findings 22, 23. The RFP did state, however, that the Government would not exchange a significantly higher cost for slightly superior technical features. In this particular case, in the absence of a meaningful price/technical tradeoff analysis, the IRS in using a fifteen percent technical difference to justify a price premium of $500-700 million did just that. The RFP entitled the IRS to emphasize technical over cost, but not to the degree that it actually did based on the justifications that the SEB cited. The selection decision therefore, unsupported as it is by an adequate price/technical tradeoff analysis, violated the requirement that the Government make award in accordance with the solicitation. FAR 15.608(a), .612(d). In reaching this conclusion, we do not depart from our prior holdings according considerable discretion to agency officials in setting award criteria and conducting price/technical tradeoffs. E.g., TRW, Inc., GSBCA No. 11309-P (Aug. 29, 1991) (where solicitation does not set specific weights to be accorded cost and technical, agency has great discretion in selection decision); Hughes Advanced Systems, 89-1 BCA at 107,329, 1988 BPD 253, at 49 (tradeoff need only be rational and consistent with stated criteria). We also do not hold that an agency must assign specific weights to cost and technical factors. See, e.g., Honeywell Federal Systems, 89-1 BCA at 108,045, 1989 BPD 23, at 26. Instead, we simply hold that the mere statement that cost was less important than technical is insufficient, in the absence of a meaningful price/technical tradeoff analysis, to justify an award such as that made in this case, which discounts cost beyond the degree that may be derived from the terms of the RFP. Relief ____________________ 27We are aware of the numerous instances in the RFP when the IRS informed offerors, in one way or another, that it was looking for first-rate technology. This is not particularly remarkable; what would be remarkable would be a provision stating the Govern- _____ ment would be satisfied with second-rate technology. More importantly, these provisions do not reflect the degree to which the IRS was willing to pay increased prices to obtain the technology it sought. In view of our conclusion that the IRS failed to provide a tradeoff analysis demonstrating that its selection was in accordance with the RFP, the appropriate relief is to return this procurement to respondent for further analysis. If the agency believes a suitable price/technical tradeoff analysis can be prepared that would comply with the RFP, it may proceed to make one and either confirm the previous award or make a new selection determination as appropriate. Alternatively, based on the award decision that the agency made, if it appears possible that the RFP, as written, will not permit an award in conformance with the Government's needs, respondent retains discretion to amend the RFP to provide a clear statement of its intention to emphasize technical over cost to the degree it believes necessary, and reopen negotiations or take other appropriate actions in accordance with statute and regulation. We make no inference as to what the Government's needs may be, as this determination belongs to the agency. ______________________________ WILBUR T. MILLER Hearing Examiner