GRANTED: December 4, 1992 GSBCA 12092-P CBIS FEDERAL INC., Protester, v. DEPARTMENT OF THE INTERIOR, Respondent, and COMPUTER BASED SYSTEMS, INCORPORATED, Intervenor. Rand L. Allen, Stanley R. Soya, Phillip H. Harrington, and David A. Vogel of Wiley, Rein & Fielding, Washington, DC, counsel for Protester. Alton Woods and James L. Weiner, Division of General Law, Office of the Solicitor, Department of the Interior, Washington, DC, counsel for Respondent. J. Patrick McMahon, Vienna, VA, counsel for Intervenor. Before Board Judges DEVINE, DANIELS, and WILLIAMS. DANIELS, Board Judge. CBIS Federal Inc. maintains that the Department of the Interior's United States Geological Survey (USGS) acted in violation of law in awarding to Computer Based Systems, Incorporated (CBSI), a contract for automatic data processing (ADP) facilities management services. The protest was filed on October 8, 1992. It alleges that USGS erred in evaluating proposals, as well as in balancing technical merit against cost savings to determine the awardee. We earlier denied a motion for summary relief brought by protester, to which we refer as "Cincinnati Bell" (short for the firm's parent, Cincinnati Bell Information Systems) so as to avoid confusion with the awardee and intervenor, CBSI. On the basis of evidence adduced at hearing, however, we are persuaded that USGS's evaluation of technical proposals proceeded from notions inconsistent with constraints imposed by the solicitation and was consequently arbitrary. The resulting determination that the proposals of protester and awardee were substantially equal in technical merit is without basis, and the selection of CBSI for award consequently has not been justified. We now grant the protest and direct the agency to terminate its contract with CBSI and permit renewed competition on an established, equitable footing before making another award. Findings of Fact 1. On May 11, 1992, USGS issued a solicitation requesting proposals to supply ADP facilities management services to the agency's facility in Reston, Virginia, over a five-year period. Protest File, Exhibit 1 at 2 (standard form 33), 8 ( C380), 234- 40 ( B). 2. The solicitation states that "[a]ward shall be made to that offeror whose proposal, conforming to this solicitation, is determined to be most advantageous to the Government, cost or price and other factors considered." Protest File, Exhibit 1 at 130 ( M2335(a)). The principal "other factor" is "[t]he offeror's technical evaluation and score." Id. ( M2335(a)(3)). The solicitation explains: In determining which proposal offers the greatest value or advantage to the Government, overall technical merit will be considered more important than price or cost. The degree of importance of price or cost as an evaluation factor will increase with the degree of equality in the technical merits of the proposals. Between acceptable proposals with a significant difference in technical rating, a determination will be made as to whether the additional technical merit or benefits reflected by a higher priced proposal warrants payment of the additional price or cost. Id. ( M2335(b)). 3. Cincinnati Bell and CBSI both submitted proposals in response to this solicitation. Protest File, Exhibit 2a. Cincinnati Bell received the maximum possible number of points for its technical proposal; CBSI received slightly less. Id. at 305-08. Cincinnati Bell's evaluated price was slightly higher than CBSI's evaluated price of $8,857,927. Id., Exhibit 5 at 626-28, 631. USGS awarded the contract to CBSI on September 28, after determining that there was no significant difference between the proposals in terms of technical merit. Id. at 631; id., Exhibit 9 at 1; Transcript at 100. 4. The solicitation states that in the evaluation of technical proposals, "Personnel and Experience" will be the most important subfactor, worth thirty-five percent of the total score. Protest File, Exhibit 1 at 128-29 ( M2320). The only difference in scores between CBSI and Cincinnati Bell is in this area. Id., Exhibit 2a at 306, 308. The cause of CBSI's having received less than the maximum possible number of points for "Personnel and Experience" is that nine of the key personnel proposed by CBSI were scored at less than the maximum by USGS's three-member technical evaluation committee (TEC). Id., Exhibit 2b at 329-30, 428-31, 540-41 (summarized at page 10 of Protester's Posthearing Brief). 5. The solicitation lists numerous types of positions "which are required under this SOW [statement of work]." Protest File, Exhibit 1 at 13 ( C382(c)(1)). There are thirteen of these categories, which bear titles such as "Computer Operations/ Production Control Overall Site Manager," "Shift Supervisor," "Senior Computer Operator," and "Supervisory Computer Support Specialist." Id. at 15-20, 220-21 ( C383). Among the thirteen categories, seven -- involving seventeen people -- are designated "key personnel." Id. at 79, 229 ( H1330(a)). 6. For evaluation purposes, each offeror was to supply resum s for individuals proposed to fill the key positions. Protest File, Exhibit 1 at 121-23 ( L2160(d)(1)); see also id. at 129 ( M2320(b)(1)(i)). Each resum was to "be evaluated to determine the Contractor's compliance with the minimum qualifications for each indicated skill level as required in the Position Description." Id. at 121 ( L2160(d)(1)(i)). 7. The solicitation also says, with regard to evaluation of resum s of proposed key personnel: Qualified resumes will be scored for degree of significant experience. Significant experience is that specialized experience which, (1) includes direct, participatory involvement; (2) was of duration to achieve a continuing expertise; and, (3) was of a level of responsibility appropriate to the skill level for which the resume is offered. Protest File, Exhibit 1 at 121 ( L2160(d)(1)(i)). The solicitation also provides, with regard to these resum s, "A college degree, college course work, specific training, and practical experience is highly desireable [sic]." Id. at 129 ( M2320(b)(1)(i)). In response to a question about this statement, the agency said that these items are not mandatory. Id. at 253 (amendment 1, question & answer 46). 8. For each of the categories of personnel, a job description and statement of minimum required years of experience is prescribed. Protest File, Exhibit 1 at 15-20 ( C383). For example, a shift supervisor is expected to perform the following duties (in addition to others listed): Resolves technical problems encountered by computer operators or others on the shift which are not covered by predefined procedures. Renders independent judgement during shift hours of operation in accordance with standard site procedures. Assigns work schedules to ensure full utilization of personnel and equipment. Id. at 15 ( C383(b)(1)). To be eligible for this position, an individual "[m]ust have seven (7) years general computer operations experience, including four (4) years actual supervisory experience in the areas of computer operation and support in a mainframe environment." Id. at 16 ( C383(b)(3)). For other jobs, the experience specification is phrased in a slightly different way; the position of a computer operator, for example "requires four (4) years experience as a computer operator in a mainframe environment." Id. ( C383(d)(2)). In addition, for some of the positions, USGS established very specific requirements. For example, a computer systems support specialist "[m]ust have knowledge of CICS [Customer Integrated Control System], TMON, Omegamon, TSO [Time Sharing Option], and Wylbur online systems for purpose of monitoring system operation and conducting authorized maintenance activities." Id. at 18 ( C383(i)(4)). 9. The solicitation explains further, in defining the sort of experience that will qualify an individual to work under the resulting contract, "In order to be considered for any of the required staff positions, with the exception of Computer Operator Trainee, all proposed individuals must have received their experience while working in a large mainframe computer environment similar to the USGS GPCC [General Purpose Computer Center]." Protest File, Exhibit 1 at 13 ( C382(c)(1)). The contracting officer and the three members of the TEC all understood the existence of this requirement. Transcript at 101, 175-76, 270, 313-14. 10. The large mainframe at the GPCC is an Amdahl model 5890, which is a second generation International Business Machines (IBM) plug-compatible mainframe computer with a capacity rated at roughly fifty millions of instructions per second (MIPS).1 Transcript at 379, 390-91, 399. The solicitation ____________________ 1 MIPS ratings vary among rating services; they depend on factors including the nature of the workload used when measuring (continued...) lists specific and general "areas of experience" to explain the sorts of computers and activities found at the GPCC. Protest File, Exhibit 1 at 14 ( C382(c)(3), (4)); Transcript at 101-02, 175-76, 269-70, 313-14.2 This list shows that the large mainframe at the GPCC runs under an operating system called IBM Multiple Virtual Storage (MVS). Protest File, Exhibit 1 at 14 ( C382(c)(3)(i)). 11. After the solicitation was issued, one vendor suggested that the requirement for work experience in a "large mainframe computer environment similar to the USGS GPCC" be relaxed. The vendor wrote: Since USGS has a general ADP operations and production environment consisting of several types of mainframe and minicomputers, we recommend that the requirement be changed to experience gained in a centralized computing environment. This will allow an offeror access to a wider range of skilled people who may have gained their experience in a production environment directly applicable to the particular platforms operating at USGS. The Government responded: The majority of the responsibility and activities in the SOW involve the production work of the Amdahl mainframe environment. . . . [I]t is essential that proposed employees received their experience in a computer environment similar to that of the USGS GPCC. [The requirement] remains unchanged except [for a minor change not relevant to this protest]. ____________________ 1(...continued) computer capacity. ViON Corp., GSBCA 10218-P-REM, 91-1 BCA __________ 23,469, at 117,739-40, 1990 BPD 352, at 16-18. According to Cincinnati Bell's expert witness, the model of the Amdahl 5890 which is installed at the USGS GPCC has a capacity of forty-one MIPS. Transcript at 419. According to a firm which leases computers, the rating of this model is 55.9 MIPS. Protester's Exhibit 11 at 2d unnumbered page. 2 In ruling on protester's motion for summary relief, we characterized these "specific" and "general" areas somewhat differently. The understanding reached in this opinion is based on the testimony of the contracting officer and the technical evaluators as to the relationship between these "areas of experience" and the other experience requirements stated in the solicitation. Protest File, Exhibit 1 at 244-45 (amdt. 1, Q & A 9). 12. Based on unchallenged testimony by Cincinnati Bell's expert witness, we come to the following conclusions about the impact of the solicitation's requirement that for the purpose of a proposed key person's meeting the specification for years of work experience, relevant work must have been in a "large mainframe computer environment similar to the USGS GPCC." Each large mainframe has its own operating system and architecture. Transcript at 391. Amdahl mainframes share an operating system with "plug-compatible" computers manufactured by IBM and Hitachi Data Systems. Id. at 390-91. The differences between the operating systems used by IBM and IBM-compatible mainframes, on the one hand, and those used by other manufacturers' mainframes, on the other, are so great that a person trained on one set of machines would not have skills that are transferable to the other without "a total conversion of the mind" like "learn[ing] a new language." Id. at 392. Furthermore, the difference between a first generation plug-compatible mainframe, such as the IBM 4300 series, and a second generation plug-compatible, such as the Amdahl 5890 at the USGS GPCC, is also significant. The operating systems, processing power, and peripheral devices of the two groups of computers are substantially different from one another. Id. at 394-400; see also Protester's Exhibit 11 at 1st, 2d unnumbered pages (4300 series, 3.7 to 8.4 MIPS; 5890 configurations, 17.9 to 72.2 MIPS). Amdahl itself does not consider the first generation machines to be "large." Transcript at 411; Protester's Exhibit 11 at 6th-7th unnumbered pages. An individual trained in an IBM first generation environment would take only "somewhere around 30 percent" of his skills with him to an IBM second generation environment, whereas someone trained in an IBM second generation environment would take "95 percent of his skill with him" to an Amdahl or Hitachi second generation environment. Transcript at 420-21. 13. The way in which the members of the TEC actually evaluated resum s of key personnel proposed by the offerors is not entirely clear. At deposition, each member of the committee testified that if a resum showed the number of years of experience required by the solicitation for the position for which the individual was proposed, the offeror received the maximum number of points available for that individual. As one member of the committee explained, "If for this position 40 points were allowed and a person met the requirements, [he] would have gotten 40 points. It's really just that simple." Transcript at 187.3 The members also testified that if a resum showed that an individual had more experience than required, he ____________________ 3 References to "Transcript" in this finding are to pages in the hearing transcript which contain portions of the deposition transcripts that were read into the record by protester's counsel for the purpose of impeaching the hearing testimony of the agency witnesses. still would have received the maximum; no "bonus points" were available for exceeding the requirements. Id. at 187, 275. In addition, the evaluators said that if an individual had less experience than required, the offeror received less than the maximum number of points available for that person. For example, "If 4 years of supervisory experience was required and in this category you could get 20 points, if you had . . . 3 years, you would have gotten less than 20 points." Id. at 181; see also id. at 198-99, 272-74, 318-19, 321-28. The contracting officer, at deposition, confirmed this understanding and added her own statement that if an offeror proposed an individual who did not meet the solicitation's experience requirements, the offeror "would not have been disqualified, but [it] would have received less points because of [the] deficiency." Id. at 110-12; see also id. at 163-64. 14. At hearing, some of the testimony by the TEC members supported this version of resume evaluation. The chairman of the committee confirmed that an individual who "fully met the requirements of the RFP [request for proposals, or solicitation]" received the maximum possible score, and that if the evaluator thought "that there was something less than full compliance with the RFP," a lower score was assigned. Transcript at 332-33. Another member of the committee testified similarly. Id. at 199- 201. The committee was certain that if an individual showed experience that exceeded the solicitation's requirements, no additional points could have been earned. Id. at 274-75, 332. 15. Each of the TEC members also stated at hearing, however, that his deposition testimony as to the relationship between a key person's experience and the number of points assigned was the product of confusion and an inability to understand the terminology used by Cincinnati Bell's counsel. See Respondent's Posthearing Brief at 13-14, CBSI's Posthearing Brief at 6 (proposed finding of fact 21), 19-21. The Board asked that the transcripts of the depositions of the evaluators and the contracting officer be supplied for the record so that we could review all the depositions and make a fully informed judgment as to how the evaluations were actually performed. See Protester's Exhibits 14-17. We conclude, having made this review, that the questions at deposition were straightforward and should have been easy to understand, and that the testimony given there is most likely a fair representation of what actually happened. We agree with Cincinnati Bell that the deposition testimony is consistent with statements made by USGS in a debriefing that followed contract award (Protester's Exhibit 18), oral representations to the Board at the initial prehearing conference in this case (Board's Memorandum of Conference of Oct. 15, 1992, at 2-3), discovery responses (Protester's Exhibit 2 at 3), and opposition to protester's motion for summary relief (at 4). See Protester's Posthearing Brief at 4. 16. At the time of the hearing, the contracting officer and the chairman of the TEC both acknowledged that the solicitation was phrased so that if even one of the individuals proposed by an offeror for a key position did not meet the relevant years of experience requirement, that offeror should have been ineligible for award. Transcript at 104, 117-18, 315-16. The evaluators and the contracting officer said that neither CBSI nor Cincinnati Bell ran afoul of this rule because all of the people they proposed were fully qualified. Transcript at 104, 106, 271, 316; Protest File, Exhibit 2a at 305, 307. 17. At hearing, the members of the TEC offered several versions of how they actually evaluated resum s of proposed key personnel. Two of these versions are not credible. One is that scoring was based in part on how each resum showed that the individual in question met the "requirements" of the job description paragraph for the relevant position. Transcript at 160-62, 197, 298-302; see Finding 8. The contracting officer conceded, however -- as is apparent from even a cursory look at the job descriptions -- that these paragraphs do not list any requirements. Id. at 165. Nor do they set out the sort of specification compliance with which may be demonstrated through resum s. Further, the chairman of the TEC testified (as had one of his colleagues at deposition) that the evaluators looked only to compliance with the years of experience requirements in assigning points to resum s. Id. at 180-81, 315. Also contradicted by the testimony of agency witnesses is the theory that when the evaluators met to discuss the scores they had assigned, they may have allayed each other's concerns about various proposed key personnel. See id. at 233, 238, 245. Two of the evaluators admitted that they did not change their scores as a result of the meeting. Id. at 245, 287-88, 292. The Government has produced no evidence even suggesting that such changes were made. As Cincinnati Bell suggests, given the lack of data in the resum s upon which the concerns were based, it is difficult to see how any group meeting could have resolved the questions without further information from CBSI. 18. Some of the testimony given by the members of the TEC at hearing is helpful, however, in understanding how the evaluations were performed. One evaluator said that when she graded resum s, she considered as acceptable for meeting the experience requirement "the months that this person was employed in the computer field in whatever capacity." Transcript at 259. If someone "gave us the impression that [he] had the skills," he was considered acceptable. Id. at 242; see also id. at 278. This evaluator said that she was "really looking for someone . . . who had basic computer experience, but someone who . . . could adapt to the work environment at the [USGS] . . . [--] [s]omebody who showed that [he was] educable and could learn." Id. at 252. Another evaluator said that as a result of having worked at the GPCC for many years, "I knew about basically what we were looking for." Id. at 291. 19. An example of this way of thinking involves the evaluators' reaction to the solicitation's specific requirement that a computer systems support specialist "have knowledge of CICS, TMON, Omegamon, TSO, and Wylbur online systems." See Finding 8. An individual proposed for this position by CBSI did not state in her resum whether she had such knowledge, yet received the maximum possible score from all three evaluators. Protest File, Exhibit 2b at 330, 430, 547; Exhibit 8. One evaluator testified that his score for the individual resulted from his having "a good feeling technically that this person could perform well in that position." Transcript at 353. Another evaluator said that she "probably subjectively from what I read came to the decision that this person could perform this position." Id. at 247-48. 20. Another example of the way in which the evaluators arrived at "good feelings" about proposed key personnel focuses on the analysis of whether experience claimed in working on various computers should have counted toward meeting solicitation requirements. The TEC chairman understood that experience gained by an individual in a mainframe environment centered on a machine manufactured by a firm other than IBM, Amdahl, or Hitachi should not have been counted, because "what we were looking for is somebody who's applicable to that IBM environment." Transcript at 352-53. One of the other evaluators, however, testified that he did not know anything about the size or operating systems of computers manufactured by Unisys, Honeywell, or Sperry; he appears, nonetheless, to have considered experience on these machines to be acceptable for meeting the solicitation requirements. Id. at 282-84. All the evaluators considered that experience with any IBM plug-compatible mainframe was relevant, even if the mainframe was a first-generation machine. Id. at 178-79, 220, 280, 371. The group did this after determining that IBM 4300 series computers were mainframes; no consideration was given as to whether they were "large." Id. at 371. The TEC chairman is convinced that a computer in the 4300 series is "equivalent" to an Amdahl 5890. Id. at 379. A second member of the committee is uncertain about this conclusion. Id. at 280. The third member understands that the two generations of computers have different operating systems; she does not know whether an individual with experience in one system "would be able to come in and pick up and be productive right away" on another. Id. at 227-28, 248-49. 21. After initial proposals were evaluated, the contracting officer initiated discussions with CBSI by sending that firm a letter identifying concerns about the qualifications of proposed key personnel. The letter mentions three individuals, only two of whom were among the nine who ultimately received less than maximum points. None of the individuals named were among the key people discussed in the next paragraph, each of whom received less than the maximum possible number of points. Protester's Exhibit 6; Transcript at 104-05. 22. According to comments written by the members of the TEC during the actual evaluation of resum s, and hearing testimony, each of these people proposed by CBSI as key personnel received less than a maximum score: Person A had worked on mainframes in the IBM 4300 series, rather than large mainframe computers with the MVS operating system, and he did not show any "supervisory experience in large data centers." Transcript at 215, 277, 329, 348; Protest File, Exhibit 2b at 329, 431, 546. The resum of Person B claimed experience on "IBM mainframes," without identifying the model numbers or operating systems of those machines. Transcript at 226-27, 278, 351; Protest File, Exhibit 2b at 329, 431, 546. Person C did not indicate with which IBM mainframes or operating systems she is familiar. Transcript at 232, 279; Protest File, Exhibit 2b at 431, 546. Person D "does not have IBM/Amdahl mainframe experience" and showed "[n]o MVS experience . . . on any large IBM or IBM-like mainframes." Transcript at 237-40, 351-53; Protest File, Exhibit 2b at 329, 432. Person E does "[n]ot [show] sufficient experience on large mainframe computers," other than those which are not IBM plug- compatible. Transcript at 244, 286; Protest File, Exhibit 2b at 329, 432, 546. Person F does "not [have] a lot of mainframe like USGS." Transcript at 349; Protest File, Exhibit 2b at 329. 23. The contracting officer directed the TEC not to compare proposals to one another, and the committee scrupulously complied with this instruction. Protester's Exhibit 3 at 1; Transcript at 99-100, 118-19, 265, 268-69, 311. Thus, the TEC never considered whether the small difference in point scores between CBSI and Cincinnati Bell accurately reflects the real difference between the proposals. Id. at 338-39. The contracting officer did not attempt to compare proposals with one another, either, because "I wouldn't have known what to look for and how to compare." Id. at 100. She consequently did not understand the extent to which CBSI and Cincinnati Bell personnel met or exceeded solicitation requirements. Id. at 167-68. Because the scores as to the technical merits of the two proposals were so close, she simply made the award to the lower-priced CBSI, rather than analyzing whether the greater technical merit of the Cincinnati Bell proposal was worth the slightly greater cost. Id. at 100, 151- 52. 24. Cincinnati Bell's strategy for proposed staffing and pricing of its proposal was predicated on its belief that USGS would value technical merit more than price and score proposals qualitatively. Transcript at 27-28, 31-34, 62-63, 68-70, 87. If Cincinnati Bell had known that USGS intended to award to the technically acceptable offeror who proposed the lowest cost, it could have revised its strategy to offer minimally qualified personnel at a lower cost. Id. at 39-40, 71-76, 89-91; see also Protester's Exhibit 9, Transcript at 72 (stipulation by respondent and CBSI). 25. For the past eight years, Cincinnati Bell has been performing under predecessor contracts the sort of work which will be performed under the contract to be awarded through the protested procurement. Protest File, Exhibit 7 at page 1 of executive summary. Every individual proposed as a key person by Cincinnati Bell received the maximum possible score on the basis of USGS's evaluation of resum s. Protest File, Exhibit 2b at 339-42, 448-51, 556-59; Transcript at 337. The chairman of the TEC characterized Cincinnati Bell's proposal as "fantastic." Transcript at 383. At hearing, Government counsel called to our attention the fact that the resum s of some individuals proposed as key personnel by Cincinnati Bell showed work on IBM 4300 series computers. Id. at 363-81. A review of the resum s reveals that none of these individuals relied exclusively on his work on a 4300 series mainframe as relevant experience, and that each of them used these computers in the course of his employment at the USGS GPCC. Protest File, Exhibit 7b at Appendix A. Discussion The protest as filed complains that USGS failed to evaluate Cincinnati Bell's and CBSI's technical and pricing proposals in accordance with the terms of the solicitation, made an improper price-technical tradeoff in concluding that award to CBSI was most advantageous to the Government, and acquiesced in a "bait and switch" scheme by CBSI that tainted the award. At hearing and in its posthearing brief, Cincinnati Bell focuses only on the allegations that technical proposals were misevaluated and the award decision was consequently without basis. These allegations are fully supported by the record. Scoring of technical proposals was dependent in large measure on the merits of resum s submitted by the offerors for their key personnel. Findings 4, 6. In our interlocutory opinion in this case, we held that the solicitation dictated two critical elements in the evaluation of these resum s. First, "the solicitation indicates that resum s will be qualitatively scored -- the document says so expressly in at least two places [Finding 7] and implicitly by indicating that a technical proposal may be superior to another even if both meet minimum standards [Finding 2]." CBIS Federal Inc. v. Department of the Interior, GSBCA 12092-P, 1992 BPD 340, at 7 (Nov. 6, 1992). Second, compliance with certain requirements regarding each key person's years of experience in specified fields of computer- related employment is essential to an offeror's eligibility for award. Id. at 6; Findings 8, 15. The solicitation makes clear that this experience must have been gained "while working in a large mainframe computer environment similar to the USGS GPCC." Finding 9. The large mainframe at the USGS center is a second- generation Amdahl computer which runs under the MVS operating system and has a capacity in the neighborhood of fifty MIPS. Finding 10. Before offers were due, a vendor asked USGS to relax this requirement to permit any "experience gained in a centralized computing environment" to meet the previous employment specification. The agency refused to do so, on the ground that "it is essential that proposed employees received their experience in a computer environment similar to that of the USGS GPCC." Finding 11. The three members of the technical evaluation committee which reviewed resum s submitted as part of the offerors' proposals have given the Board different pictures of how they actually performed their work. Findings 13-20. Two of the depictions are unbelievable, Finding 17, and we do not analyze them. The others all demonstrate that the committee did its work oblivious to the two critical elements set out in the solicitation and identified in our interlocutory opinion. At deposition, all three members of the committee testified that if a proposed key person's resum demonstrated that the individual had the minimum experience required for his job, the offeror received the maximum possible score for that individual. Finding 13. If the person was better than minimally qualified, that did not matter; even though "[q]ualified resumes [were to] be scored for degree of significant experience," no credit was given for superior proficiency. Findings 7, 13. On the other hand, if a person was found to have insufficient experience to meet the solicitation's requirements, the offeror was simply assigned a score of less than the maximum available; though disqualification should have been indicated, it was not. See Findings 13, 21. Thus, both critical elements were ignored. We have already said that this version of how the evaluations were performed is most likely correct. Finding 15. It squares with all the other statements made by respondent about the evaluations -- until the Board issued its interlocutory decision. Id. It is consistent with the actions of the TEC during the procurement in marking down, but not disqualifying, CBSI personnel whose resum s did not demonstrate that the individuals met the experience requirements. Finding 22. This version is also consistent with the agency's determination not to discuss the low scores of many individuals proposed by CBSI, Finding 21 -- notwithstanding the regulatory mandate that all deficiencies in a proposal be brought to the attention of the offeror, 48 CFR 15.610(c)(2) (1991). Finally, this version was confirmed by the evaluators in some of their hearing testimony. Finding 14. Even if we were not to conclude that the deposition testimony is the true explanation of how resum s were evaluated, however, we would still find that the evaluation of proposed personnel did not follow the solicitation's rules and was therefore arbitrary. At hearing, two of the evaluators testified that they scored proposals in accordance with their view of whether an individual had "basic computer experience" and appeared to be "educable" or to have "what we were looking for." Finding 18. The third evaluator scored candidates not on whether they showed that they had particular knowledge, but rather, on whether their resum s gave him "a good feeling" that they could do the job. Finding 19. Thus, the evaluators applied their own subjective standards, rather than the solicitation's standard, in deciding what experience was relevant. This problem was acutely demonstrated in the evaluation of experience in working with mainframe computers. The solicitation clearly stated that only experience gained "while working in a large mainframe computer environment similar to the USGS GPCC" would count in determining relevant experience. Findings 9, 11. Evaluators ignored this restriction, however, and counted experience in working with mainframes generally, no matter whether those machines ran under operating systems that were altogether or markedly different from the "large mainframe" at the USGS center. Findings 12, 20. The evaluators' testimony indicates that they did not appreciate the differences between operating systems, and that one member of the committee did not even understand the concept of the differences. Id. The evaluators' failure to adhere to the solicitation's standard had a clear, beneficial impact on the scores of personnel proposed by CBSI. Finding 22. USGS and CBSI offer two defenses to this last conclusion. First, they maintain that the solicitation's standard is unclear because "large" may modify "environment," rather than "mainframe computer." On its face, this argument is without merit: the notion of a "large environment" is meaningless in the context of this procurement. Second, the Government and the awardee contend that Cincinnati Bell may not complain about the evaluators' consistent application of an understanding that any experience on any mainframe was acceptable. This is so, the argument goes, because some of the key people proposed by protester, like some of the key people proposed by CBSI, showed experience on computers in the IBM 4300 series, which are of a different generation from and much smaller than the Amdahl mainframe at the USGS center. See Xerox Corp., GSBCA 9862-P, 89-2 BCA 21,652, at 108,923, 1989 BPD 68, at 21 ("The integrity of the protest process does not permit a protester to argue the unreasonableness of an interpretation of a solicitation provision which the protester itself held during the procurement."). The two situations are not analogous because the people proposed by Cincinnati Bell received their requisite experience at the USGS center itself, and were not qualified exclusively because of their experience on smaller computers. Finding 25. In our interlocutory decision, we stated: The evaluation of resum s of an offeror's key personnel is highly subjective, and we must consequently give deference to the agency in this matter. For Cincinnati Bell to prevail as to this allegation, it must prove that USGS's evaluation was clearly erroneous -- an abuse of discretion demonstrating a "gross disparity or unfairness." CBIS Federal, 1992 BPD 340, at 7 (citations omitted). In light of this analysis of what actually happened, we are persuaded that protester has met its burden. We also recognize that in this instance, whether a resum meets a mandatory requirement is susceptible to less subjectivity than is the case for other kinds of evaluation. Protester has provided convincing evidence that several key personnel proposed by CBSI do not show sufficient experience in a large mainframe computer environment to meet mandatory experience requirements prescribed by the solicitation. The result of the clearly erroneous evaluations of resum s of key personnel is that the selection determination lacks a reasonable basis, so the award is not justified. The solicitation requires that if the difference in the quality of technical proposals is significant, and the highest-rated technical proposal is not also the least expensive, the agency must determine whether the proposal with greater technical merit is worth its additional cost. Finding 2. The agency never made any comparison of technical proposals here, however, other than relying on the point score differences. Finding 23. The point scores themselves were of little help in judging relative merit, for the evaluators never assessed the quality of proposed personnel, as the solicitation directed them to do. To the extent that USGS erred in not taking quality of personnel into account, it converted a procurement in which technical merit was to be more important than cost into one in which award was made to the lowest-cost, technically acceptable offeror. We have expressly found this sort of action to be impermissible. DALFI, Inc., GSBCA 8755-P, 87-1 BCA 19,552, at 98,807-08, 1986 BPD 228, at 24, reconsideration denied, 87-1 BCA 19,584, 1987 BPD 15; see Finding 2. Had the solicitation promised a different evaluation scheme, there is no doubt that Cincinnati Bell (or any other offeror, for that matter) might have proposed a different group of personnel and different prices. Finding 25. Furthermore, to the extent that USGS erred in considering CBSI to be eligible for award, notwithstanding the failure of half its proposed key people to meet mandatory minimum experience requirements, it ignored essential evaluation factors. This, too, is contrary to statute and regulation. Sysorex Information Systems, Inc., GSBCA 10642-P et al., 90-3 BCA 23,181, at 116,371-72, 1990 BPD 193, at 20, aff'd, sub nom. SMS Data Products Group, Inc. v. Austin, 940 F.2d 1514 (Fed. Cir. 1991). At this point in the procurement, it is altogether impossible to say which offeror should have been selected for award. We have no way of knowing the relative merits of the personnel proposed by Cincinnati Bell and CBSI, for scoring basically ignored the quality of the individual key personnel. Revised scoring based on existing proposals would be inappropriate, however. As matters now stand, the CBSI proposal is ineligible for award because the resum s of several CBSI individuals do not demonstrate compliance with minimum requirements. CBSI was never advised of this, though; it was never given an opportunity to supplement the resum s or to replace proposed personnel. Finding 21. Thus, we cannot tell whom CBSI would actually have offered if it had learned of the deficiencies. Additional discussions -- and a consequent additional round of best and final offers -- are necessary before any meaningful scoring can be done. We also note that one of the evaluators testified that all portions of technical proposals -- not just personnel -- were scored in accordance with the philosophy that if minimum requirements were met, the maximum possible score should be assigned. Transcript at 275. We have taken no additional evidence on this matter, but the mention alone triggers a concern that all parts of the proposals should be rescored, prior to award, so that a realistic assessment of relative merits may be made. Decision The protest is GRANTED. The contract awarded to CBSI must be terminated. The Department of the Interior may continue this procurement in accordance with the requirements of statute and regulation and our directions expressed in the two preceding paragraphs. _________________________ STEPHEN M. DANIELS Board Judge We concur: _________________________ _________________________ DONALD W. DEVINE MARY ELLEN COSTER WILLIAMS Board Judge Board Judge