THIS OPINION, INITIALLY ISSUED UNDER PROTECTIVE ORDER, IS BEING RELEASED IN REDACTED FORM ON MAY 25, 1994 ___________________ DENIED: May 9, 1994 ___________________ GSBCA 12777-P COMPUTER BASED SYSTEMS, INC., Protester, v. DEPARTMENT OF HEALTH AND HUMAN SERVICES, Respondent, and R.O.W. SCIENCES, INC., Intervenor. John R. Tolle and William T. Welch of Barton, Mountain & Tolle, McLean, VA; and J. Patrick McMahon, Attorney at Law, Vienna, VA, counsel for Protester. Barbara Robbins, Office of the General Counsel, Department of Health and Human Services, Washington, DC; and Richard S. Brown, Office of the General Counsel, Department of Health and Human Services, Rockville, MD, counsel for Respondent. Joseph J. Brigati and Matthew D. Anhut of Kirkpatrick & Lockhart, Washington, DC, counsel for Intervenor. Before Board Judges PARKER, WILLIAMS, and VERGILIO. VERGILIO, Board Judge. On March 4, 1994, Computer Based Systems, Inc., filed this protest with the Board contesting actions of the respondent, the Department of Health and Human Services, Food and Drug Administration, the National Center for Toxicological Research, in awarding a cost-plus-award-fee contract to the intervenor of right, R.O.W. Sciences, Inc. The protester's allegation that the agency failed to establish either minimum requirements or minimum mandatory requirements for acceptable offers is not borne out by the record. The facts are contrary to the protester's assertion that the agency utilized undisclosed evaluation criteria and assigned weights inconsistent with the solicitation. Although protester faults the discussion process, it has not established that the agency failed to conduct meaningful discussions--the agency informed the protester of the failings in its initial proposal. Further, the record does not support the protester's assertion that the agency improperly evaluated the proposals of the protester and the intervenor, or that the selection determination was flawed. Accordingly, the Board denies the protest. Findings of Fact The solicitation The requirements 1. In June 1993, the agency issued the request for proposals underlying the protested cost-plus-award-fee contract. Protest File, Exhibit 5. The solicitation enunciates the "background and objectives" of the procurement: "The objective of this contract is to provide Federal information processing support services to the National Center for Toxicological Research (NCTR) using state-of-the-art technologies and methodologies. In performing the work described in Attachment 1, 'Statement of Work', the Contractor shall consider the background, objectives, and references stated therein." Id. at B-1 ( B-1). The solicitation details the relevant Government- furnished property--equipment and software. Id., Attachments 2, 3; Exhibit 6, Attachment 2. 2. The above-referenced statement of work contains a "background" section that concludes with the following paragraph: "The NCTR management and FIP [federal information processing] staff recognize the need for a strong viable contractor which can rapidly adapt to major technological changes and provide support and implementation of these new technologies to the NCTR user community." Protest File, Exhibit 5, Attachment 1 at 3 ( I). Under the "scope of work" section, the solicitation states: This contract shall provide Federal information processing support services to the [NCTR] using state-of-the-art technologies and methodologies including image processing, parallel processing, neural network analysis, artificial intelligence, animated 3D computer graphics, simulation and numerical modeling techniques, molecular modeling, and CAD/CAE. Id. ( II). The statement of work specifies nineteen "technical requirements" (e.g., assistance in retrieving and statistically analyzing research data; experiment support services for specific studies; development, enhancement, conversion and maintenance of application systems designed to support NCTR research and research support activities) with descriptions as to what the contractor shall provide to satisfy each. Id. at 5-14 ( IV). 3. In response to questions concerning state-of-the-art technologies and methodologies, a solicitation amendment provides the following response: There are research projects being developed at a basic level . . . . Other projects have not yet begun in these areas of expertise. We do not anticipate requirements in these areas at the beginning of this new contract. However, the successful Contractor must have experience in all areas and be able to provide the required expertise when these requirements are initiated. We will expect the successful Contractor to provide us with the technical information as to which hardware and software platforms are required for these technologies and methodologies, and to have the expertise in using the recommended hardware and software. These state of the art technologies and methodologies span across the technical requirements listed in the RFP Statement of Work. Protest File, Exhibit 6 at 15 of 25. 4. In an amendment, in response to an inquiry, the agency describes the differences between the protested procurement and the existing contract (under which the protester is the contractor): "The proposed contract requires implementation of state-of-the-art technologies and methodologies, and a greater emphasis on Technical Requirements Numbers 5, 6, 7, 8, 11, and 12. (See RFP Attachment 1.)" Protest File, Exhibit 6 at 20 of 25. 5. The solicitation requires each offeror to specify personnel--with resume and by labor category--and to propose an organization and structure for the personnel. Protest File, Exhibit 5 at L-6 to L-15 ( L-9). As stated in an amendment, "The Government will not provide a staffing plan to offerors. Offerors must make their own determination based on their evaluation of the Statement of Work and historical and other technical data provided under the solicitation and this solicitation amendment." Id., Exhibit 6 at 5 of 25. The solicitation does not dictate minimum education or experience requirements for any personnel; it is for the offeror to propose personnel capable of performing within the category assigned by the offeror. Id. at 19-20 of 25; Exhibit 8 at 4 of 7. Proposals 6. The solicitation directs each offeror to submit as separate documents a "Business Management and Cost Proposal" (volume one) and a "Technical Proposal" (volume two). Protest File, Exhibit 5 at L-6 ( L-9). 7. Volume one is to contain a separate, detailed cost proposal for the phase-in period and for each contract year, plus an overall summary of the total cost. The business portion is to detail cost and pricing data, other administrative data, and representations and certifications. Protest File, Exhibit 5 at L-6 ( L-9, Vol. I.A). 8. Regarding volume two, the solicitation states: "Technical proposals shall be organized into the following four parts to correspond to the technical evaluation factors stated in Section M of this solicitation:" A. Technical Understanding and Approach B. Resources C. Management Structure and Organization D. Experience Protest File, Exhibit 5 at L-11 ( L-9, Vol. II). 9. The solicitation specifies that the first stated element (technical understanding and approach) "shall cover your understanding of the Statement of Work and your proposed plan for accomplishing each of the technical requirements." In particular, in this part of the proposal, the offeror is to: 1. Describe your overall understanding of the technical requirements. 2. Describe what you see as the inherent problems in performing these requirements in a scientific research environment. 3. Describe your methods or plans to accomplish the work under this contract. Specifically address each technical requirement (nos. 1- 19) within the Statement of Work. 4. a. Provide your plan for receiving, tracking, and reporting work requests. Provide your plan for insuring adherence to applicable Government guidelines, procedures, and standards. b. Describe any innovative techniques proposed for maximizing productivity and efficiency. c. Identify, by DOL [Department of Labor] labor categories, the types of personnel required on a full time basis to perform the work. Address the availability of additional qualified personnel for support of or assignment to the contract as needed for specific projects. Provide the proposed staff hours, by DOL labor category, for each of the technical requirements (nos. 1-19). Note. Do not include cost data in your technical proposal. Provide your equipment and software requirement and plan for supporting the requirements of this contract. d. Provide your plan for personnel security clearances. Protest File, Exhibit 5 at L-12 ( L-9, Vol. II.A). 10. The solicitation provides similar detail regarding the other three elements of the technical proposal. Id. at L-12 to L-15. For example, as to "resources" the offeror is to: "Describe how you will ensure the availability and continuity of the staff assigned to this contract. Address the availability of additional qualified personnel for support of or assignment to the contract as needed for specific projects." Further, "Resumes should reflect not only academic qualifications but length and variety of experience in similar tasks and clearly demonstrate relevant training and experience." Id. at L-12 to L-13. 11. As another example, as to "experience" the offeror specifically is to show its experience as a company in performing work similar to that being procured, Protest File, Exhibit 5 at L-14 to L-15 ( D.1), and Show experience and capability in state-of- the-art technologies and methodologies, including image processing, parallel processing (super computer), neural network analysis, artificial intelligence, animated 3-D computer graphics, simulation and numerical modeling techniques, molecular modeling, and CAD/CAE. Id. at L-15 ( D.2). Additionally, the offeror is to provide its "approach for applying corporate experience to accomplishing the work requirements. Address the availability of additional qualified personnel for support of or assignment to the contract as needed for specific projects." Id. ( D.3); Exhibit 6 at 15- 16 of 25. Evaluations and selection 12. Section M of the solicitation states: "Primary consideration shall be given to the evaluation of the technical proposals rather than cost. It is pointed out, however, that should technical competence between offerors be considered approximately the same, then cost will become primary." Protest File, Exhibit 5 at M-1. 13. Section M contains two paragraphs regarding the evaluation of the business management and cost proposal: The factors for evaluation of your general financial and management capability are as set forth in the Business Management and Cost Proposal Instructions contained in Section L. Costs will be evaluated on the basis of cost realism which is defined as the offeror's ability to project costs which are reasonable and indicate that the offeror understands the nature and extent of work to be performed. Protest File, Exhibit 5 at M-1 ( M-2). Section L adds guidance: "In evaluating the offeror's proposed cost, the Government's concern is to determine the prospective contractor's understanding of the proposed contract requirements and its ability to organize and perform the proposed contract." Id. ( L-9, Vol. I). 14. Additional guidance on the evaluation process is contained in solicitation amendments issued prior to initial proposal submission. As to the evaluation of personnel: Each person proposed must be identified by an appropriate DOL labor category. The Government will evaluate proposed personnel on the basis of information provided by the offeror. Offerors should note the requirement (RFP Page L-13, Paragraph 2) for resumes to reflect not only academic qualifications but length and variety of experience in similar tasks and clearly demonstrate relevant training and experience. Protest File, Exhibit 6 at 3 of 25. 15. As to the evaluation of technical proposals, The offeror shall show that the technical requirements are understood and offer a logical plan for their achievement. The following factors will be weighted as indicated in establishing a numerical rating for technical proposals. Evaluation Criteria Weight A. Technical Understanding and Approach 270 B. Resources 280 C. Management Structure and Organization 200 D. Experience 250 Protest File, Exhibit 5 at M-2 ( M-3). For each of the four identified evaluation criteria, the solicitation specifies "factors" on which the evaluation would be based. The "factors" are drawn from and reflect the elements enumerated in the section L instructions regarding the technical proposal, Finding 9. Compare id. at M-2 to M-4 with id. at L-12 to L- 15.[foot #] 1 16. Prior to the submission of initial proposals, the solicitation was amended to incorporate the following questions and response under the heading "technical evaluation." Question: 1. The referenced paragraph states that "personnel capabilities, as evidenced by the resumes, will be compared to the technical requirements set forth in the Statement of Work ...". The Statement of Work does not provide quantifiable technical requirements for the personnel's experience or capabilities. What are the specific mandatory minimum requirements for personnel capabilities by which personnel will be evaluated? Please clarify what criteria will be used to evaluate the proposed personnel. 2. The offeror needs a definition of the minimum requirements for each position ----------- FOOTNOTE BEGINS --------- [foot #] 1 Under the first criterion, the solicitation lists four items, the fourth with five sub- parts. Under the second criterion, the solicitation lists three items, the second and third each with two sub-parts. Under the third criterion are five items, the third and fifth each with four sub-parts. Under the fourth criterion, the solicitation lists three items, the first with five subparts, the second identifying the eight state-of-the-art technologies and methodologies referenced in Finding 2. Protest File, Exhibit 5 at M-2 to M-4. ----------- FOOTNOTE ENDS ----------- required to include the minimum years of experience in the relevant hardware and software that the position would be involved in. Also, please provide the minimum level of training and education requirements of each position. This information is needed to ensure that all offerors are on the same footing. 3. The Government has provided the number of points or weight for technical evaluation criteria. Please provide the weighting factors and points for the sub-criteria in each of the major criteria categories or please provide a priority of each of the subfactors. Answer The minimum requirements for personnel under this contract are stated in the U.S. Department of Labor Occupational Definition in RFP Attachment 5; also see RFP Page L-7, Paragraph (a), Direct Labor, "Note." All offerors are required to identify all proposed personnel by the applicable DOL category/definition. The solicitation provides the requirements that apply to each of the four evaluation categories and numerical weights that show their relative importance. (See RFP Section M.) Protest File, Exhibit 6 at 19-20 of 25. 17. The same amendment addresses the technical evaluation of resources: Evaluation of offerors proposed resources (See RFP Pages M-2 and M-3, Paragraph B, Resources.) will constitute a substantial portion of each offeror's technical score. The term 'available' as used in the RFP reference means that offerors must determine that the personnel they propose are, in fact, available to work directly on this contract. (Also see 'Cost Evaluation', Page 19 of this document.). Protest File, Exhibit 6 at 5 of 25. The protester's alleged approach 18. The protester's president testified as to the protester's interpretation of the solicitation and its approach and proposal strategy. [ ] 19. The protester contends that, from the solicitation itself and throughout the procurement process, it viewed the solicitation as lacking technical discriminators; that is, the solicitation does not contain criteria enabling one to make a determination that there is a degree of technical excellence in one offeror's proposal over another, or in one resume as opposed to another. Transcript at 37-38. The president testified as to why the protester did not object to any terms of the solicitation: "[ ]." Id. at 38-39. 20. The protester contends that it did not view state-of- the-art technology as integral to the procurement as detailed in the solicitation: Well, we knew that the customer had a high degree of interest in state-of-the-art technology. However, the apparent weighting of that factor in the context of these procurement documents did not support that interest and, as a consequence, we, to be responsive to this procurement, had to view it in that context. I think if you did a straight mathematical calculation it would show that the state-of-the-art technology, under the assumption that these were equally weighted factors and sub-factors, was approximately 8.3 percent of the total score. Transcript at 25-26. Out of all of the evaluation criteria, the protester viewed state-of-the-art technology as affecting the evaluations only under the second of three elements of "experience"--D.2, Findings 11, 15. Id. at 26-27. Further, in the absence of specific weights or relative importance, the protester assumed that the elements within each identified technical criterion would be equally scored (and that the sub- elements within each element would be equally scored). Id. at 23. Element D.2 is one of three elements for "experience" (250 out of 1000 points); one third of the 25% for "experience" is 8.3%. 21. In contrast to the testimony, the initial paragraph of the executive summary of protester's technical proposal contains the following: NCTR has set a new direction to accomplish this goal [of leveraging its information technology resources to the fullest extent]-- a direction that mandates an increase in direct computing support to NCTR's scientists with a concurrent emphasis on aggressive pursuit of state-of-the-art technologies. The active advocacy and joined commitment of the Federal Information Processing (FIP) support contractor is central to NCTR's new information resource management strategy. Protest File, Exhibit 37, Technical Proposal at 1-1. The "Technical Understanding and Approach" summary in the protester's technical proposal states: [Protester] fully appreciates the difference in the Scope of Work for this contract and our existing contract, i.e., the emphasis on infusing state-of-the-art technologies through NCTR's information systems, particularly in support of NCTR's scientific community. We welcome this change and the opportunity to provide a new dimension to our FIP support of NCTR. Id. at 1-3. Initial evaluations 22. Utilizing evaluation score sheets, five individuals (constituting the project advisory group) separately evaluated the initial technical proposals, scoring and writing comments for each evaluated item. Protest File, Exhibit 16 at 29-670. 23. The evaluated items noted in the evaluation score sheets parallel, and are directly derived from, the section M evaluation criteria, Finding 15. Compare Protest File, Exhibit 5 at M-2 to M-4 with id., Exhibit 16 at 30-32. The points allotted to each of the four evaluation criteria are as described in section M, Finding 15; the elements and sub-elements are not all identically or similarly weighted. Id., Exhibit 16 at 30-32. Thus, for example, 270 points are allocated to the technical understanding and approach criterion; the four elements have maximum scores of 40, 40, 108, and 82 points. The third element (feasibility of the methods or plans proposed to accomplish the work under this contract) is divided with unequal point maximums (ranging between 2 and 9) among each of the nineteen technical requirements identified in the statement of work. The five sub- elements constituting the fourth element have maximum points of 5, 10, 17, 20 and 30. Id. 24. The intervenor's initial proposal received scores of zero on particular sub-elements by four of the evaluators, three of whom assigned a single zero. For no sub-element did it receive a score of zero from four of the evaluators; three evaluators assigned a zero on one sub-element, and two assigned a zero on another sub-element. Protest File, Exhibit 16 at 351-53, 415-17, 483-85, 546-48, 609-11. The protester's initial proposal received no score of zero. Id. at 30-32, 94-96, 162-64, 225-27, 288-90. 25. By a memorandum dated December 16, 1993, the chair of the five-member evaluation group transmitted to the contracting officer/source selection official a report on the technical evaluations of the proposals. Protest File, Exhibit 16. In addition to a summary of the procurement action, the report provides a score for each proposal--representing the average of the scores of the five evaluators. It also categorizes the proposals-- intervenor's as "fully acceptable," protester's as susceptible of "being clarified or amended so as to warrant further consideration from a technical viewpoint." Id. at 3. For each offeror, for each of the four technical criteria, the report identifies the strengths and weaknesses of the initial proposal as found by the various evaluators. Id. at 4-10. Further, for each offeror, the report identifies proposed personnel by name, Department of Labor category, and position, and provides an evaluation of either "qualified" or a narrative expressing some concern. Id. at 12-28. The report also contains the score sheets and comments of the individual evaluators. Id. at 29-670. 26. The strengths and weaknesses identified and the narratives on personnel not designated as "qualified," mentioned in the above finding, do not represent a group consensus. Rather, they represent a narrative summary of the written comments submitted by the individual group members. Protest File, Exhibit 16 at 4, 8. 27. The agency also evaluated the business management and cost proposals. Protest File, Exhibits 17, 18. Discussions 28. By letter dated December 20, 1993, the agency informed the protester that its proposal is within the competitive range and that negotiations are occurring. Specifically, "Enclosed is a list of questions which we would like answered and/or areas of your proposal where we require further clarification." The questions pertain to the business management and cost proposal and to the technical proposal. Protest File, Exhibit 19. The questions represent the entirety of substantive discussions held with the protester, Transcript at 223, although further information was passed along to the protester in the request for best and final offers. Protest File, Exhibit 27. 29. One question with sixteen sub-parts, which addresses the technical proposal, begins: "The following are areas of concern regarding your proposal." Five of the items read as follows: a. The resumes of proposed management personnel do not reflect experience in managing state- of-the-art technology projects. b. The proposed onsite staff does not reflect experience that appears sufficient to support state-of-the-art technology projects started or completed by [teaming partners]; it appears they would require training in these areas. c. The experience of the proposed LAN administrator is weak. d. The proposed database administrator is lacking in database design experience. i. Technical Requirement Nos. 5 and 6: Explain the proposed employee assignments. The proposed personnel do not appear to have the necessary expertise for these requirements. Protest File, Exhibit 19 at CBSI-3 to CBSI-4. 30. The questions do not reflect every weakness, or every aspect raised in the narrative on personnel not designated as "qualified," which are identified in the report to the contracting officer. The questions represent the summary opinion of the technical evaluators, not necessarily individual opinions. Transcript at 189. The concerns of the group as a whole are identified. Best and final offers and selection 31. By reports dated January 5, 1994, the Defense Contract Audit Agency provided the agency with audits of the initial pricing proposals of the protester and intervenor. Protest File, Exhibits 22, 23. 32. In light of responses to the discussion questions, each member of the project advisory group individually rescored those portions of the technical proposals which the evaluator deemed affected by the responses. From each evaluator the protester received a lower overall score than it had on its initial proposal, while the intervenor received a higher score than it had on its initial proposal. Protest File, Exhibits 16, 25. The protester has demonstrated no impropriety in the downgrading. The zero scores in the intervenor's technical proposal were unaffected, there being no new zeroes or raised scores for those sub-elements. Id., Exhibit 25. 33. The project advisory group provided to the contracting officer a report which summarizes the analysis and determinations of the group and provides the determinations of the group particular to the responses of the individual offerors. The report also contains the scoring by the individual evaluators of the technical proposals read in conjunction with the offeror responses. Protest File, Exhibit 25. 34. The agency made another competitive range determination. Protest File, Exhibit 26. Thereafter, it requested and received best and final offers from the protester and intervenor. Protest File, Exhibits 27, 28, 37, 38. 35. Having completed its review of the best and final offers, the technical advisory group issued a memorandum to the contracting officer. Regarding the protester's offer, the group states that it is of the "opinion that there was nothing new that warranted a change in the overall score." Protest File, Exhibit 32 at 2. The intervenor did not submit additional technical information. Id. In a separate memorandum, the group recommends that the intervenor be awarded the contract; the rationale: the intervenor's "technical understanding and proposed resources made their proposal technically superior." Id. at 1.[foot #] 2 36. In a memorandum to the file (curiously dated December 17, 1993, before all events had occurred), the contract specialist provides an overview of the procurement, addressing pre-negotiation actions and summarizing negotiations, best and final offers, the various evaluations, and the acceptability of the intervenor's offer. The memorandum makes a recommendation for award to the intervenor. The memorandum indicates contracting officer approval with a signature and date of February 20, 1994. Protest File, Exhibit 33. The technical score of the intervenor is [ ] above that ----------- FOOTNOTE BEGINS --------- [foot #] 2 The agency introduced documentation and testimony to reflect evaluation scoring utilizing score sheets which provide for equal points for the elements and equal points for the sub-elements within each criterion. Protest File, Exhibit 45; Transcript at 210-13, 217. The material is of limited value principally because the agency has not demonstrated that the altered point distribution reflects its true requirements and because it is based upon the scoring of initial proposals, not best and final offers. ----------- FOOTNOTE ENDS ----------- of the protester. The proposed cost of the intervenor is approximately twenty-two percent above that of the protester. Id. at 16. 37. In making the selection determination, the contracting officer considered "[w]hether or not the costs were reasonable and realistic for the contractor to perform the job as they had proposed." Cost proposals had been audited and compared with Government estimates. Transcript at 227-28. 38. As part of his selection determination, the contracting officer re-totaled the scores of the individual evaluators, checking the final averages. The protester's technical score was significantly inferior to that of the intervenor. Further, the contracting officer dropped out, one by one, each evaluator's score in calculating the average. This was done to highlight any inconsistencies or anomalies in scoring of an evaluator, which could affect the selection determination. The procedure reveals no significant variation in the technical ranking of the protester and intervenor. Transcript at 229, 231. Further testimony 39. During the hearing, some of the evaluators' testimony focused on the evaluation process. Testimony reveals that evaluators considered personal likes and dislikes nowhere apparent in or supported by the terms of the solicitation. However, the protester has not demonstrated that the instances of sometimes over-reaching, subjective evaluations are more than minor in number and nature, or that they detrimentally impacted upon the score of the protester compared to that of the awardee. When reviewing the resumes of incumbent personnel, the evaluators considered the actual experience of those personnel for the proposed positions. The resumes expressly indicate the present positions of such individuals; the evaluators put that information in context. The evaluators down-graded the protester's technical proposal for failing sufficiently to explain its staffing structure using teaming partners. Contrary to the protester's assertions, the evaluators interpreted the very information provided by the protester; they did not have to speculate separately on the application of state law. The protester has failed to demonstrate factually that the composite scores of the group reflect an inaccurate assessment of the best and final offers of the protester or of the intervenor. 40. Contrary to the protester's suggestions, each evaluator appears to have evaluated the proposals fairly and consistently, while aware of the stated requirements and criteria in the solicitation. Although the protester is critical of the fact that the evaluators would not specify a threshold level of education or experience necessary to achieve a particular evaluated score, true to the terms of the solicitation, the evaluators considered the actual education and experience of each resumed individual for the proffered position. 41. The Board qualified a protester witness as an expert in three areas: automatic data processing support services; the evaluation of the requirements of a request for proposal; and, the adequacy of discussions. Transcript at 271, 311, 312-13, 316. The testimony has not proven to be particularly credible or helpful. Without adequate explanation, the individual ignores or reads so narrowly the requirements and provisions of the solicitation as to fail to give the requirements and provisions meaning. Given that the individual lacks experience in procurements involving the application of computer technology in research environments and has scant experience in state-of-the- art technology for any of the nineteen statement of work items, the individual has not explained how or why his knowledge sheds useful light on the aspects of this procurement and issues of this protest. Transcript at 273-75, 277, 304-05. Discussion The protester pursues six counts of protest. One, the agency failed to establish either minimum requirements or minimum mandatory requirements for acceptable offers. Two, the agency utilized undisclosed evaluation criteria and assigned to factors and sub-factors weights inconsistent with the solicitation. Three, the agency failed to conduct meaningful discussions. Four, the agency improperly evaluated the technical proposal of the awardee-intervenor. Five, the agency improperly evaluated the technical proposal of the protester. Six, the agency conducted an improper cost-technical trade-off. Minimum requirements The protester maintains that the agency failed to establish or apply minimum requirements during the evaluation process such that the agency violated provisions of the Federal Acquisition Regulation (FAR), 48 CFR 15.606(a), (c) (1992). The protester faults the agency for failing to have objective evaluation criteria, particularly with respect to the evaluation of education and experience of resumed personnel, and concludes that the evaluations were arbitrary and unreasonable. The protester-referenced regulations relate to instances when the agency "changes, relaxes, increases, or otherwise modifies its requirements," FAR 15.606(a), or the most advantageous offer departs from the stated requirements, FAR 15.606(c). The protester has demonstrated no alteration in the requirements of the agency or any departure from the requirements by the intervenor. Unchanged throughout the procurement is the agency's attempt to obtain support services using state-of-the- art technologies and methodologies. The protester's proposal suggests that the protester shared this interpretation of the solicitation. Finding 21. The solicitation requires offerors to propose a staffing plan and to specify personnel to perform within that plan. The solicitation does not dictate minimum education or experience requirements for any personnel. Finding 5. The solicitation specifies that the evaluation process will focus upon the offeror's technical understanding and approach to the procurement. Finding 9. Although, during the evaluation process, evaluators considered personal likes and dislikes nowhere apparent in or supported by the terms of the solicitation, the protester has not demonstrated that the instances of sometimes over-reaching, subjective evaluations were more than minor in number and nature, or detrimentally impacted upon protester's score compared to that of the awardee. Finding 39. The evaluations overall were conducted consistent with the dictates of the solicitation. When one considers the composite scores of the evaluators, the record shows no hint of arbitrariness or unreasonableness. The scoring and agency determinations regarding the protester are consistent with the testimony of the protester that it attempted to obtain the award with a minimally acceptable offer, Finding 18. Evaluation criteria The protester maintains that the agency violated FAR 15.605(e) ("solicitation shall clearly state the evaluation factors . . . and any significant subfactors, that will be considered in making the source selection and their relative importance") as well as the parallel provision FAR 15.406- 5(c).[foot #] 3 The protester raises specific instances of agency actions allegedly violative of the provisions. First, the protester notes that the agency "assigned unequal weights to the factors and subfactors listed under each of the four technical evaluation criteria," even though the solicitation contains no weights for these items. Third Amended Protest at 12. Section M states that technical proposals shall be given primary consideration rather than cost. This is consistent with regulation. 48 CFR 15.605(d) (1992) ("In awarding a cost- reimbursement contract, the cost proposal should not be controlling."). Section M establishes the specific numerical weights available for each of the four evaluation criteria of the technical proposal. Finding 15. By solicitation amendment, although requested to provide the weighting factors and points for the sub-criteria in each of the major criteria categories or the relative importance for such items, the agency provided no additional detail and instead simply referenced the existing clause. Finding 16. This response suggests (consistent with the terms of the solicitation) that technical and cost constitute the factors, and the four point-scored technical criteria constitute the sub-factors for purposes of the FAR provisions. The protester has failed to establish the reasonableness of its alleged assumption that all items and sub-items within the evaluation criteria would be scored equally. Integrated Systems Group, Inc., GSBCA 12417-P, 93-3 BCA 26,225, 1993 BPD 178. However, the analysis of Integrated Systems is not dispositive; the simple labels of "factor" and "subfactor" do not determine compliance with the regulations. Protester asserts that, in fact, there existed significant subfactors that were considered in making the source selection and whose relative importance were not revealed. Despite the unequal scoring of elements and sub-elements within criteria, the protester has not demonstrated how their actual assigned relative importance is ----------- FOOTNOTE BEGINS --------- [foot #] 3 Although the agency moved for summary relief for a portion of this basis of protest, no party addressed the impact, if any, of the provisions of the agency's regulations (HHSAR) implementing and supplementing the cited provisions of the FAR. 48 CFR 315.06, 315.406-5(c) (1992). The Board reserved ruling on the motion until the parties had the opportunity to consider these provisions in the post-hearing briefs. ----------- FOOTNOTE ENDS ----------- inconsistent with the solicitation provisions which place the evaluated elements and sub-elements in the context of the technical requirements. Further, given the protester's general approach to the procurement, it has not persuasively demonstrated that it would have altered its offer if explicit detail had been provided. Second, the protester asserts that the agency placed greater emphasis on state-of-the-art technology resources than the evaluation criteria indicated. In pursuing this allegation, the protester fails to read the solicitation as a whole. The background and objectives in the original solicitation and statement of work reveal that through the procurement the agency will obtain support services using state-of-the-art technologies and methodologies. Findings 1, 2. By amendment, the agency emphasized the importance of state-of-the-art technologies and methodologies--they "span across the technical requirements listed in the RFP Statement of Work" and, in contrast to the then existing contract, "the proposed contract requires implementation of state-of-the-art technologies and methodologies." Findings 3, 4. State-of-the-art technologies and methodologies permeate the procurement. The protester's narrow interpretation of the solicitation, which is inconsistent with its proposal, Finding 21, is unreasonable. Third, the protester claims that it was down-graded for several evaluation factors not disclosed in the solicitation. Factually, the protester has not demonstrated the foundation for its allegation. The evaluators considered the protester's offer against the evaluation criteria. The protester's offer merited the comments and scoring which ultimately resulted. Any quibbles the protester may have with respect to individual sub-elements, and the award or not of one or more extra points by one or more of five evaluators, do not affect the general evaluation of the protester's proposal. [ ] Fourth, the protester maintains that the evaluators lacked guidance to deem the proposed personnel as qualified or not, Finding 25. The solicitation provides the technical requirements which offerors are to satisfy. The offerors are to identify the approach and structure, as well as the personnel and positions, for accomplishing the contract work. The solicitation, together with the offeror's proposal, provide the bases for evaluating the experience and education of the personnel. The evaluations did not occur in the abstract, but rather, in this context. The protester has shown no agency impropriety. Fifth, evaluators applied criteria based upon their own observations of protester's personnel performing under the incumbent contract. When reviewing the resumes of incumbent personnel, the evaluators considered the actual experience of those personnel for the proposed positions. The resumes expressly indicate the present positions of such individuals; the evaluators put that information in context. Finding 39. The protester has neither demonstrated a violation of statute or regulation by this evaluation process nor shown that any loss of points was more than insignificant. Finally, the protester claims that the agency utilized undisclosed sub-factors in a portion of the technical proposal evaluations. In particular, under the first technical criterion (technical understanding and approach), the third element is listed as feasibility of the methods or plans proposed to accomplish the work under this contract. The evaluation sheets further subdivide this element by identifying for separate point scoring the nineteen technical requirements identified in the statement of work--that is, the evaluations focused upon "the work under this contract." Findings 15, 23. Not only has the protester failed to show that the elements and sub-elements equate to factors and sub-factors under the protester-referenced FAR provisions, but also, the protester's reading of the criteria and element ignores the very language at issue--the evaluators focused upon the work under the contract. Discussions The protester alleges that the agency failed to engage in meaningful discussions with the protester. In particular, it asserts that the agency did not point out deficiencies, weaknesses and uncertainties in protester's initial proposal, and misled the protester into making detrimental revisions to its proposal. The agency did not at any time inform the protester that it had submitted a fully acceptable proposal. The questions posed during the discussion process were not identified as all being simple clarification questions. The significance of the agency's concerns regarding technical aspects of the protester's initial offer are apparent from the questions which specifically focus upon an apparent wide-spread lack of adequate experience in state-of-the-art technology projects, and particular weak experience of given individuals. Finding 29. The questions expressed true concerns of the evaluation group. The protester ignores the focus of the questions and contends that through the discussion process the agency should have assisted the protester in formulating its approach. Despite the questions, [ ]. In short, the protester maintains that the agency was obligated to engage in discussions to dissuade the protester from its proposal strategy. The protester's tacit contention that the agency should have instructed the protester on how to approach the procurement and structure its personnel discounts the portion of the solicitation which evaluates the offeror's approach and structuring of the personnel, Findings 8, 9, 15, and discounts the express statements in the protester's initial proposal indicating that the protester was well aware of the significance of state-of-the- art technology in the procurement. Evaluation of intervenor The protester has failed to point factually to a single instance of the agency misevaluating the intervenor's initial or amended proposal. Allegations that the agency scored intervenor's technical proposal much higher than it should have been are completely unfounded in the record developed and arguments presented. The belatedly raised allegation that the intervenor's best and final offer was unacceptable because it had received some zeroes in the individual scoring by evaluators in some sub- elements is also meritless. Although individuals did the evaluations, the group provided composite scoring for each offeror. Findings 22, 32, 35. The intervenor's offer did not receive a zero on any sub-element from all members of the group. Findings 24, 32. The group deemed the intervenor's best and final offer to be "technically superior." Finding 35. Moreover, the contracting officer/source selection official deemed the proposal to be acceptable. Findings 36, 38. The record amply supports that determination. Evaluation of protester The protester contends that the agency improperly evaluated the protester's technical proposal, which resulted in a lower than proper score. As noted in the findings and discussion above, the record does not support the protester's assertions. The protester has not demonstrated that the overall scores for best and final offers misrepresent or inadequately reflect the proposals of the protester or awardee. The ultimate score of the protester's technical proposal is consistent with the terms of the solicitation and true to that proposal. Cost-technical trade-off As a group, the technical evaluators concluded that the intervenor's best and final offer is "technically superior" to the other offers in the competitive range. Finding 35. The contracting officer confirmed this conclusion with his review of the evaluation scorings. Finding 38. The agency and contracting officer also concluded that the proposed costs of the intervenor are reasonable and realistic. Finding 37. The contracting officer selected the intervenor for the award. Given the determinations that the intervenor's best and final offer is technically superior and is reasonable and realistic in terms of cost, the selection is fully consistent with the selection criteria of section M of the solicitation, Finding 12. The protester has not demonstrated any unreasonableness or impropriety in the conclusions of the contracting officer. No alternative or additional action was required to constitute a valid selection determination. It is the protester which persists in its misunderstanding of the selection criteria established in the solicitation. Decision The Board DENIES the protest. On March 10, 1994, without agency objection, pursuant to statute, 40 U.S.C. 759(f)(3) (1988), the Board suspended the relevant procurement authority of the agency. The suspension lapses by its terms. __________________________ JOSEPH A. VERGILIO Board Judge We concur: __________________________ __________________________ ROBERT W. PARKER MARY ELLEN COSTER WILLIAMS Board Judge Board Judge