Uncategorized

More RFP Madness: IPAM

The Finding:

The Department of Central Management Services (Department) allowed a vendor to extensively revise its proposal during the best and final process after initial scoring evaluations were completed. Several items deleted by the vendor during the best and final process eventually were added back into the agreement, in the form of contract amendments, subsequent to the awarding of the contract, potentially costing the State $5.75 million.

Documentation contained in the procurement files for the Asset Management professional services procurement opportunity showed that the Department evaluated proposals and summarized the information on November 4, 2003. The table below shows the Department?s evaluation summary for the Asset Management procurement.

The Request for Proposals (RFP) for the Asset Management professional services procurement opportunity informed proposers that the Department ??may request best & final offers if deemed necessary, and will determine the scope and subject of any best & final request.? On December 8, 2003, only one proposing vendor, Illinois Property Asset Management, LLC (IPAM) was provided the opportunity to submit a best and final offer.

There was no documentation in the procurement file addressing why other responsive proposers were not provided a best and final opportunity. The Department?s December 8, 2003 correspondence to IPAM states, ?The purpose of
this BAFO is to provide you with an opportunity to enhance the pricing and to improve any of the services offered within your original proposal.? While the price decreased from $35.9 million to $24.9 million as a result of the best and final process, IPAM?s technical proposal also significantly changed. Our review of the original proposal and BAFO submitted by IPAM noted: (see report for the details)

CMS Response: In one instance, the Department did?during a permitted Best and Final Offer (?BAFO?) process, clarify pricing. As a result of that process, it
became clear that one vendor?s proposal was superior, and this was documented in the procurement and contract files

Which completely misses the point that only one vendor was given the same opportunity even though that vendor significantly changed the quality of the proposal in reducing costs. No other vendor was asked for similar information.

To make matters worse, after reducing the price, the chosen vendor received contract amendments potentially making the contract nearly $6 million more expensive.

This is the worst I’ve seen yet and one should expect a whole lot of press attention on this point as well as potentially some legal attention from the AG. This stinks of an insider deal.

The Auditor Comment to sum up the problem: Comment 67: The auditors do not contend that IPAM is being allowed to charge twice for the same service; rather, the auditors contend that services that were deleted from IPAM?s original proposal during the best and final process have subsequently been amended back into the contract as sole source, non-competitive procurements

RFP Madness

I know that the initials RFP usually produce snores, but the rank incompetence is amazing. Going to page 21a, the RFP response by CMS starts.

The essential issue here is that if you change the RFP process, you have to give everyone a fair chance to respond or you can game the system towards favored companies. What’s stunning is that most state agencies know how to write an RFP without having to worry about adjusting things afterward to get the company they want—CMS couldn’t even game the system within the rules as most agencies do already.

Auditor Finding:The Department used evaluation criteria not stated in the RFP.

CMS Response:

Each of the evaluations sheets shows that CMS did use the same evaluation criteria in the RFP.

? This is demonstrated in Attachment 04-4 A, which compares the criteria in the RFP with the criteria used in the evaluation score sheets.

? The use of sub criteria is cited as a ?best practice? of the National Association of State Procurement Officers (NASPO). (NASPO Principles, Chapter 9, p. 67 See 04-4 Attachment B) Given that the auditors used the NASPO principles as part of their audit criteria, they should have applied these principles here, but did not.

? The auditors? criticism of the Department is disingenuous and hypocritical. The OAG routinely uses sub criteria in its procurement evaluations even though the sub criteria are not delineated in the RFP.

? Each of the awards was clearly documented and was made to the vendor, which offered the best value to the State

Auditor Comment:

Comment 58: It is a fundamental principle of competitive procurement, recognized by NASPO guidelines and required by Illinois law, that contract awards must be made based on the evaluation criteria set forth in the solicitation document. Sub-criteria, by their definition, should be derived of, not depart from, the evaluation criteria set forth in the RFP. In Finding 4, the uditors noted instances in which CMS departed from its stated criteria and/or failed to maintain documentation necessary to demonstrate its compliance with those criteria.

Comment 59: Simply stated, CMS? statement is not correct. The criteria used to evaluate proposals received through the RFP process are set forth in the RFP document. These criteria can be linked to the evaluation team scoring forms. Firms participating in the OAG procurement process have never expressed any concern about the OAG using sub-criteria not delineated in the RFP.

Auditor Finding: The Department changed the scoring methodology without communicating the changes to bidders.

CMS Response:

The Department did not change scoring methodology without communicating changes to the bidders.
? In one instance, the Department did?during a permitted Best and Final Offer (?BAFO?) process, clarify pricing. As a result of that process, it became clear that one vendor?s proposal was superior, and this was documented in the
procurement and contract files.

Original Finding

FINDING: (Changes in Award Evaluation Criteria not Communicated to Proposers) The Department of Central Management Services (Department) used evaluation criteria to evaluate vendor proposals that were not stated in the Request for Proposals (RFP). Changes in scoring methodology were not communicated to proposing vendors or reflected in an addendum to the RFPs. Additionally, in one of these instances, the Department awarded a contract to a vendor that had not received the highest scoring total based on evaluation criteria set out in the RFP. The Illinois Administrative Code states that proposals shall be evaluated only on the basis of evaluation factors set forth in the RFP. Price will not be evaluated until ranking of all proposals and identification of the most qualified vendors (44 Ill. Adm. Code 1.2035
(h)(2)).

In 44 percent (4 of 9) of the contracts we reviewed, the Department used different criteria when evaluating the price component of the proposals. The results are summarized below:

Go to the full Compliance Report for the examples

Holy Agency Batman! CMS Strike Back

Instead of simply addressing the findings, CMS attacks the Auditor General’s operations starting on Page 19 of the CMS Response

CMS Response:

First, CMS did not use any of the firms listed in the finding to develop specifications. This is clear in the work papers (Meeting minutes, CMS and OAG: 12/20/04, 1/13/05, 1/20/05, 1/24/05) and it is clear in the RFPs and contracts themselves. Furthermore, although permitted to do so, the Department does not use contractors to develop specifications and then bid on the RFP for which they developed the specification.

Rather, as CMS has repeatedly stated and demonstrated, it used these firms to gather factual information that was included as background information in these RFPs?and which was shared with all other bidders and publicly disclosed. See the face sheets from each RFP. 2 This undisputable fact alone removes the stated basis for the finding and requires its removal. CMS is providing with this response an affidavit signed by the CMS Assistant Director that attests to the veracity of the Department?s claims. Second, although the use of these firms to collect data and identify opportunities for improvements within the organization is entirely permissible?and the auditors do not contend otherwise?the Department nonetheless went above and beyond any requirements to ensure that the procurement process for these contracts was transparent.

It required these firms to fully disclose the information they provided the State to their competitors, negating any de facto advantage in the procurement process. This transparency went beyond not only the requirements of the Procurement Code and Administrative Rules, but it exceeded National Association of State Procurement Officials Guidelines.

Moreover, the auditor?s assertion that CMS has not followed procurement ?best practices? is disingenuous and hypocritical. As part of the Legislative Audit Commission (?LAC?) ?Audit Review Program,? the Office of the Auditor General participates with certain accounting firms relating to their audit programs. Interestingly, the firms who participate in this Program receive an overwhelming number and amount of auditing contracts from the Auditor General.

Auditor Comment:

Comment 55: The finding acknowledges that using potential vendors to develop RFP specifications is permissible under CMS? procurement rules if the agency head determines in writing that it would be in the State?s best interest to accept a proposal from such a vendor, and if a notice to that effect is published in the Procurement Bulletin. 44 Ill. Adm. Code 1.2050 (i). The auditors were not provided with any such written determination by the Director of CMS, and no notice to that effect was published in the Procurement Bulletin. The auditors believe that the type of information provided by potential vendors constitutes ?specifications? as that term is defined in the Procurement Code, and that is the basis of our finding. Please see Auditors?Comment 50.

CMS continuing (If you look at the PDF’s you’ll see how this works):

While CMS has no reason to believe that these decisions were anything other than entirely proper — as were CMS? procurements — these actions are inconsistent with the auditors? statements in this finding.

Third, the finding omits the following, relevant facts:
? The finding is based on the statistically and otherwise invalid sample of 9 contracts as referenced in response to Finding 04-2. Thus, the finding excludes contracts, like the legal services efficiency contract awarded to Hildebrandt, in which one of the other bidders provided pro bono background information, but was not selected. It also omits the other efficiency contracts?not to mention both: (1) the 25 contracts the external auditors tested, but omitted from their report, and (2) the thousands of other contracts CMS awarded during the audit period?in which no contractor provided information. Thus, it is highly misleading for the finding to use this improperly selective group of contracts to tout percentage statistics that would only lead a

Auditor Responds:

Comment 56: CMS? response here reflects a fundamental lack of understanding about the Legislative Audit Commission process. The accounting firms listed in CMS? response attended LAC hearings and provided testimony pertaining to audits those firms had conducted as Special Assistant Auditors to the Auditor General. Such testimony is completely unrelated to our procurement process since the testifying firms are already under contract with our Office at the time their testimony is given. For additional information, please see Auditors?Comment 49.

CMS Response Continued:

reasonable reader to conclude that most of the Department?s contracts are awarded to vendors who have provided background information. It simply is not true.

? None (0%) of the selected contracts reviewed by the auditors involved a contractor winning a bid it wrote the specifications for, and
? None (0%) of all Department contracts involved such a contractor winning
such a bid.

There were multiple potential vendors who provided background information, and not all of them were selected for an award. This fact was conveniently omitted from the finding, including the table on page 20. (i.e. Procurement Assessment- BearingPoint, Accenture; Strategic Marketing- IEG, Promotion Group Central, Civic Entertainment Group, Sustain Communications, SponsorAid, The New England Consulting Group; Software Review- McKinsey, IBM; Server Consolidation- McKinsey, IBM.)

Auditor Comment:

Comment 57: The use of judgmental selection is consistent with generally accepted government auditing standards. In this audit, the auditors judgmentally selected large contracts related to CMS? efficiency initiatives. It was a deliberate process set forth in an audit program at the outset of the engagement. That audit program was discussed with CMS personnel at the audit entrance conference held on June 14, 2004, and a copy of the audit program was provided to CMS at its request. At the time these 9 specific contracts were selected for testing by the auditors, we had no idea what we would find. Somehow CMS seems to be saying that we purposefully selected contracts for which our findings would cast CMS in a bad light. While we certainly agree the
results of our testing are not favorable to CMS, the Department does not explain ? short of our being psychic ? how the auditors might have known which CMS contracts to select to achieve such a result.

I read one comment over at Capitol Fax by Ralph that it’s very strange that this is so political. Even DCFS under Edgar’s wasn’t this over th tope, but it becomes clear once CMS started trying to attack the Auditor, the Auditor realized what was going on and got his ducks in a row.

Tone Deaf Whining

Original Auditor Statement:The Department had a non-State employee review the RFP prior to the release of the RFP. A memo was in the file from this individual suggesting benchmarking as a goal in the RFP. This individual was subsequently named as partnering with the winning vendor.

CMS Response: CMS provided clear documentation confirming that any involvement with this individual was prior to the contract award. The recommendation made by this individual would not have provided any benefit to the winning vendor or any vendors bidding on the procurement.

Auditor Comment:Comment 54: This is the situation referenced in CMS?Footnote 1 in its letter dated April 14, 2005. The auditors noted that a non-State employee had submitted comments on an RFP that had not yet been issued by the Department. CMSwas unable to tell the auditors in what capacity this person was working when he provided comments on the draft RFP to one of CMS? Deputy Directors. The person?s comments were received on May 4, 2003; the RFP was issued on May 14, 2003; and the winning vendor?s proposal was submitted on June 12, 2003. Sometime after submitting comments to CMS on the draft RFP (May 4) and before the winning proposal was submitted (June 12), this non-State employee established a business relationship with the vendor who was eventually awarded the contract. Further, in his comments on the RFP to CMS dated May 4, the non-State employee stated that he ?understand[s] one of the objectives in this RFP is to not exclude McKinney & Company [sic] from participating in this procurement simply because they participated in gathering background statistics.? The winning vendor, with whom this individual soon after partnered, was McKinsey and Company. Please see also Auditors?Comment 4.

That the agency even highlighted this instead of just saying they’d enact reforms to ensure it didn’t happen again without clear guidelines boggles the mind.

No Paper Trail

One finding is:

CMS’ contract files lacked individual scoring sheets for 6 of 9 efficiency initiative contracts we tested. Eight of 9 contract files we tested lacked evidence of a written determination for contract award. The 9 contracts totaled $69 million.

The back and forth is fascinating

CMS Repsonse:CMS contract and/or solicitation files always contain such a written determination. This information was provided to the auditors.

Auditor: Comment 29: To CMS?credit, the auditors believe that the procedures CMS now claims it follows represent good business practices. Notwithstanding the fact that CMS did not adopt specific policies in this regard until October 2004,
good business practices were applicable ? but not always utilized ? during the current audit

Back At You

Jeesshhhh is this brutal

CMS Response:Based on the comments during the entrance conference, CMS was concerned that the information in the draft report might not comply with these standards, perhaps not because there was a deliberate effort to do so, but because it would either be expedient to do so, or that the issues involved, e.g. the billings and accounting of the Efficiency Initiatives Revolving Fund, were understandably complex and would require extensive discussion between the auditors and the auditee to ensure accurate and balanced presentation of these issues in the report.

Auditor Comment: Comment 9: We agree the issues surrounding the Efficiency Initiatives Revolving Fund are complex. However, the auditors have developed a high level of expertise in this matter by virtue of the fact that we have had similar findings in 20 other agency audits to date. Sixteen of those 20
other agencies agreed with the auditors that CMS had not provided adequate documentation with the efficiency billings.

Snarkiness Deserved

CMS Response: Indeed, in 2004, CMS received the Cronin Gold Award, the
highest award for innovation in state procurement from the National Association of State Procurement Officers. CMS was also recently notified that it has won a national award from the E-Gov Institute, also for its innovative practices in government.

Auditor Comment: Comment 7: In Finding 16, the auditors cite CMS for not filing reports with the General Assembly regarding the status of its reorganizations, as required by the Executive Reorganization ImplementationAct. CMS?argument against Finding 16 is that such reports need not be filed until its reorganizations are in ?full force? and that, to date, none of its reorganizations are in ?full force? or ?executed.? Nevertheless, CMS has managed to file a report with the National Association of State Procurement Officials (NASPO) to obtain an award in 2004 for its procurement initiative.

Bizzare

CMS Response:Conferences refused to provide its basis for findings that CMS challenged, either telling us to ?save our argument for our response? or, in the case of legal issues, ?get an opinion from the Attorney General,? knowing full well that there was inadequate time to do so.

Auditor: Comment 5: Generally, the auditors and audited agencies are able to agree on matters of statutory interpretation. In those instances, however, where agreement cannot be reached, it is the auditors? standard practice to suggest the agency refer the matter to the Attorney General who, by law, is charged with rendering opinions to State officials on matters of statutory interpretation. 15 ILCS 205/4. In areas of disagreement over statutory interpretation, the Auditor General’s Office defers to a formal written opinion from theAttorney General on the matter. CMS?objection to this suggestion simply reveals its ignorance of standard audit practices.

High Comedy of Audits

Best Response by the Auditor to a Runman charge (well that I found yet)

Comment 8: We?re puzzled as to how Director Rumman could have had a discussion with the auditors at the entrance conference about unsubstantiated media allegations when, in fact, Director Rumman did not attend the entrance conference. Furthermore, no auditor in attendance could have been ?slightly embarrassed? by a point that was not made by a person who was not there. At any rate, as pointed out previously, the auditors are responsible under generally accepted government auditing standards to assess the risk involved in an agency?s operating environment at the outset ? as well as periodically throughout ? an audit engagement. However, media reports are
never used as ?source materials? or support for audit findings.

Attack the Auditor?

That’s really stupid. Rich Miller points out part of the CMS response here. He points out that the reporters love the office and that’s true–the auditor essentially writes stories for them by doing the job of auditor, but more importantly Holland can’t be painted as a partisan hack given his reappointment was unanimous and he was Chief of Staff to Phil Rock when Rock was Senate President.

Rich also reports that he think he heard:

Listening to the auditor general’s press conference. What I just heard, I think, is that CMS apparently conducted an investigation of Bill Holland’s office to see if they could impugn his integrity before the release of the audit.

Holland has a long history working with Lege Members and an attack on him will only hurt CMS–he’s worked both as state lobbyist in DC and for Dems in key committees. His relationships across the aisle are very strong too. He’s solid.

The only criticism one can make of him as 12 years as Auditor is that the auditor has an incentive to keep the Lege happy. Simply put, attacking him says nothing about what he found. The worst is when the CMS response is to whine that he didn’t say anything good about the agency’s practices.

Largely, an audit finds area of non-compliance and points them out–pointing out that someone is in compliance is done with one sentence. It’s a PR ploy and not a serious rebuttal.

If the Governor tries to turn this into a PR fight, he’ll lose bad. He should accept the report, fire the appointed employees and move a new team in accepting responsiblity. It’ll be the only way to take the bite out of this.

This report gives the media a tool which to dissect every agencies dealings with CMS and keep an eye on what is occurring. Ignoring it will just keep a constant drip of revelations coming for the next year and a half.