With more evidence surfacing that the Internal Revenue Service had targeted more conservative groups applying for tax exempt status, President Obama announced the acting head of the IRS had offered his resignation, as Republicans in the Congress said even more changes are needed at the tax agency.

"I will not tolerate this kind of behavior in any agency, but especially in the IRS, given the power that it has and the reach that it has into all of our lives," the President said at the White House, as he announced the acting IRS chief had been pushed out of his job.

Here are some of the President's comments:

As for Steven Miller, the Acting IRS Chief, he didn't mention anything about being forced to resign in a message to IRS employees:

http://www.flickr.com/photos/95476565@N08/8743557298/sizes/l/in/photostream/

"It is with regret that I will be departing from the IRS as my acting assignment ends in early June. This has been an incredibly difficult time for the IRS given the events of the past few days, and there is a strong and immediate need to restore public trust in the nation’s tax agency. I believe the Service will benefit from having a new Acting Commissioner in place during this challenging period. As I wrap up my time at the IRS, I will be focused on an orderly transition.

"While I recognize that much work needs to be done to restore faith in the IRS, I don’t want anyone to lose sight of the fact that the IRS is comprised of incredibly dedicated and hard-working public servants. During my 25-year IRS career, I am profoundly proud to have worked alongside you and to be part of an institution that has worked hard to support the nation. I have strong confidence in the IRS leadership team to continue the important work of our agency.

"I want to thank everyone for all of their support and friendship during my career in government service. And I especially want to thank each and every one of you for your continued commitment to the nation’s taxpayers.

The decision to push out Miller from his Acting Commissioner post didn't placate Republicans.

"This resignation does nothing to change the culture of discrimination at the IRS," said Rep. Dave Camp (R-MI), who will lead the first House hearing on the IRS scandal on Friday.

"And, it certainly does nothing to change the fact that the tax system is targeting honest, hardworking taxpayers instead of working for them," Camp added.

Meanwhile, other GOP lawmakers spent Wednesday afternoon sparring with Attorney General Eric Holder over the IRS revelations, making clear they want more than just a review of what happened.

"The facts will take us wherever they take us," Holder said. "We'll go wherever the facts lead."

But Republicans weren't calmed by those assurances from Holder, as they argued that more than just one official should be thrown overboard.

"I know for a fact Lois Lerner lied to me," Rep. Jim Jordan (R-OH) said to Holder, referring to the head of the IRS tax-exempt unit.

"She lied to our personal staff, she lied to our committee staff," Jordan added. Lerner is one of several officials invited to testify next week before the House Oversight Committee.

Outside the hearing room, Republicans were even more blunt about what they want from the FBI investigation of the IRS.

"The question isn't about who is going to resign," said Speaker John Boehner, "My question is, who is going to jail."

Along with the Friday hearing before the House Ways and Means Committee, there are already two hearings set for next week, one before House Oversight, and another before the Senate Finance Committee.