Less than two weeks after leading IRS efforts to apologize for the targeting of conservative groups, IRS official Lois Lerner invoked her Fifth Amendment rights against self-incrimination, refusing to answer questions about what went on at the Internal Revenue Service.

"I have not done anything wrong; I have not broken any laws," Lerner said in a short statement to the House Oversight Committee.

"After very careful consideration, I have decided to follow my counsel's advice and not testify or answer any of the questions today," Lerner said, before being dismissed from the hearing.

Lawmakers in both parties had accused Lerner of lying to them over the past year about the targeting of political groups, but they were left steaming on the sidelines, unable to get any answers out of someone in the middle of this matter.

Lerner's refusal to answer questions sparked outrage in both parties, with one Democrat accusing the IRS of deliberately stonewalling the investigation of Congress.

"So, I hope that's not the approach of the IRS going forward," said Rep. Stephen Lynch (D-MA), "because there will be Hell to pay."

After Lerner departed the committee room, much of the committee's angst was directed at ex-IRS chief Doug Shulman, who for a second straight day, seemed to have few details to offer.

"Mr. Shulman knows nothing," fumed Rep. Darrell Issa (R-CA), the Chairman of the House Oversight panel.

At one point, the top Democrat on the panel, Rep. Elijah Cummings (D-MD), remind Shulman that he was under oath, saying that his answers - or lack thereof - had been "troubling," as lawmakers piled on the former IRS Commissioner.

"Sir, you misled Congress, make no question about it," thundered Rep. Lynch at one point.

While Shulman acknowledged "there was a serious breakdown" on the political targeting issue, his answers often felt lawyerly, like "I don't have a recollection."

"Come on, Mr. Shulman," said a frustrated Rep. Cummings. "Help us help the taxpayers."

But for a second straight day of testimony, Shulman had no names to offer lawmakers, no real guidance on who called the shots and allowed conservative groups to be targeted.

"They expect us to believe that the systematic targeting of conservative groups was just the work of two rogue agents in Cincinnati," complained Rep. Jim Jordan (R-OH).

One new item that was unearthed today was that the IRS itself undertook a review of the issue that concluded in May of 2012, that from a committee interview yesterday with an IRS official.

That was a full year before the inspector general of the IRS had written up a report, raising questions about whether officials knew about this internal probe - but didn't tell the Congress.

Even the Inspector General for the IRS admitted to lawmakers that he had become frustrated with the answers of both IRS management and staff, telling Rep. James Lankford (R-OK) in one exchange that there had been a number of examples of where IRS workers had not been "fully forthcoming" with investigators.