Only 0.6 percent of business returns were audited, but the rate varied greatly depending on the size of the business. About 16 percent of corporations with more than $10 million in assets were audited.
Most people don't have much of an opportunity to cheat on their taxes, said Elizabeth Maresca, a former IRS lawyer who now teaches law at Fordham University.
Your employer probably reports your wages to the IRS, your bank reports interest income, your broker reports investment income and your lender reports the amount of interest you paid on your mortgage.
"Anybody who's an employee, who gets paid by an employer, has a limited ability to take risks on their tax returns," Maresca said. "I think people who own their own business or are self-employed have a much greater opportunity (to cheat), and I think the IRS knows that, too."
One flag for the IRS is when your deductions or expenses don't match your income, said Joseph Perry, the partner in charge of tax and business services at Marcum LLP, an accounting firm. For example, if you deduct $70,000 in real estate taxes and mortgage interest, but only report $100,000 in income.
"That would at least beg the question, how are you living?" Perry said.
Koskinen said the IRS could scrutinize more returns — and collect billions more in revenue — with more resources. The president's budget proposal says the IRS would collect an additional $6 for every $1 increase in the agency's enforcement budget.
Koskinen said he makes that argument all the time, but for some reason, it's not playing well in Congress.
"I say that and everybody shrugs and goes on about their business," Koskinen said. "I have not figured out either philosophically or psychologically why nobody seems to care whether we collect the revenue or not."