Layoffs and voluntary resignations at the Internal Revenue Service have significantly reduced the agency’s revenue agents responsible for conducting audits, the Treasury Inspector General for Tax Administration said Monday.