The first woman to permanently serve as chief counsel of the Internal Revenue Service began her tenure Wednesday.