A former senior appraiser for the Internal Revenue Service was appropriately fired for golfing on company time, a federal appeals court affirmed Wednesday.