A Virginia man petitioned the U.S. Tax Court too late to contest his $86,000 liability, the court ruled Monday as it backed the Internal Revenue Service in the dispute.