A Georgia federal judge on Thursday refused to disqualify himself from presiding over a defamation case arising from a family dispute related to a tax preparation business, while also rejecting a bid to transfer the matter to a federal court in California.