Shevuot (Oaths) 47 - One, Who Has to Take an Oath but Cannot, Pays
If the defendant has to take an oath, such as when he agrees that he owes half of the amount and has to swear about the other half, but cannot, because he once swore falsely, the oath is shifted to his opponent, the plaintiff. If, however, the plaintiff is also not trustworthy, then the rule is that "the oath returns to its place."
What does "the oath returns to its place" mean? The Teachers in Babylon, that is, Rav and Shmuel, say it means that the oath returns to Sinai, where it was stated, "do not steal." The courts do not get involved, and the one who is dishonest must fear Divine retribution.
The teachers in Israel, that is, Rabbi Abba, says that the oath returns to the one who was initially obligated to take it, in order to free himself from liability. And, since he cannot swear, he remains liable and has to pay.
Art: Rembrandt Van Rijn - Anna Accused by Tobit of Stealing the Kid