And yes, you'll end up paying less!
I don't think so. The only way everyone ends up paying less is if the usual deficit goes up.
If I understand it right, when FairTax works properly, the lower class above the poverty line pay more. They spend nearly 100% of their income on perceived necessities, and 23% of nearly 100% of their income is more than their current net (weighted) tax bracket.
Meanwhile, upper middle and upper class, who consume a lot, also pay more.
Everyone else pays less, because they limit consumption and their current effective income taxes are larger than 23% of their spending.
There are other consequences: since only
retail goods and services would be taxed, I would think there would be a sudden increase in the market for person-to-person sales of used goods.
Even though FairTax might not substantially affect the GNP and/or government's collected taxes, it would dramatically change the structure of the economy, and it would reduce the government's ability to crack down on "evil" things like gambling and drugs. Without income reporting for individuals, there would be no need to launder money. Drug and gambling enforcement would become much more difficult, and the prospect of being able to tax drugs and gambling directly might make the government decide to legalize them.
Taxing gambling is extremely problematic, though. If you lose $50k of $100k one day, and turn your remaining $50k into $150k the next day, you've made 50% overall. However, if you're not a professional gambler, technically your gain according to the IRS is $100k. In other words, your losses aren't tax deductable, but your gains are taxable.
There's the same problem if the government taxes gambling winnings directly.
When do you tax them? And how would the casino be able to report such things in a meaningful way, so that the IRS would know the casino wasn't skimping on taxes (and passing those savings on to the gamblers)?