An IRS Penalty is a fine levied by the IRS. You may pay a flat dollar fee or a fee based on an interest charge for unpaid taxes, failure to pay taxes, failure to make estimated tax payments, failure to make federal tax deposits, or filing late.