What penalties and interest does the IRS charge?

Penalties

Paying late - If you file a return but don't pay all tax owed on time, you'll have to pay a late payment penalty. The failure-to-pay penalty, as it is called, is one-half of one percent (0.5%) on any payments made after April 15 for each month, or part of a month, up to a maximum of 25%. Note: The one-half of one percent rate increases to one percent if the tax remains unpaid 10 days after the IRS issues a notice of intent to levy property. Also read IRS took money out of bank or paycheck

If you file your return by its due date and request an installment agreement, the one-half of one percent rate decreases to one-quarter of one percent for any month in which an installment agreement is in effect.

Filing late - If you owe tax and don't file on time (that includes extensions) there's a penalty for not filing on time. The failure-to-file penalty is five percent (5.0%) of the tax owed for each month, or part of a month that your return is late, up to a maximum of 25%. If your return is over 60 days late, there's a minimum penalty for filing late of $435 or 100 percent of the tax owed, whichever is less.

Underpayment of estimated tax - Currently the estimated tax penalty is 8.0% of the underpaid amount that should have been paid on each of the 4 estimated tax payment dates and computed for the number of days each payment is late. This penalty stops being calculated with the 4th payment.

Interest

Interest accrues on any unpaid tax from the due date of the return until the date of payment in full. The interest rate is determined quarterly and compounds daily. You can find a chart of historical rates here.

Note that interest is also charged on unpaid penalties.

How payments are applied

The IRS applies payments to the unpaid tax, first to the year that has the oldest assessment date (that's not necessarily the oldest tax year if returns were filed late, was filed on time), then to penalties, and then to interest. Any penalty amount that appears on your bill is generally the total amount of the penalty up to the date of the notice, not the penalty amount charged each month.