Pollution is cheap, for the polluter. Releasing sulfurous fumes into the air or dumping radioactive water into the ocean is ostensibly the easiest and cheapest way to deal with unwanted by-products. The current solution to pollution is generally dilution.
But starting with Arthur Pigou in the early 20th century, economists have argued that if we really want less of something then we should make it cost more, not hide it in a public good like the atmosphere. How? Taxes!
For example, if we want less odorless, colorless carbon dioxide billowing into the air from coal-fired power plants and vehicle tailpipes, then a tax should cause people to cut down on this undesirable activity. The most common examples of this method are probably the various "sin" taxes put on things like alcohol or tobacco. A pollution sin tax could also offset current taxes, such as income tax.
Of course, adding a tax to fossil fuel burning would just begin to compensate society for the other, hidden costs it imposes on us—such as higher health care spending. We do pay for pollution, one way or the other.