Persistent bad breath or a bad taste in the mouth may be a warning sign of gum (periodontal) disease. Gum disease is caused by the buildup of plaque on teeth. Bacteria cause the formation of toxins to form, which irritate the gums. If gum disease continues untreated, it can damage the gums and jawbone.