While pepper may not be considered to be poisonous for dogs, it is in fact not a part of their natural diet and should be given to them in moderation. Most dogs do not like to eat pepper because of the fact that it has such a strong taste and smell. When dogs go through obedience training, pepper is often used as a deterrent.
If your dog enjoys eating pepper though, it is highly recommended that you not give too much to them on a regular basis. Doing this could result in health problems with them later on.