I don't know of a good word for what you describe. There is a distinction between "clean jokes" and "dirty jokes," but a dirty joke is specifically one that's about sex or bathroom matters. "Offensive" is the word that covers "jokes that might make someone feel uncomfortable or angry," for whatever reason. "Yes, 'Toy Story' is a very funny movie, and don't worry, none of the humor in it is offensive." There are of course many kinds of ways in which jokes can offend someone. "Ethnic jokes" make fun of a person's race, religion, or national origin. In the U.S. jokes that begin "A priest, a minister, and a rabbi" are always going to end by making fun of the rabbi (Jewish clergyman) for example. "Sick jokes" specifically means jokes that make fun of people who are in terribly tragic situations--being blind, perhaps.

You could say an "off-color joke" (something not quite politically correct or embarrassing).