Hi Sid. If as you say, people don't seem to have a problem understanding you then you're probably OK at that speed. You could try a little experiment to see at what speed you seem to lose people- you'll be able to tell from their eyes! Comprehensibility comes from being able to maintain the stress pattern and intonation of a language which is familiar to a native speaker's ear. Generally for Germans speaking English it's differences in intonation that cause difficulties.
I think German people have a hard accent because of their native language. Anyway, you don't need to speak slowly. In my opinion, the best think you can do is to practice your pronuntiation. I say this, because most Spanish speakers (I'm Spanish) have a terrible pronuntiation, and when they speak in English they use the same Spanish sounds. So English speakers have to make a big effort to understand us.