"A lot of" and "lots of" are quantifiers, or a sort of determiner, and are essentially an informal alternative to using "many" (for plural nouns) or "much" (for singular, mass nouns). ("Much" as a quantifier, in particular, sounds a bit stilted in everyday use, and so "a lot of" or "lots of" is more usual.) Note that although the former looks singular and the latter looks plural, you can use them both with either singular or plural nouns (so unlike "much"/"many" in that respect), with logical verb agreement, thus: "there's lots of tea left" or "a lot of people are going".
Without "of" , these phrases can be used more as pronouns or adverbs rather than determiners: "there is lots left", "a lot are going" or "I am travelling a lot".