I agree, they are almost the same. But "rudimentary" suggests that something is only the very beginning of a subject and not much more. A rudimentary book on English grammar might only have the present tense and a small vocabulary, for example. The other words, basic, primary, and fundamental, suggest that you are getting more than that and mean the same as each other. Another way to think of it: a person with a rudimentary knowledge of English knows less than a person with a basic knowledge of English. (You wouldn't use primary or fundamental to talk about someone's abilities, though; just to describe a text or class).

Those terms are all very similar and the differences between them are slight.

Those are synonyms! They all mean basically/fundamentally/primarily the same thing... fun·da·men·tal : basic, underlying, core, foundational, rudimentary, basal, root

Rudimentary is more simple than basic.