"Queen Elizabeth II" is correct and by far the most common way to write it. Write it that way.
According to Wikipedia and some other sources,
https://en.wikipedia.org/wiki/List_of_titles_and_honours_of_Queen_Elizabeth_II ,
her official title in the United Kingdom--it varies by country!--is:
"Elizabeth the Second, by the Grace of God, of Great Britain, Ireland and the British Dominions beyond the Seas Queen, Defender of the Faith."
In all of the variations, the word "Second" is capitalized, and generally the entire title is capitalized in the same style as book titles--important words capitalized, unimportant words in lower-case.
So, in the rare cases where "Elizabeth the Second" is written out, the word "Second" is capitalized.