The risk you take in dating someone who is attractive to many people is that there is a greater chance they will be a narcisist.  https://psmag.com/narcissism-linked-to-attractiveness-dbee656774aa#.77lamaq07

https://www.psychologytoday.com/blog/fulfillment-any-age/201304/why-are-narcissists-so-darn-sexy

I am a psychotherapist and I have come to believe that people who generally are not found to be so attractive to many people often treat their partners better than people who most people find very physically attractive. People who do not think they themselves are very physically attractive tend not to take their partners for granted as much.  I have worked with too many people who are very unhappy in marriages with physically attractive people who tend to be charmers in groups, but at a more intimate level in a long term relationships behave in ugly ways.

However, it is important that someone you date be attractive TO YOU.

So I guess the answer depends on what you want from a date.  If you want to be the envy of your friends or have a one night stand, go for physically attractive one.  If you are dating to find a long term partner, do not let physical appearance have so much influence on your choice. 

I think it's best to be with someone who is kind, warm, funny, attractive, interesting and intelligent, which is exactly what my wife is. Lucky me!

In my opinion, if you are trying to think this through logically, you are making a mistake. You should be focussing on how you feel about the other person, and how the other person feels about you. Date people who make you feel good when you are near them, people who make you laugh at silly things. What matters is whether you and the other person are attracted to each other, not whether either of you is "attractive" to anybody else.

An English poet, George Wither, said it well:

Shall I wasting in despair 

Die because a woman's fair? 

Or make pale my cheeks with care 

'Cause another's rosy are? 

Be she fairer than the day, 

Or the flow'ry meads in May— 

If she be not so to me, 

What care I how fair she be?