Hello. They're both correct. However, here're the differences:
1. ......ACROSS + NOUN... --> Think of this as '"flying" over the NOUN' or 'walking past the NOUN'.
The supermarket is ACROSS the BANK. -->
If you fly over the bank, then you will see the supermarket. / If you walk past the bank, then you will see the supermarket.
A: Excuse me, where's the hospital?
B: It's across the school. (= Go past the school, then you will see the hospital)
Teacher: ....Hm. Where's my marker?
Student: It's across the room. (= The teacher has to walk past all the desks and chairs to get to the other side of the room.)
Teacher: ...How'd it get there??...
2. .....ACROSS FROM + NOUN --> Think of this as 'in front of' the NOUN.
The supermarket is ACROSS FROM the BANK. -->
The supermarket is IN FRONT OF the bank. (= If you walk out of the supermarket, then you will see the bank straight ahead.)
I like where I live, because my house is across from the Police Department. (= If you open your front door, you can see the Police Department.)
My best friend lives across from me. (= I can go to my friend's house anytime. I just cross the street and I'm there.)
**Those are only examples to help you better understand the differences. Please let me know if you have any further questions or comments. :)