The point is what comes after "に" or "で". に=place, location で=action, movement, behavior eg: 都会に暮らす ( I live in the city) Where you live is the city.→place 都会で暮らす Living in the city is your movement. 彼はベッドに横になった ( He is lying in his bed.) Where he is lying is in his bed. 彼はベッドで横になった Lying in his bed is his action. Therefore, "ここに入らないで下さい" should be that they don't want you to come in that place. "ここで写真を撮らないで下さい" should be that they don't want you to make an act of taking photos. There are some exceptions but a simple explanation would be like that.

You are on the right track! You have to look what verb you are using. There are certain verbs that calls [target]に。For example, ・・・に　行く/来る/帰る/ ・・・に　入る/出る ・・・に　あげる/貸す 　(These verbs must have ・・・を, as well.) ・・・に　書く/置く/入れる/出す/送る/届ける etc. 　(These verbs must have ・・・を, as well.) When you use these verbs, [target]に is a MUST information that you should include in your sentence, unless it's implied. There are other に that doesn't mark target. Still you can consider they always come together. [location] に　ある/いる [giver/render]に　もらう/借りる On the other hand, で doesn't have any specific types of verbs that it should accompany with. It marks where you are when you do something or where things happen. This location information is optional, so you don't need to include [location]で unless it's necessary. Also, it's very possible to have both [target]に and [location]で for one verb. 私は　トイレに　行きました。correct 私は　映画館で　トイレに　行きました。correct 私は　映画館の　トイレに　行きました。This is also correct. *私は　映画館で　行きました。 incomplete. It's missing に. *私は　入れました。Incomplete. It's missing を and に. *私は　部屋で　入れました。Incomplete. It's missing を and に. *私は　部屋で　本を　入れました。Still incomplete. It's missing に。 *私は　部屋で　かばんに　入れました。Still incomplete. It's missing を. 私は　かばんに　本を　入れました。Correct. The sentence is complete without で。 私は　部屋で　かばんに　本を　入れました。Correct. ここに入らないでください。 In this sentence, ここ is the target place you are prohibited to enter into. You are not at ここ yet. Therefore you use に。 ここで写真を撮らないでください。 In this sentence, you are already at ここ。That's the location you do the action. Therefore you use で。

に introduces the target of an action and therefore is used with verbs of displacement like 入る、行く、置く etc... で introduces the setting where the action takes place and works for verbs that don't express displacement like 撮る