I would personally just treat whatever direction I'm facing at the time as North and go from there.

Do I look like the person who would get lost in a familiar place?

Actually, don't answer that.

No. I keep trying to go but its always too far!

It hurts me that people don't realise you know where the sun rises and sets (roughly), anywhere, by looking up and roughly knowing what time it is. Other than midday, then fair enough.