A lot of people think that to get to orbit, you just have to go up, but actually you need to go sideways.
Imagine throwing a ball that leaves a visual trail behind it. You throw it straight up, it comes straight back down and just leaves a vertical line. Throw it across the room, and it makes an arc. Take it outside, throw it really hard, and it makes a bigger arc. Zoom the camera out, and throw it so hard it goes over the horizon. It leaves a pretty long arc right? If you throw it hard enough, that arc goes farther and farther past the horizon until it misses the ground entirely and comes right back around to you. That's an orbit!
But that's only part of it. You see, any time you impart force on an object in orbit, you only change its trajectory, not its current position. Since your arm is now the lowest part in the ball's orbit, you can never raise that point above where your arm is. But you can affect the other side of its orbit–the faster you throw the ball, the higher the opposite side of the orbit gets. Let's head up to the highest point in the ball's orbit, and give it another push. Again, that doesn't affect its current position, but it does affect its trajectory. Making the ball go faster forward increases height at the opposite side of its orbit, so if we push it with the right amount of force, we can make its orbit circular!
Now you know enough to get a rocket to space! Well, kind of. You also need to know about fuel and the tyranny of the rocket equation, but that can wait until you play Kerbal Space Program or get a job at NASA
Let's imagine we're in a rocket ship in a circular orbit, and we want to go back to earth. You might think you need to point towards the ground and turn your engine on, but remember how we got up here–we're in orbit because we're going sideways very fast. The most efficient way to come down is to point backward along our orbit and slow ourselves down, to lower the height at the opposite side.
What happens if we do point straight downward? Well, we would start going downward, but because we aren't pointing straight backward, we aren't actually reducing our speed, only changing the direction of the orbit. It would take much more energy to come back to earth this way, and because we aren't actually reducing our speed, it would be much more dangerous, because we would be entering the atmosphere faster than if we had pointed backwards instead.
In a worst case scenario, we would run out of fuel before re-entering the atmosphere. This is very bad, because as we fall towards the earth, we start moving faster. Remember how moving faster at the lowest point in an orbit increases the height of the highest point? If we don't hit the atmosphere, the top of our orbit will end up even higher than it was before!