I often use Ctrl-B to chase chains to help understand code. Then I want to get back quickly to where I was. Similarly, I want to go back to the place I just did a replace on.
It would be nice if there were something like Undo, that just jumped you back to where you were without undoing any edits. It might logically consider clicking the mouse as adding a location to the stack, and when popping off, always return to a point at least an "inch" away from where you are now by bypassing points in the snapshot stack that were close by.