Thinking about the example of traveling to the plane if fire, it isn't that it lets you go where the plot is. It is that it lets you go where the fiery enemy's boss is to try to go over his head. Maybe the DM meant for you to go there; maybe not. But when you can just do it, then you can do it on your own terms and when it is use full to you.