If you are not using your current location, then it is not faster one way or another. Either get addresses from contacts or manually enter (or drop pin, as you said).If you want it do automatically and easier then turn on location services. If you want to do it harder then you would need to type in manually for both address (or use a drop pin for the end). Your choice.
I must be doing something wrong then, this does not work for me with location services off.I just turned off location services on my iP4S and got directions to and from 2 of my contact's addresses.