I had three big babies, the first was 8lbs 11oz and I was tiny and very sore afterwards. 6 years later my son was 8lbs 9oz and popped right out and then 3 years later my daughter was born 8lbs 14oz and I had no trouble at all.
Your body does stretch and each birth is easier at least it was for me. If you are healthy and the baby is healthy I would wait but definitely talk with your doctor about your concerns and see what he says.