Trust the tale, not the teller.
Whatever the opposite of regret is best describes how I've always felt about that decision - it opened me up to a million creative opportunities I needed to experience away from the bull and distorting mirrors that fame engenders.
If you feel driven and compelled to make your work and to be fiercely original and have something unique to say, in a compelling way, then chances are the helpers will be there for you the doors will open some, the ice will crack.