Place sweetened condensed milk and peanut butter in a small saucepan over low heat. Stir until smooth. Add water and stir until it is incorporated. Remove from heat then stir in the vanilla.

Transfer sauce to a jar to cool. It will thicken as it cools. To make it pourable, just heat it in the microwave for a few seconds, then stir and drizzle or pour over your favorite ice cream. Store covered at room temperature.