Place eggs yolks in a small bowl and lightly beat with cornstarch-water mixture until smooth. Set aside.
Combine sugar and milk in a medium saucepan and heat over medium heat until steaming. Slowly drizzle 1/2 cup warm milk mixture into egg yolk, beating constantly until smooth.
Add egg yolk mixture and cornstarch-water mixture to saucepan and cook over medium heat, whisking occasionally until bubbly and thickened.
Remove pudding from heat and whisk in butter and vanilla extract. Allow pudding to cool slightly, then transfer to a bowl and press plastic wrap directly on surface. Allow to chill in refrigerator 30 minutes.
Once vanilla pudding has chilled, remove from refrigerator and gently fold in whipped cream.
Pour pudding mixture into popsicle molds and freeze 4 hours or overnight until popsicles are solid.
Place chocolate chips and coconut oil in a microwave-safe bowl. Microwave 1 minute on high, stirring at 15-second intervals until chocolate is smooth and melted.
Line a sheet pan with parchment paper. Remove popsicles from their molds and dip in melted chocolate mixture. Place popsicles in a single layer on sheet pan and immediately sprinkle with sprinkles. Freeze popsicles 10 minutes until chocolate coating is firm. Store popsicles in freezer until ready to serve. Enjoy!