Place butter, milk and cream in a medium saucepan. Bring to a low simmer over medium high heat, whisking occasionally.
Meanwhile, in a small bowl, whisk agave, eggs, and cornstarch until smooth. Whisk in a 4 tablespoons hot milk mixture, 1 tablespoon at a time until smooth, then add all back into pan. Whisk occasionally over medium heat until pudding starts to thicken and coats the back of a spoon. Remove from heat and cool slightly.
Fill 6 popsicle molds with vanilla pudding and insert popsicle stick in center. Freeze popsicles until completely solid, about 5-6 hours.
Once popsicles are frozen, melt chocolate in a double-boiler until smooth. Remove popsicles from mold and quickly dip the end of each popsicle in melted chocolate. Set popsicles on wax paper and return to freezer to set for 5-10 minutes. Store popsicle in freezer until ready to serve.