Preheat the oven to 350°F. Line 2 standard-size muffin pans with cupcake liners. Light spray the liners with nonstick spray. Set them aside.
Add the brownie mix to a medium-sized mixing bowl.
Add in the oil, water, and the beaten eggs. Stir just until combined. Do not overmix.
Place 1 peanut butter cup in the bottom of each cupcake liner.
Spoon 1½ tablespoons of the brownie mix on top of the peanut butter cup.
Bake for 20 minutes. Allow the cupcakes to cool completely before frosting.
Using either a stand mixer or a medium-sized mixing bowl and a handheld mixer on medium-high speed, beat together the butter, peanut butter, and vanilla extract for 1½ to 2 minutes, until smooth.
Add the powdered sugar, ½ cup at a time, alternating with the milk until completely incorporated and smooth.
Once the cupcakes are completely cooled, fill a decorator’s bag or piping bag fitted with an open star tip or a gallon-size Ziploc with a corner snipped off with frosting.
To frost the cupcakes, hold the decorator’s bag about ½ inch from the center surface of the cupcake. Use even pressure, and squeeze to form the center of the star-swirl. Keeping the tip elevated and using the same pressure, follow the outer line of the star-swirl, making a C shape around the star-swirl until the outer swirl lines connect. Repeat for the remaining cupcakes.
Sprinkle a few of Reese’s mini unwrapped candies over the frosted cupcakes.