Combine condensed milk and chocolate in a medium saucepan over low heat. Cook, stirring constantly, until chocolate is just melted.
Stir in most of the chopped candy, reserving about 2 tablespoons for topping.
Transfer the chocolate mixture into a parchment-lined loaf pan using a flexible spatula to press it in evenly. Top fudge with reserved candy and press it gently to adhere. Chill until firm before slicing, about 30 minutes.