Preheat oven to 350ºF. Pile pears into an 8x8" glass baking dish and toss with the brown sugar and vanilla. In a medium bowl, combine the topping ingredients using your fingers to squeeze it all together until no dry flour remains. Bake in centre of oven until filling is bubbling at the edges, about 50 minutes, tenting with aluminum foil if topping is browning too quickly.

To make custard sauce, stir milk and sugar in a saucepan over medium heat.  When bubbles start to form at the edges, slowly pour a stream of about 1/2 a cup into the beaten egg yolks, whisking constantly.  Gradually whisk the yolk mixture into remaining hot milk and return to medium-low heat until it thickens enough to coat the back of a spoon, about 5 minutes.  Strain through a fine-mesh sieve into a bowl set within an ice bath to cool a bit, then stir in vanilla extract. You can serve it warm or cold.