Preheat the oven to 350 degrees fahrenheit and prepare a baking sheet with parchment paper and set aside.
In a mixing bowl, beat the butter and sugar together until combined.
Add the vanilla and lemon extracts and beat again.
Add the flour to the wet ingredients and mix together until a dough forms
Roll out balls of dough about 1.5 x 1.5 inch round and place on the baking sheet.
Slightly flatten each cookie with the palm of your hand.
Use your thumb to press an indentation in each cookie to fill with lemon curd filling.
Put the lemon curd filling in a piping bag and cut the tip. Fill each hole in the unbaked cookies with lemon curd filling.
Place in the preheated oven for 10-12 minutes or until the cookies are firm and barely browned.
While the cookies are in the oven, make the lemon icing by adding the powdered sugar, lemon juice and milk to a bowl and mixing together until smooth. Transfer the icing to a piping bag.
Remove the cookies from the oven and cool on a wire rack.
Snip the end of the lemon frosting piping bag and drizzle the glaze over all the cookies.
For extra lemony flavor, top with lemon zest.