Make the cake: Heat oven to 350°. Butter and flour two 9" round cake pans and set aside. Whisk flour, baking powder, and salt in a bowl; set aside. Whisk milk and vanilla in a bowl; set aside. In the bowl of a stand mixer fitted with a paddle, cream butter, sugar, zest, and juice on medium-high speed until fluffy, about 3 minutes. Add eggs, one at a time, beating well after each addition. On low speed, alternately add dry ingredients in 3 batches and wet ingredients in 2 batches. Increase speed to high and beat until batter is smooth, about 5 seconds. Divide batter between prepared pans, and smooth tops with a rubber spatula; drop pans lightly on a counter to expel any large air bubbles. Bake until a toothpick inserted in middle comes out clean, 25-30 minutes. Let cakes cool for 20 minutes in pans; invert onto wire racks, and let cool to room temperature. Using a serrated knife, halve each cake horizontally to produce four layers.
Make the curd filling: Whisk sugar, zest, juice, and eggs in a 4-qt. saucepan until smooth. Place saucepan over medium heat; cook, stirring, until mixture thickens to the consistency of loose pudding, about 10 minutes. Remove from heat and slowly whisk in butter until melted and smooth. Pour curd through a fine-mesh strainer into a bowl and chill until ready to use.
To assemble, place one cake layer on a cake stand and spread with 1⁄3 cup curd; repeat with remaining layers and curd, ending with a cake layer. Cover top and sides with 3⁄4 of the White Chocolate Buttercream. Place remaining buttercream in a pastry bag fitted with a plain 1⁄2" tip; pipe 1" mounds along the top edge of cake. Mix graham crackers and melted butter in a bowl and sprinkle along bottom edge of cake. Garnish the top of the cake with candied lemon slices, if you like, and chill cake to firm frosting. Serve at room temperature.