Melt butter in a small pot over medium-high heat. Remove pot from heat, add lavender, cover, and set aside to let infuse for 30 minutes. Warm butter again and strain through a fine sieve, pressing lavender with the back of a spoon to extract as much as possible; discard solids. Set lavender butter aside.
Arrange 14 of the apple halves vertically around the sides of a 9" cast-iron skillet, all facing in the same direction, nesting them as tightly as possible. Arrange remaining apples halves in center of skillet, rounded side down. Brush apples evenly with reserved lavender butter; sprinkle all over with sugar. Cook over medium-low heat, without stirring, until apples are deep brown on the bottom and softened about halfway up, about 1 1⁄2 hours. Let cool for 30 minutes.
Meanwhile, heat oven to 425°. Prick puff pastry all over with a fork and arrange on top of apples, punctured side down; tuck edges in around apples. Bake until golden brown all over, about 15 minutes. Let rest for 5 minutes, then run a knife around the edge. Place a large plate over the skillet; invert tart onto the plate. Garnish with lavender sprigs and serve with vanilla ice cream, if you like.