For the stewed quince: Using a vegetable peeler, peel the zest from the orange and lemon in strips and cut the strips lengthwise into a fine julienne. In a medium saucepan, combine the julienned zests with the quince, wine, sugar, salt, cinnamon, cloves, rosemary, star anise, and ginger. Juice the lemon and add the juice to the pan along with 2 cups water. Bring the syrup to a boil, and then reduce the heat to maintain a slow simmer. Cook until a paring knife inserted through the middle of each quince comes out easily and the cooking liquid is reduced to a loose glaze, about 1 1⁄2 hours. Remove from the heat and keep warm.
For the shallot sauce: In a 12-inch skillet, heat 1 tablespoon of the olive oil over medium. Add the shallots, 1 bay leaf, 1 rosemary sprig, 1 thyme sprig, and the garlic and cook, stirring, until the shallots start to caramelize, 6 to 8 minutes. Stir in the stock and cook until reduced by half, about 5 minutes. Remove from the heat and stir in the cream and 2 1⁄2 tablespoons butter. Discard the bay leaves and herb stems and purée the sauce in a blender until smooth. Scrape into a bowl and season with kosher salt and pepper.
In a 12-inch skillet, heat the remaining 2 tablespoons olive oil over high. Season the venison with salt and pepper and add to the skillet. Cook the loins, turning, until browned all over, about 6 minutes. Add the remaining 5 1⁄2 tablespoons butter, rosemary and thyme sprigs, bay leaf, and the orange zest and cook, spooning the melted butter over the loins, until medium-rare, about 10 minutes (or until an instant-read thermometer inserted into the thickest part of the loins reads 140°)
Transfer the loins to a cutting board and let stand for 10 minutes to rest. Using a slotted spoon, lift the quince and transfer to a cutting board. Remove the cores, and then cut each quarter into 4 wedges. Place the quince in a serving bowl and pour its cooking liquid over top, discarding the herbs and spices. Cut the loins across the grain into 1⁄2-inch-thick slices and arrange on a serving platter. Season the venison with flaky sea salt and pepper and serve alongside the quince and shallot sauce.