To roast plums: Preheat oven to 400 degrees F. Place plums and 2 rosemary sprigs in an 8-inch-square baking dish. Whisk water, vinegar, 4 tablespoons sugar and peppercorns in a small bowl until the sugar dissolves. Scrape seeds from vanilla bean; add the seeds and bean to the vinegar mixture. Pour the mixture over the plums. Sprinkle with the remaining 2 tablespoons sugar.

Roast the plums, uncovered, until tender and beginning to break down, 20 to 25 minutes. Discard the rosemary and the vanilla bean. Transfer the plums to a serving platter and cover with foil. Strain the roasting liquid into a small saucepan and bring to a boil. Reduce heat to medium-high; cook until reduced to 1/2 cup, 6 to 8 minutes. Pour the sauce over the plums; keep warm.

To prepare pork: Meanwhile, heat oil in a large ovenproof skillet over medium-high heat. Sprinkle pork with pepper and salt. Add to the skillet and brown on all sides, 5 to 8 minutes.

Transfer the pan to the oven; bake at 400 degrees until an instant-read thermometer registers 155 degrees and the pork has just a hint of pink in the center, 10 to 15 minutes. Transfer the pork to a cutting board and let rest for 10 minutes. (The internal temperature will increase to 160 degrees during resting.) Cut the pork into thin slices and serve with the roasted plums.

Substitution Tip: You can use 1/4 teaspoon vanilla extract instead of the vanilla bean.

MAKE AHEAD TIP: Cover and refrigerate the roasted plums for up to 2 days.