Preheat the oven to 400 degrees F. Slice the potatoes in half lengthwise and bake, cut side down, on a baking sheet for about 45 minutes, until soft. Remove from the oven.
Saute the garlic and pepper flakes in the oil for 1 minute. Add the broccoli and water, cover, and simmer for about 15 minutes.
When the potato halves are cool, scoop out the centers, leaving 1/4 inch of pulp on the skin. Mash the scooped-out potato pulp and stir it into the stuffing. Add salt and pepper. Refill the potato skins and bake for 30 minutes. Sprinkle with sliced olives and serve.