Slice oranges and lemons. Reserve 1/2 of slices for garnish.
In large saucepan, over high heat, bring apple cider, water, tea bags, remaining orange slices and lemon slices, whole allspice, cloves, and cinnamon sticks to a boil. Reduce heat and simmer 3 minutes.
Remove tea bags, orange and lemon slices, allspice, and cloves. Simmer additional 5 minutes. Stir in ground allspice.
Garnish with fresh slices of orange and lemon, if desired. Serve hot or chilled.