Cut slits in the surface of chestnuts with a sharp knife. Place chestnuts in a medium saucepan with enough water to cover and bring to a boil. Cook until tender, about 25 minutes. Drain, peel, and chop.
Soak fruit in brandy.
Preheat the oven to 350 degrees.
Melt butter in a medium saucepan over medium heat. Stir in chestnuts, onion, herbs, salt, and pepper, and cook until onions are tender; add chopped fruit and brandy and cook about 3 minutes more.
Transfer to a medium baking dish and mix with cubed bread.
Beat together eggs and milk in a small bowl. Drizzle over cubed bread mixture. Bake until the surface is crisp and lightly browned, 30 to 45 minutes.