In a medium bowl, whisk together the flours, baking powder and salt. Set aside
In the bowl of a stand mixer with the paddle attachment, mix together the butter and sugars on medium speed until light in color and fluffy, about 5 minutes. Add the eggs and vanilla and mix until smooth, about 1 minute more. With the mixer off, scrape the bottom and sides of the bowl with a spatula
With the mixer on low speed, add in ⅓ of the flour mixture and mix until incorporated, about 15-30 seconds. Combine the milk and oil and add half to the batter, mixing until smooth, about 30 seconds. Add another ⅓ of the flour mixture, mixing again for about 15-30 seconds, followed by the remaining milk/oil. Add the remaining flour and mix until completely smooth, about 30 seconds more, stopping to scrape the sides and bottom of bowl at least once to ensure all of the batter is evenly incorporated
Pour the cake batter over top the fruit and caramel, making sure it is evenly distributed. Smooth with a spatula
Place the cake pan on a baking sheet to catch any spillover, as the fruit and caramel tend to bubble up while baking. Bake for 40-45 minutes or until a toothpick inserted in the center comes out clean
Remove the cake from the baking sheet and place it on a cooling rack. Let the cake cool for just a minute, until the fruit stops bubbling. Place a cake plate (or cutting board) over the cake and, using oven mitts, carefully flip the cake over. Gently remove the cake pan and parchment, if it’s stuck to the cake.
Using a fork, rearrange any peaches that may be out of place or have fallen off the sides. Let the cake cool until still warm but not hot. Serve warm with whipped cream