When I started messing with cakes, I had the very same problem, and actually thought the very same things you addressed were the issue, but after much

I reached out to those in the know!
Many of you here know Laurie Biere (LPBiere) form "that other" forum, and she gave me a piece of advice that made all the difference in the world. When she was in pastry school, she said they were taught to not rely on "timers" when baking a cake. She told me that once its in the oven, when you can smell it, you know its time to start watching it. Keep an eye on the sides-when it starts to pull away, its getting close, and to test for doneness, lightly press it in the middle with your finger. When it springs back, its done-take it out of the oven!!
I know the toothpick method works, but I just hate poking holes in my cakes!
Since that day, I have never even used the timer. I'll get an idea of how long it should back from the recipe, but I am always checking it around the halfway point. Even the best ovens temps can vary, and the type of pans used etc all make a difference. A good example of this is that two days ago I made a loaf of bannana bread for my grandson, and the recipe called for baking it @350 degrees for 55-75 minutes. It was done perfectly in 43 minutes! So it just goes to show you.....
Also, another tip Laurie gave me was that when baking cakes, seek out "professional recipes" you can recognize them because quantities of ingredients are always given in weights (grams/ounces as opposed to cups etc) digital kitchen scales are relatively cheap-under 20$, and they are useful for so many things in the kitchen, so they really are "multi-taskers"!