Hi Katy!
Plenty of locations in the Golden Circle and South Coast are accessible with a push chair. (By the way, I assume you’re from the UK and that a push chair is what we call a stroller in the United States.)
But, locations are not “child proofed” for young kids to visit and run around. For example, at the Geysir geothermal area, there is at most a low rope separating you from very very hot water. Here’s a sample image:
That’s right next to the walkway; if your children are likely to run off and explore, you’ll need to keep hold of them, or keep them in the stroller!
Seljalandsfoss and Skogafoss are both very visible from the parking lots, and so you won’t have trouble getting close to them, with or without strollers. But you can walk behind Seljalandsfoss, and it’s a slippery path that again isn’t child proofed. Iceland expects you to be responsible for yourself (and in this case your kids!)
So, you’ll be able to get to most everything on the Golden Circle in a stroller; park in the lower parking lot at Gullfoss to avoid the huge set of stairs. But you won’t want to let the kids run free!
Hope that helps.