Hi, just wanted to update this post since there is finally a specific baby equipment rental service in Iceland.
You can rent strollers, cots, carriers, high chairs, car seats and more. The name of the company is MiniRent. You can check out their website http://www.minirent.is.