I think it's a little brash, but I do think that if we could accurately measure "care" that people with 1 child care statistically slightly more for them than those with 5 might care about any individual one. Not that they don't care, but that the one with 1 would worry more. Like how families with 1 daughter and multiple boys might dot on the only daughter a bit more.
I mean historically this is exactly how it worked. When infant mortality was 25-50%, people often had 10 children or so. Maybe 5 would make it to be adults, but people back then didn't view death like we view it now. Sure, they mourned, but losing a child was not a "stop the world" event, because you could expect to lose many. So, it couldn't be.
That was a little over 100 years go. So, a long time, but not that long.