I semi-recently read Dr. Walter Schumm’s book Same-Sex Parenting Resarch: A Critical Assessment. My thoughts? Meh.
There are a few problems I had in this book. The first is not really a data-driven issue but rather the way Schumm cites research. See, Schumm has done a lot of research on the topic of same-sex parenting and so he has a lot of reviews of the literature out there. So, when he cites a study he’s previously talked about in a journal article, he cites not just the study but also the paper he reviewed it in. Why? I’d understand maybe if it was along the lines of “also see Schumm x” and maybe he re-analyzed the data but it regularly just goes like this:
As noted earlier, Sarantakos (1998) found that 55% of the gay men in his study reported they were a woman in a man’s body while 45% of the lesbians reported being a man in a woman’s body (Schumm, 2015b, p. 7).p. 153
There are other areas where it is more intrusive than this. I just see no reason for doing it. While we are on the topic of syntax and grammar, it does seem like Schumm didn’t have much of an editor. There are regularly run-on sentences or sentences which clearly needed to be revised. But, moving on: how’s the data argumentation in this book?
It’s okay. Schumm’s main point seems to be that the authors of studies on same-sex parenting abuse p values and ignore the relevance of Cohen’s d. When he looks at the Cohen’s d of differences between kids raised by same-sex parents compared to those raised by opposite-sex parents, he finds the differences are regularly clinically relevant, regardless of the p value. This is kind of the old issue of p values. There can be substantial correlations which still appear relevant but they are still not great enough to not be reduced to random chance.
What Schumm doesn’t talk about is why these studies get bad p values in the first place: tiny samples. It’s very difficult to find large, representative samples on this topic; very few studies seem to. Many of the studies both he and the people he criticizes cite are n<100. Really not great for observational data. Furthermore, he makes little mention of the issue of convenience sampling which seems to afflict a large proportion of studies on same-sex parenting. This is a huge burden to same-sex parenting studies as convenience samples are useless.
Another thing I took note of was Schumm’s usage of a one-tailed significance test at one point. His defense of this was in the notes:
Blanchard and VanderLaan (2015, p. 1504) have discussed the usefulness of one-tailed statistical tests when theory of the preponderance of research suggest a direction to the expected effects of the test involved.p. 138, nn. 3
Not the greatest defense, especially if Schumm uses two-tailed statistical tests elsewhere. Just seems like p-hacking. Other than these issues, I still see this book as a useful contribution to the current research trend on the topic. Would like to see some better research done.
Anyways, this post is going to be a response to one of the chapters in the book where Schumm uses longitudinal data to calculate the effect of same-sex marriage on fertility rates. I oppose his thesis on three grounds. 1) His own data does not support the claim. 2) It doesn’t even make sense theoretically. 3) Lack of control for confounding variables.
Here’s what Schumm does. He conducts simultaneous regression between the amt. of years since same sex marriage was legalized in a state, fertility rate, age at marriage, percent urban, median income, and women’s education. He also shows the model controlling for the fertility rate in the year 2000. Then he shows the regression output:
Technically the Cohen’s d was moderately sized, yes, but the effect size was still statistically insignificant at p<0.10. Yet, Schumm writes,
These results indicate that fertility rates may be influenced by changes in same-sex marriage law over time.p. 290
But, we really can’t generalize off of one statistically insignificant finding. Second of all, as I said, it really doesn’t make much sense theoretically. Allowing gay people to get married isn’t going to limit who gay people have sex with. If they’re homosexual, they’ll continue to have sex with people of the same sex no matter what. Schumm even seems to concede the possibility of reverse causality as well,
An argument can be made that our model reverses the causality – that fertility predicts changes in law or earlier age at marriage.p. 290
Different hypothesis: liberalism predicts lower fertility rates and earlier legalization of same sex marriage. Areas that are more liberal will also probably decline in fertility rates faster than areas which maintain conservative ideals. So, there is some uncontrolled confounding. And considering the results already weren’t statistically significant, I find it unlikely Schumm’s model would hold up against controlling for another confounding variable.