- Holm et al (2023) use an RDD design to compare the test scores of Danish children whose parents divorce right after & right before the dates of national achievement tests. They estimate that divorce has an immediate -0.03 SD impact on test scores. (https://osf.io/preprints/socarxiv/p2qgk)
Thanks for these. I like the sound of the Kendler study. The other one sounds a bit harder to interpret: I guess it is estimating the immediate impact of divorce among families who are going to divorce anyway.
one good point I haven't often seen brought up (education was untypically uniform, though it probably still is in most places IMO).
the rest isn't particularly well supported by the evidence. read up on protzko's work on fadeout before you put your money on childhood interventions (here's a hint from the review you cite "the majority of trials were not preregistered and did not have a study protocol of prespecified outcomes to assess whether selective outcome reporting bias or selective analysis reporting bias was present").
as soon as you leave aside the silly position "parenting doesn't matter" (don't know if anyone really believes it) and contend with the nuanced position "parenting doesn't matter as much as most think" or even "parenting among well-adjusted and well-nourished westerners explains a small portion of variance for most outcomes, but not e.g. years of education, so relax about all the inconsistent silly advice you're getting on everything from babywearing to multivitamins" there's much less of a gap to explain between twin study behavior genetics and the small effects found in RCTs and adoption studies.
You also offhandedly mention the attachment literature. It hasn't been swallowed whole by beh gen not because it's such a different story, but because the literature has walled itself off from the broader discourse. Very insular, most studies simply ignore genetic confounding. For an attempt at a bridge, you can read https://psyarxiv.com/836md/
No, just that the whole theoretical rationale is built on sand.
The RCTs suffer from outcome switching and other manners of p hacking. The reason effects fade out may be because they actually fade but even simper, because you can't switch the outcome again for wave 3 after switching to it in wave 2, so there's less room to p hack. It hasn't dawned on anyone that soft skills are equally or more helpful in life (the data don't support it at all, intelligence reigns supreme), it's just that people have switched from primary to secondary outcomes a lot.
You say: It can be hard to extract advice from science.
Following that thought process, maybe accept the uncertainty that still have given the suboptimal quality of the work and all the contradictory results. Then, you're left with providing adequate care and perhaps less stressed out.
There's a lot of claims here that I think need more than a comment to back up. Have you got a reference for this specific set of RCTs having the problems you mention, or is this more of a general issue with work done before the past few years? I would certainly agree with the idea of accepting scientific uncertainty, though.
Re the RCTs: The systematic review you cite mentions lack of preregistration being widespread. Preregistration was invented in part to reduce outcome switching. For early childhood RCTs, I've heard specific tales about outcomes being switched that I found believable, but it's hard to conclusively show when they're not even preregistered.
in addition to Protzko maybe
D. Bailey, G. J. Duncan, C. L. Odgers, W. Yu, Persistence and Fadeout in the Impacts of Child and Adolescent Interventions. J. Res. Educ. Eff. 10, 7–39 (2017).
N. R. Kuncel, S. A. Hezlett, D. S. Ones, Academic performance, career potential, creativity, and job performance: can one construct predict them all? J. Pers. Soc. Psychol. 86, 148–161 (2004).
B. W. Roberts, N. R. Kuncel, R. Shiner, A. Caspi, L. R. Goldberg, The power of personality: The comparative validity of personality traits, socio-economic status, and cognitive ability for predicting important life outcomes. Perspect. Psychol. Sci. 2, 313–345 (2007).
N. R. Kuncel, R. J. Kochevar, D. S. Ones, A meta-analysis of letters of recommendation in college and graduate admissions: Reasons for hope. Int. J. Sel. Assess. 22, 101–107 (2014).
F. L. Schmidt, J. A. Shaffer, I.-S. Oh, Increased accuracy for range restriction corrections: Implications for the role of personality and general mental ability in job and training performance. Pers. Psychol. 61, 827–868 (2008).
Generally, if you haven't read it, I think the Genetic Lottery is clearer, and more nuanced than e.g. Plomin, but still quite skeptical of quick fixes from the RCT literature.
Mmm. I'm skeptical of the idea that education doesn't work in general. For particular instances it might be true, or it might work only via signalling.
A fast track of rendering this type of signaling moot is through online certification and requirements for "X years of experience in relevant field". The former is a return to preference + intelligence, and the latter LinkedIn status immobility + Guanxi "nepotism".
This post commits the Missing Effect Size fallacy, and it's a strawman. Shared environment typically accounts for 5-10% of the variance of a trait. Plomin doesn't deny this. You cite evidence like "Adopters made a difference in other ways too: for instance, mothers who drank were about 20% more likely to have an adoptive child who drank", but while you give us the tiny OR of 1.2, you fail to say how much the absolute difference in terms of overall variation is. For example, d here could be <0.10. You are hiding the fact that d is tiny to argue against a strawman that d=0.
I disagree that a 20% increase in the chance of drinking is small! I don’t recall how “drinking” was defined in the paper, but I would think it’s either a) serious or b) common. More generally, part of our point is that studies using other methodologies give higher estimates of effect sizes than twin studies, and Plomin ignores these.
A few additional studies lend credence to the idea that parenting matters:
- Kendler et al (2016) estimate large reductions in criminal offending using an adoption study w/co-sibling control (https://www.cambridge.org/core/journals/the-british-journal-of-psychiatry/article/criminal-offending-and-the-family-environment-swedish-national-highrisk-homereared-and-adoptedaway-cosibling-control-study/43B4EAE8189993694B50CB33F3AA9926)
- Holm et al (2023) use an RDD design to compare the test scores of Danish children whose parents divorce right after & right before the dates of national achievement tests. They estimate that divorce has an immediate -0.03 SD impact on test scores. (https://osf.io/preprints/socarxiv/p2qgk)
Thanks for these. I like the sound of the Kendler study. The other one sounds a bit harder to interpret: I guess it is estimating the immediate impact of divorce among families who are going to divorce anyway.
one good point I haven't often seen brought up (education was untypically uniform, though it probably still is in most places IMO).
the rest isn't particularly well supported by the evidence. read up on protzko's work on fadeout before you put your money on childhood interventions (here's a hint from the review you cite "the majority of trials were not preregistered and did not have a study protocol of prespecified outcomes to assess whether selective outcome reporting bias or selective analysis reporting bias was present").
as soon as you leave aside the silly position "parenting doesn't matter" (don't know if anyone really believes it) and contend with the nuanced position "parenting doesn't matter as much as most think" or even "parenting among well-adjusted and well-nourished westerners explains a small portion of variance for most outcomes, but not e.g. years of education, so relax about all the inconsistent silly advice you're getting on everything from babywearing to multivitamins" there's much less of a gap to explain between twin study behavior genetics and the small effects found in RCTs and adoption studies.
Thank you for the hint about Protzko.
You also offhandedly mention the attachment literature. It hasn't been swallowed whole by beh gen not because it's such a different story, but because the literature has walled itself off from the broader discourse. Very insular, most studies simply ignore genetic confounding. For an attempt at a bridge, you can read https://psyarxiv.com/836md/
Right, but we mention RCTs in that context. There's no reason to think they suffer from genetic confounding.
No, just that the whole theoretical rationale is built on sand.
The RCTs suffer from outcome switching and other manners of p hacking. The reason effects fade out may be because they actually fade but even simper, because you can't switch the outcome again for wave 3 after switching to it in wave 2, so there's less room to p hack. It hasn't dawned on anyone that soft skills are equally or more helpful in life (the data don't support it at all, intelligence reigns supreme), it's just that people have switched from primary to secondary outcomes a lot.
You say: It can be hard to extract advice from science.
Following that thought process, maybe accept the uncertainty that still have given the suboptimal quality of the work and all the contradictory results. Then, you're left with providing adequate care and perhaps less stressed out.
There's a lot of claims here that I think need more than a comment to back up. Have you got a reference for this specific set of RCTs having the problems you mention, or is this more of a general issue with work done before the past few years? I would certainly agree with the idea of accepting scientific uncertainty, though.
Re the RCTs: The systematic review you cite mentions lack of preregistration being widespread. Preregistration was invented in part to reduce outcome switching. For early childhood RCTs, I've heard specific tales about outcomes being switched that I found believable, but it's hard to conclusively show when they're not even preregistered.
in addition to Protzko maybe
D. Bailey, G. J. Duncan, C. L. Odgers, W. Yu, Persistence and Fadeout in the Impacts of Child and Adolescent Interventions. J. Res. Educ. Eff. 10, 7–39 (2017).
or check out one of the trials you liked here https://evidencebasedprograms.org/
Re predictive validity of IQ >> soft skills:
N. R. Kuncel, S. A. Hezlett, D. S. Ones, Academic performance, career potential, creativity, and job performance: can one construct predict them all? J. Pers. Soc. Psychol. 86, 148–161 (2004).
B. W. Roberts, N. R. Kuncel, R. Shiner, A. Caspi, L. R. Goldberg, The power of personality: The comparative validity of personality traits, socio-economic status, and cognitive ability for predicting important life outcomes. Perspect. Psychol. Sci. 2, 313–345 (2007).
N. R. Kuncel, R. J. Kochevar, D. S. Ones, A meta-analysis of letters of recommendation in college and graduate admissions: Reasons for hope. Int. J. Sel. Assess. 22, 101–107 (2014).
F. L. Schmidt, J. A. Shaffer, I.-S. Oh, Increased accuracy for range restriction corrections: Implications for the role of personality and general mental ability in job and training performance. Pers. Psychol. 61, 827–868 (2008).
Generally, if you haven't read it, I think the Genetic Lottery is clearer, and more nuanced than e.g. Plomin, but still quite skeptical of quick fixes from the RCT literature.
Counter conjecture: parenting (emotional care & socialization) matters, but parental "investment" (education & "prep") does not. Discourse about the former often devolve into arguing about the latter, as (a) genetics are a stronger constant to achievement than merit, (b) The Cult of Smart needs educational assistance "equal opportunity" to disguise oneself against scrutiny, (c) aristocratic education is only reserved for Old Money, (d) anything genetic correlate or direct parent is rhetorically reinterpreted as equivalent vices through "keeping it in the family". https://astralcodexten.substack.com/p/non-cognitive-skills-for-educational https://freddiedeboer.substack.com/p/education-doesnt-work https://erikhoel.substack.com/p/why-we-stopped-making-einsteins
Mmm. I'm skeptical of the idea that education doesn't work in general. For particular instances it might be true, or it might work only via signalling.
I think that any magical "signaling" fails one standardized tests and class environment is controlled for.
Upper class status cannot be faked, at best the working class can fake their way into being middle class (professional adjacent), and the professional class proper can fake their way into being "rich". The UMC are the most economic fragile, culturally superficial people there is (by obligation rather than choice), and James F Richardson has paywalled a lot of relevant material on the matter. https://bowtiedbull.substack.com/p/going-through-levels-of-wealth https://jasonstanford.substack.com/p/umc-entitlement-creep https://annehelen.substack.com/p/the-anxious-style-of-american-parenting
Maybe honest signaling with SATs and fluid intelligence tests are useful. But "Sounding smart" signaling has a wordcel-like effect, where crystalline intelligence and covert neuroticism can be spotted once someone has worked in a project for 3+ years. This is why "midwit" liberal arts degrees exist as status inflation. Peter Turchin would like a word on the social instability that can cause. https://kirkegaard.substack.com/p/the-verbal-tilt-model https://maxread.substack.com/p/wordcels-and-shape-rotators-in-four https://roonscape.substack.com/p/a-song-of-shapes-and-words https://cactus.substack.com/p/its-the-midwits-stupid https://archive.ph/6dHdy https://archive.ph/sL3zb https://archive.ph/zmwI7
A fast track of rendering this type of signaling moot is through online certification and requirements for "X years of experience in relevant field". The former is a return to preference + intelligence, and the latter LinkedIn status immobility + Guanxi "nepotism".
This post commits the Missing Effect Size fallacy, and it's a strawman. Shared environment typically accounts for 5-10% of the variance of a trait. Plomin doesn't deny this. You cite evidence like "Adopters made a difference in other ways too: for instance, mothers who drank were about 20% more likely to have an adoptive child who drank", but while you give us the tiny OR of 1.2, you fail to say how much the absolute difference in terms of overall variation is. For example, d here could be <0.10. You are hiding the fact that d is tiny to argue against a strawman that d=0.
I disagree that a 20% increase in the chance of drinking is small! I don’t recall how “drinking” was defined in the paper, but I would think it’s either a) serious or b) common. More generally, part of our point is that studies using other methodologies give higher estimates of effect sizes than twin studies, and Plomin ignores these.
Nature or Nurture . . . . whatever . . .