Eric Chyn on the Impacts of Removing Children from Abusive or Neglectful Homes

Chancellor: Hello and thanks for joining us for the Poverty Research and Policy Podcast from the Institute for Research on Poverty at the University of Wisconsin–Madison. I’m Dave Chancellor. For this episode we’re going to be hearing from an economist named Eric Chyn about the impact of home removal—for reasons like neglect or abuse—on children’s later outcomes. Dr. Chyn is an assistant professor of economics at Dartmouth College and a faculty research fellow at the National Bureau of Economic Research or NBER. And the paper he’s going to be talking about is an NBER working paper—number 25419—called the Causal Impact of Removing Children from Abusive and Neglectful Homes, which he cowrote with Anthony Bald, Justine Hastings, and Margarita Machelett. And their main finding, for me at least, was really surprising. But before we get there, when we started talking, I asked him to give us a basic overview of what home removal is as a policy and what we know about it from past research.

Chyn: What we need to know about this is that in the United States every year, child protection agencies investigate about four million children associated with cases of abuse and neglect and as a result of those investigations, about 200,000 children per year are removed from home. And despite the prevalence of this policy of removing kids from home, we have relatively little research that tries to understand the causal impact of removing kids from their families. And so this work is amongst a few studies that really tries to take seriously, trying to get at causal effects of home removal. And we use administrative data to rigorously look at a number of different outcomes and see how children are faring many years after the removal is actually happening.

Chancellor: I asked Professor Chyn, for someone who is not a social scientist or doesn’t have that background, what does he mean when he talks about causal impacts or causal effects?

Chyn: The key obstacle that we’re often confronted with in social science is, ‘I want to study some program or policy and the problem I face is I see people who do go to college and don’t go to college. And I don’t like making comparisons across those two different groups of people. Like, maybe people who go to college, you see them later on in life and they earn more money, but that may not actually be the causal impact of going to college. So, if I look at those people who went to college and earned more money, maybe they’re just different in fundamental ways relative to someone who doesn’t go to college. Maybe the comparison between someone who does and does not go to college doesn’t necessarily reveal what is the true causal impact of going to college. So, similarly, in our context here, our concern is that children who are removed from their home versus some other child who is not removed from their home, we’re very concerned that a comparison of outcomes of those two groups—so let’s say on test scores or something like this, children who are removed are going to do worse on test scores relative to a child who is still in school and is never removed from their home because the child who is removed from their home is much more likely to disadvantaged status on a number of different characteristics. They might come from a household that has lower income, they come from a worse neighborhood, all those sorts of factors. So what we don’t want to do is confound removal with all of these differences that make children who are removed different from children who are not removed. And so when we’re getting at causal impacts, what we really want to do is come up with a comparison that actually does make a really compelling comparison between a child who is removed and another child who is not removed, but is similar in all other ways except that they’re not removed. And so in this work we’ve spent a lot of time trying to come up with a research design, research strategy, that allows us to form comparisons that really are compelling in that way.

Chancellor: So, the way that Professor Chyn and his colleagues do this in this paper is to compare kids who are similar in all ways except for the fact that they were removed from their home.

Chyn: And the way that we try to approach this in this paper to get at the causal impact is we create a research design where we fundamentally leverage the fact that in the child protection system, investigators differ in their tendency to remove children. We use that in the technical speak as an instrument to try to get around this selection bias problem. And basically fundamentally what we’re doing is we’re relying on comparisons between children who are all being investigated by child protective services but some children are removed from home because they’re seeing a caseworker who is a little bit more strict, they tend to remove children a little bit more frequently relative to another caseworker that removes children less frequently. And we rely on that variation across different caseworkers to make comparisons across children who are otherwise equivalent, except for the fact that the caseworkers themselves differ in the likelihood of being removed. And so we have a child who is very similar to another child but one sees the strict investigator and they are subsequently removed. That’s sort of the basis of the variation that we end up relying on to try to get at a really credible comparison across children and looking at the impact of home removal.

Chancellor: To date, a lot of the research that has looked at home removal has been based on datasets with older children, but for this study, Professor Chyn and his colleagues are able to look at younger kids.

Chyn: In our dataset, we’re actually going to see children who are less than age 6 who are being removed from their home. And this is a very policy relevant group where almost half of all children who are removed from their home nationally in instances of abuse and neglect are below age 6, so we’re really studying a group of children that are very policy relevant, and relative to prior research, we really don’t have a good sense of how home removal may impact their lives. And there’s a lot of child development literature and other work by economists that also suggest that programs or interventions that happen earlier in life may have distinct and large benefits for children in ways that are different from programs that may have been administered later on in life. So early life is a period of time that we really want to think about learning something about different programs and policies and how they’re operating. What we’re going to do be doing is using some new data that does have information on young children who are removed and we take data on these kids who are being removed and are otherwise being investigated for instances of child maltreatment and we’re going to link all of this data to a number of other administrative sources, school records, etcetera, to study things like test scores, attendance in school. Whether or not children are receiving special education and whether or not they are proceeding on time with grades and not being held back. So we’re going to study all of these different schooling outcomes for these young kids and try to use those as what we’re looking at in terms of the metric for when we’re evaluating whether or not home removal is having a positive or negative impact on these children’s lives.

Chancellor: As Chyn mentioned, in this paper they are studying instances in which children are being removed from their homes when they’re pretty young—under age 6. And the simple timeline of this is that after removal, they spend some time outside of their home in some sort of a foster care environment—Professor Chyn says a bit over a year on average for the children in their dataset—and then they eventually get reunited with their parents.

Chyn: And where we pick up the story and we look at impacts is we study schooling outcomes, so this is after the removal has happened and they’ve already been reunited with their parents for the most part. And when we’re looking at these test scores, it’s many years after the removal has happened and we’re studying these test scores and seeing whether or not, ok, this temporary sort of separation that you have with parents, did it seem like that had some beneficial impacts on young kids. And we’re going to separate girls and boys because there is some literature that suggests that there are gender differences in the responses to different types of interventions. There has been some other prior research that has found some heterogeneity along that dimension. So when we take the data, we’re going to look at boys and girls separately, and we see that home removal seems to have large positive benefits, improving schooling outcomes for girls, improving their test scores, lowering the likelihood that they’re repeating grades and we see those benefits for young girls, we don’t see any corresponding evidence that there are similar benefits for young boys. So it seems like girls are responding in a very different way relative to their counterpart young boys who are also being removed from home.

Chancellor: Professor Chyn says that one of the ways they try to understand these differing results for girls and boys is to make sure that they’re comparing kids who are really similar and from similar situations. One of the ways they can do this is by looking at siblings.

Chyn: So, they come from the same household and in all likelihood they have the same type of foster care experience. They go to the same foster care household together and so by focusing on these siblings, what we can do is try to look at whether or not brothers and sisters, young brothers and sisters who are being removed, do they seem to have different responses to home removal? And in this subanalysis which is a supplementary analysis to what we do in the main part of the paper, what we end up finding is that it looks very much similar to our main results in that we see that young girls who are being removed seem to be doing much, much better in areas of achievement and other schooling outcomes and then their young brothers who are coming from the same household, they don’t seem to be benefiting the same way and so this is a piece of evidence that we try to include in our analysis to try to try to really, really narrow the scope of possible stories that you might have in mind about why girls and boys might be differentially responding to the treatment. So we’re here, we’re looking at the same households who have similar foster care experiences and we’re still seeing the same pattern that we see in our main results so this rules out a number of other possible stories that one could have in mind like differences in the types of foster care placements, that’s not what’s going on because these children are overwhelmingly having the same sort of foster care experiences when they’re coming from the same home.

Chancellor: And these different results for girls and boys is something that Professor Chyn says he really wants to learn more about.

Chyn: One of the outstanding questions that I think is open for future work by myself and other people who are interested in child maltreatment literature is to do more investigations along the lines of looking at, heterogeneity in different interventions for abused and neglected girls versus boys and trying to understand, dig deeper into whether or not girls and boys are responding differently to the different array of policies that we could imagine trying to implement for children in these very special circumstances in this child maltreatment world, so trying to understand more about this gender heterogeneity is certainly the next thing that we need much more research on and the work that we have done is just to provide some new evidence that brings up the question for future research to try to take this on in much more detail and understanding about why you would see these differences for girls versus boys.

Chancellor: Chyn says that a main takeaway from this work is an underscoring of the importance of early life interventions.

Chyn: And so a lot of literature outside of the child maltreatment context has studied different types of enrichment and education programs that are delivered early in life for children seeing very robust evidence that those type of programs really, really when delivered early in life have a lot of beneficial impacts. We see a similar theme in our own results in that we see really positive benefits for doing home removal early in life for children and that this seems to be really be improving school outcomes for these young girls. In our own paper when we try to look at older children , which we don’t spend a lot of time talking about, but if you try to look at older girls who are being removed from their home, you don’t see any positive benefits for older girls. So this theme in our work that early life home removal matters for young girls, that sort of corresponds to this broader literature talking about the importance of early life interventions. So that’s been a key theme and contribution that we have that we’re trying to contribute more to the literature on early child life interventions and we’re innovating relative to this literature by looking at this in this specific context of child maltreatment and this very prevalent intervention that’s being done for children at risk of child maltreatment which is removing them from their homes.

Chancellor: Thanks to Eric Chyn for sharing this work with us. Again, if you would like to read their whole paper it is NBER working paper number 25419. This podcast was supported as part of a grant from the U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation but its contents don’t necessarily represent the opinions or policies of that Office, any other agency of the Federal government, or the Institute for Research on Poverty. To catch new episodes of the Poverty Research and Policy Podcast, you can subscribe on Apple, Stitcher, or Google Play Podcasts. You can also find all of our past episodes on the Institute for Research on Poverty website. Our theme music for this episode is “Staring Straight” by Maarten De Boer. Thanks for listening.