I feel like some of this needs splitting up and clarifying.
JH does seem to have stood by EG prior to DM coming on the scene. He seems to have been in disbelief that EG was involved. That may or may not have been in combination with episodes of "what if she did..."
But after DM and the discovery of Lucas' body, JH has not explicitly stated support for EG. He has not said that he buys the story of her just coming upon a natural death scene and panicking. He has said he doesn't understand it, and I think he's said that he wanted to find out the true story (paraphrasing, but I think a fair paraphrase).
His behavior, both before and after, has made people get suspicious feelings. But I think we need to separate out the before and after, and be clear on where he's been explicit and where it's more about our impressions and conclusions based on his words or behavior.
For instance, I never heard him say that she "only" moved the body, therefore that's okay. We might think he's saying that by allowing her to stay in the house, but I don't think it's a given, only a possibility from personal interpretation of baffling behavior. And that possibility does go against his own words....which might not be 100% forthcoming. He might have more than one reason for letting her stay there.
He might not have the time and space to explain all his feelings and thoughts in a quick answer to NG that's going out to thousands of strangers to dissect his every word. I think that part might be in part why some of his attitude sounds distanced and a bit cold. I don't want to be sexist, but this exists in some women too, but a lot of men feel that emotions should be kept inside and not made too public, and I think a lot of boys are brought up that way (boys don't cry), so there might be a bit of that in the way he comes over and appears relatively calm and collected on the outside even when he's talking about finding EG in that condition that night or about his son being found. I think these things have affected him far deeper on the inside than he's really letting out to NG on the radio.
Regarding EG going back to the house on Edgemoor and JH saying that her relatives no longer wanted her back after she led DM to Lucas' body.... Do you remember that after EG was released the final time that the news reporters said they followed the car with EG and her attorney to the offices of the public defenders? Could that have been to discuss where she was going to stay? Could someone have phoned JH and said she really doesn't have anywhere to stay, is there any chance she can stay in the house in Edgemoor if you don't want to stay there...if not we're afraid she'll go on the streets, dangers for her there....and then when JH is asked he explains it that he didn't want her to end up on the street and she might as well stay there as he wasn't really living there?