What hinders Joe from working with Black Beauty and Ginger in Anna Sewell's Black Beauty?

In Anna Sewell's Black Beauty, Little Joe Green is hired as the stable boy to replace James Howard once John Manly, the coachman at Birtwick Park, recommends James as head groom at a different estate. Once Joe is hired, except for the time he accidentally makes Black Beauty seriously ill, Joe is kept on as the stable boy and not hindered at all from workingwith Black Beauty or Ginger, up until the moment...

In Anna Sewell's Black Beauty, Little Joe Green is hired as the stable boy to replace James Howard once John Manly, the coachman at Birtwick Park, recommends James as head groom at a different estate. Once Joe is hired, except for the time he accidentally makes Black Beauty seriously ill, Joe is kept on as the stable boy and not hindered at all from working with Black Beauty or Ginger, up until the moment Squire Gordon must sell off the estate and relocate to care for his ailing wife.

Within the first few days of Joe replacing James, Black Beauty, called Beauty for short, must run very hard to fetch the doctor to save Mistress Gordon's life. Joe, thinking Black Beauty was overly hot, did not cover Beauty up with his warm blanket when he was drenched in sweat. As a result, Beauty was inflicted with hypothermia, which led to a severe lung infection. During Beauty's illness, Ginger and Merrylegs were moved to the neighboring stable to be taken care of by Joe, while John and Joe's father, Tom Green, tended to Beauty along with the veterinarian, who came daily. Joe was brokenhearted and devastated by the consequences of his foolish actions.

Nevertheless, Beauty reports that Joe learned quickly, and Beauty becomes very proud to be under the care of Joe, especially when Joe testifies in a court of law against horse abuse he had witnessed while riding Beauty.

After three years of having lived at Birtwick Park, Beauty explains that Miss Gordon must be moved to warmer climates for her health. As a result, Squire Gordon must break up his estate, finding new posts for his servants and selling all of his horses. It is naturally when Black Beauty and Ginger are sold to the Earl of W-- that Joe can no longer work with them. He is instead hired by the vicar to take care of Merrylegs, who was sold to the vicar.