Casper refused Bumble's invitation to move back into the Mansion. He knew that with Big Kitty's temper tantrums, the stay could be short lived.
Seen here, waiting for his chance to nab the sparrow that's sitting on the rail above, Casper is capable of making it on his own.
That's the great thing about Barn Cats.
If for some reason, nobody feeds them, they will survive. (But we always feed them)