The same thing that happens to a lot of extremist groups, they refuse to adapt and align with others that aren't the same as them, but share their viewpoint. The Panthers did not hold women in the same regard as men, and thus, their female membership was non-existent. Towards the end, there are stories of brutality against women, and during the late 70s and early 80s, women were attempting to gain more rights for themselves. Imagine being a member of the Panthers and having a wife that was a feminist. It doesn't take violence to overthrow a movement, simply reducing their numbers at the street level can be enough to ground them. With fewer and fewer men joining or staying with the panthers, and the issues surrounding Huey P. Newton, the power of the party declined and eventually collapsed under itself with no one to support it.

This is a super simplified and very narrow argument for the downfall of the Panthers. I would suggest reading up more on it. There are plenty of books on the subject. Just remember that the more you read and study, the greater an informed position can be reached. Don't rely on just one narrative or one viewpoint. Read, analyze, and discuss with others. Knowledge is gained slowly and deliberately and rarely completed.