Fighting back is a often debated aspect of dealing with a bully.

The case against it is that it can lead to escalation, making the situation worse for the bully.  The bully is often larger, sometimes multiple bullies, and with the possibility of a weapon being carried at older ages can prevent a real threat.

Telling a scared child to simply hit the bully in the face could very easily make the situation worse.

That said the right to self-defence is something that should never be taken away from a child, and in some cases may be the only remaining choice at a given time.

Children should be taught to use other options whenever possible, and not to escalate the situation whenever possible.

So step 1 should be tell the adults in charge.  Same as in the adult world, we attempt to bring in law enforcement or security when there is a problem.  Taking matters into their own hands when not necessary can escalate a situation.

The next thing they need to try and do, which can be hard for anyone, is to ignore the problem and not respond.  Bullies feed off of the reactions, if they get upset or worked up the bully was successful and will continue.  If they leave the bully hanging with deflection or ignoring them the bully will end up looking silly and hopefully move on.

But when a situation becomes an assault and the adults in charge are unable to, or unavailable at the time the right to self-defence of any person should never be taken away.

But without giving them the tools to do so is unfair to the victim.  Martial arts training can definitely be one avenue to provide those tools, as well as provide the confidence to avoid the situation in the first place.

The other problem with a "fight back" strategy is not all bullying is physical, so much of it is done through words, both in person and nowadays through the internet.  "Defend yourself" doesn't work quite as well when the bully is behind a screen or across the play ground.

Physical self-defence skills are a part of the answer, but not the whole answer.  The confidence from having them can do a lot to prevent being a victim in the first place, and be prepared to act in defence when the bullying is in the form of assault, but kids need more tools then just boxing and wrestling.