A federal appeals court on Monday dismissed a landmark decision that decriminalized polygamy in Utah, marking a legal defeat for the family from the reality TV show "Sister Wives."

Kody Brown and his four wives cannot sue the state over its ban on plural marriages because the family never faced charges and prosecutors later said they would not prosecute consenting adults with multiple wives, the 10th U.S. Circuit Court of Appeals in Denver ruled.

The decision reverses a 2013 ruling that removed the threat of arrest for polygamous families. U.S. District Judge Clark Waddoups had found that the threat of prosecution alone drove the Browns out of the state and that key parts of Utah’s bigamy law violated their right to privacy and religious freedom.

Utah Attorney General Sean Reyes appealed Waddoups’ ruling. The state has a longstanding policy against prosecuting consenting adult polygamists, but prosecutors argued that the ban should stay on the books to help authorities go after polygamists who commit other crimes, such as sexual assault, statutory rape and exploitation of government benefits.

Prosecutors also pointed to imprisoned leader Warren Jeffs, who was convicted of assaulting underage girls he considered wives.

The Browns said other laws exist to target crimes linked to plural marriages and that banning the practice can sow distrust of authority. They say their show is evidence that polygamous unions can be as healthy as monogamous marriages.

Bigamy, or holding multiple marriage licenses, remained illegal amid the legal fight. Brown has a license for only one of his marriages and says his other unions are spiritual.

There are about 30,000 polygamists in Utah, according to court documents. They believe polygamy brings exaltation in heaven — a legacy of the early Mormon church. The mainstream Church of Jesus Christ of Latter-day Saints abandoned the practice in 1890 and strictly prohibits it today.

The Browns’ lawyer and the state attorney general did not immediately have comment.

(Copyright (c) 2016 Sunbeam Television. All Rights Reserved. This material may not be published, broadcast, rewritten, or redistributed.)

Join our Newsletter for the latest news right to your inbox