Several former U.S. attorneys general and two ex-FBI directors have urged the U.S. Supreme Court to review a ruling that allowed immigrants detained in the wake of 9/11 to sue top officials in former President George W. Bush’s administration for alleged civil rights violations.

In an amicus brief filed Wednesday, the group said the Second Circuit’s decision last summer to revive the detainees’ claims could restrict federal officials’ qualified immunity and burden them with defending against even frivolous allegations.

Qualified immunity can shield officials from suits alleging rights violations and, in some instances, allow them to resolve claims in the early stages of litigation.

The case involves plaintiffs who were arrested on immigration violations and detained after the Sept. 11, 2001, terrorist attacks. They were allegedly subjected to a policy under which Muslim and Arab men who had violated their visa terms were arrested and deemed to be “of interest” in terrorism investigations.

The judges said the plaintiffs could sue under the 1971 high court precedent Bivens v. Six Unknown Named Agents, which allows plaintiffs to sue individual federal officials for monetary damages when no other remedy is available to protect a constitutional right.

In May, ex-Attorney General John Ashcroft and ex-FBI Director Robert Mueller appealed their cases to the Supreme Court, as did ex-Immigration and Naturalization Service Commissioner James Ziglar, and Dennis Hasty and James Sherman, the former warden and associate warden of the Metropolitan Detention Center in Brooklyn, where the plaintiffs were held.