The CFPB could lose its first battle as early as Friday, the earliest that a U.S. District Court could rule on whether the CFPB's single-director structure is unconstitutional. The lawsuit, brought by mortgage servicer PHH Corporation over a fine for allegedly taking kickbacks for referring customers to an insurance company, is the most serious legal test the agency has faced.

PHH contends that the CFPB's structure is unconstitutional because the agency's director is not directly answerable to the president or to Congress.

Should PHH prevail, the CFPB would likely appeal. But any ruling against the agency would give its critics - primarily regulated industries and Republicans on Capitol Hill - ammunition in the longer-term fight to dismantle or reorganize the agency.

"It's a full-frontal constitutional attack on the CFPB ..., and it's in front of a panel of judges in the D.C. circuit who seemed very hostile toward the CFPB during the oral arguments," said Alan Kaplinsky, who chairs the Consumer Financial Services Group at law firm Ballad Spahr.

A decision against the agency "will literally cast a cloud over everything the CFPB has done since it was created," he added.

The losing side could appeal to the Supreme Court or ask for a review from the entire district court. The CFPB declined to comment for this story.

For now, the agency is pushing ahead on several fronts, including proposing a rule that would ban forced arbitration clauses from financial contracts.