Bankers say that the so-called qualified residential mortgage rule proposed in the Dodd-Frank Act—just one of a host of proposed federal policy changes—would discourage banks from lending to minorities and the poor.

The rule includes a provision that would essentially require higher down payments from borrowers. Bankers say that rule and other policy changes, ranging from Basel capital standards to rules governing a borrower’s ability to repay guarantee fees to Fannie Mae and Freddie Mac, are so onerous that they could be forced out of the mortgage business.

The banks say they’re caught in a catch-22: if the rules are too strict, they will have little choice but to restrict lending, but if the rules are too loose or poorly defined, banks say they will face higher compliance costs and increased litigation risks. “On one side they’re telling us things are crazy and we need to get our underwriting in check, and then they come after us for imposing standards,” said Richard Andreano, a partner at Ballard Spahr. “It’s the Goldilocks issue. When is it just right?”