The Supreme Court heard arguments on Wednesday that could decide whether hundreds of thousands of immigrants will lose deportation protections.

Justices from the court’s conservative majority seemed skeptical that lower courts had the authority to stop the Trump administration from ending Temporary Protected Status for Haitians and Syrians.

The justices’ decision, expected to come at the end of the term in June, would have direct repercussions for more than 330,000 Haitians and nearly 3,900 Syrians legally authorized to live and work in the U.S. through TPS. But it would also affect how courts consider the program as a whole, which the Trump administration has sought to scale back in his second term.

Congress created TPS in 1990 to provide deportation reprieve for immigrants from countries facing extreme hardship, such as wars and natural disasters. Roughly 1.3 million people had the status and work permits before the Department of Homeland Security started scaling back the designations by ending TPS for 13 countries, including Venezuela, Afghanistan and Honduras.

U.S. Solicitor General John Sauer has argued in response to lawsuits that the courts have no authority to review decisions over whether to designate or terminate TPS.

At the hearing, conservative justices suggested some sympathy for that view, given that the suspension of TPS for Haitians and Syrians is currently on hold based on lower court rulings.

“How can it not be judicial review of the determination if you’re postponing the determination?” Justice Neil Gorsuch asked, regarding the lower courts’ decisions to postpone the revocation of TPS for the two countries.

“It’s difficult for me to answer that question without pointing out —” Geoffrey Pipoly, the attorney representing the Haitian immigrants, began to reply before Gorsuch interrupted him, saying, “It’s difficult for me to answer that question too.”

The Homeland Security secretary is tasked with determining whether it’s safe for immigrants with TPS to return to their home countries at the end of the designation period. Congress did not set a limit on how many times TPS can be renewed, and in some cases it has remained in place for decades.

In the case of Haitians and Syrians, then-Homeland Security Secretary Kristi Noem deemed the countries safe even though the State Department advises Americans not to visit them due to ongoing violence and conflict.

Trump has repeatedly made disparaging comments about Haitians, as Justices Sonia Sotomayor and Ketanji Brown Jackson noted in questioning Sauer, who said those statements should not be considered.

“The court should conclude that those comments are unilluminating,” Sauer said.

Pipoly and Ahilan Arulanantham, a UCLA law professor arguing on behalf of the Syrian TPS holders, said that courts could tell the Trump administration it must follow the steps Congress outlined in the creation of TPS, such as consulting with other agencies about the safety of the immigrants’ home country.

They argued Noem did not conduct an appropriate review of the countries’ safety, making the rescission of protections a preordained outcome that would put the immigrants in peril.

Arulanantham also said that should the court side with the administration, it would give future administrations the power to grant TPS to populations that don’t fit the parameters without judicial review.

Immigrants from four other countries — El Salvador, Lebanon, Sudan and Ukraine — are set to lose protections this year unless Homeland Secretary Markwayne Mullin extends them.

The only way to ensure immigrants with TPS can remain in the U.S. long term is for Congress to create a pathway to permanent status, Arulanantham said during a rally after the hearing.

“Win, lose or draw, the ultimate solution to ensure that the TPS holder communities are given what you all deserve, the dignity and respect that you deserve, it lies in the hands of the United States Congress,” he said.