Havana, Cuba (CNN) -- A young Cuban man slouched against his city's famous sea wall, enjoying fall's cool breeze and thinking about the world little more than 90 miles north.
"A lot of people died in that sea trying to make it to the other side," Yoandri Perez, 20, said, while resting along the Malecon, a concrete partition and six-lane highway that holds back the Florida straits from the Cuban capital.
"It's very difficult here. The economy is bad and now they're cutting jobs," he said, enjoying the seasonal shift of cooler weather and rough seas. "But at least the Malecon is a place where we can come to relax."
More than 1,300 miles north, another possible shift is under way. In Washington, as midterm campaigning is peaking, powerbrokers are discussing the effects of a possible change in the balance of power in Congress.
One thing that isn't being discussed: Cuba.
"People on the Hill are just not focused on Cuba," said Sarah Stephens of the Center for Democracy in the Americas, a Washington-based group that advocates an end to the 48-year-old U.S. trade embargo.
"Right now they have the votes [to end the travel ban], but after November it's a whole new ball game."
And President Barack Obama, who pledged "a new beginning" in relations with Cuba, has made few changes since loosening Treasury restrictions in 2009.
"We already initiated some significant changes around remittances and family travel. But before we take further steps, I think we want to see that in fact the Castro regime is serious about a different approach," the U.S. president said last week.
Obama said he was interested in more openings with Cuba, but the Castro government must first do some shifting of its own.
Last year, the president made a similar pledge.
"What we're looking for is some signal that there are going to be changes in how Cuba operates that assures that political prisoners are released, that people can speak their minds freely ... and do the things that people throughout the hemisphere can do and take for granted," he said during a 2009 interview with CNN en Espanol.
The island has since released dozens of political prisoners and announced massive public sector layoffs to pave the way for free market enterprise intended to create new jobs for its former state workers.
"What we're now left with is a president on the hook," said Phil Peters of the Washington-based Lexington Institute. "He said if there were positive developments he would respond, and now we're seeing the release of political prisoners and some pretty significant economic changes."
Senior U.S. officials and congressional sources told CNN the White House had been considering further relaxing regulations, but had been persuaded to hold off until after the November midterm elections.
Republicans are expected to pick up seats in both the House and Senate, leaving the White House with the possibility of facing a Congress more opposed to changing U.S.-Cuba policy.
"The political costs of getting these [changes] out are higher," said one congressional source, suggesting the administration might now trim the package that was being fashioned over the summer. "The question is how much stomach at the White House is there to take that hit?"
Peters said that with the U.S. still engaged in Iraq and Afghanistan and still reeling from a global financial crisis, "Cuba is not a high priority."
"But I think the White House will respond because Obama's word is on the line," Peters added.
In September, Cuban Foreign Minister Bruno Rodriguez accused Obama of failing to keep his promise, saying that far from easing regulations, the U.S. administration had tightened the enforcement of trade restrictions.
"The president has fallen far short of the expectations created by his speeches," Rodriguez said in the Havana news conference, stressing the reach of U.S. sanctions on international business and Cuban trade.
"The true impact of the embargo is not just a bilateral impact," said John Kavulich of the U.S.-Cuba Trade and Economic Council. "It isn't Cuba's inability to access U.S. markets, it's Cuba's inability to access foreign exchange and the U.S. ability to manipulate private companies and some governments in their relationship with Cuba."
Despite the U.S. trade embargo, which Cuba calls "a blockade," the United States is the island's leading source of food and agriculture.
In 2000, the U.S. allowed American farmers to sell food and farm products directly to Cuba. A bill passed eight years earlier permits the shipping of medical supplies although red-tape has often slowed the delivery of goods.
While the White House cannot lift sanctions without congressional approval, some analysts believe the real obstacle to improved relations is Alan Gross, an American jailed in Cuba on suspicion of spying.
Gross, 60, had been working for a USAID subcontractor called Development Alternatives Incorporated (DAI) when he was arrested at Havana's international airport on December 3, 2009.
His continued imprisonment -- although he has not been charged -- prompted one of the highest-level diplomatic exchanges between the two countries in recent years.
During the U.N. General Assembly in New York last month, U.S. Assistant Secretary of State Arturo Valenzuela spoke with Cuban Foreign Minister Rodriguez in a meeting intended to "encourage the release" of Gross.
Cuba is one of a handful of places -- including Iran and Myanmar -- where the U.S. funds what it calls democracy-building initiatives without the host country's permission.
USAID -- the U.S. Agency for International Development -- came under intense scrutiny in 2006 and 2008 as a result of reports by the U.S. Government Accountability Office that identified potential misuse of U.S. grant money to promote Cuban democracy.
DAI, where Gross was working, does not receive those grants but is a USAID subcontractor engaged in Cuba to "strengthen civil society in support of just and democratic governance," according to a statement from the company's president and chief executive Jim Boomgard.
Gross' continued imprisonment and the potential fallout from the upcoming U.S. election may already have cooled what had once appeared to be a warming of relations.
CNN's Deirdre Walsh and Dan Lothian contributed to this report