President Donald Trump departs on the South Lawn of the White House, on December 12, 2020 in Washington, DC. Trump is traveling to the Army versus Navy Football Game at the United States Military Academy in West Point, NY. (Photo by Al Drago/Getty Images)
PHOTO: Al Drago/Getty Images
President Donald Trump departs on the South Lawn of the White House, on December 12, 2020 in Washington, DC. Trump is traveling to the Army versus Navy Football Game at the United States Military Academy in West Point, NY. (Photo by Al Drago/Getty Images)
Now playing
03:05
Trump's future Palm Beach neighbors seek legal action to keep him out
Now playing
01:35
Laughter follows awkward moment between GOP leaders
PHOTO: AFP/Getty Images/CNN
Now playing
03:11
Cabrera: GOP suddenly cares about mean tweets ... just not Trump's
Now playing
03:20
Avlon on Ron Johnson: Hyperpartisan denial is a hell of a drug
WASHINGTON, DC - JANUARY 06: Pro-Trump supporters storm the U.S. Capitol following a rally with President Donald Trump on January 6, 2021 in Washington, DC. Trump supporters gathered in the nation
PHOTO: Samuel Corum/Getty Images
WASHINGTON, DC - JANUARY 06: Pro-Trump supporters storm the U.S. Capitol following a rally with President Donald Trump on January 6, 2021 in Washington, DC. Trump supporters gathered in the nation's capital today to protest the ratification of President-elect Joe Biden's Electoral College victory over President Trump in the 2020 election. (Photo by Samuel Corum/Getty Images)
Now playing
03:04
Capitol officials say riot was planned and involved white supremacists
Then-President Donald Trump addresses supporters during a Make America Great Again rally in Erie, Pennsylvania, October 20, 2020.
PHOTO: SAUL LOEB/AFP/Getty Images
Then-President Donald Trump addresses supporters during a Make America Great Again rally in Erie, Pennsylvania, October 20, 2020.
Now playing
02:44
What Trump's released tax records mean for DA's criminal case
Now playing
02:51
'This is incredible': Burnett explains Trump's reported offer to Kim Jong Un
From left, President Joe Biden, First Lady Jill Biden, Vice President Kamala Harris and her husband Doug Emhoff, bow their heads during a ceremony to honor the 500,000 Americans that died from COVID-19, at the White House, Monday, Feb. 22, 2021, in Washington. (AP Photo/Evan Vucci)
PHOTO: Evan Vucci/AP
From left, President Joe Biden, First Lady Jill Biden, Vice President Kamala Harris and her husband Doug Emhoff, bow their heads during a ceremony to honor the 500,000 Americans that died from COVID-19, at the White House, Monday, Feb. 22, 2021, in Washington. (AP Photo/Evan Vucci)
Now playing
05:45
Biden leads nation in mourning 500,000 US Covid-19 deaths
PHOTO: CNN
Now playing
01:06
Honig: Public won't see Trump's taxes unless this happens
Now playing
02:00
Former FBI Deputy Director: Trump taxes 'massive trove of information'
Now playing
01:31
Lara Trump: Donald Trump may run in 2024 and beyond
gop rep michael mccaul texas winter storm response bash sotu vpx_00041202.png
gop rep michael mccaul texas winter storm response bash sotu vpx_00041202.png
Now playing
04:12
Texas GOP rep.: When a crisis hits my state, I don't go on vacation
PHOTO: CNN
Now playing
04:18
GOP governor reacts to his nephew leaving the Republican Party
PHOTO: Getty Images
Now playing
03:00
'Forged over decades': WH reporter on Biden, Dole relationship
Now playing
01:26
Did Pence feel betrayed after riot? His former chief of staff responds
PHOTO: Getty Images
Now playing
03:34
'Real sense of betrayal': Reporter on GOP facing backlash over impeachment vote
(CNN) —  

First lady Melania Trump plans to slip down to Palm Beach via government jet on Friday for a weekend walk-through of renovations underway at the Trump family’s private quarters at Mar-a-Lago, according to a source familiar with the first lady’s schedule, to make sure it’s to her liking before she and President Donald Trump move to the private club after January 20, when President-elect Joe Biden moves into the White House.

But beyond the new paint, fabrics and finishes and general expansion of the approximately 3,000 square foot private quarters, there looms a much larger issue with the permanency of the move: whether or not it is legal. When he turned the private residence into a club, Trump had agreed with the town to limit his stays there, and now some Palm Beach residents say he might be violating that agreement.

Trump bought the former estate of Marjorie Merriweather Post in 1985, and subsequently turned it into a members’ only club in 1993. The plan was – as with most Trump deals – to turn a profit. Yet in order to transform the private residence into a revenue-generating business, Trump had to agree to certain limitations, based on guidelines presented as dealbreakers from the Town of Palm Beach. For example, there could be no more than 500 members, there were rules concerning parking and traffic, and club members – Trump included – could not spend more than seven consecutive days at Mar-a-Lago, for no more than three weeks total a year. At the time, following several appearances at town council meetings by Trump and his lawyers to plead his case for approval on the evolution of the property, Trump assented to abiding by the 21-day rule.

However, he has been exceeding that number by staying at Mar-a-Lago far more often while president, and is apparently hoping to make the club his permanent home after he leaves office – something nearby residents in posh Palm Beach are not interested in supporting in a neighborly manner.

West Palm Beach-based attorney Reginald Stambaugh wrote to Palm Beach officials on Tuesday saying that Trump has already violated the agreement for visitor stays at Mar-a-Lago, noting the President’s plans to move to the club in January. The letter was first reported by The Washington Post.

Stambaugh said he represents a client who may have been exposed to a so-called microwave fence used to secure the property, and that he still has security concerns with the fence and with how Trump’s usage of Mar-a-Lago might devalue neighboring properties.

“It is the Town Council’s responsibility to right these wrongs and restore safety and security to the neighborhood by upholding its Use Agreement,” Stambaugh wrote. “In order to avoid an embarrassing situation for everyone and to give the President time to make other living arrangements in the area, we trust you will work with his team to remind them of the Use Agreement parameters.”

“Palm Beach has many lovely estates for sale and surely he can find one which meets his needs,” the letter continues.

According to Brian Seymour, a Palm Beach County land use expert and attorney, the allowances the town gave Trump during his presidency could end once he becomes a private citizen again.

If the club doesn’t comply, the town can try to enforce its regulations and agreements it made, such as by suing or fining Mar-a-Lago to enforce the residency limits, Seymour said.

But ultimately, it would be up to Trump to comply. “It could get very complicated if the town really was enforcing this,” Seymour said.

When asked for the town’s response to the complaint Wednesday morning, Palm Beach officials didn’t immediately respond. CNN also reached out to the White House for comment and did not immediately receive a response.

“There is no document or agreement in place that prohibits President Trump from using Mar-A-Lago as his residence,” a spokeswoman for the Trump Organization said in a statement. CNN has also inquired with the office of the first lady.

Frustrated neighbors

Neighbors in Palm Beach have made clear they’re frustrated with Trump’s tactics.

“It’s been a circus there for four years and they’re fed up with it,” one Palm Beach homeowner aware of the neighbors’ thinking told CNN.

Another neighbor with knowledge of the efforts to raise the legality of permanent residency said complaints about disruption to normal life when Trump stays at Mar-a-Lago have been lodged to the town council and the mayor consistently for years, including issues about amped up security, bottleneck traffic and noise violations. However, the pleas for assistance from local government were overlooked, said the neighbor, because Trump was president. In short time, that will no longer be the case, and the neighbor said this is in part why there is renewed effort to highlight the agreement that was sealed into record when Trump was allowed to create his club. “It has been a nightmare,” the neighbor said.

The neighbor added the mash-up of old-money Palm Beach residents and the flash-and-cash persona of Donald Trump has always been “like oil and water – it just doesn’t mix.” Over the last 30 years, Trump has pushed the town to the brink of frustration with several legal clashes, including flouting height regulations for a giant flagpole he had installed on Mar-a-Lago’s grounds, which was 80-feet tall and flew a 375-square foot American flag.

After filing a lawsuit against the town, which levied a daily fine for each day the flagpole remained, Trump eventually agreed to a settlement (and to lower it, slightly). Over time, Trump complained to the town about air traffic routes, expansion rules, member prohibitions and other grievances, and the town complained back – once even over a photo shoot in the 90s for a Versace ad campaign featuring Madonna that was shot on Mar-a-Lago’s grounds, citing zoning restrictions that did not allow for commercial photography at a private residence, which it was at the time. Trump’s more recent demands for a helipad also consumed several months of contentious back-and-forth before the town’s Landmarks Preservation Commission eventually allowed for one to be installed in 2017. (The helipad will be no more when Trump’s presidency ends.)

For many of the uber-wealthy denizens of Palm Beach, helping to get rid of Trump, and using a document he himself agreed to to do so, would be the cherry on top of years of mogul-induced headaches. “It’s what the law says,” says Larry Leamer, who wrote “Mar-a-Lago: Inside the Gates of Power at Donald Trump’s Presidential Palace,” and who owns a home in Palm Beach. “It’s an agreement. Now, whether the town goes forward with enforcing it, that remains to be seen.”

Trump does have a handful of allies in Palm Beach, mostly members of his club, some of whom are politically and financially influential. The post-presidential staff and security footprint is also smaller, which could help alleviate some of the disruptions of the last four years, and sway other neighbors to possibly overlook the specifics of the agreement, but Leamer said, “That seems unlikely.”

Leamer noted Trump has also wedged himself into a corner of another Palm Beach pet peeve: he has legally claimed Florida as his residence and even listed Mar-a-Lago as his address to vote this year, meaning he must prove he is there for six months out of the year. “There is nothing true local Palm Beach residents hate more than New York City people coming down and claiming they live here to get the tax breaks,” said Leamer. Whether Trump stays in Palm Beach more than at any of his other residences per year, as required by law, is up in the air, but Leamer noted, “they’ll watch,” referring to the disgruntled neighbors.

Trump, who has for several years defied adherence to norms and customs, and his wife appear to be demonstrating little concern for the agreement signed in 1993. According to a source familiar with the security apparatus already being established to accommodate a former president’s new place of residence, the United State Secret Service has for the last several months been organizing and devising a plan to protect Trump post-presidency at Mar-a-Lago.

On her trip to Florida, the first lady also plans to check on the suite of rooms occupied by her parents, which is also receiving a facelift. In addition to putting her personal interior decorator in charge of the refresh, she has also seen to updates to the quarters’ bathrooms.

In an effort to utilize more space, Trump has also authorized the takeover of what was one of the Secret Service’s “down rooms,” a place adjacent to the President’s quarters where agents can rest, watch television, use a computer, or catch up on correspondence between shifts. According to a source familiar with the expansion, the additional square footage was requested by the Trumps, meaning the Secret Service will have to find another respite.

Jamie Gangel and Cristina Alesci contributed to this report.