Several members of Congress from Florida are accusing the Veterans Affairs officials of trying to limit oversight into department operations by evicting them from offices at local VA medical centers, limiting their ability to interact with veterans.

But VA Secretary Robert Wilkie insists there is no truth to that claim, and that the move is being made to provide more space for veterans medical services. He accused lawmakers of making “misleading public statements” about the moves in an effort to create unneeded controversy.

The dispute began last week, when Rep. Brian Mast, R-Fla., went on Fox News to complain that he had been told to clean out his satellite congressional office at the West Palm Beach VA Medical Center by December, so officials could reclaim the space.

Mast, an Army veteran who lost both his legs in an explosion in Afghanistan in 2010, blasted the move as an attempt to cut down on lawmakers’ visibility of potential problems at the facility.

“We need to flip that place upside down in order to change the whole climate of what’s going on there,” he told the network. “And if the people in charge aren’t willing to do that, then they’re not doing the right thing.”

Fellow Florida Democratic Reps. Darren Soto and Stephanie Murphy said they both have been given similar reasons for VA shutting down their offices at the Orlando Veterans Affairs Medical Center. They have also requested top VA officials reconsider the decision.

VA officials said six lawmakers with offices at local medical centers have been told their spaces will be reclaimed, as part of an effort to “maximize clinical space.” In a letter to Mast last week, Wilkie criticized the congressman’s reaction as off-base and unhelpful to veterans.

“You falsely implied that you could not conduct congressional oversight and constituent services from your district or Washington-based offices,” Wilkie wrote.

“You are one of only six members of Congress who currently have office space in a VA facility. None of the other 529 members, all of whom conduct oversight and constituent services, are making such demands.”

Lawmakers conceded that VA is not required to make the space available to them, but said the eviction runs counter to publicly stated goals of transparency and reform at the department. Wilkie dismissed those concerns, saying the department “is more transparent and accountable than ever before.”

The issue is likely to come before the House Veterans’ Affairs Committee, which in recent months has raised its own complaints about a lack of cooperation from VA officials in providing information on key programs and personnel. VA leaders have disputed that, saying they have provided all of they documentation available upon request.

Mast and other lawmakers said they are also considering legislation to force a return of the space to interested members of Congress.