Defense Secretary Jim Mattis says he isn’t going anywhere, yet.

That was his quick response to questions from reporters outside the Pentagon on Tuesday about a slew of stories wondering if he is a short-timer, and whether he still has President Donald Trump’s confidence.

In the last week or so, Politico has reported that Trump now calls Mattis “Moderate Dog,” in what they reported as a growing frustration by the president on Mattis' slow-rolling on some of Trump’s initiatives, like the Pentagon’s transgender policy. The New York Times and Washington Post have also reported, citing multiple defense officials, that Mattis' days are numbered.

On Tuesday, Mattis provided his own take, in an exchange that was filmed by the Independent Journal Review’s Julio Rosas.

“Like most of those kinds of things in this town, somebody cooks up a headline, they then call to a normally chatty class of people, they find a couple other things to put in, they add their rumor, somebody on the other coast starts writing the same thing, next thing you know you’ve got a story," Mattis said. "It’ll die down … how many times have we been through this now, just since I’ve been here.”

“It’ll die down soon and the people who started the rumor will be allowed to write the next rumor, too. That’s the way the town is, keep a sense of humor about it,” Mattis said.

He then counter-punched with a little humor of his own. The defense secretary does love Washington ... state. That’s where he grew up. Still, he reveled for a moment in the idea of never going away.

“Of course I don’t think about leaving. I love it here. I’m thinking about retiring right here. I’ll get a little place down on the Potomac,” Mattis said.