If the speculation bears out and the White House's chief economic advisor, Gary Cohn , a Goldman Sachs (NYSE: GS) alum, were to become the next Federal Reserve chairman, the Trump administration would be gaining a steady hand at the central bank, but perhaps losing support from one of Wall Street's most influential firms, veteran banking analyst Chris Whalen told CNBC on Tuesday.

"If you see Cohn go to the Fed, to me, that's an escape path for him and that means that Goldman has given up on the Trump administration," said Whalen, a former New York Fed official whose father was an advisor to past Fed Chairman Alan Greenspan . Goldman Sachs was not immediately available to respond to CNBC's request for comment.

At the time same, the Whalen Global Advisors chairman said on "Squawk Box" that Cohn would be a refreshing change at the Fed, which "needs to have more non-economists in its leadership."

With President Donald Trump unlikely to nominate current Fed Chair Janet Yellen to another term, Cohn as her successor has been talked about for months by CNBC and other media outlets. Last week, citing four sources, Politico said Cohn is now the leading candidate to succeed Yellen once her term ends in February. For its part, the White House told CNBC that Cohn is focused on his responsibilities as director of Trump's National Economic Council.

Whalen's thesis about what a Cohn Fed appointment would signal about how Goldman Sachs feels about President Trump is significant because Cohn left the No. 2 spot at Goldman to join the administration and the NEC.

Cohn isn't the only former Goldman executive in Trump's administration: Treasury Secretary Steve Mnuchin also once worked at the venerable investment bank, which has been a feeder for decades on both sides of the political aisle in Washington.

Regardless of how Goldman would feel if Cohn, who made his bones on Wall Street as a trader, were to leave the White House for the Fed, "I'm one of those who believes that there's nothing wrong with having bankers, people from industry on the Fed board," Whalen said. "We have been dominated by academic economists. And you know what, they've gotten it wrong."

Whalen said nobody should be surprised that inflation has yet to rise to the Fed's 2 percent target level. "Low interest rates [and] quantitative easing are deflationary. So, of course, we haven't hit our inflation targets. But nobody at the Fed understands that."