LONDON, March 22 (Reuters) - Ratings agency Moody's said on Tuesday it saw "clear downside risks" if Britain votes to leave the European Union in a referendum in June, and repeated its warning that this would make the country's debt more vulnerable to a downgrade.

Moody's concerns come alongside a report from consultants Oxford Economics which estimated that Britain's economy could be almost 4 percent smaller than otherwise if it curbed inward migration after leaving the bloc.

Prime Minister David Cameron wants Britons to endorse continued membership of the 28-nation bloc in a referendum in three months' time, but both his Conservative Party and public opinion are deeply divided.

Moody's currently rates British government debt one notch below triple-A with a stable outlook, but said voting to leave the EU could lead to a negative outlook.

"This is because we believe a Brexit vote would lead to heightened uncertainty, which would manifest as modestly weaker economic growth in the UK over the medium term," Moody's said in a note to investors.

"However, there are clear downside risks to this assessment, especially if new arrangements led to a substantial disruption in trade or capital flows," it added.

Oxford Economics said it expected British economic growth to be weaker if the country voted to leave the EU, but the damage would depend on how sharp a rift there was with the bloc.

If Britain continued to enjoy good access to EU markets, cut tax and regulation -- a goal championed by many who want to leave the EU -- and did not reduce the number of EU immigrants it let in, then the economy would be less than 1 percent smaller in 2030 than if it had stayed in.

However, reducing immigration from the EU is a key aim of those who support "Brexit". If leaving the EU led to a halving of the number of migrants, and trade and regulatory agreements were less favourable, then economic output would suffer by as much as 3.9 percent, Oxford Economics said.

"The long-term impact of Brexit on the UK need not be severe. But benign scenarios involve retaining some of the least popular aspects of EU membership," associate director Henry Worthington said.

Moody's said the impact on British banks and insurers would be limited, unless they faced barriers to operating in the EU.

"Brexit is unlikely to have significant negative consequences unless the current 'passporting' regime for financial services were changed, which would be costly and disruptive," it said.

(Reporting by David Milliken; Editing by Mark Heinrich)