Edward joked that he had not been taking notes during Gaston Browne’s opening remarks so could not respond to all the points he had made.
The prime minister did not laugh at the comment.
It follows protests on the couple’s visit to St Vincent and the Grenadines on Saturday, where banners were held aloft saying “compensation now” and “Britain your debt is outstanding”.
Mr Browne also told the earl and countess it is the country’s wish to “one day become a republic”.
The prime minister’s remarks came at a meeting between Edward and Sophie, Mr Browne and his cabinet during their visit to the island on Monday.
The earl nervously laughed after being asked to give words in response to Mr Browne’s speech.
Despite stating the country would one day want to change position, the prime minister acknowledged that it is “not currently on the cards”.
He told Edward and Sophie: “You will have noticed there are no protestations here,” adding that they were not “holding placards”.
Mr Browne said the decision not to protest was because they believed in having an “open and very objective discussion”.
The prime minister said he understood the royal family did not get involved in “contentious issues” but said he wanted them to “understand these issues… so you can use your diplomatic influence in achieving the reparatory justice that we seek”.
Mr Browne added: “The reality is we have been left and bereft of modern institutions such as universities and medicinal facilities.”
He said: “We continue to have the Queen as our head of state, even though I should say we aspire at some point to become a republic.
“But that is not currently on the cards so she will remain as head of state for some time to follow.
“We’re not trying to embarrass you, we’re just trying to build awareness.
“You may not necessarily comment on this issue as you represent an institution that doesn’t comment on contentious issues.
“Our civilisation should understand the atrocities that took place during colonialism and slavery and the fact that we have to bring balance by having open discussions.
“We believe in constructive engagement so you’ll notice there weren’t any placards because for us it’s about having open and objective discussions.
“We understand that nature of your job is not to get involved in contentious issues but at the same time it’s important for you to understand these issues.
“You can even use your, let’s say, diplomatic influence to build bridges in achieving the reparatory justice that we seek here in the Caribbean.
“Because the reality is we have been left and bereft of important institutions such as universities and good medicinal facilities.”