Prince Harry may be a newly minted California resident, but he’s not a U.S. voter yet—nor, for that matter, is he in his native England.

“Many of you may not have known that I’ve never been able to vote in the U.K. my entire life,” he said in a September 22 Time 100 video that sparked a thousand British think pieces about Harry’s alleged breach of royal protocol.

It is unusual, of course, for a British royal to speak at all about another country’s election. But married to an American citizen and no longer a senior royal himself,

Harry is busy undoing all sorts of family norms. And in his frankness he’s confirmed something that has often been written about but never quite nailed down: that the Windsor family really does prohibit itself from voting, even if there’s no specific British law against it.

It turns out that there is actually some debate about whether or not Harry could be allowed to vote, but it’s not a surprise that he never has. During Queen Elizabeth’s 68 years on the throne, maintaining her political neutrality has been one of her main priorities, though she has encouraged the act of voting in the past. That neutrality is something that Prince Charles and Prince William have carried on, though Charles’s environmental activism has veered close enough to politics to garner some criticism. The Buckingham Palace website is open about her role in the British political system. “By convention, The Queen does not vote or stand for election,” it reads, “however Her Majesty does have important ceremonial and formal roles in relation to the government of the U.K.” Though British law doesn’t clearly demand it, tradition seemingly extends her political neutrality to her entire family.

Even British politicians have disagreed about whether or not the royal family is allowed to vote. In his quest to restore voting rights to the members of the House of Lords this February, Michael Morris, who chose the name Lord Naseby when he became a life peer in 1997, mentioned the uncertainty around the matter. “We know that members of the inner circle of the royal family are encouraged not to vote—but they can vote,” he said. “In fact, we do not really know if they vote or not, but all other members of the royal family can.”

In 2010, Jacob Rees-Mogg, who later made a global name for himself as a henchman of current prime minister Boris Johnson, drew the line in a slightly different place. “Our sovereign lady cannot vote,” he said on the floor of the House of Commons, “but members of the royal family who are not peers in their own right can.”

It makes intuitive sense that the royals would be disinclined to vote. In a system where a king has autocratic powers, the idea of a monarch’s vote is an oxymoron. Besides, the two chambers of the U.K. Parliament—the House of Commons and the House of Lords—both emerged more than 500 years ago as a check to the power of the monarch, so it’s reasonable to think that most monarchs never even thought to vote. On the other hand, Britain consistently ranks as one of the Economist Intelligence Unit’s top 15 most democratic societies. That a seemingly egalitarian society could disenfranchise a class of people—even if they are from a remarkably privileged class—seems off.

But there are plenty of rights that royals don’t enjoy. As Robert Hazell, coeditor of the new book The Role of Monarchy in Modern Democracy, points out, royal family members are often restricted in their freedom of speech, freedom of religion, right to privacy, and freedom of travel. The European Court of Human Rights has even taken up a few cases about legal restrictions due to royal status. In 2001, Sigvard Bernadotte, a Swedish prince and designer, filed a suit arguing that parliamentary restrictions on whom he could marry were a violation of his rights. (He died the next year, and the court declined to settle the case on its merits.)

In the book, Hazell and his coeditor Bob Morris point out that Queen Elizabeth and her ancestors reaching back to Queen Victoria have overseen Britain’s transition from a government where monarchs wielded considerable power to a social democracy where the queen’s role is largely ceremonial. In fact, the family’s desire to avoid political stances dates back to an era before the vast majority of British adults were able to vote. Because England’s formal constitution is a collection of a dozen pieces of legislation passed over the course of a few centuries, most of the norms surrounding the monarch’s involvement in politics come from tradition and scholarly writing. No one has had more influence on the Windsors than scholar Walter Bagehot, who argued the monarch has rights that are more important than voting.

In his 1867 book, The English Constitution, Bagehot coined the oft-repeated idea about the role of a monarch in a representative system: “the right to be consulted, the right to encourage, the right to warn.” In Bagehot’s formulation, the monarch has to remain above party loyalty precisely because their role is so important to maintaining order and mutual trust in the British political system. When Victoria was queen, this state of affairs mostly stood. Her biographer, A.N. Wilson, described the political system as “representative paternalism,” where a group of elites purport to act in the best interests of the populace. However, Victoria was reported taking sides in political disputes and hoped that the Whigs would win out over their Conservative opponents.

Experts on the English Constitution have noted that perhaps the first monarch to actually practice neutrality of political parties was the queen’s grandfather, King George V. In 1967, the scholar Harry Hearder wrote that George actually read Bagehot’s book closely. The “convention” that the royal family doesn’t vote most likely dates to the end of World War I, when most British adults gained the right to vote just as George implemented changes to bolster the monarchy after watching others around Europe collapse. First, George changed the family’s surname from Saxe-Coburg-Gotha to Windsor, and changed the names of other peerages to be more English. This is also when the day-to-day of royal work became focused on charitable support and being seen by the public.