Lord Lipsey’s main objection is that it is right that (only) British citizens living here (in the UK, paying council tax & spending money in our shops etc.) get to choose the government of Great Britain, which implies that he also does not agree with expats having a vote at all, irrespective of how long they have been away.
The tax aspect is not valid as Brian Cave pointed out since he continues to pay UK income tax on his teacher’s pension. The British government has also responded that the right to vote is not necessarily related to paying tax since some foreign nationals resident in the UK and paying taxes also cannot vote. Interestingly, for historical reasons some other foreign nationals e.g. Irish or Commonwealth citizens resident in the UK can vote but not, according to election law, British citizens who have lived outside the UK for more than 15 years.
Removing the tax element from the discussion then, Lord Lipsey’s objection is that only if you live in the UK do you as a British citizen get to choose the government of Great Britain, but which denies the right to vote for democratic representation for many of his fellow but expatriate British citizens.