The 46-year-old - who was forced to quit as Deputy Prime Minister after failing to pay enough tax - was quizzed on BBC Radio 4's Today programme about out of control Government spending.
But when the Housing Secretary was pressed about whether she backed welfare cuts, she replied: "Well, I, again, what I'm saying is that what Milburn is saying himself is that we have to create opportunities for young people to get into employment and it's too difficult for some of these young people.
She added: "The Government has already changed, like I say, making more apprenticeships available ..."
At this point, presenter Nick Robinson intervened and pointed out that Rayner had failed to answer the question about whether she supported cuts.
She replied: "I think the big difference, and across the floor of Labour, was that you can't put the cart before the horse."
The British taxpayer is acting as the welfare state for the world. That’s unfair. We all know it.”
At the same time, the Timms Review - co-led by Social Security and Disability Minister Sir Stephen Timms - is examining Personal Independence Payment, or PIP, which is claimed by 4.1 million people.
Reform UK has already unveiled bold plans to restrict welfare benefits only to British citizens, arguing that the system must prioritise British taxpayers and those who genuinely need support.
Reform Shadow Chancellor Robert Jenrick and Deputy Leader Richard Tice announced the policy - Making Welfare Work - in August. Jenrick said that a Reform government would begin by “ending the foreign welfare rip-off”.
"Only British citizens will be entitled to benefits”, he said, with limited exceptions including Armed Forces service and contributory state pensions for people who have “paid their way”.
He added: "The line will be clear. The welfare state is there for Britain. No British citizenship. No entitlement. That’s fair.”
Jenrick said one in six Universal Credit claimants was not a British or Irish citizen and claimed £55 billion had gone to foreign nationals over the past four years.
“The British taxpayer is acting as the welfare state for the world”, he said. “That’s unfair. We all know it.”
Reform UK estimates that restricting access to benefits in this way would save taxpayers £21 billion a year by 2029, equivalent to around £700 per household.
He asked: "Is it fair that overseas nationals come to the UK, don’t work and ultimately cost the British taxpayer a lot of money?
“I think most people would reasonably say, actually, that’s not fair."
Reform UK has pledged to protect British pensioners, protect the triple lock and ensure the welfare system continued to support those genuinely in need of support.
As of August 2025, 9.3 million working-age people in England and Wales were claiming some form of welfare. This is the highest number on record, surpassing the Covid-era peak. Of this, 6.6 million are on ‘out-of-work’ benefits.
A least 1.3 million people who are not British or Irish citizens are believed to be claiming Universal Credit.