The prospect that lower-income earners could enjoy a pad at Barangaroo is in doubt after it emerged Lend Lease is wavering on a commitment to build affordable housing next to James Packer's luxury casino.

The change in stance follows an admission by the state government that a taskforce set up to help solve Sydney’s housing affordability crisis has not met for more than a year, or delivered the housing policy it promised, despite previously saying "doing nothing" was not an option.

Fairfax Media has learnt that Lend Lease has been eyeing off locations away from Barangaroo on which to build the affordable housing component it pledged in return for developing public harbourside land.

Under current approvals for Barangaroo south, 2.3 per cent of 100,000 square metres of residential floor area must be “key worker housing” – homes rented to lower-income public sector workers such as police, nurses, teachers and paramedics.

However it has emerged Lend Lease has been in talks with not-for-profit housing groups about building the homes off-site – potentially allowing it to reap a greater profit from the Barangaroo residential floorspace.

It is understood the developer has considered nearby Millers Point as a potential site. The government is controversially relocating 465 public housing tenants from that suburb as part of a property sell-off.

A source at one community housing provider confirmed the organisation had met with Lend Lease, but questioned whether heritage restrictions at Millers Point would allow affordable housing development.

The source said there had always been questions over how affordable housing, which is often set as a percentage of market rent, would be provided at Barangaroo.

“If market rent is $1000 for a one-bedroom unit, 80 per cent of [that] is still not affordable,” the source said, adding “you’d have to have very small units”.

A Lend Lease spokesman confirmed it had held discussions with government agencies and community housing groups about off-site housing at Barangaroo, but no "deal" had been reached at Millers Point.

University of Sydney urban planning professor Peter Phibbs said the affordable housing commitment at Barangaroo was very low by international standards. If low-cost homes were to be built off-site, they should remain in the inner city, he said.

Professor Phibbs, a member of the affordable housing taskforce established in 2011, said the government had “basically given up on it”, adding “I wouldn’t describe it as active”.

A report by the taskforce in 2012 said Sydney faced an "acute" housing affordability issue and "doing nothing" was not an option.

A Planning Department spokesman did not explain why the taskforce had not met since June last year, nor why it had not delivered the policy it promised. He said affordable housing was being delivered under existing policies.

Independent Sydney MP Alex Greenwich said Lend Lease would enjoy a "huge windfall" if it was allowed to dodge its affordable housing provision at Barangaroo. If the government “is going to let them get away with" building low-cost homes at Millers Point, the rent should be cheap enough to allow existing public housing tenants to live there, he said.