We hear Artificial Intelligence (AI) bandied about a lot in recent times as well as innovation and agility and more recently we have been hearing terms such as robo-debt recovery, algorithms and malware. The Income Security Integrated System (ISIS) ISIS was set up in 1983 and oversaw welfare payment deliveries, customer service, support and compliance activities for Centrelink. In 2015 Marise Payne, former Human Services Minister (and now Defence Minister) called for an overhaul of the system: ‘To deal with the increased demands over the years the original system has literally had another 350 systems bolted on. To put it simply, we are running a turbo-charged Commodore 64 with a spoiler in the age of the iPhone.’

In the 2015-16 budget, the Welfare Payment Infrastructure Transformation (WPIT) program was announced as the replacement for ISIS. The 2015-16, budget measure worth $60.5m is part of a $1.5 billion, seven-year program. The program was described by the government as one of the world’s largest social welfare ICT transformations.

In September 2015, the Department of Human Services (DHS) asked for expressions of interest (EOI) for the first tranche of WPIT for a core software provider. As part of tranche one a panel of members was also to be formed to compete for the other tranches. In a statement, Ms Payne said that: “Finding innovative and expert industry partners is the first step in providing a modern platform that will make interacting with government services easier for our customers,” the Minister added. “Over the next year, the department will commence two major procurement activities to secure a Core Software Vendor and Systems Integrators.”’

“The new system will reduce red tape for customers, lower the costs of administering welfare payments and save taxpayers money,” Payne said. “Customers can expect to see improvements to our payment systems by the end of 2016 with enhancements that will make online interactions quicker and easier.”

On March 2^nd 2016, legislation was introduced to parliament to assist the government in chasing welfare debt by Social Services Minister Christian Porter. The changes allow interest to be charged on debts, ends the six-year limit on when debt can be pursued and stops debtors from being able to travel overseas. The new interest charge is around nine-percent and applies to social security, family assistance, child care, paid parental leave and student assistance debt. It won’t be imposed on those that have an approved repayment plan. The six-year limit brings it in line with tax debt and the travel ban brings it in line with child support debtors.

And by March 20^th it was reported in the media that the DHS had partnered with the Australian Federal Police (AFP) in a venture called Taskforce Integrity. Welfare recipients in areas identified as high-risk, received letters with the AFP logo alongside the Centrelink logo. This is a first, using a police logo on a welfare letter. The first batch of letters was sent to South Queensland and will be rolled out to other geographical areas around Australia considered high risk or noncompliant. The letters warn that the taskforce was “currently working in your community” and that providing the wrong information could constitute welfare fraud, resulting in a “criminal record or a prison sentence”.

The government’s new automated compliance system to detect overpayments began on the 13^th of July last year. The system compares Centrelink information with records such as tax records, saving the government money on employing staff. The first error to come to light was it not computing the difference between 52 weeks in a year and 26 fortnights. And in December last year stories from the public began trickling through to the media.

In early August 2016, German software company SAP was selected to be the government software provider and tranche two was opened up for bidding. In the MYEFO published in December 2016 it was revealed that tranche two of the WPIT will cost $313.5 million over four years. The panel is to consist of IBM, HP, Capgemini, and Accenture; with the latter two currently competing for tranche two below:

• Tranche 2 – Student payments
• Tranche 3 – Job seeker payments
• Tranche 4 – Family payments, including disability and carer payments
• Tranche 5 – Seniors, pensioners and any remaining payments.

It is of note that IBM was also awarded a five-year contract by the DHS in March 2016 worth $484 million. DHS CIO Gary Sterrenberg said: “This innovative and flexible agreement allows the Department to use products, services and expertise through an on- demand model. It ensures value for money for government in maintaining the Department’s existing spend with IBM, with the opportunity to realign technology and services to areas which provide better outcomes for our customers over the five-year term.” And that: “This will also ensure the Government is prepared to transition to new infrastructure with more dynamic capability to support future programmes.”

This week Centrelink’s new robo-public servant was introduced in the media, and it is being tested on the public next month in February. Two robo-assistants will answer questions from the public, one will focus on the National Disability Insurance Scheme, (NDIS) and the other one on student payments. The plan is that if the trials are successful, they will be rolled out to replace traditional public servant roles behind the desk and on the phone in the DHS. Human Service’s Chief Technology officer Charles McHardie, also believes that virtual assistance will have a central role in the future of claims processing at the DHS or in WPIT.

Concerns about the right use of AI are real and there are many examples of it helping to increase inequality in many areas of our lives. Sexism, racism and other forms of discrimination are being built into the machine-learning or predictive algorithms, either intentionally or unintentionally. Machines are taught by humans and this includes any bias that may have. An example of predictive algorithms is Pro Publica’s study of an algorithm, built by a private company, it incorrectly flagged black defendants as “future criminals” more than twice that of white defendants. “The reason those predictions are so skewed is still unknown, because the company responsible for these algorithms keeps its formulas secret,” wrote Microsoft Research principal researcher Kate Crawford in “Artificial Intelligence’s White Guy Problem.”

Australian businesses spend an average of $6 million a year on AI technologies according to recent research by Infosys. Algorithms are being developed for more and more things such as predicting investor responses to market shocks and offering financial advice. The research also found that: “Happily, Australia was the most ethically conscious country of the seven surveyed, with 69% [of businesses surveyed] saying ethical concerns were a major barrier to their organisation’s AI deployment plans, compared to just 33% in the US.”

To date 230,000 debt recovery letters have been sent out to Australians. There’s been countless articles written about it and there are 350 individual stories shared on the Not My Debt web site and a false debt tally of $2,124, 501. Thousands of Indigenous Australians have been sent letters with some just paying it off despite knowing it is wrong. Daniel Hayes told NITV News he was repaying the debt but when he started seeing news articles he stopped paying Centrelink. He said in early January that: “I’m in the middle of repaying them $3350 for apparently not declaring correctly in periods where I didn’t even have a job. When I asked for proof, they told me I had to go through my bank records, so I’ve paid it for a year down to $1600,” he said.

In early January, independent MP Andrew Wilkie, said: “I have had at least four people now approach me in my office who I would describe as presenting suicidal and in all those cases we’ve taken what action we thought was appropriate.”

Mr Wilkie requested an investigation into Centrelink by the Commonwealth Ombudsman before Christmas, they agreed on January 9^th. Deputy ombudsman, Richard Glenn told the Guardian that the matter was “of significant interest to this office”.

“I can certainly say the ombudsman has approved an own-motion investigation into the matter… this one will be self-initiated because we have a number of complaints and there is significant public controversy about the issue. So, it is an inquiry into the issue at large, rather than into a specific complaint,” Mr Glenn said.

“Certainly, there’s enough information from complaints we’ve received and … that it’s an issue of significant interest to this office, and we’ll be pursuing it.”

The focus will be on three areas: the data-matching process used to compare Centrelink records with those of the tax departments; how Centrelink communicated with clients and how the agency managed the fallout.

Centrelink has been referring distraught people to Lifeline and several current and ex-Centrelink employees have told Mr Wilkie that there was little to no training for the recovery program. Mr Wilkie has written to the Ombudsman this week sharing what he has been told. It all reads badly but what jumps out at me, is that it has been alleged that senior departmental staff have been encouraging officers to compete with themselves over who can achieve the highest debt recovery quotas.

While all of this has been going on for weeks, the government denies that there is a problem, although they have agreed to soften some wording in the letters. And they’ve agreed to start sending letters by registered mail so that Centrelink can track if letters are being received. Many people have been unaware of any alleged debts until a debt collector was knocking on their door.

So far only The Australian has reported that there will be a senate inquiry into Centrelink. Perhaps last year’s failed Census inquiry report can assist them with it. The Turnbull and the Abbott governments don’t have a great record with technology. News about Australia’s biggest infrastructure, the National Broadband Network (NBN) is reduced to tiny PR pieces talking up their rollout but neglecting to tell the rest of the story. The census fallout may not be felt now but it will, that data has been compromised and is vital for planning things like infrastructure. Seeing this play out and knowing that the government is nowhere finished with their five-seven year WTIP plan, sends a shiver down my spine. We can take comfort in the fact that it has united us, so many are fighting for those affected but it is bittersweet, because it feels intentional and a government at war with its own people will never end well.

This article was originally published on Political Omniscience.