What to never put into an AI tool
I was talking to a business owner recently who had Claude's Cowork running on his laptop, using it to sort out years of accumulated files. Sensible use of the tool. He was thrilled with it.
He also had no idea there was a setting that governs whether his conversations get used for training, no idea what he'd selected when he signed up, and no idea how long any of it was being kept. His laptop, like most business owners' laptops, has things on it he wouldn't email to a stranger.
He hadn't done anything reckless. He'd done what the tool invited him to do. And that's the part worth sitting with, because everything else in this article assumes you're the one deciding what goes in — and with this class of tool, you aren't.
What follows is the list I give clients: what shouldn't go into a general AI tool, why, and what to do instead. I've written it for Australian businesses, because the legal picture here is different from what you'll read on American blogs, and the difference matters.
The scale of it
A July 2026 survey of 500 employed US adults, commissioned by business-litigation firm Kolmogorov Law and run through the Pollfish platform, found 38% had entered work information into a personal AI account their employer didn't control. Nearly a quarter had pasted internal emails, memos or documents. Around one in ten had put in customer information, financial figures, contracts, or HR records.
The part that should worry business owners most is that only about a third of those workers said their employer had a clear written policy on what could be shared with AI tools. Roughly two-thirds were improvising.
That's American data and I'd expect Australian numbers to look similar rather than identical. But it matches what I see, and in almost every case there's no malice involved. It's a helpful employee, a deadline, and a tool that makes the task easy.
What shouldn't go in
Customer personal information, particularly in bulk. Names on their own are usually fine. A name plus an address plus a phone number plus a date of birth is a different beast, and a spreadsheet of two hundred of them is different again. Volume changes the risk profile completely.
Payment and banking details. Card numbers, account details, anything that would let someone move money. There is no summarising task worth this.
Employee records. Tax file numbers, superannuation details, medical certificates, performance management notes, incident reports. HR data is some of the most sensitive material in any business and it gets pasted into AI tools constantly, because writing HR documents is unpleasant and AI makes it faster.
Anything under a confidentiality agreement. A client's drawings, their tender pricing, their unreleased plans. If you signed something promising to protect it, uploading it to a third-party service carries contractual consequences on top of the privacy ones. On large construction and infrastructure projects this is the one I flag hardest.
Login credentials and API keys. Sounds obvious, happens all the time — usually when someone's troubleshooting a system and pastes in a config file without reading it properly.
Safety incident detail involving named people. Injury descriptions attached to names are sensitive information under Australian privacy law and attract stricter handling than ordinary personal information.
Anything currently in dispute. If there's a legal matter running, or one that might, talk to your lawyer before that material goes near a general AI tool.
When you're not the one choosing
Pasting is a decision. You look at the text, you copy it, you make a judgement. Even a bad judgement is a judgement.
File agents work differently. You point one at a folder and ask it to organise, summarise or find something, and it decides what to open. The old superannuation file, the folder of scanned licences, the tender pricing from a job that didn't go your way, the personal tax return that ended up in the business drive years ago. You're not reviewing any of it. You're delegating the review.
That's not an argument against these tools. They're the most useful thing to arrive in small business software in a decade, and the file-sorting job is one they're very good at. But it does mean the safeguards have to move from "think before you paste" to "decide in advance what the tool can see, and on what account".
Two settings do most of the work.
The training setting
On Claude's Free, Pro and Max plans, whether your conversations are used to improve the model is a choice you were asked to make once, and most people don't remember what they picked. You can change it any time in your privacy settings. OpenAI's consumer tiers have an equivalent control. Business and team plans on both platforms don't train on your content by default, which is a large part of why they're worth the twenty dollars.
Retention
This is the number almost nobody knows. On Claude's consumer plans, if the training setting is on, Anthropic can keep conversations in de-identified form for up to five years. If it's off, the retention window is 30 days. Same product, same subscription price, one toggle, and a difference of nearly five years in how long your files sit on someone else's system. Team, Enterprise and API accounts sit outside that arrangement entirely.
The third safeguard isn't a setting, it's a habit. Don't point an agent at your whole drive. Make a working folder, put in it only what the task needs, and run the tool there. It takes two minutes and it converts an open-ended risk into a bounded one.
Three that catch people out
Beyond the obvious list, there are three quieter risks I flag on every engagement.
Shared chat links. Both major platforms let you share a conversation by link, and those links have, at various points over the last few years, ended up more publicly accessible than the people creating them expected. Treat a shared chat link the way you'd treat a public web page.
Meeting notetakers. An AI notetaker joining your call is recording a conversation. Recording consent rules in Australia vary by state and territory, and "everyone could see the bot in the participant list" is a weaker consent position than most people assume. Ask at the start, out loud.
Browser extensions and free tools. The AI feature bolted onto a free browser extension has a terms of service too, and it's usually far less protective than the major platforms'. Free tools with unclear ownership are where I'd expect problems to surface first.
The Australian legal picture, and the bit most people get wrong
The Privacy Act 1988 has long exempted most businesses turning over $3 million or less. As at mid-2026 that exemption is still in force. It has not been repealed, despite a great many blog posts claiming otherwise, and while the government has committed to a second tranche of reform it hasn't published a firm timeline.
Treating the exemption as a shield is still a mistake, for three reasons.
It's being dismantled around the edges
From 1 July 2026, businesses providing designated services under the anti-money-laundering regime — accounting, legal, conveyancing, real estate and others — came under the Privacy Act for their AML-related data handling regardless of turnover. That pulled in more than 100,000 small businesses, plenty of whom haven't noticed. Health service providers, businesses trading in personal information and Commonwealth contractors were already covered.
The statutory tort doesn't care what you turn over
This is the important one. Since 10 June 2025, an individual has been able to sue directly for a serious invasion of privacy, and the tort explicitly reaches beyond the entities the Privacy Act covers. The plaintiff doesn't have to prove financial loss. So a small business that's technically exempt from the Australian Privacy Principles can still be taken to court by the person whose information it mishandled.
Enforcement has teeth now
The first civil penalty under the Privacy Act landed on Australian Clinical Labs at $5.8 million. Maximum penalties for serious or repeated breaches reach $50 million, three times the benefit obtained, or 30% of adjusted turnover, whichever is greatest. Those are big-end-of-town numbers, but they tell you about a regulator that has shifted from educating to enforcing.
One more date for the calendar. From 10 December 2026, businesses covered by the Act must disclose in their privacy policies where computer programs make or substantially inform decisions that significantly affect people's rights or interests. If you're using AI to screen job applicants or assess customers, that's you. The OAIC has said it will publish guidance ahead of the commencement date.
The OAIC has also published guidance specifically on using commercially available AI products, with a checklist at the end of it. Free, readable, and the best starting point I can point you to.
What to do instead
Learn the swap
Before you paste anything, replace the identifying details with placeholders. Client A. Site 1. The supervisor. Most of the time the AI doesn't need real names to do the job — it needs the structure and the facts. Summarising a contract works just as well with the parties renamed.
Know which account you're in
Consumer plans on the major platforms use your content for training by default with an opt-out available, while business and team plans don't train on your content by default. For most small businesses the fix is a $20 business seat and ten minutes in the settings, not a ban on AI.
Check where the data goes
Under Australian privacy law, if you disclose personal information to an overseas recipient you generally remain accountable for how they handle it. Read the provider's terms on data location and retention before you decide what's allowed in.
Seven things to do this month
Write a one-page AI policy. Not twenty pages — one, listing what's fine, what needs the swap, and what's never allowed. Keep the "never" list short enough that people remember it.
Audit your accounts. Find out who's using AI for work and on what plan, then move work use onto business accounts.
Check the training setting on any consumer account being used for work, and turn it off. On Claude that also drops retention from five years to 30 days.
Scope your file agents. Point them at a working folder, never the whole drive.
Rule out shared chat links for anything work-related you wouldn't publish on your website.
Set a notetaker rule and ask for consent verbally at the start of every recorded call.
Read the OAIC's guidance on commercially available AI products and work through its checklist. Half an hour, and you'll be ahead of almost every business your size.
None of this is an argument against using AI. The businesses that get burned are the ones that never sat down and decided what was allowed in. Decide it, write it down, and tell your team.
Questions owners ask
Is it safe to use the free version of ChatGPT or Claude for work?
For anything containing no business information, yes. For anything else, the consumer tiers are the wrong tool — they don't come with the same commitments about training and retention that a business account does. If AI is genuinely useful in your business, a business seat is the cheapest risk control you'll ever buy.
Does the Privacy Act even apply to my small business?
Possibly not, and that's less reassuring than it sounds. Most businesses under $3 million turnover are still exempt. But since June 2025 an individual can sue directly for a serious invasion of privacy whether or not the Act covers you, from 1 July 2026 the exemption no longer covers accounting, legal, conveyancing or real estate for their AML-related data, and it never covered health service providers. It also does nothing about your contractual obligations to clients, which is usually what bites first.
What if someone has already pasted customer data into a chatbot?
Don't panic, and don't ignore it. Work out what went in, which account it went into, and what that provider's terms say about retention. Delete the conversation if you can. If the exposure is significant, take advice on whether it triggers notification obligations — and treat it as the prompt to sort out the approved-tool question rather than a disciplinary matter.
Can I put a client contract into AI to summarise it?
It depends on what you've promised that client. Many commercial contracts and NDAs restrict disclosure to third parties, and a third-party AI service can be exactly that. In a business-tier tool that doesn't train on your content the risk is much lower — but check the confidentiality clause before it becomes a habit.
This article is general information, not legal advice. Privacy obligations turn on the specific facts of your business — get advice from a qualified practitioner if you're unsure whether the Act applies to you.
