Generative AI and customer information
Can staff put customer information into ChatGPT?
Not into a public chatbot as a matter of habit, no. The OAIC recommends as best practice that organisations do not enter personal information, and particularly sensitive information, into publicly available generative AI tools.
That is a recommendation about public tools rather than a legal prohibition on all AI, and the honest answer for a business account with assessed terms is different. Which is exactly why the organisation has to decide, rather than leaving it to whoever has a tab open.
General information about Australian obligations, not legal advice. Applies equally to Claude, Gemini, Copilot and similar services.
Is it against the law to put customer details into an AI chatbot?
There is no clause that names ChatGPT. What applies is APP 6, which allows personal information to be used or disclosed for the purpose it was collected for, or for a related secondary purpose the individual would reasonably expect. Putting customer information into a service the provider can access is generally a disclosure, and it has to survive that test.
The OAIC's position is that it may be difficult to establish a reasonable expectation for an AI-related secondary use, and that where an organisation cannot clearly establish one it should seek consent or offer a meaningful ability to opt out. That is a high bar for something being done casually.
The OAIC gives a worked example that maps onto a lot of Australian businesses. Employees at an insurance company experiment with a publicly available chatbot, enter a customer's claim details including health information, and ask it to assess the claim. By entering that information the company is disclosing it to the owners of the chatbot, and for that to comply with APP 6 the purpose has to match what the customer was told at collection.
Two further points sit behind the headline. Once personal information is in a generative system it is very difficult to track or control, and potentially impossible to remove. And privacy obligations apply to the output as well as the input, so a generated summary about an identifiable person is itself personal information you now hold.
Source: Guidance on privacy and the use of commercially available AI products (OAIC, 21 October 2024, updated 17 January 2025).
Does a business account change the answer?
It changes some of it, and not the part most people assume. Business and enterprise tiers commonly differ from consumer products on whether inputs are used to train models, on retention controls and on administrative visibility. Those are real differences and they matter.
What a business tier does not do is decide whether the disclosure fits the purpose you collected the information for, whether the customer would reasonably expect it, whether processing offshore is acceptable, or whether the output was checked. Those remain your decisions regardless of which plan you are on.
One concrete example, because it is the platform most Australian businesses already have. Microsoft states that prompts, responses and data accessed through Microsoft Graph are not used to train foundation large language models, including those used by Microsoft Copilot. Microsoft also states that customers outside the EU may have their queries processed in the United States, the EU or other regions, so an Australian business should not assume the processing stays onshore.
Both of those are useful facts and both are the kind of fact that changes. Provider terms, defaults and regional arrangements are revised regularly. Verify against the provider's current documentation on the day you make the decision, record where you found the answer, and re-check it when your plan or their terms change. That record is the difference between an assessment and an assumption.
What a personal subscription actually means
- The contract is between the provider and the employee, not your business.
- It sits outside your identity system and your MFA policy.
- You cannot see what has been entered, and cannot delete it.
- Nothing happens to it when that person leaves.
- The consumer product's data handling may differ from the business tier.
What an organisational account gives you
- A contractual relationship your business is party to.
- Administrative controls over access, retention and features.
- The ability to remove a person's access when they leave.
- A defined position on training use you can assess and record.
- Somewhere to point when a client asks which service you use.
Can this information go into AI? A decision flow
Six questions, in order. It is deliberately not a shortcut around the organisational decisions: questions three, four and five depend on work the business has to have already done.
- 1. Is it already public? Your published marketing copy, a public standard, a page from your own website. If yes, and it contains nothing about an identifiable person, proceed. Most day-to-day AI use lands here.
- 2. Does it contain personal or sensitive information? Names, contact details, health, financial or employment information, anything identifying a customer, patient, candidate or employee. Also information that identifies someone in combination with what else is in the prompt. If yes, this is a privacy decision, not a productivity one, and it continues below.
- 3. Is the service approved by your organisation? Approved means a named person decided it, on a recorded basis, for this kind of use. If your business has no approved list, the honest answer is no, and the fix is a decision rather than a judgement call in the moment.
- 4. Has the provider and this use been assessed? What the service receives, where it is processed, how long it is kept, whether inputs train models, who else receives it. Assessed once and recorded, with a review date. An unassessed provider is not made safe by being well known.
- 5. Does the purpose match what the information was collected for? This is the APP 6 test and it is the one most often skipped. Would this customer reasonably expect their information to be handled this way, given what you told them when you collected it? If you cannot answer confidently, the OAIC's guidance points toward consent or an opt-out rather than proceeding.
- 6. Who checks what comes back? A named person verifies output before it is relied on, sent or published, particularly where it concerns a person or feeds a decision. The OAIC's position is that a human user should be responsible for verifying accuracy and able to overturn decisions the system produced.
Why this cannot be reduced to a wallet card
Three of the six questions are about decisions the organisation makes once and communicates, not about judgement in the moment. A flow that skipped them would let a staff member reason their way to yes on a service nobody had assessed, which is the exact failure it is supposed to prevent. If your people cannot answer questions three and four, that is the finding.
Five things that are less obvious than they look
Consent cannot be assumed
Sensitive information generally requires consent to be handled, and the OAIC is explicit that consent cannot be implied merely because someone was notified of a collection. Health information, and photographs or recordings from which sensitive information can be inferred, sit in this category.
Removing the name is often not enough
Information can be personal information where it can reasonably be linked with other information to identify someone. A de-identified case summary that names a suburb, a condition and a date may still identify a person in a small community.
Generated content about people is a collection
If a system infers or generates information about an identifiable person, that is a collection under APP 3 and it must be reasonably necessary for your functions. This holds even where the output is wrong, because information can be personal information whether or not it is true.
Cross-border disclosure is the default
Most widely used AI services process outside Australia, which engages APP 8. Your privacy policy should say that information may be disclosed overseas, and you generally stay accountable for what the recipient does.
Third parties you did not choose
Web search, plugins, connectors and integrations can pass what you entered to parties beyond the provider. The OAIC specifically flags interfaces with search engines as a route by which prompt content reaches a third party.
A file is not a prompt
People calibrate their caution to what they type. An uploaded document carries everything in it, including the parts nobody re-read, and it is where the volume of exposure actually comes from.
Does this apply to Claude, Gemini and Copilot as well?
Yes. Australian privacy law is not vendor-specific. The obligations attach to what you do with personal information, not to which company processes it. The ACSC's small business guidance treats ChatGPT, Gemini, Claude and Copilot together as cloud-based AI raising the same categories of risk.
What varies between providers is the detail your assessment has to establish: training use, retention, administrative controls, processing location, subprocessors and incident notification. Those differ between vendors, between tiers of the same vendor, and over time. That is an argument for assessing each service you approve and recording the answer with a date, not for treating one brand as safe and another as risky.
The same applies to the AI features appearing inside software you already licence, which is where most businesses now have AI they never chose. The supplier questions worth asking.
The same action, two different positions
A financial advice practice summarising client meeting notes
An adviser records a client meeting and uses AI to produce a summary for the file. The practice has strong MFA and endpoint controls. The question is whether this is a governed use or an ungoverned one, and the action looks identical either way.
Governed
- An approved service on a business account
- Provider assessed and the assessment recorded
- Clients told at engagement that AI assists with note-taking
- The adviser reads and corrects the summary before it is filed
- The service is in the supplier register with a review date
Ungoverned
- Whichever account the adviser is signed into
- No assessment of retention or training use
- Clients not told, so no reasonable expectation established
- Summary filed as recorded, errors and all
- Nothing in any register, so nobody else knows it happens
Same adviser, same client, same tool. The difference is four organisational decisions and about a day of work, and it is the entire difference between a practice that can explain itself and one that cannot. AI compliance for financial services.
What to do when customer information has already gone in
This is common and it is usually nobody's fault in particular. Handle it as an incident rather than a disciplinary matter, because the first case sets whether you hear about the second.
-
Establish what went where
Which information, when, into which service, on which account, and whether it was typed or uploaded. Ask without consequences attached, because an incomplete answer here makes everything after it guesswork.
-
Find out what the provider does with it
Retention, training use, whether history can be deleted and whether deletion reaches backups. The answer depends on the product and tier, so check the documentation for the account that was actually used.
-
Assess it as a possible breach
Where personal information was disclosed without authorisation, work through the notifiable data breach criteria. Record the assessment and its outcome at the time, including where the conclusion is that it does not meet the threshold. How the NDB scheme works.
-
Reduce what you can
Delete history where the product allows it, turn off training use if it is on, and rotate any credential that was pasted anywhere.
-
Fix the cause, which is almost never the person
Decide what is approved, say what may go into it, tell everyone, and record the decision. In most cases the staff member was doing sensible work with no rule to follow.
The answer should not depend on who you ask
In most businesses, whether client information may go into an AI tool gets a different answer from the owner, the manager and the person doing the work. That is not a training problem. It is a sign the decision was never made in one place.
The decision, written where staff see it
Cleverer's Policy Builder produces an AI Governance Policy from the answers your organisation gives about tools in use, data entry and output review, with adoption, audience and acknowledgement tracked around it.
The provider recorded as a provider
The supplier register captures data types received, storage country, offshore disclosure, sensitive and health information flags, subprocessor visibility, contract status, an owner and a review date.
Somewhere for it to go wrong
An incident register and a structured notifiable data breach assessment, so a case of information going where it should not have gets assessed and recorded rather than discussed and forgotten.
Find out what your business would answer today
Ask three people in your organisation whether client information can go into an AI tool. Then take the Readiness Check and see how the rest of the position looks.
Questions about customer information and AI tools
Can employees put personal information into ChatGPT?
The OAIC recommends as a matter of best practice that organisations do not enter personal information, and particularly sensitive information, into publicly available generative AI tools, because of the significant and complex privacy risks. Where a business uses an approved, assessed service and the use fits the purpose the information was collected for, the position is different. The decision belongs to the organisation, not to the individual employee.
Is entering customer data into an AI tool a disclosure?
Generally yes, if the provider can access it. Entering personal information into an AI system may be a use, if the data stays within your control, or a disclosure, if it is made accessible outside your effective control. For most cloud-based AI services it is a disclosure, and APP 6 applies to it.
Does it help if we remove names first?
It helps and it is not a complete answer. Information can be personal information where a person is reasonably identifiable, including by combining it with other information. In small communities, specialist practices or narrow client bases, a de-identified summary can still identify someone. Judge it on whether the person could realistically be picked out, not on whether the name was deleted.
What about AI meeting transcription with clients?
It captures a conversation involving people other than your staff, then processes and stores it somewhere. Consider what the client was told, whether recording consent is required in your state, where the recording is processed, how long it is kept, and whether anyone corrects the summary before it is filed. It is one of the fastest-spreading AI uses and one of the least assessed.
Is Microsoft Copilot safer than ChatGPT for client data?
The comparison is not really between brands. Microsoft states that prompts, responses and data accessed through Microsoft Graph are not used to train foundation models, which addresses one specific risk. What still has to be decided is which staff may use it, what information may go in, whether processing outside Australia is acceptable to you and your clients, and who checks the output. Verify any vendor claim against the provider's current documentation, because these arrangements change.
Do we have to tell customers we use AI?
The OAIC's guidance is that businesses should update their privacy policies and notifications with clear and transparent information about their use of AI, and that public-facing AI such as chatbots should be clearly identified as AI. Where AI use is a purpose or a disclosure connected to information you collect, it also belongs in your collection notices. More on privacy policies and AI.
