AI helpers and health privacy

Most psychologists assume their digital tools provide reasonable privacy, not perfect, not impregnable, but private enough for everyday clinical work. When we think about the risk of privacy breaches, we tend to think of either the skilled teen hacker, or omnipresent state level surveillance.

The skilled teenage hacker is mostly defeated with decent password hygiene, up to date software, and perhaps even a VPN.  Nation state intelligence (e.g. Office of National Intelligence, or the Central Intelligence Agency) is rarely interested in psychologists, but importantly we know there is little we can do to defend against this threat anyway, so we chalk this risk up as “acceptable” and go about our day.

AI and specifically Large Language Models (LLMs) have shifted this landscape by introducing a new threat; a hybrid between spyware and state level surveillance.  We are now being surveilled by our digital tools, mostly for advertising and potential marketing purposes, but the methods used to collect and store this data, opens up a whole new risk vector for health professionals. 

Have you ever asked ChatGPT: how to change a washer on a tap, or how to make a massman curry, or what type of flower is in this photo?  AI answers these questions shockingly well not because it is smart, but because it has had exceptional training.  AI training takes massive data centres, extra-ordinary processing power and access to the knowledge of the world; the library of congress, Wikipedia, and yes even Reddit and Twitter.  Your AI doesn’t know how to change a tap washer because it has specialist plumbing training, it knows because the massive blob of data ingested as part of its training included a few people who knew how to change a tap washer.

So how does this affect us in healthcare?  

Health data is special.  The Australian Privacy Principles (APP) see https://www.oaic.gov.au/\_\_data/assets/pdf\_file/0019/258121/Consolidated-APP-guidelines.pdf treat health data as a special category of information, by definition it is considered sensitive. Chapter 6 APP defines the use and disclosure of personal information.  If a shoe retailer knows your shoe size and uses that information to market you a discount on shoes in that size, that is a perfectly legitimate business practice.  If a podiatrist knows your shoe size and gives that information to the same shoe retailer, this is a major violation of the Australian Privacy Principles. This health data exists in a different category to typical business data.

Right now, there is an arms race going on between AI companies, as each of them tries to make their model better and thus indispensable to society.  Alphabet (Google), Meta (facebook), OpenAI (ChatGPT), Microsoft, Apple, X, Amazon are all reaching deeper and wider into the sum of human knowledge to train their next model to be fractionally better than their competitors. To do this, they need data, and each company is looking for where they can get more.

Wearable devices, free email accounts, casual conversations overheard by your voice assistant, all of these are being fed into the LLM data centre blender to make AI smarter.

Some of these companies have access to data that you as a psychologist (and the Australian Privacy Principles) would consider sensitive and protected.

To what extent does Microsoft read the contents of your “one drive” in order to train its AI? Does Apple train apple intelligence with iCloud data and how about Google drive?

Like many things when answering this question, it depends. With Google it depends on whether you are talking about your commercial google drive account, or your free Gmail with a few GB of storage, or are you talking about the chat you had with Gemini.  The answer in each of these cases is different, but it is important to understand, these companies that we have placed at the centre of our IT ecosystem are now incentivised by the AI arms race to find any loophole to grab a richer data set to train their next model or product. 

Apple has been forward thinking on this front and recently added a clause to their terms and conditions for iCloud specifically stating that iCloud is not suitable for storage of sensitive health data.  The reason Apple uses HIPAA-specific language is to legally shield itself from being considered a “business associate” liable for data breaches, reinforcing the point that they are not accepting the responsibility required for handling sensitive information.

I(c) iCloud terms and conditions

“If you are a covered entity, business associate or representative of a covered entity or business associate (as those terms are defined at 45 C.F.R § 160.103), You agree that you will not use any component, function or other facility of iCloud to create, receive, maintain or transmit any “protected health information” (as such term is defined at 45 C.F.R § 160.103) or use iCloud in any manner that would make Apple (or any Apple Subsidiary) your or any third party’s business associate.”

While this clause uses HIPAA specific language such as “covered entity” and “business associate”, it is effectively a fairly blunt warning that iCloud storage should not be considered safe for health data.

Just think about that for a second.  

While iCloud is probably not the number one cloud server option for business, iPhones are the number one consumer telecommunications device.  iCloud is turned on by default when setting up an iPhone, many iPhone users also have a Mac laptop or iPad, and turning on the sync with iCloud feature is fairly common due to its convenience.  Do any mac using clinicians ever leave client reports in on their “Desktop” folder, or download GP health care plans into their “Download” folder?  

But these concerns are not limited to Apple.  Microsoft was subject to a huge online backlash about a controversial new feature quietly rolled out in Windows 11 called “Recall”.  Recall uses what is called “see what you see” technology.  Which is a fancy way of saying that it takes a screen shot of your desktop every few seconds, feeds that image to its local AI and neural processor which takes a look and describes what it is seeing.  This is super useful when you are struggling to do something, because Co-pilot already knows what you are struggling with and can give tailored advice.  It’s also risky in terms of client privacy.  Co-pilot is literally screenshotting your desktop, client names and all and feeding that into its AI engine. Following this backlash, Microsoft changed tack and recall is now “off” by default, but will any health professionals turn it on without realising the privacy implications?

Finally, Open AI, the parent company of ChatGPT was recently the subject of a US Federal court ruling forcing them to store chat records indefinitely so that they may be subject to subpoena.  Indeed their privacy policy included the following clause;

Privacy Policy

• Legal Requirements: We may share your Personal Information, including information about your interaction with our Services, with government authorities, industry peers, or other third parties (i) if required to do so by law or in the good faith belief that such action is necessary to comply with a legal obligation, (ii) to protect and defend our rights or property, (iii) if we determine, in our sole discretion, that there is a violation of our terms, policies, or the law; (iv) to detect or prevent fraud or other illegal activity; (v) to protect the safety, 

security, and integrity of our products, employees, or users, or the public, or (vi) to protect against legal liability.

Those of us who work with families or with clients who are subject to litigation such as through third party payment schemes like workers compensation know just how easy it is for a lawyer to obtain a subpoena.  We should expect our ChatGPT research to be subject to court scrutiny in the not too distant future.

In this landscape, psychologists and all health professionals need to evolve our view of privacy breaches as well as their defences.  We have rushed to adopt these AI systems for the obvious benefits they provide, but now we need to slow down and think about how we implement them in a way that is safe for our clients.

Clinical Governance and Digital Competence

Governments are also interested in the benefits and risk in digital tools, AI and privacy considerations. Healthcare professionals vary in their digital competence and there has been little guidance as to standards until recently.

The Psychology Board of Australia has released a Professional Competencies for psychologists Fact Sheet on compliance with expectations of digital competence as psychologists https://www.psychologyboard.gov.au/Standards-and-Guidelines/Professional-practice-standards/Professional-competencies-for-psychology.aspx. The Fact sheet notes that digital tools that are used for information, communication, client record keeping, storing data and practice management systems and any use of AI are included in digital health.

Digital competency and clinical governance in terms of safe, effective and competent practice (as described in Principle 1 of the Code of Conduct) will require:  

• an annual audit of gaps in technology and how you can improve in your practice. 
• A focus on developing an understanding of staff and clients’ needs and inherent risks in current digital platforms
• Consideration of compliance with the Australian Privacy Principles and expectations of health confidentiality in the use of telehealth platforms, practice management software, online booking systems and cloud storage and back ups.
• Be sure that only information that is necessary for clinical care is collected 
• Clear privacy policies in terms of data collection, digital consent, cyber security and data breach incident prevention and response 
• New staff onboarding processes that outline the how and why of digital system safety for clients and the practice.

These digital clinical governance recommendations are significant and we should all have them place as soon as possible. 

Where to next for Psychologists

What follows is a series of potential immediate actions, starting with the lowest level of easy to implement, and progressing to more advanced.  It is suggested that clinicians consider their risk profile and consider which level of response is right for them.  For those who run teams, it is suggested that not only strategic clarity is needed, but also policy, induction of all staff and on-going training.

1.  Understand your tech stack

Many readers will baulk at the term “tech stack”, but in truth it’s just a nerdy way of saying what software you use.  It’s called a stack because some software has to be built on other software; you can’t have an application without an operating system.

Operating systems: Windows 11, MacOS, Linux. Each of these operating systems has its own challenges in terms of privacy, however it is probably fair to say that Linux is the one least likely to be actively trying to undermine your privacy by invasive use of AI.  

Internet Browser: most software we use daily is delivered to us via a browser (MYOB, Pracsuite, Halaxy, NovoPsych and others).  Browser preferences matter. Largely speaking, Chrome should be viewed as spyware that exists primarily to gather data on you so it can be sold to advertisers. Firefox is a lot better, Brave is better still. All of these browsers have preferences that can be set by the user and even Chrome can offer greater levels of privacy by carefully selecting these preferences.

Software as a Service (SaaS). Have you noticed that you don’t really “buy” software and install it anymore?  Instead, you subscribe to someone who produces that software and runs it on their servers.  Many of these SaaS companies are also rushing to implement AI within their software.  From the AI help menu through to full AI based analysis and insights of your data.  Many of these SaaS products collect client data via an AI assistant (e.g. Jax in Xero). Your practice management software for client files, your accounting software for invoices, your marketing software for newsletters and reminders, your secure messaging software for referrals and letters may all collect client data. 

Immediate actions that are recommended for all  

• Look at Operating Systems, Browser and SaaS products that touch ANY client data, and disable AI by default.
• Consider using more secure versions of all of the above, Linux instead of Windows, Brave instead of Safari etc
• Read the terms and conditions of any SaaS product that has access to client data, do not enable AI or cloud features unless you are comfortable with the risk outlined

The next step: The two browser rule

Most of us have a dozen different services we use regularly.  Some of the potential data leaks will happen due to one piece of software gaining access to data in another piece of software (think about how often you use google to login to different websites).  But let’s not forget human error, a quick copy and paste of the wrong block of text can be a disaster, as can forwarding an email from your email software to your AI software.

In the medium term: Recommendation

We strongly recommended that at the bare minimum clinicians use the two-browser rule.  One browser for any software that touches personal client data, and another one for watching YouTube, scrolling Facebook and use of your preferred LLM. This created two clear contexts and reduces the risk of data leakage through both human error and software design.

For advanced users: Local AI

It is hard to deny the incredible benefits derived from AI usage, but many of the risks come not from the AI itself, but from the companies serving these AI products to us. While training LLMs uses more electricity than a medium sized European country, running them is achievable on a local machine.  Latest cutting-edge models cannot be run on consumer machines, but most of us don’t need the absolute greatest AI, we just need something useful. This is achievable (albeit a bit slow) on a reasonably powered Intel, AMD or ARM machine with decent RAM and preferably a good GPU.

While this solution is appealing, it lacks the convenience of simply logging into your favourite LLM. The cost of many of these machines is still at a level that is just out of reach for most people, there is a technological learning curve associated with running AI locally and the performance is also just a little bit below what everyone really wants.  

We expect this to change in the next 2 years, as newer more powerful machine decrease in price. Both the privacy risk and risking subscriptions costs may drive many businesses toward local models

For the extra cautious: The Two computer rule (one of them Linux)

To be extra sure that your handy personal assistant and your health files don’t paths, the simplest solution is to have two laptops. Many will baulk at the idea of paying for 2 expensive computers, but the good news is that the most secure laptop is also the cheapest.

As noted above, Linux is one of the most secure operating systems in terms of LLM data breaches, there is no company called “Linux” that wants your data, Linux is just an operating system that has been around for 30 years, it is the leading operating system in data centres because it is quickly and reliable, and growing privacy concerns have seen it double its market share from 2% in 2023 to over 5% in 2025.  Linux never adds software you don’t want, in fact it pretty much does nothing unless you expressly instruct it to.

While Linux has evolved a long way from an operating system for geeks (in fact modern Linux options look a lot like windows or MacOS), it does still require a small amount of tinkering.  This means it may not be the ideal software for daily use for you.  However, because it is so light on resource use, Linux can turn a 5 year old windows machine that is ready for e-waste into a modern snappy laptop.  

Rather than throwing away your current laptop when it gets slow, stick Linux on it and have a play.  Consider using it for your sensitive health records.  There are hundreds of tutorials on how to do this so we won’t go into that here, other than to say that one of the barriers to Linux is that there are a dozen of more different versions of it and people get overwhelmed by the choice.  Download Ubuntu if you want the most stable and popular version, Download Fedora if you want the prettiest, Download Mint if you want the one that is marketted as beginner friendly.

Summary

The principles in this paper are not prescriptive.  The authors are not saying that anyone using Windows is doing the wrong thing.  Rather we are highlighting an issue with creep in the current technology.  While these companies are jostling for dominance, privacy and particularly health privacy is not a priority in their reasoning. Governance guidance will help you consider what you think is your threshold for acting on these dilemmas. At a minimum it is important to audit your systems. It is not unreasonable for the health professional to take that as a given and to start thinking about how they can ensure protection of client data and confidentiality in their practice and business.

Kaye and I have put a practical privacy checklist and short videos on a free PractAus page.

References

Apple Inc. (n.d.). iCloud terms and conditions.

(2024, November 22). Microsoft rolls out controversial Recall feature for Windows 11 beta testers. PCMag Australia.

The New York Times Company v. Microsoft Corporation, No. 1:23-cv-11195 (S.D.N.Y., filed December 27, 2023). CourtListener.