Your Privacy and AI Tools

Think

Do you understand how AI tools use your data and impact your privacy? How carefully do you read the privacy policies of the tools that you use?

If you haven’t read the privacy policies in detail, you are not alone. These policies are often long, detailed and inaccessible, but they contain a lot of information about how your data is collected and used.

To practice parsing privacy policies, and to learn more about how chatbots might use our data, lets take a closer look at a real privacy policy (in this case, for ChatGPT).

Before starting the quiz below, you could read through OpenAI’s privacy policy – or just take a guess, and see if your expectations line up with reality!

Explore

AI-based tools have certaintly become widespread across many aspects of our personal, professional and academic lives during the past few years!

The quantity of data typically required to train AI models, and the data processing capabilities of these models make AI unlike previous digital technologies. These properties are part of what makes AI (and in particular Generative models and LLMs) so powerful, but they also present serious privacy implications.

Should I be concerned about the privacy risks posed by AI?

AI technologies pose new threats to personal privacy. Don’t just take it from us:

• Everyday Citizens: Public sentiment is cautious when it comes to artificial intelligence, with around half of Canadians regarding AI as a threat to humanity. Privacy in particular is a key concern: a 2025 survey found that 38% of Canadians were concerned about AI systems exploiting their personal data.
• The Canadian Government: The federal Canadian government also acknowledges that AI has the potential to threaten personal privacy: in the words of Mark Carney, because of AI, “the privacy of Canadians is under threat“. Addressing and mitigating threats to personal privacy is part of one of six pillars in Canada’s National Artificial Intelligence Strategy.
• Privacy Watchdogs and Commissioners: As discussed in the video, Canadian privacy commissioners (Federal, Quebec, BC and Alberta) found that ChatGPT had violated Canadian privacy laws. The website of the federal privacy commissioner also cautions users on the privacy risks posed by AI, including through resources on protecting your privacy, and related research articles.
• Scientists and Researchers: Many researchers are also raising alarms when it comes to AI’s privacy impacts. For example, the director of The University of Toronto’s Citizen Lab spoke in a recent interview about how AI-based technologies can facilitate mass surveillance, and post a threat to democratic processes. Stanford University’s Human Centered Artificial Intelligence Center has also released several reports, includng on how AI relates to privacy legislation, and on the privacy implications of AI chatbots.

What are these privacy risks exactly?

1. Data harvesting: AI is data extremely hungry. Training an AI model requires enormous amounts of data, which is often scraped from all across the internet, without the permission of the people who created or uploaded the data. This sort of unrestrained data harvesting was the focus of the Canadian Privacy Commissioners’ ruling for ChatGPT.
2. Input retention and use: Not only are many AI models trained on data scraped from the internet, they may also retain any data you enter or upload to refine future models or to sell. For an example of exactly what data is stored and how it is used, you could read ChatGPT’s privacy agreement. In short, ChatGPT collects and stores personal account information, location information, your browser cookies, your IP address, all chat logs and all information you upload to ChatGPT, and personal information that it might buy or receive from other vendors. It then uses your personal information to train new models (although you can opt-out of this), and shares it with various undisclosed online vendors and other parties.
3. Data re-identification: AI models may have access to a great deal of online knowledge, and can identify patterns and create linkages across disparate datasets. These capabilities can enable ‘re-idenitification’ of anonymous data. In other words, AI may be able to deduce the identity of a person from data that might appear anonymous, blurring a distinction between personal data and ‘de-identified’ data that is central to existing privacy laws.

Who would want my information – and how might they use it?

• State Surveillance: As privacy researcher Jennifer King explains, “Today, it is basically impossible for people using online products or services to escape systematic digital surveillance across most facets of life—and AI may make matters even worse.” Without restrictions and protections of citizen rights, AI-based tools could allow governments and leaders to identify and monitor dissidents, to automatically censor communications or online content, and otherwise infringe on civil liberties and undermine democratic processes. A recent paper from the UN expands on these risks, and found that “at least 75 countries had employed Artificial Intelligence (AI) technologies for surveillance”.
• Law enforcement: Law enforcement might use AI to identify and locate suspects, including through AI-based facial recognition systems, through AI-based monitoring of social media and surveillance footage, and though the analysis of enormous datasets of emails and communications. For example, in the United States, Immigration and Customs Enforcement (ICE) extensively uses AI-based tools to locate and track people. Law enforcement agencies across Canada (including the Toronto Police Service and Vancouver Police Department) have also begun to make use of AI-based algorithmic policing. These applications of AI are particularly problematic given that these facial recognition systems have been demonstrated to exhibit racial bias, and have led to wrongful arrests.
• Corporate Surveillance: Businesses and corporations may be particularly interested in your spending habits and interests. AI enables algorithmic pricing, which could for example, allow grocery stores to charge you different prices based on your personal information, your interests and your shopping history. Similarly, insurance companies and banks could use AI (and its knowledge of your habits and attributes) to evaluate your deductibles or credit score.
• Your Intellectual Property: AI models may use your intellectual property (your poetry for example, or your art) and your likeness, to generate new content, changing modifying and reusing your intellectual property for profit.
• Identity Theft: Given photographs, recordings or videos of you, Generative AI could facilitate identity theft and other forms of cyber crime. For example, generative AI could be used to generate deepfake images or videos that purport to show you, while AI voice cloning could be used to impersonate you over the phone.

What can be done to mitigate these risks?

Researchers, developers, policy makers and educators are all working to address the privacy risks associated with AI. For example:

• Privacy Legislation: In the last few years, Canadian privacy legislation hasn’t changed, but the technological landscape that it governs certainly has. The Canadian governments “National Artificial Intelligence Strategy” outlines several updates to privacy legislation.
• Privacy By Design: Privacy by Design centers around the idea that privacy considerations should be proactively integrated into various aspects of a technology or system throughout the design process. The privacy and data sovereignty challenges associated with modern AI are not limited to its use, implementation or regulation – these concerns are inextricably linked to the way existing technologies work on a technical level. In line with Privacy by Design, researchers and developers are working on AI systems and technologies that inherently integrate privacy and data sovereignty.
• Protecting Privacy through Technology: Researchers and software developers have developed creative tools to help users protect their privacy in the face of AI. Check out CrankGPT, which runs AI models on a local, disconnected (private) platform. Take a look at Nightshade, which slightly alters your photographs or art so that they can’t be used to train AI models. Or look into Reflectacles sunglasses, which are designed to fool facial recognition systems!

What can *I* do to protect my privacy?

• Use trusted tools: Ideally it would be possible to find AI tools that are tightly bound by Canadian privacy laws and regulations, but the international and distributed nature of the internet, and of AI corporations, and the shortcomings and gaps in existing privacy laws makes this difficult. You could start by researching whether your workplace or field has any more specific reccomendations or requirements. For example, healthcare workers need to use PIPEDA-compliant AI tools, and UBC has cleared specific AI tools for instructional use.
• Change privacy controls: Check the privacy agreements of AI chatbots and other AI applications. In some cases you are able to change how your personal data is collected, retained or used.
• Watch what you share with AI: You should be careful not to share any personal or sensitive information with GenAI tools. Even though some GenAI providers may not use your input to refine their products, your data is still subject to data breaches if hackers target these tools. Remember also that AI re-identification capabilities might mean that ‘anonymous’ data could still be traced to you. Canada’s Privacy Commissioner has a list of specific reccomendations of things to keep in mind when interacting with AI chatbots.
• Be careful what you upload elsewhere: Remember that AI models might be trained on data uploaded across the internet. Do you upload your personal poetry to a blog? Do you upload your academic work to a public portfolio? Do you have a Nest Camera? Do you upload photographs of yourself and your friends and family online? That data might all be used to train new AI models.
• Advocate for your rights: What do you think of the use of AI and facial recognition by your local police force? What changes would you like made to Canadian privacy legislation? Stay informed and advocate for your right to privacy!

Links

Government Resources

• Your Privacy and AI Chatbots | Office of the Privacy Commissioner of Canada
• Privacy Research | Office of the Privacy Commissioner of Canada
• Canada’s National Artificial Intelligence Strategy: AI For All | Government of Canada
• Joint Investigation of OpenAI OpCo, LLC | Office of the Privacy Commissioner of Canada

Research Papers (and research summaries)

• AI Ethics: Integrating Transparancy, Fairness and Privacy in AI Development | Radanliev
• Algorithmic Policing in Canada Explained | Citizen Lab
• Navigating the Intersection of AI, Surveillance, and Privacy: A Global Perspective | Francis (UN Science Policy Brief)
• Rethinking Privacy in the AI Era: Policy Provocations for a Data Centric World | King
• Be Careful What You Tell Your AI Chatbot (User Privacy and Large Language Models: An Analysis of Frontier Developers’ Privacy Policies) | King
• Optimism Meets Uncertainty and Even Fear in Canada’s AI Landscape | abacus data

News Articles and Websites:

• How to Use Generative AI Tools While Still Protecting Your Privacy | WIRED 
• Generative AI: Privacy and tech perspectives | iapp
• Generative AI and Data Privacy | PEBC
• When Artificial Intelligence Gets it Wrong | Innocence Project
• ICE Uses a Growing Web of AI Services to Power Its Immigration Enforcement and Surveillance | American Immigration Council
• Is Data DeIdentification Dead? Why the AI privacy risk isnt what it leans but what it figures out | Geist
• Canada isn’t doing its part to stop AI government surveillance | Citizen Lab
• Can Mark Carney get Canadians to Trust AI? | CBC News

Discuss

The Digital Tattoo Project encourages critical discussion on topics surrounding digital citizenship and online identity. There are no correct answers and every person will view these topics from a different perspective. Be sure to complete the previous sections before answering the one or both questions below.

• Did anything in the privacy policy exercise surprise you? Is the data that you share with chatbots as private as you had imagined?
• How concerned are you about the privacy risks associated with AI on a scale of 1 (not concerned at all) to 10 (super worried)? Are there any aspects of the situation that you find particularly worrying?
• What are some steps that you can take to protect your privacy when using AI tools? Do you plan to change the type of information that you share with AI chatbots or post online?
• What actions do you hope to see from AI companies and from the government? Can you advocate for or contribute to those changes?