Privacy | Safety

October 15, 2026·9 min read

On August 5, 2021, Apple announced a new proposed feature to combat CSAM (Child Sexual Abuse Material)¹, by which a detection algorithm would scan every single photo in a user’s library and flag if any CSAM content was detected. It was a safety feature, one that could help identify people who may pose a threat to our society. To achieve that end, Apple had to compromise a user’s privacy (after all, their algorithm had to look at photos stored in a user’s personal device and make a judgement on what was being detected), to facilitate safety. Over the next two months, privacy and security researchers raised objections², highlighting how such a feature could pave the path towards mass surveillance at a broader scale. By September 3, Apple paused the rollout of the feature³, more than a year before ultimately abandoning the effort in December of 2022^{4, 5}.

In the Fall of 2013, Pavel Durov launched Telegram⁶. In more than a decade that has since followed, the platform has faced multiple lawsuits across the United States, Russia, Brazil, India, Germany, Australia, among others^{7, 8, 9, 10, 11}, pertaining to the way Telegram handles privacy, and the security risks they pose. There is a case to be made that due to Telegram’s privacy guarantees, the platform has enabled state and non-state actors to carry out illicit operations¹².

Such incidents remind us of the inherent tension between the two desirable outcomes: safety and security of our collective societies, and the human psychological need for privacy necessary for a dignified and meaningful life. Most would agree that an Orwellian Society is dystopian, and yet, it could be maximally safe for the collective society. This begs the question, how do we define utopia, or its inverse variant, dystopia?

What is Utopia?

In his 1516 novel Utopia, Thomas More describes utopia¹³ (satirically meaning “nowhere”) as an idealistic place that cannot exist. In Physics terms, this is what one might refer to as an unstable state of a system. The state may exist, and yet, given the forces at play, the state may not continue to exist in a stable manner.

It is the same mechanism that drives phenomena such as the Tragedy of the Commons, as described by William Forster Lloyd in 1833¹⁴. In this problem, given a finite pool of regenerating resources, if a certain number of consumers continue consuming from the resource pool at a lower rate than that of regeneration, the resource pool continues to exist, and if not, the resource pool continues to deplete, and leads to net negative outcomes for all consumers. The tragedy here is one very human, combined with a lack of trust in the commitment of every participant in the agreement, striving for a personal sense of security, and the competitive human spirit to have more than the rest. In this case, the stable state of every participant honestly subscribing to the agreement of using the shared resource responsibly is utopian, since there are not enough forces to keep the system from becoming unstable.

This same mechanism has presented itself in political systems: monarchs and dictators can arguably perform better and execute their vision more effectively without the checks and balances of the democratic process. Without hindrances to executing absolute policies, the monarch can potentially diminish debates and disagreements, leading to a more coherent society, one where subjects can feel at ease, without questioning what alternative viewpoints can exist. This works, as long as one, the monarch / dictator is capable of understanding what would lead to a better society, and two, the monarch / dictator has the best intentions for their people at heart, and is not plagued by the same sense of mistrust, greed, and insecurity as the rest of their subjects. This is utopian because those are very human tendencies, and there is no good way to stabilize the system against the forces at play. Over a longer span of time, the benefits of a coherent society are outweighed by the lack of normalizing effects that debates and discussions bring. The exact same argument also applies to Communism, where the vision of the system is one that can exist, but not enough forces exist to preserve that state, and usually ends up in an authoritarian / egalitarian rule. Of course, there are flavors of communism (such as certain socialist systems) where this effect is better mitigated than others.

Is Safety utopian?

I defined Safety with a capital S here, to indicate the concept of maximal safety for every individual participant in a system. Safety, however, is hard to define in any particular way: what we find unsafe is a cumulation of our lived experiences colored with exposure to risks and traumas. For now, let’s take two things we may find unsafe as a collective society: CSAM and terrorism, to start with. To attain Safety, we must make sure those elements are entirely absent from the workings of our society. Question is, how do we architect this system? We could have an algorithm to scan personal documents to detect CSAM or terrorism plots. Can this algorithm be 100% accurate? What are the repercussions of a false positive? The suspect may have to disclose their private data while being innocent. Is that acceptable? Once this algorithm exists to detect CSAM pictures and terrorism plot texts, can this same mechanism also not be used to target political dissidents or opponents? Would that scenario be acceptable? What if an elected populist government decided to target religious minorities using the same system in order to “achieve coherence”? Would that be acceptable? The further we go down this road, the more we realize how delicate the democratic system really is. In the times of yore, the right to bear arms in America originated as a counter measure to an authoritarian government. Since then, the rapid expansion of military forces has made the point moot. The right to bear arms is still celebrated by many, but not as a counter measure to the rise of authoritarianism. The concept of Safety, then, might be utopian: while an Orwellian society might have sacrificed their privacy for Safety, the definition of Safety can evolve over time to serve authoritarian purposes.

Is Privacy utopian?

Imagine a world where everyone has absolute privacy: every property has high walls, no monitoring algorithms and systems exist, every car has a tinted window, and every piece of data on the internet cannot be traced back to any source. Most would think this is also a very unsafe world: democratically written policies cannot be enforced due to difficulties in detecting activities. In this world, everyone would need to fend for their own selves: the benefits we derive from a shared society and the contracts that define them, do not exist. Without regulations of food, supermarkets might stock lower quality produce (at a higher profit margin) that pose severe health risks. Without proper enforcement of crimes, criminal activity might go through the roof, impacting every person and business that might want to operate in that system. Imagine not being able to trust anybody in a wild west festered with danger. Is that a stable system? Sure. Is this a world most of us want to live in? Probably not. It seems absolute privacy is not desirable either.

So where do we draw the line?

I’ll take a page out of my long time hobby as a street photographer. In Street Photography, for the most part, we cannot ask for explicit consent for clicking someone’s picture. For most people new to street photography, this is something that they use some time to internalize: the concept of implicit consent. It relies on sound human judgement of what is okay, and what is not. When outside of our homes, in public places, we subscribe to being observed implicitly. If someone explicitly asks to not be photographed, the photographer should comply and delete any pictures they might have captured. This may not feel comfortable to some, and that is the heart of the debate around AI powered glasses which may capture and record events¹⁵. The nuance lies in how explicit it is: if someone pulled out a phone and started recording or clicking pictures, most of us would find that noticeable, same as a street photographer with a dedicated camera in hand. With glasses, this is less obvious, and the contention lies in the subjects being able to tell they are being recorded.

Law enforcement then, applies outside of a certain boundary, which in my opinion should be held sacred. Inside that boundary, lies the messages people send to one another, the things they think about, the ideas they choose to express inside private spaces, their personal lives and habits that describe their lived experiences. Outside of this boundary lies the actions they take in public spaces or more accurately, under public contracts: a guest at one’s private residence still operates under a public contract of being a guest. If the said guest divulges content they consumed under the public contract with the host, even when divulged under confidence, it reaches law enforcement under that public contract. If CSAM or terror motives are detected in the public realm, law enforcement should be able to act on them. While it would be ideal to detect CSAM or terror motives in the private realm to better mitigate the risks the subjects may pose to society preemptively, for the aforementioned reasons, this may not be a net positive for the society such measures plan to serve. The ideal society, in my view, grants adequate privacy to individuals while implementing effective measures to address illicit activities outside of the private boundary.

The boundary between private and public realms, ultimately, is a policy question: to be enacted by elected officials representing the people they serve. Increasingly however, the boundary has been decided by private for-profit corporations legally shielded by terms and conditions their users sign upon using the platforms. Most social media platforms of today operate on a freemium model: the users get to access the platform free of charge, and instead are served ads based on their preferences. I would argue that this breaches the boundary between private and public lives of people: collaborative filtering algorithms effectively map how groups of people think, and use this model of an individual identity to serve ads that are likely to be more effective. The problematic part is, they do so under the guise of keeping people connected. Some AI labs have, in the last few years, offered free access to their Large Language Model applications to users, positioning it as a helpful assistant. To many, such applications play the role of a confidant: users divulge secrets in hopes of getting an outside perspective into a thought, users use them to reflect on their lives, think through their troubles, and confide in them with details they are not comfortable sharing even with the closest of their friends. This trust is broken however, in the event of a lawsuit, when the labs freely divulge the details their users trusted them with¹⁶, in an embarrassing public spectacle of a private moment.

Where do we go from here?

Currently, there is no consensus on where this line should be drawn. This led to the debacle against Flock cameras being operationalized to better address crimes¹⁷, while positioned as license plate readers. Several jurisdictions are now rescinding their programs in the face of mass outcry¹⁸. First of all, we need to define where the border should exist, at a policy level. Then, we might evaluate both public and private programs on where they fall, and if the distinction is made obvious, beyond a 15 page terms and conditions that users agree with to use the platform. The choice architecture that our society subscribes to does not allow a corporate platform to offer a service that violates policy (such as discrimination based on race) if the user agrees to terms and conditions; why should they allow another corporate platform to breach their users’ privacy under agreement?

The societies we build for tomorrow need to be capable of tackling some of the bigger challenges humanity faces. To be best prepared to do so we need to ensure that it’s safe, while ensuring the dignity of a private life.