کار درست را انجام بدهDoing the Right Thing

This closing chapter is unlike the rest of the book: instead of architecture and algorithms, it talks about consequences. Its path can be summarized in five stops:

1. The engineer’s responsibility — technology is not good or bad by itself; what matters is how it is used and how it affects people.
2. Automated decisions about people — from predictive analytics and the algorithmic prison to bias rebuilt from proxies.
3. Feedback loops — when a system’s output becomes its own next input: the exclusion spiral and the echo chamber.
4. Surveillance and consent — from behavioral tracking to mass-surveillance infrastructure, and why “consent” is usually not meaningful.
5. Power and the way out — data as a toxic asset, the lesson of the industrial revolution, law and self-restraint.

Throughout the book we compared data-system architectures and learned techniques for building reliable, scalable, maintainable applications — but one fundamental piece was missing. Every system is built for a purpose; the purpose may be as simple as “making money”, but the consequences can reach far. We who build these systems are responsible for weighing those consequences carefully and making sure our decisions do no harm.

We talk about data as if it were an abstraction, but remember that many datasets are about people: their behavior, interests, identity. Such data must be handled humanely and respectfully; users are humans too, and human dignity comes first. Guidance exists for engineers — such as the ACM Code of Ethics and Professional Conduct — but in practice it is rarely discussed, applied, or enforced; the result is sometimes a careless attitude from engineers and product managers toward privacy and a product’s negative consequences.

Technology is not good or bad by itself — what matters is how it is used and what effect it has on people. That is as true of a search engine as of a weapon. The burden of moral responsibility is on us; it is not enough for a software engineer to focus only on the technology and ignore its consequences.

Unlike most of computing, the concepts at the heart of ethics have no fixed, precise meaning; they need interpretation, and interpretation can be subjective. “Good” and “bad” have no crisp definitions, and serious discourse about them is rare among computing professionals. Reasoning about ethics is hard — but too important to ignore. Ethics is not passing a compliance checklist; it is a participatory, repeating process of reflection, in dialogue with the people involved, with accountability for outcomes.

Predictive analytics is a big part of the “big data and AI” excitement — and full of moral dilemmas. Predicting the weather or the spread of a disease is one thing; predicting that a convict will reoffend, a loan applicant will default, or an insurance customer will file an expensive claim is another. The latter directly shape a person’s life.

Payment networks want to catch fraud, banks want to avoid bad loans, airlines want no hijackings, and companies want no unproductive hires. From their view the cost of a missed opportunity is small but the cost of a bad loan or a problematic employee is large — so caution, saying “no” when in doubt, is natural. But when the algorithmic decision becomes pervasive, someone labelled “high-risk” by the algorithm (rightly or wrongly) faces a flood of such “no”s: systematic exclusion from jobs, flights, insurance, rentals, financial services and the rest of society — a constraint on individual freedom so large it has been called the “algorithmic prison” (algorithmic prison). In rights-based countries the justice system presumes innocence; yet automated systems can shut a person out of society with no proof of guilt and almost no chance of appeal.

Automated decisions open the question of responsibility and accountability. If a human errs, they can be held to account and the harmed person can appeal. Algorithms err too — but who answers for it when they fail? Who is liable when a self-driving car crashes? If automated credit scoring systematically discriminates against a race or religion, is there any remedy? If an ML system’s decision comes under judicial review, can you explain to the judge how the algorithm decided? Nobody should be able to dodge their own responsibility by blaming the algorithm.

Compare classic credit bureaus: a bad score makes life hard, but at least it rests on relevant facts from the person’s own borrowing history, and file errors can be corrected (however grudgingly). Machine-learning scoring instead takes far broader, far more opaque inputs; understanding how any particular decision was made — and whether it was unfair — becomes much harder.

A credit score summarizes “how did you behave in the past?”, but predictive analytics usually runs on “who is like you, and how did people like you behave?” — that is stereotyping people, for instance by neighborhood (a close proxy for race and class). What about whoever lands in the wrong bucket? And if the decision was wrong because of bad data, remedy is nearly impossible. The same power can do good (focusing help on the neediest) or harm (predatory businesses finding vulnerable people to sell overpriced loans or worthless credentials).

Much of data is statistical in nature: even if the overall distribution is right, the individual case can be wrong. If average life expectancy is 80, that does not mean you die exactly on your 80th birthday — mean and distribution say little about any specific person. A prediction system’s output is likewise probabilistic and can be wrong for individuals. Blind faith in data’s superiority in decision-making is not just an illusion but dangerous: we must learn how not to amplify existing bias, how to make algorithms accountable and transparent, and how to fix them when they inevitably err.

An algorithmic decision is not necessarily better or worse than a human one. Every person likely carries bias, even while actively resisting it, and discriminatory practices can become culturally institutionalized. There is hope that data-driven decisions, replacing subjective gut feeling, could be fairer and give a better chance to those overlooked by the traditional system. But when we build prediction and AI systems, we automate not just the human’s decision — we hand over the rules themselves to be inferred from data. The learned patterns are opaque: even where data shows correlation, we may not know why. If the algorithm’s input carries systematic bias, the system will most likely learn that same bias into its output and amplify it.

In many countries anti-discrimination law bans differential treatment based on protected traits (ethnicity, age, gender, orientation, disability, belief). The remaining traits may be analyzed — but what if they correlate with a protected one? In racially segregated neighborhoods, the postcode or even the IP address is a strong predictor of race. Given that, the belief that an algorithm turns biased input into fair, neutral output seems absurd — yet data-driven decision advocates implicitly hold it; an attitude wryly dubbed “machine learning is like money laundering for bias”. A prediction system only extrapolates the past; if the past was discriminatory, it encodes and amplifies it. If we want a better future than the past, “moral imagination” is needed, and only humans can supply it. Data and models must be our tools, not our masters.

Even in predictive uses with less direct impact on people — like recommenders — there are hard problems: once a service gets good at predicting what the user wants to see, it may show only views the user already holds; an echo chamber where stereotypes, misinformation and polarization grow. We already see social networks’ echo chambers acting on election campaigns.

But when predictive analytics touches people’s lives, especially vicious self-reinforcing feedback loops appear. The book’s example: an employer screening hires by credit score. A good worker with a good score hits money trouble through no fault of their own; payments slip, the score sinks, job chances shrink; unemployment leads to poverty, the score sinks further, finding work gets harder — a downward spiral from toxic assumptions, camouflaged as mathematical precision and data. Another example: when German petrol stations adopted algorithmic pricing, competition fell and consumer prices rose, because the algorithms learned to collude.

Such loops cannot always be predicted in advance; but many consequences can be foreseen by thinking about the whole system (not just the computerized parts, but the people interacting with it) — an approach called systems thinking (systems thinking): how does the data-analysis system react to different behaviors and structures? Does it reinforce and amplify existing differences between people (rich get richer, poor get poorer), or does it fight injustice? Even with the best intentions, unintended consequences are to be feared.

Apart from predictive analytics (automated decisions about people), data collection itself is a moral problem: what is the collecting organization’s relationship with the data’s owners? When a system keeps only data the user explicitly entered for the system to store and process in a specific way, the system serves the user and the user is the customer. But when user activity is tracked and recorded as a side effect of other things the user does, the relationship is murky: the service no longer just does what the user asked; it develops interests of its own that may conflict with the user’s.

Behavioral tracking has become important for the user-friendly features of many online services: which search result got clicked improves ranking; “people who liked X also liked Y” aids discovery; A/B tests and funnel analysis improve the UI. These need some tracking, and the user benefits. But depending on the business model, tracking does not stop there: if the service is funded by advertising, the advertisers are the real customers and the user’s interests come second. Tracking data gets more detailed, analyses reach further, and data is kept for a long time to build a detailed profile of each person for marketing. Now the relationship looks different: free service plus nudges toward maximum engagement; tracking serves first not that person but the advertisers’ needs. The fitting, darker name for this relationship: surveillance (surveillance).

The book proposes a thought experiment: replace the word data with surveillance and see whether familiar phrases still sound fine: “in our surveillance-driven organization we collect real-time surveillance streams and keep them in our surveillance warehouse; our surveillance scientists build fresh insight with advanced analytics and surveillance processing.” This tone is unusually sharp for this polite book — Designing Surveillance-Intensive Applications — but the emphasis is needed: in the rush for software to “eat the world”, we built the largest mass-surveillance infrastructure in history, and we are fast reaching a world where every living space has at least one internet-connected microphone: phones, smart TVs, voice assistants, baby monitors, even children’s toys with cloud speech recognition — many with appalling security records.

What is new versus the past is that digitization made collecting huge volumes of data about people easy: tracking of location and movement, relationships and communications, purchases and payments, and health has become nearly unavoidable; the surveilling organization may know more about a person than they know themselves — diagnosing an illness or money trouble earlier than they do. Even the most totalitarian past regimes only dreamed of a microphone in every room and making everyone permanently carry a tracker; but digital technology’s benefits are so great that we have now voluntarily accepted that same total-surveillance condition — with the difference that companies collect the data to provide services, not governments to control. Why do we seem content with corporate surveillance? Perhaps you feel you have nothing to hide — meaning you are fully aligned with existing power structures, not a marginalized minority, and fear no harassment. Not everyone is that lucky. Or perhaps because the purpose looks benign: just better recommendations and more personal marketing. But with predictive analytics in the picture, the distinction blurs: recorded driving data collected without consent affects insurance premiums, health coverage gets tied to wearing a fitness tracker, and a smartwatch motion sensor can guess with fair accuracy what you type (passwords, for instance) — and sensors and algorithms only get better.

Privacy harm is not just “data leaked”; it takes its own shape at every stop of the data flow. This concept visual arranges four harm families in flow order: collection (surveillance, interrogation, insecurity, over-collection), processing (aggregation, re-identification, secondary use, exclusion from decisions), dissemination (disclosure, exposure, blackmail, distortion) and invasion (intrusion, decisional nudging, the exclusion spiral, the chilling effect). Each column also names its defense: minimize at the source, limit the purpose, audit access, and provide appeal paths that actually work.

One might say users voluntarily choose the tracking service, agree to the terms and privacy policy, and consent to collection; even that they get a valuable service in exchange for data, and tracking is needed to deliver it. Social networks and search engines are undeniably valuable to users — but the argument is flawed. First ask why is the tracking needed? Some tracking flows straight back into features (click rates for ranking, co-purchases for related-product suggestions); but when interaction tracking feeds content recommendation or an advertising profile, it is far less clear that it benefits the user — or whether it is only “needed” because advertising pays for the service.

Second, most users know little about what data they feed into our databases and how it is stored and processed — and most privacy policies hide more than they reveal. Without understanding the data’s fate, the user cannot give meaningful consent. Often one user’s data also speaks about other people who are not users and agreed to nothing. The derived datasets of recent chapters — all users’ data combined with behavioral tracking and external sources — are exactly the data no user can meaningfully grasp. Moreover, data is extracted from the user through a one-way process, not a genuinely fair exchange: there is no dialogue, and the user cannot bargain over how much data to give and what service to get. The relationship is asymmetric and one-sided; the service sets the terms, not the user.

In the EU, the GDPR says consent must be “freely given, specific, informed and unambiguous”, and the user must be able to “refuse or withdraw without detriment” — otherwise it does not count as “freely given”; the request must be “in an intelligible and accessible form, using clear and plain language”, and “silence and pre-ticked boxes are not consent”. Consent is of course not the only lawful basis for processing (complying with another law, saving a life, and legitimate interests such as fraud prevention also count), but it is the most-used basis in internet services.

You might say a user who won’t consent can simply not use the service — but that choice isn’t free either. If a service is so popular that “most people consider it essential for basic social participation”, people can hardly be expected to quit — using it is effectively mandatory. In most Western societies carrying a phone, using social networks, and searching with Google are the norm; and when a service has network effects, non-use carries a social cost. Quitting over a tracking policy is easier said than done: platforms are engineered to keep you engaged (game mechanics and gambling tactics to bring you back); and even past that, quitting is only possible for a privileged minority with the time and knowledge to understand privacy policies and the ability to forgo social and professional opportunities. For the less privileged there is no meaningful freedom; surveillance becomes unavoidable.

People sometimes say “privacy is dead” because some users happily post their whole lives on social networks — but that is wrong, and comes from misunderstanding the word privacy. Having privacy does not mean hiding everything; it means the freedom to choose what to show to whom, what to make public and what to keep hidden. The right to privacy is the right to decide: each person decides, in each situation, where to stand on the secrecy-to-transparency spectrum; a major part of individual freedom and autonomy. Someone with a rare medical condition, for example, may gladly give private data to a researcher if it helps a cure — but must have the choice of who accesses the data and for what purpose. If the illness information wrecks their access to insurance or jobs, they become far more cautious.

When data is extracted from people through surveillance infrastructure, the right to privacy is not necessarily eroded — it is transferred to the data collector: companies effectively say “trust us to do the right thing with your data” — the right to decide “what to whom” moves from the person to the company. Companies also prefer to keep this surveillance’s output hidden, because revealing it looks “creepy” and harms their business model (knowing more than everyone else). Intimate information leaks only indirectly — e.g. an ad-targeting tool for a specific group (such as people with a particular illness). Even if no specific user is identifiable, they have lost agency over the disclosure of intimate facts: it is the company exercising the privacy right to maximize profit, not the user.

Many companies want to avoid looking “creepy” and focus on “managing user perception” instead of asking “how intrusive is the collection” — and they manage even that badly: something may be factually correct yet revive a painful memory the user never asked to be reminded of. With any data we must expect it can be wrong, unwanted or inappropriate, and build machinery for handling such failures; and “unwanted” is a human judgment — unless explicitly programmed to respect human needs, algorithms know nothing of these concepts. We engineers must be humble: accept failures and plan for them. Privacy settings controlling which facets of a user’s data other users see are a start at returning some control — but the service itself, regardless of settings, has unconstrained access and is free to process and analyze internally however the policy allows, often far beyond what the user sees. This wholesale transfer of the privacy right from person to company is unprecedented in history: surveillance always existed but was expensive and manual, not scalable and automatic; and trust relationships (doctor-patient, lawyer-client) were bound by strict ethical and legal constraints. Internet services made amassing huge volumes of sensitive information without meaningful consent easy.

Not every “yes” is consent. This concept visual lines up five points of the spectrum: genuine consent (free, informed, specific, with costless refusal), informed opt-in (plain notice and level choice — valid if honest), take-it-or-leave-it (refusal means no service — invalid, because not free), dark patterns (pre-ticked boxes and nagging — not consent at all) and no choice at all (tracking by default — call it surveillance). The bottom strip is the five-condition GDPR test: consent is valid only when all five conditions — freely given, specific, informed, unambiguous, withdrawable — hold at once.

Because behavioral data is a by-product of users interacting with a service, it is sometimes called “data exhaust” (data exhaust) — as if it were worthless waste and behavioral, predictive analysis a kind of recycling. The truth is closer to the reverse: if targeted advertising pays for the service, the user activity producing behavioral data is a kind of labor; one could even say the app the user works with is just bait to get people to feed more and more personal information into the surveillance infrastructure. The creativity and warm human relationships expressed on online services are cynically exploited by the data-extraction machine. Personal data is a valuable asset — witness the data brokers quietly buying and selling it — and startups are valued by user counts (“eyeballs”) — that is, by their surveillance capacity.

Because data is valuable, many want it: companies, of course (that is why they collect it); but governments too (via secret deals, coercion, legal compulsion, or even theft); in bankruptcy, the amassed personal data is among the assets sold off; and because securing data is hard, breaches are alarmingly frequent. Critics therefore say data is not just an asset but a “toxic asset” (toxic asset) — or at least hazardous material; perhaps data is not the new gold or the new oil but the new uranium. Even if we believe we can prevent misuse, at collection time we must weigh the benefit against the risk of falling into the wrong hands: criminals or foreign intelligence break in, an insider leaks, the company falls to unethical management, or the country is taken by a regime with no qualms about forcing us to hand the data over. So at collection time we must consider not just today’s political climate but all future governments; no guarantee exists that every later government will respect human rights — as Bruce Schneier puts it, “installing technology that could one day facilitate a police state is poor civic hygiene”. “Knowledge is power”; and “watching others while escaping watch yourself is among the most potent forms of power” — which is why totalitarian governments want surveillance. Today’s tech companies are not overtly after political power, but the data and knowledge they have amassed — much of it secret and outside public oversight — gives them great power over our lives regardless.

Data is the defining feature of the information age; the internet, storage, processing and software automation have hugely affected economy and society — and the comparison with the industrial revolution comes to mind. The industrial revolution arrived with technological and agricultural progress, bringing sustained economic growth and better living standards — but also grave problems: horrific air pollution (smoke and chemical processes) and water pollution (industrial and human effluent); factory owners in luxury and urban workers in cramped, unhealthy housing, working long hours in harsh conditions; child labor was common, including dangerous, low-paid mine work. The antidotes took a while: environmental protection rules, workplace safety protocols, child-labor bans, food hygiene inspections. Business costs surely rose once factories could no longer dump effluent into rivers, sell rotten food, or exploit workers — but society as a whole gained enormously, and few would return to before.

Just as the industrial revolution had a dark side that had to be managed, our transition to the information age has grave problems we must face and solve; collecting and using data is one of them. In Schneier’s words: “data is the pollution problem of the information age, and protecting privacy is its environmental challenge. Nearly every computer produces information; it lingers and rots. How we contain it and how we dispose of it is central to the health of our information economy. Just as today we look at the early decades of the industrial age and wonder how our ancestors ignored pollution in the rush to industrialize, our grandchildren will look at us in these early decades of the information age and judge us by how we solved the challenge of data collection and misuse.” Let us try to make them proud.

Data-protection law can help preserve individual rights: the GDPR, for instance, says personal data must be “collected for specified, explicit and legitimate purposes and processed compatibly with those purposes” and be “adequate, relevant and limited to what is necessary for the purpose”. But this data minimization principle (data minimization) runs exactly against the big-data philosophy: maximize collection, combine with other datasets, and probe and explore for fresh insight. Exploration means use for unforeseen purposes — which the GDPR calls the opposite of “specified and explicit”. These rules did affect the online-advertising industry, but were weakly enforced and do not seem to have changed the whole tech industry’s culture and practice.

Heavy data collectors usually oppose regulation as a burden and a brake on innovation — and they have a point: sharing medical data, say, has clear privacy risks but also opportunities (how many deaths would better analysis prevent through better diagnosis?). Excessive regulation may block breakthroughs; balancing opportunity and risk is hard. Fundamentally we need a culture change in the tech industry around personal data: stop seeing the user as an optimization metric, and remember they are humans deserving respect, dignity and agency. We must self-regulate collection and processing to earn and keep the trust of those who depend on our software; and educate end users ourselves about their data’s fate, instead of keeping them in the dark.

We must let each person keep their privacy (their control over their own data) and not steal it from them through surveillance. The individual right to control data is like a national park’s natural environment: without explicit protection and care it is destroyed; a tragedy of the commons, leaving everyone worse off. Pervasive surveillance is not inevitable; we can still stop it. First step: don’t keep data forever, purge it as soon as it is unneeded, and collect little in the first place. Data you don’t have cannot leak, cannot be stolen, and no government can force you to hand it over. Overall a change of culture and attitude is needed: if we in technology ignore our work’s social impact, we haven’t done our job.

In this final chapter the book reviews the whole journey — the same map we travelled over thirteen chapters:

• Chapter 1: analytical vs operational, cloud vs self-hosting, distributed vs single-machine, and balancing business needs with user needs.
• Chapter 2: nonfunctional requirements — performance, reliability, scalability and maintainability.
• Chapter 3: the spectrum of data models — relational, document, graph, event sourcing and DataFrames — and query languages (SQL, Cypher, SPARQL, Datalog, GraphQL).
• Chapter 4: OLTP storage engines (LSM and B-tree) and analytical (columnar) ones, plus retrieval indexes (full-text and vector).
• Chapter 5: encoding and evolution, plus dataflow paths (databases, service calls, workflow engines, event-driven architecture).
• Chapter 6: single-leader, multi-leader and leaderless replication, plus read-your-own-writes consistency and the offline-capable sync engine.
• Chapter 7: sharding — rebalancing, request routing and secondary indexes.
• Chapter 8: transactions — durability, isolation levels (read committed, snapshot, serializable) and distributed atomicity.
• Chapter 9: the troubles of distribution — network faults and delays, clock skew, process pauses, and why even locking is hard.
• Chapter 10: consensus and linearizability.
• Chapter 11: batch processing — from Unix tool chains to the distributed processor on a distributed filesystem.
• Chapter 12: stream processing — message brokers, CDC, fault tolerance and patterns like the streaming join.
• Chapter 13: the philosophy of streaming systems — easier integration, evolution and scale.

And finally this last chapter: data can do good, but it can also do grave harm — life-altering decisions that are hard to appeal, discrimination and exploitation, normalized surveillance and intimate disclosures; plus breach risks and the unintended consequences of well-meant use. Given the enormous effect software and data have on the world, we engineers must remember our responsibility to work toward the world we want to live in: one that treats people humanely and respectfully. Let us work toward that goal together.

Before the interview pack, we tie all the threads together. This concept map arranges the chapter’s seven concepts around one hub: predictive analytics (scores people), bias (launders bias), feedback loops (amplify difference), surveillance (watches), consent (asks — but is there a choice?), data as power (controls), and law and self-restraint (responds: minimize, purge, change the culture). Read each edge: every concept is one way a data system touches human lives.

1. “The model is mathematical, so it’s neutral” — input bias gets amplified into output; being mathematical guarantees no fairness.
2. “We removed the protected column, so it’s fair” — proxies (postcode, IP) rebuild that same column.
3. “Model accuracy is high, so it’s right for each person” — a correct overall distribution doesn’t erase individual error.
4. “We have a consent checkbox, so we’re ethical” — unfree/uninformed/unspecific consent isn’t consent; ethics isn’t a checklist.
5. “We’re GDPR-compliant, so our culture is fine” — formal compliance differs from culture change; enforcement has been weak too.
6. “More data is always better” — against minimization; every extra byte is future breach and abuse risk.
7. “An appeal path exists, so there’s no problem” — if the appeal’s judge is the same score, the appeal is theater.
8. “The data is aggregated/anonymized, so it’s safe” — joining with external sources and proxies brings back re-identification and discrimination.
9. “We’re a tiny startup, our impact is negligible” — feedback loops and data combination work at small scale too; and data persists.
10. “Ethics slows us down; we’ll think about it later” — unintended consequences cost more later; social impact is part of the job.
11. “Privacy means hiding things; honest users have nothing to hide” — privacy is the right to decide, not total secrecy.
12. “We focus only on technology; use isn’t our business” — the burden of responsibility is on the builder, as the book’s example shows.

1. Say the algorithmic prison in two sentences (high-risk label → systematic exclusion → no proof, no appeal).
2. Have the proxy example (postcode/IP → race) plus the “laundering bias” line ready?
3. State “what you did” (classic score) vs “who you resemble” (ML score) in one line.
4. Recap the score-job spiral in five steps (shock → missed payment → score → filter → poverty).
5. State the systems-thinking question (“amplify or fight?”) plus two examples (price collusion, echo chambers).
6. Run the surveillance substitution test in one sentence and give its verdict.
7. Know the four GDPR consent conditions (free, specific, informed, unambiguous + costless refusal) by heart?
8. Explain privacy = right to decide (not secrecy) plus the rare-illness example.
9. Say “data you don’t have can’t leak” plus two actions (minimization, timely purges).
10. Prepare one closing line: “Data is our tool, not our master — the future needs moral imagination.”