Your Knowledge Gaps Are the Most Intimate Data You Have

In February 2012, the New York Times Magazine published Charles Duhigg's "How Companies Learn Your Secrets," describing how Target's data science team had built a model that assigned shoppers a "pregnancy prediction" score from purchasing behavior alone: unscented lotion bought around the start of the second trimester, a larger bag of cotton balls, hand sanitizer and washcloths as a due date approached.¹ The piece became famous for a different, more dramatic claim riding alongside that one — that the model had revealed a teenage girl's pregnancy to her father before she had told him herself, via a mailer of baby coupons. That specific scene is Duhigg's own retelling of an anecdote he did not independently verify; a later, closer look at the reporting found no confirmation that any single, identifiable father-and-daughter case actually happened the way it is usually repeated, and noted that Target's own practice of interspersing baby offers among unrelated ones was designed precisely to prevent a customer from ever noticing the pattern in the first place.² The signature the model used — the underlying purchasing pattern — was real. The viral confrontation scene built on top of it was not independently confirmed, and this paper does not repeat it as fact.

The story became canonical anyway, reproduced in business-school cases and conference keynotes for a reason that has nothing to do with whether the father-and-daughter scene really happened: it crystallized an argument about purchasing behavior as a proxy for life state — that what you buy reveals more than you intended to reveal.

But purchasing behavior is, in the hierarchy of intimate data, not the most sensitive category. It is what you do in the world. It is behavioral, observable, often shared with others who were present for the same transaction. There is a more intimate category that nobody has thought to name as a category: what you know and what you don't.

I want to argue that epistemic data — data about the boundaries of your knowledge — is more intimate than location data, more sensitive than purchase history, and more revealing than almost anything else you generate online. And that the tool category being built to help people read and understand complex material sits at the exact center of this.

Where you were and what you bought are both on record. What confused you is the denser trail.

1. What grounds this argument, and what would undo it

This paper makes a normative argument — that a particular category of behavioral data deserves a particular kind of architectural protection. The category itself is still mostly undiscussed, so it is worth being explicit about what grounds that argument before building on it, rather than letting the argument run ahead of what is actually established.

What it argues from. A documented, decades-old professional and legal literature on reading privacy: library confidentiality policy and statute,³⁴⁵ and the legal-scholarship literature on a "right to read anonymously" and on the dangers of "social reading."⁶⁷ The published, current data-retention and training policies of the two major commercial AI API providers, read directly from their own documentation.⁸⁹ The text of the one data-protection statute with the broadest international reach that defines legally "special" categories of personal data.¹⁰ A single, widely-cited journalistic account of a purchasing-behavior prediction model, corrected where a part of its popular retelling turns out not to be independently verifiable.¹²

What it does not argue from. No study has measured how often, or for which populations, an "epistemic lookup" — a search or query driven by confusion rather than by an instrumental need — actually occurs, at what scale, or with what content. No AI reading-tool vendor's query logs have been the subject of a public subpoena fight that would test whether the legal protections this paper discusses actually hold under pressure. No data broker's price sheet for "comprehension" or "confusion" signals, as distinct from general search-interest data, was found. These are named directly in Appendix B rather than assumed.

What it does not claim. It does not claim epistemic data is currently being collected and sold at scale today; the argument is that the tools now being built create the technical capability to do so, and that the choice of architecture made now will set a default. It does not claim BYOK (bring-your-own-key) architecture is risk-free; §7 states directly that it depends on a provider's own retention and training policy remaining what it is today, and on that provider never being the compelled party in a legal proceeding. It does not claim Flux's own product, AnswerTab, currently implements the architecture this paper recommends — §9 states the opposite directly, because it is true and because a paper arguing for an architectural standard its own author's product does not yet meet is exactly the kind of inconsistency that should be named rather than quietly dropped.

What would refute it. Given in full in §11. The sharpest version: if epistemic data, once collected at scale, turns out not to be meaningfully more revealing or differently sensitive than ordinary search-history data already is — if the "cognitive state" distinction this paper leans on collapses into a difference of degree rather than kind — the central claim of this paper fails, and the right response is to drop the special-category argument, not to restate it more forcefully.

2. What Epistemic Data Is

Epistemic data is the record of your knowledge state. It includes:

• What questions you asked when you didn't understand something
• What terms you looked up and how often
• Which parts of which documents you read versus skipped
• What you knew before you asked and what you knew after
• The domains where you are confident and the domains where you are lost

This is not the same as search history, though it overlaps. A search history includes everything you looked for — product research, directions, news, entertainment, health symptoms. Most of what you search for is instrumental. You need a plumber. You're looking for a restaurant. You want to know who won a match.

Epistemic data is the subset of information-seeking behavior that maps the edges of your understanding. It's what you searched for because you were confused, not because you needed a fact. It is the record of where your knowledge runs out.

The boundary between epistemic and non-epistemic search is not always clean. "What time does the pharmacy close?" is instrumental. "What is the difference between a statin and an ACE inhibitor?" could be either — you might be a medical student doing revision, or a patient who just received a new prescription and doesn't understand why. But the directional claim holds: a significant subset of search and lookup behavior reveals the edges of your understanding, and that subset is meaningfully different from the rest.

No published study known to this paper has measured what fraction of lookup behavior, for any real population, is actually epistemic in this sense rather than instrumental. That is registered in Appendix B as a genuine gap, not papered over with an invented percentage.

3. The Intimacy Spectrum of Data

It is useful to think about data intimacy as a spectrum, not a binary. Different data types reveal different aspects of a person, at different depths.

What makes epistemic data categorically different from the others is that it maps your internal cognitive state, not your external behavior. Location data records where your body was. Purchase data records what your wallet did. Epistemic data records what your mind was doing — specifically, where it ran into limits.

This is not a trivial distinction. Your knowledge gaps are deeply professional in a way that your location or purchase history usually isn't. A cardiologist who keeps looking up drug interaction tables is revealing something about the limits of her clinical memory that she would not share with a recruiter. A policy analyst who searches for the definition of "crowding-out effect" in the middle of a budget analysis is revealing something about her economics training that she would not broadcast to her colleagues. A journalist who looks up the same historical detail three times across a six-month period is revealing that he doesn't fully understand the context he's reporting on — which is fine, that's how learning works, but it is not something he'd expose voluntarily.

Your knowledge gaps are the negative space of your competence. They tell anyone who can read them exactly where your expertise ends, how quickly you learn, what domains you're moving into, and what you're working on hard enough to be confused by.

4. The History of Library Privacy

It is tempting to tell the library profession's privacy history as a single clean story: McCarthyism, the FBI pressuring libraries to report "subversive" readers, and a resulting legal commitment. The real history runs through two distinct episodes rather than one, which matters for understanding why the resulting protection is as broad as it now is.

The first episode is squarely McCarthy-era. In June 1953, with censorship fights over "subversive" books in State Department overseas libraries at a fever pitch, the American Library Association and the American Book Publishers Council jointly adopted the Freedom to Read Statement, opening: "The freedom to read is essential to our democracy."¹¹ That statement is the origin of the profession's broad commitment to intellectual freedom — but it is a statement about censorship of what libraries carry, not, on its own, a confidentiality rule about who reads what.

The specific commitment to keeping circulation records confidential has its own, separately documented history, and it predates the FBI episode most people now associate with it. The ALA adopted its Policy on Confidentiality of Library Records on January 20, 1971 — more than a decade and a half before the episode that is usually credited with triggering the modern wave of state statutes.⁵

That later episode was the FBI's Library Awareness Program: a counterintelligence effort that had run quietly since as early as 1973, seeking information from librarians about the reading habits of library patrons the Bureau suspected of being foreign intelligence agents or their recruits — not, as the popular retelling often has it, a hunt for domestic political dissidents reading "subversive" material. It remained secret until the New York Times disclosed it on September 18, 1987, after which congressional hearings followed and the library profession organized, state by state, around explicit statutory confidentiality.¹² Today, forty-eight states and the District of Columbia protect the confidentiality of library circulation records by statute, with the remaining two covered by attorney-general opinions reaching the same result.⁴

The ALA's own current definition states the resulting ethic plainly: "Confidentiality is when a library has users' personally identifiable information (PII) and keeps it private."³ The reason isn't simply respect for privacy as a value. It is recognition that what you are reading is a signal of what you are thinking — and that the freedom to think requires the freedom to read without being watched.

The interesting thing about this history is that libraries protected reading records, in a formal national policy, more than a decade before the FBI episode most people now cite as the reason — and well before the data existed in a form that was easy to analyze at scale. Circulation records were paper. The analytical threat was still mostly hypothetical in 1971. Librarians made the ethical commitment in advance of the technology that would make it matter.

The digital reading environment has done the opposite: collected enormously rich behavioral data about reading, at massive scale, across millions of users, before anyone had a serious conversation about whether that was appropriate. The Kindle knows which pages you highlighted. Every e-reader platform knows your reading speed, your finish rate, which parts you re-read. Every browser extension knows every tab you've opened. The data collection preceded the ethics by decades this time, not years, and the ethics are still catching up.

Epistemic data — data about what you understood and what you didn't — is the next frontier of this. It doesn't exist yet as a widely collected category, because the tools that would collect it at scale don't exist yet at scale. But they are being built. And the question of who owns that data, who can access it, and what they can do with it is exactly as live as any other privacy question — more live, in my view, because epistemic data is more intimate than most of what people currently debate.

5. The BYOK Architecture as Ethical Position

Bring Your Own Key is usually described as a product feature. You connect the tool to your own API account, you pay for your own inference, the tool doesn't bear the cost of model usage. This is accurate but it undersells what BYOK actually is.

BYOK is an architectural commitment: your queries go directly from your browser to the model provider, without transiting any server that the tool builder controls. The tool builder never sees the content of your questions. They never see which terms confused you. They never see which documents you were reading when you asked. The query is yours, the response is yours, and the entire exchange is invisible to the intermediary.

This is not what most "AI reading tools" do, including, as §9 states directly, Flux's own. The typical architecture routes your query through the product's backend, where it is logged, stored, and used at minimum for debugging and billing and at maximum for model training, product analytics, and data partnership arrangements that may be buried in a terms of service document you didn't read.

The distinction between "your API key, direct to provider" and "our backend handles the model call" is invisible to most users. Both look like a box where you type a question and text appears. The difference is whether a third party sits between your question and the answer, and what they can see and retain. This is exactly the kind of architectural choice that is invisible until it matters — and for epistemic data, it matters more than for most.

For a researcher asking about the methodology section of a paper on Kenyan water governance, BYOK means that nobody outside the researcher-to-model dyad knows what the question was, why it was asked, or what documents prompted it. For a journalist asking about the legal structure of a company they're investigating, BYOK means the tool builder can't be subpoenaed for records of what the journalist was curious about. For an analyst at a financial institution asking about a term in a regulatory document, BYOK means no third party has a log of what the analyst found confusing about compliance documentation.

These are not edge cases. They are the use cases that matter most.

6. What the Data Would Reveal, At Scale

Let me push this further, because the individual-level sensitivity is the beginning of the argument, not the end of it.

At scale, a platform with access to epistemic data from a large population of knowledge workers would have something genuinely unprecedented: a map of what educated, informed people don't know. Not a survey. Not self-report. A behavioral log of where comprehension breaks down, across millions of document-reader pairings.

This data would be extraordinarily valuable in ways that have nothing to do with serving the individual user. It would tell an education technology company which concepts have the highest confusion rates across which populations. It would tell a media company which assumptions in their articles readers can't follow. It would tell a pharmaceutical company which clinical terms physicians look up most frequently — which is a competitive intelligence signal about prescribing patterns and knowledge gaps. It would tell a government which parts of proposed legislation are least understood by the policy analysts reviewing it, which is a signal about where regulatory capture is most likely.

None of this requires identifying individual users. Aggregate epistemic data is commercially useful without being individually attributable — which is precisely why "we anonymize everything" is insufficient protection for this category. No published source documents an actual market price for "comprehension" signals specifically, distinct from general behavioral data; this is a plausible commercial incentive, not a confirmed market, and Appendix B registers it as such.

The only architecturally sound position, in the absence of that market being regulated, is to not collect the data in the first place. To not have the server in the middle. To route the query directly to the model and let the model provider's own retention and training policy govern what is retained.

That policy, read directly from both major commercial providers rather than assumed, currently holds in the researcher's favor. OpenAI states that API data "is not used to train or improve OpenAI models" by default, with abuse-monitoring logs retained up to 30 days.⁸ Anthropic states that "by default, we will not use your inputs or outputs from our commercial products... to train our models."⁹ Both are contractual commitments, not architectural guarantees: they can change, they apply only to the commercial/API tier, and neither prevents a provider being compelled by a court to produce records it holds within its retention window — a different risk than training.

7. What the Law Does Not Yet Cover

It is worth being precise about how far existing data-protection law actually reaches here, because the honest answer is: not very far.

The European Union's General Data Protection Regulation — the data-protection statute with the broadest practical international reach — names a specific, closed list of "special categories" of personal data subject to heightened protection: data revealing "racial or ethnic origin, political opinions, religious or philosophical beliefs, or trade union membership," genetic data, biometric data used for identification, health data, and data concerning a person's sex life or sexual orientation.¹⁰ Nothing in that list covers a person's epistemic state — what they understood, what confused them, where their knowledge ran out. A regulator applying the GDPR today would have no textual basis for treating a log of someone's reading confusion as a "special category" the way it would treat a log of their health conditions, even though the argument of this paper is that the former can be just as revealing, and sometimes more professionally consequential, than the latter.

This is not a defect in the GDPR's drafting so much as evidence that the category this paper is naming is still genuinely new to the law. Ordinary data-protection principles — purpose limitation, data minimization, the general requirement of a lawful basis for processing — still apply to epistemic data as to any other personal data. But the heightened protection reserved for the categories law already recognizes as especially sensitive does not currently extend to it, which means the protection this paper is arguing for has to come from architecture and professional norms before it can come from statute, exactly as library confidentiality did in 1971, more than a decade before most state legislatures caught up.

8. The Library Precedent, Applied Forward

The library profession made a commitment to reading privacy before the analytical threat was at its most serious, before the data was digital at scale, and before anyone was trying to systematically monetize reading behavior. The commitment held because it was codified in professional ethics and, eventually, in law — ahead of, not behind, the technology.

The comprehension tool category is at a comparable moment now. The tools are just becoming technically viable. The data collection norms haven't been established. The regulatory frameworks don't exist, as §7 just showed directly. The first generation of products will set the defaults that subsequent products inherit — which is always how defaults work.

The default that matters most is whether epistemic data is treated as belonging to the reader or to the platform. A tool that asks for your API key and routes your queries directly to the provider is asserting that the data belongs to you. A tool that routes through its own backend is asserting the opposite, whatever its privacy policy says. Ownership of epistemic data is the foundational question here; everything else — licensing, retention, access — follows from how that question is answered architecturally, not from how it is answered in a policy document.

The users who care most about comprehension tools — researchers, senior analysts, investigative journalists, policy professionals — are exactly the users who have thought about data sovereignty and who will ask the right questions before they adopt a tool that sits in the middle of their reading. BYOK is not a barrier to adoption for this population. It is a signal that the tool was built by people who understood the stakes — which is exactly why §9 does not soften the fact that Flux's own current product does not yet send that signal.

The web never built the comprehension layer. The comprehension tools that are now being built have a choice to make about data ownership that the web never had to make, because the web collected behavioral data incidentally rather than as a core product function.

A tool designed for comprehension sits closer to a library than to a browser. It should be designed with library ethics, not browser economics.

9. Objections I take seriously

"Flux has a commercial interest in this conclusion, and its own product does not meet the standard it argues for." Both halves of this are true, and I am not going to soften either. Flux builds AnswerTab, a comprehension tool of exactly the kind this paper describes — and the normative argument this paper makes (BYOK is the only architecturally sound default) is, if anyone acts on it, good for Flux's own competitive position relative to products that do not offer it. More importantly: as currently shipped, AnswerTab's query path is proxied through Flux's own infrastructure to the underlying model rather than running the direct browser-to-provider architecture this paper recommends, though answers are not stored server-side. That is a real inconsistency between the paper's own argument and its author's own current product, not a hypothetical one, and the honest response is to name it rather than to quietly scope the paper around it. A standalone BYOK client is a stated direction for the product, not a shipped fact.

"Epistemic data isn't really different in kind from ordinary search history — this is a difference of degree, not a new category." This is the objection that, if correct, would do the most damage to this paper, and §11 names it as the primary falsification condition for exactly that reason. My response is that the cardiologist-drug-interaction and policy-analyst-crowding-out-effect examples in §3 are meant to show a difference in what the data licenses an observer to infer — professional competence, not just behavior — but I have not found, and this paper does not claim to have found, a measurement that cleanly separates "epistemic" lookups from instrumental ones at scale. Until that measurement exists, this objection cannot be fully answered, only argued against on the strength of the examples.

"Current AI-provider no-training policies are contractual promises, not architectural guarantees, so BYOK's privacy benefit is weaker than this paper implies." Correct, and §6 says so directly. A provider can change its policy, a policy applies only to the tier a user is actually on, and neither OpenAI's nor Anthropic's policy prevents a lawful compelled disclosure of records held within the retention window. BYOK removes one intermediary — the tool builder — not every intermediary. It is a meaningfully smaller attack surface, not a privacy guarantee.

"The GDPR gap identified in §7 could be closed by regulators extending 'special category' treatment to cognitive/epistemic data — and if that is plausible, this paper's call for architecture-first protection is premature." This is a fair point about sequencing, not substance. Regulatory extension of special-category protection typically follows, rather than precedes, public recognition that a category of data is sensitive — which is part of this paper's own argument for why the architectural default matters now, before any such recognition exists. If a regulator does extend special-category protection to epistemic data before this paper's falsification window closes, that is a point in favor of the underlying claim, not against the timing argument.

10. Limitations

Epistemic data is not, on the evidence here, already being collected and sold at scale. The narrower claim is that the technical capability is being built now, and that the default chosen at this stage will likely persist.

It does not claim any specific vendor, including Flux's own AnswerTab, is currently misusing epistemic data; no evidence of misuse was sought or found, and §9 states plainly that AnswerTab does not yet implement the architecture this paper recommends, which is a different claim from misuse.

It does not claim BYOK architecture is a complete privacy guarantee; §6 and §9 both state that it depends on a provider's current, changeable policy and does not prevent lawful compelled disclosure.

It does not claim existing data-protection law (the GDPR specifically) currently treats epistemic data as a protected special category; §7 states the opposite directly.

It does not claim a measured, population-level frequency for "epistemic" versus "instrumental" lookups; no such measurement was found, and the distinction in §2 and §3 rests on illustrative cases, not a quantified base rate.

It does not claim the Target/Duhigg anecdote's father-and-daughter confrontation scene is confirmed fact; the opening and footnote 2 state it is not independently verified, and no claim in this paper relies on it beyond illustrating the purchasing-pattern model itself, which is well documented.

11. What would falsify this

Stated as commitments, in decreasing order of how much damage each would do to this paper's claims.

A rigorous measurement shows that "epistemic" lookups, once isolated from ordinary instrumental search, are not meaningfully more revealing of professional competence or cognitive state than general search-history data already is. That would collapse the difference-in-kind claim this paper's entire argument depends on into a difference of degree, and the right response would be to withdraw the special-category argument, not restate it with softer language.

A major commercial AI provider is shown, through litigation, regulatory action, or its own disclosure, to have trained on or retained API-tier data in violation of its own stated no-training policy. That would not falsify the architectural argument for BYOK, but it would falsify this paper's reliance on current provider policy as a meaningful protection, and would require the paper's confidence in §6 to be substantially revised downward.

AnswerTab, or a comparable Flux product, ships a BYOK-equivalent architecture and user adoption among the target population (researchers, analysts, journalists) shows no measurable preference for it over the proxied alternative. That would directly undercut §8's claim that this population cares about data sovereignty enough for it to function as a product differentiator, and would require this paper's market-facing argument, though not its privacy argument, to be revised.

A regulator extends "special category" protection under the GDPR or an equivalent statute to cognitive/epistemic data specifically. This would not falsify the paper; it would be the point conceded in §9 as support for its underlying claim. It is listed here because it is the condition most likely to actually occur, and the paper commits in advance to treating it as vindication rather than obsolescence.

Appendix A — Evidence table

Appendix B — Citation register

CITATION-NEEDED — measured base rate of epistemic vs. instrumental lookups. Appendix A row 14. No study was found that isolates "confusion-driven" search or lookup behavior from ordinary instrumental search at any meaningful scale; the distinction in §2–3 rests on illustrative cases, not a quantified population measurement.

CITATION-NEEDED — a public legal test of compelled AI-provider disclosure. Appendix A row 15. No subpoena fight or equivalent public legal proceeding testing whether a major AI provider's retention window can be pierced by compelled process was found; §6's "subpoena" scenario is a structural possibility under current retention policy, not a documented event.

CITATION-NEEDED — a market price for epistemic/comprehension data specifically. Appendix A row 16. No data-broker price sheet or market report naming "comprehension" or "confusion" signals as a distinct, priced product (as opposed to general search-interest or behavioral data) was found; §6's commercial-value argument is inferential, not evidenced by an actual market transaction.

Correction, not merely flagged. The previous version attributed "Ownership is the foundational question. Everything else — licensing, access, use — follows from it" to Kenneth Crews in a book titled Copyright and Your Job: The Truth About Works Made for Hire. No such book could be found; Crews's actual works on this subject (Copyright Law for Librarians and Educators; a chapter on "works made for hire" at universities) do not contain this sentence. The attribution is removed; §8 restates the point as this paper's own, unattributed.

Correction. The previous version dated the Target/Duhigg story to "January 2013." It ran February 16, 2012; corrected throughout, with the father-daughter scene additionally flagged as unverified (opening; Appendix A row 2).

Correction. The previous version merged the library profession's privacy ethic into a single "McCarthyism / FBI pressured libraries" narrative. §4 separates this into the two actually-documented episodes — the 1953 Freedom to Read Statement and the FBI's 1973–1987 Library Awareness Program (a foreign-counterintelligence effort, not domestic political surveillance) — rather than repeating the conflation.