Worst-Case Membership Inference of Language Models
An in-depth analysis of the feasibility of worst-case membership inference in large language models, challenging prevailing assumptions about data privacy.
A Google TechTalk, 2025-06-11, presented by Ashwinee Panda
Privacy in ML Seminar. ABSTRACT: It is widely believed that membership inference of the pretraining data of language models is impossible. In this work, we show that worst-case membership inference is possible. We identify key characteristics of documents that make them more susceptible to membership inference. We find that while the membership inference test statistic is uninformative when computed over the entire training document, it can be informative when computed over a specific span of the document. We observe that the test statistic distribution of members quickly converges to that of non-members when we update the model on documents where members are not present, giving the impression that the model has "forgotten" the members. We propose using finetuning to separate these distributions, which we show can produce successful membership inference of certain documents that exhibit "worst-case" memorization.