William Seymour
Lecturer, King's College London
https://www.kcl.ac.uk/people/william-seymour
william.seymour@kcl.ac.uk
Lecturer, King's College London
https://www.kcl.ac.uk/people/william-seymour
william.seymour@kcl.ac.uk
I conduct interdisciplinary work at the intersection of security, privacy, HCI, ethics, and law using a combination of computational and social science research methods. My work explores people’s concerns about using AI systems, what values those systems should embody, and how they can better meet the needs of the people who use them. I've had the privilege to have worked with a wide range of public sector and industry partners including Microsoft, BRE Group, and the Information Commissioner’s Office.
View publications on Google Scholar
Abstract: User-configured chatbots built on top of large language models are increasingly available through centralized marketplaces such as OpenAI’s GPT Store. While these platforms enforce usage policies intended to prevent harmful or inappropriate behavior, the scale and opacity of customized chatbots make systematic policy enforcement challenging. As a result, policy-violating chatbots continue to remain publicly accessible despite existing review processes. This paper presents a fully automated method for evaluating the compliance of Custom GPTs with its marketplace usage policy using black-box interaction. The method combines large-scale GPT discovery, policy-driven red-teaming prompts, and automated compliance assessment using an LLM-as-a-judge. We focus on three policy-relevant domains explicitly addressed in OpenAI’s usage policies: Romantic, Cybersecurity, and Academic GPTs. We validate our compliance assessment component against a human-annotated ground-truth dataset, achieving an F1 score of 0.975 for binary policy violation detection. We then apply the method in a large-scale empirical study of 782 Custom GPTs retrieved from the GPT Store. The results show that 58.7% of the evaluated GPTs exhibit at least one policy-violating response, with substantial variation across policy domains. A comparison with the base models (GPT-4 and GPT-4o) indicates that most violations originate from model-level behavior, with customization tending to amplify these tendencies rather than create new failure modes. Our findings reveal limitations in current review mechanisms for user-configured chatbots and demonstrate the feasibility of scalable, behavior-based policy compliance evaluation.
Array Vol. 30, July 2026
https://doi.org/10.1016/j.array.2026.100834
Abstract: LLM-based Conversational AIs (CAIs), also known as GenAI chatbots, like ChatGPT, are increasingly used across various domains, but they pose privacy risks, as users may disclose personal information during their conversations with CAIs. Recent research has demonstrated that LLM-based CAIs could be used for malicious purposes. However, a novel and particularly concerning type of malicious LLM application remains unexplored: an LLM-based CAI that is deliberately designed to extract personal information from users. In this paper, we report on the malicious LLM-based CAIs that we created based on system prompts that used different strategies to encourage disclosures of personal information from users. We systematically investigate CAIs’ ability to extract personal information from users during conversations by conducting a randomized-controlled trial with 502 participants. We assess the effectiveness of different malicious and benign CAIs to extract personal information from participants, and we analyze participants’ perceptions after their interactions with the CAIs. Our findings reveal that malicious CAIs extract significantly more personal information than benign CAIs, with strategies based on the social nature of privacy being the most effective while minimizing perceived risks. This study underscores the privacy threats posed by this novel type of malicious LLM-based CAIs and provides actionable recommendations to guide future research and practice.
Proceedings of the 34th USENIX Security Symposium, 2025.
https://www.usenix.org/conference/usenixsecurity25/presentation/zhan
Since 2023/2024 I have taught Cybercrime and Forensics (7CCSMCFC) for MSc Cybersecurity students in semester 2. Since 2024/2025 I have also coordinated final year Informatics MSc projects (7CCSMPRJ). I previously taught Security Management (7CCSMSEM) in 2021/2022.