Listen
Neueste Veröffentlichungen
- ItemLost in moderation: How commercial content moderation apis over- and under-moderate group-targeted hate speech and linguistic variations(Association for Computing Machinery, 2025) Hartmann, David; Oueslati, Amin; Staufer, Dimitri; Pohlmann, Lena; Munzert, Simon; Heuer, HendrikCommercial content moderation APIs are marketed as scalable solutions to combat online hate speech. However, the reliance on these APIs risks both silencing legitimate speech, called over-moderation, and failing to protect online platforms from harmful speech, known as under-moderation. To assess such risks, this paper introduces a framework for auditing black-box NLP systems. Using the framework, we systematically evaluate five widely used commercial content moderation APIs. Analyzing five million queries based on four datasets, we find that APIs frequently rely on group identity terms, such as “black”, to predict hate speech. While OpenAI’s and Amazon’s services perform slightly better, all providers under-moderate implicit hate speech, which uses codified messages, especially against LGBTQIA+ individuals. Simultaneously, they over-moderate counter-speech, reclaimed slurs and content related to Black, LGBTQIA+, Jewish, and Muslim people. We recommend that API providers offer better guidance on API implementation and threshold setting and more transparency on their APIs’ limitations.Warning: This paper contains offensive and hateful terms and concepts. We have chosen to reproduce these terms for reasons of transparency.
- ItemExploring Prompt Generation Utilizing Graph Search Algorithms for Ontology Matching(IOS Press, 2024) Sampels, Julian; Efeoglu, Sefika; Schimmler, Sonja; Salatino, Angelo; Alam, Mehwish; Ongenae, Femke; Vahdati, Sahar; Gentile, Anna-Lisa; Pellegrini, Tassilo; Jiang, ShufanThe interoperability of domain ontologies, developed by domain experts, necessitates their alignment before attempting to match them. Within these ontologies, defined concepts often encounter an ambiguity problem stemming from the use of natural language. This interoperability issue raises the underlying ontology matching (OM) challenge. OM might be defined as the identification of correspondences or relationships between two or more entities, such as classes or properties among two or more ontologies. Rule-based ontology matching approaches, e.g., LogMap and AML have not outperformed machine learning based matchers on the Ontology Alignment Evaluation Initiative (OAEI) benchmark datasets, especially on the OAEI Conference track since 2020. Supervised machine or deep learning approaches produce the best results but require labeled training datasets. In the era of Large Language Models (LLMs), robust zero-shot prompting of LLMs can also return convincing responses. While prompt generation requires prompt template engineering by domain experts, contextual information about the concepts to be aligned can be retrieved by leveraging graph search algorithms. In this work, we explore how graph search algorithms, namely (i) Random Walk and (ii) Tree Traversal can be utilized to retrieve the contextual information to be incorporated into prompt templates. Through these algorithms, our approach refrains from considering all triples connected with a concept to be aligned in its contextual information creation. Our experiments show that including the retrieved contextual information in prompt templates improves the matcher’s performance. Additionally, our approach outperforms previous works leveraging zero-shot prompting.
- ItemSilencing the Risk, Not the Whistle: A Semi-automated Text Sanitization Tool for Mitigating the Risk of Whistleblower Re-Identification(ACM, 2024) Staufer, Dimitri; Pallas, Frank; Berendt, BettinaWhistleblowing is essential for ensuring transparency and accountability in both public and private sectors. However, (potential) whistleblowers often fear or face retaliation, even when report- ing anonymously. The specific content of their disclosures and their distinct writing style may re-identify them as the source. Legal measures, such as the EU Whistleblower Directive, are limited in their scope and effectiveness. Therefore, computational methods to prevent re-identification are important complementary tools for encouraging whistleblowers to come forward. However, current text sanitization tools follow a one-size-fits-all approach and take an overly limited view of anonymity. They aim to mitigate identification risk by replacing typical high-risk words (such as person names and other labels of named entities) and combinations thereof with placeholders. Such an approach, however, is inadequate for the whistleblowing scenario since it neglects further re-identification potential in textual features, including the whistleblower’s writing style. Therefore, we propose, implement, and evaluate a novel classification and mitigation strategy for rewriting texts that involves the whistleblower in the assessment of the risk and utility. Our prototypical tool semi-automatically evaluates risk at the word/term level and applies risk-adapted anonymization techniques to produce a grammatically disjointed yet appropriately sanitized text. We then use a Large Language Model (LLM) that we fine-tuned for paraphrasing to render this text coherent and style-neutral. We evaluate our tool’s effectiveness using court cases from the European Court of Human Rights (ECHR) and excerpts from a real-world whistleblower testimony and measure the protection against authorship attribution attacks and utility loss statistically using the popular IMDb62 movie reviews dataset, which consists of 62 individuals. Our method can significantly reduce authorship attribution accuracy from 98.81% to 31.22%, while preserving up to 73.1% of the original content’s semantics, as measured by the established cosine similarity of sentence embeddings.
- Item“Guilds” as Worker Empowerment and Control in a Chinese Data Work Platform(Association for Computing Machinery, 2024) Yang, Tianling; Miceli, MilagrosData work plays a fundamental role in the development of algorithmic systems and the AI industry. It is often performed in business process outsourcing (BPO) companies and crowdsourcing platforms, involving a global and distributed workforce as well as networks of collaborative actors. Previous work on community building among data workers centers organization and mutual support or focuses on the structuring and instrumentalization of crowdworker groups for complicated projects. We add to these lines of research by focusing on a specific form of community building encouraged and facilitated by platforms in China: guilds. Based on ethnographic work on a Chinese crowdsourcing platform and 14 semi-structured interviews with data workers, our findings show that guilds are a form of both worker empowerment and control. With this work, we add a nuanced empirical case to the interconnection of BPOs, online communities and crowdsourcing platforms in the current data production sector in China, thus expanding previous investigations on global perspectives of data production. We discuss guilds in relation to individual workers and highlight their effects on data work, including efficient coordination, enhanced standardization, and flattened power structure.
- ItemDocumenting Computer Vision Datasets: An Invitation to Reflexive Data Practices(2021) Miceli, Milagros; Yang, Tianling; Naudts, Laurens; Schüßler, Martin; Serbanescu, Diana; Hanna, AlexIn industrial computer vision, discretionary decisions surrounding the production of image training data remain widely undocumented. Recent research taking issue with such opacity has proposed standardized processes for dataset documentation. In this paper, we expand this space of inquiry through fieldwork at two data processing companies and thirty interviews with data workers and computer vision practitioners. We identify four key issues that hinder the documentation of image datasets and the effective retrieval of production contexts. Finally, we propose reflexivity, understood as a collective consideration of social and intellectual factors that lead to praxis, as a necessary precondition for documentation. Reflexive documentation can help to expose the contexts, relations, routines, and power structures that shape data.