Too Far Away from the Job Market – Says Who? Linguistically Analyzing Rationales for AI-based Decisions Concerning Employment Support

Abstract

This paper describes an AI-based decision-support system deployed by the Swedish Public Employment Service to assist decisions concerning jobseekers’ enrolment in an employment support initiative. Informed by previous research concerning explanations in relation to trust, appealability, and procedural fairness, as well as jobseekers’ needs and interests in relation to algorithmic decision-making, the study linguistically analyses the extent to which the system enables affected jobseekers to understand the basis of decisions and to appeal or take other actions in response to automated assessments. The study also analyses the degree to which rationales behind decisions accurately reflect the actual decision-making process. Several weaknesses in these regards are highlighted, largely resulting from the opacity of the statistical model and the linguistic choices behind the design of explanations. Potential strategies for increasing the explainability of the system as a means to meet the needs and interests of affected jobseekers are also discussed. More broadly, the study contributes to a better understanding of how the linguistic design of AI explanations can affect normative dimensions, such as trust and appealability.