Transparency for contesting automated decisions: impediments and affordances under the EU law

The thesis, answering the research question: How could transparency of automated decision-making be constructed as a prerequisite of contestation and what could be the main affordances and impediments under EU law? consists of four peer-reviewed articles.

The research does not provide a full account of the entire regulatory landscape of transparency but rather focuses on essential requirements of the concept in the context of automated decision-making (ADM) — analysing the relevant provisions of the GDPR and intellectual property laws as the most prominent body of rules relevant to the legal scrutiny and contestation of ADM systems.

Since the thesis is focused on a specific type of transparency, that is, an actionable one for the purposes of contestation, the first paper “The ‘Rule of Law’ implications of data-driven decision-making: a techno-regulatory perspective” 1 (co-authored with R. Leenes) sets the ground by conceptualising datadriven ADM as the new horizon of techno-regulation. The paper puts forward the perspective that transparency in automated decisions is not about reading of computer code but rather relates to the question how these systems make up the normative landscape that we are subjected to. As an extension of Lessig’s “Code as Law”, the thesis rests on the premise that by sorting, classifying and predicting these technologies impose or facilitate certain norms, values or criteria. As such, the paper defines automated decision-making (ADM) as a regulatory technology and identifies three impairments (normative, causal, moral) which undermine the principal of rule of law. This theoretical stance allows for a conceptualisation of ADM and the surrounding transparency debate as a procedural, or we may say, as a due process problem.

Next, the thesis develops a transparency model, laying out the required forms and degrees of transparency necessary to contest automated decisions. The second paper2 initially explains how i) technical complexities; ii) epistemological flaws (spurious correlation or weak causation); and iii) biased processes inherent in machine learning (ML) create obstacles in terms of interpreting automated decisions. The analysis illustrates that a conception of transparency aiming to see the entire system ‘at work’ is an ever-expanding territory, that is, as you open black-boxes, you may just find more black-boxes. Instead, the outcome of ML-based systems may eventually be attributed to the values and assumptions that underlie the response of the system to a given input. Accordingly, the transparency (contestation) model developed in the second paper is a reconstruction of ADM as a ‘rule-based’ process where certain input lead to certain results—akin to the decisions in a legal system based on facts, norms and the ensuing consequences.

Having identified the essential transparency requirements for effective contestation, the remainder of the thesis focuses on the relevant legal frameworks. implementation of the transparency model—exploring to what extent the relevant provisions in the GDPR could be interpreted in the direction of “contestability”. For this purpose, the third paper provides a systematic and teleological interpretation of Article 22 of the GDPR on automated decisions—focussing on the question how the rights to obtain human intervention, express one’s views and contest the decision could practically be implemented. By defining Art 22 as a general provision of due process and the right to contest as the core remedy provided by the GDPR against ADM, the paper transcends the current debates about the existence and the scope of a so-called “right to an explanation".

As the most important legal framework that could impede transparency efforts, the final part of the thesis on IP rights provides a macro-view of the potential areas of conflict between the transparency requirements and the relevant IP regimes—i.e., copyright, sui generis database right and trade secret protection. The fourth paper initially clarifies that the implementation of transparency measures and mechanisms as defined in the previous parts of the thesis require the disclosure, reproduction or modification of certain informational elements of ML systems. Following this, the paper explores i) to what extent reliance on IP rights could excuse ADM from the obligation of making transparent and contestable decisions, e.g., under Article 22 of the GDPR and ii) what are the counter-arguments based on statutory exceptions and limitations restricting IP rights. The paper analyses the IP-eligible elements in ML-based systems in a dual structure as: data and datasets (expressional elements) on one side and algorithmic techniques and ML models (utilitarian/ operational elements) on the other.