Algorithmic systems can put people at a disadvantage when it comes to lending or fraud detection. A new study in the "European Journal of Information Systems" examines how companies can promote responsible decision-making right from the development stage. Prof. Dr Martin Adam (Chair of Smart Services) explains the findings and their practical implications. Professor Adam, what were you hoping to find out with your study? We were interested in how organisations can support developers in ensuring that their decisions are guided by ethical principles. Many risks arise as early as the stage of selecting data, designing a model or deciding how to use it. At this stage, the future consequences are not yet fully apparent. It is therefore important that staff assess at an early stage whether their decisions take into account, for example, fairness and protection against discrimination. You talk about algorithmic accountability. What does that actually mean? This refers to the perceived obligation to explain and justify one’s own development decisions and their expected consequences. Anyone who uses certain data or approves a model for implementation therefore expects to have to defend this decision to others. Our study examines whether this awareness strengthens people’s commitment to fundamental moral duties and rights. We are referring here to deontological judgements: people assess whether their actions comply with ethical principles, even when a problematic system appears economically attractive. How did you investigate this? We combined a survey of 148 developers, a randomised online experiment involving 102 participants, and 15 interviews. The research focused on the USA. In the experiment, participants decided on the implementation of a credit decision-making system that incorrectly classifies some applicants as gamblers and thereby disadvantages them. We varied the extent to which the user interface made the personal nature and verifiability of their decisions visible. The interviews helped us to better understand the reasoning behind such decisions. What was the main finding? Where perceived accountability was higher, 62 per cent opposed the implementation of the problematic system. Where accountability was lower, the figure was 41 per cent. This represents a difference of around 21 percentage points. Our analyses suggest that a stronger focus on ethical principles explains this correlation. The decision related to a hypothetical banking scenario. Further studies need to be carried out in a real-world business context to assess how such measures would affect systems actually in use in the long term. What role do specialist knowledge and the working environment play? The link between accountability and ethical judgements was stronger when developers rated their knowledge of algorithmic decision-making systems more highly. Those who understand a system’s data, how it works and its potential errors are better able to assess risks in concrete terms. A working environment that upholds ethical principles also reinforced this link. The interviews highlight just how important support within the team is in this regard: employees find it easier to voice their concerns if these are taken seriously and they do not feel they are on their own. What specific changes can companies make? Based on our findings, we conclude that organisations should clarify responsibilities and document key development decisions in a transparent manner. Before a system is released, for example, explicit questions could be asked about which people might be disadvantaged by a design decision and how the team intends to address this. To this end, employees need the specialist knowledge to identify such risks. Managers should also demonstrate that ethical concerns are welcome, even if addressing them involves extra work or means a later launch. In this way, the call for responsible behaviour becomes a tangible part of day-to-day development work. What message would you like to convey? Organisations shape the conditions under which developers can take on responsibility. These include clearly defined responsibilities, technical support and a working environment in which ethical concerns are taken seriously. Anyone wishing to develop responsible AI must establish these conditions right from the outset. Thank you very much for the interview! Publication: Bartsch, S. C., Schmidt, J.-H., Alizadeh, A., Adam, M., & Benlian, A. (forthcoming). The role of algorithmic accountability in shaping developers’ deontological judgements and implementation behaviours. European Journal of Information Systems.
Responsibility in the development of algorithmic decision-making systems
New publication in the European Journal of Information Systems
DOI: https://doi.org/10.1080/0960085X.2026.2747938
