The risk of negligence following the failure of a human replacement function in an automated system

摘要

In conventional human operated systems, unsafe failure of the system due to the behaviour of the operator can result in legal charges of negligence. However, if the human is replaced in the system, and a similar failure occurred, would similar charges be brought? And if so, against whom? In this paper, we review how the responsibility for correct system behaviour shifts, from the human controller to the design chain, when implementing human replacement functions. We do this using a pseudo-legal discussion of negligence tests in a variety of hypothetical scenarios. It should be stressed that the authors are not legally trained and the contents of this paper should not be taken as legal advice. Further, the concepts and ideas described in this paper are solely those of the authors, and do not necessarily reflect the views of their parent organisations.