In the long run, the acceptance of AI technologies by the society at large depends on how much they will be trusted by the users. Beyond statistical or formal validation, trust will not exist without explainability, fairness, and robustness. This GT aims at increasing the awareness of the community to these questions, at enhancing the auditability of machine learning models, and at designing statistical control procedures to provide guarantees on machine learning pipelines.

Main topics

AI systems are nowadays used as the basis for more and more important decisions in people’s lifes, or as the source of more and more design decisions. In such context, their results must be well understood, and we must have the guarantee that they are not biased. The GT will concentrate on the following topics:
  • Explainability: all decisions can be formulated as the result of some logic reasoning using domain-specific concepts.
  • Interpretability: the whole model can be rewritten as some logical or probabilistic model allowing to trace its possible outcomes as the results of reasoning; for instance, rule-based systems or Bayesian networks are interpretable, provided a small complexity. Unfortunately, it seems that a trade-off between performance and interpretability/explainability cannot be avoided. Overall, systems should provide reliable confidence estimates on their decisions or their internal parameters when those matter.
  • Robustness: guarantees are given that the decisions will meet some constraints and not go wild, whatever the input data; guarantees can be formal (e.g., like formal proofs for critical systems) or statistical (e.g., like in aerospace industry).
  • Fairness: the outcome of the system must not discriminate groups of people for their gender, race, age, sexual orientation, etc. Given a working definition of fairness, decision system have to be designed in order to avoid biases, even when these are built implicit by combining the information available on individuals.

Examples

Here are some examples of biased outputs and practices:

  • Explanations: asking a Deep Network the reasons for its conclusions, one usually get explanations that look like « this is a cat because the cat output is 0.85, while the dog output is 0.22 », whereas human beings would expect something like « this is a cat because it has fur, whiskers and claws, and 2 triangular-shaped ears ». Whereas this might be accepted for cat recognition, it is not socially acceptable in the case of decisions that greatly impact people’s lifes, like « you have not been selected for the universities you were applying to », or « you will not get that loan », or « you will remain in jail for another 5 years ».
  • Adversarial examples: after a AI has been trained, some images, that are correctly classified by the model, can be purposely modified in such a way that no human eye can notice a difference, but the AI is unable to correctly classify them any more. Think at what could happen if, with some simple tiny bits of paint on STOP road signs, you could make all autonomous vehicles see a « Speed limit 80 » sign.
  • Gender discrimination: It might seem a good idea to remove the gender from all personal data to avoid gender discrimination. But unfortunately, it’s not that simple. Your digital behavior (favorite purchased items, journeys, age …) is enough to « guess » your gender, and thus to possibly discriminate.

Disciplines involved

Computer scientists are needed here, Data Scientists of course, but also people from Automated Reasoning as well as specialists of Formal Proofs. Mathematicians, specialists in statistics and probability, are concerned too. Finally, lawyers are needed too, regarding actions to be taken when guarantees are violated, explications unclear to the user, or biases detected at different levels of proofs.

Glossary

AI (artificial intelligence) is the simulation of human intelligence processes by machines, especially computer systems. These processes include : learning (the ability to assimilate information as well as the rules for using this information), reasoning (using rules to reach conclusions) and self-correction.