GT2 – Learning & Trust
- Explainability: all decisions can be formulated as the result of some logic reasoning using domain-specific concepts.
- Interpretability: the whole model can be rewritten as some logical or probabilistic model allowing to trace its possible outcomes as the results of reasoning; for instance, rule-based systems or Bayesian networks are interpretable, provided a small complexity. Unfortunately, it seems that a trade-off between performance and interpretability/
explainability cannot be avoided. Overall, systems should provide reliable confidence estimates on their decisions or their internal parameters when those matter.
- Robustness: guarantees are given that the decisions will meet some constraints and not go wild, whatever the input data; guarantees can be formal (e.g., like formal proofs for critical systems) or statistical (e.g., like in aerospace industry).
- Fairness: the outcome of the system must not discriminate groups of people for their gender, race, age, sexual orientation, etc. Given a working definition of fairness, decision system have to be designed in order to avoid biases, even when these are built implicit by combining the information available on individuals.
Here are some examples of biased outputs and practices:
- Explanations: asking a Deep Network the reasons for its conclusions, one usually get explanations that look like « this is a cat because the cat output is 0.85, while the dog output is 0.22 », whereas human beings would expect something like « this is a cat because it has fur, whiskers and claws, and 2 triangular-shaped ears ». Whereas this might be accepted for cat recognition, it is not socially acceptable in the case of decisions that greatly impact people’s lifes, like « you have not been selected for the universities you were applying to », or « you will not get that loan », or « you will remain in jail for another 5 years ».
- Adversarial examples: after a AI has been trained, some images, that are correctly classified by the model, can be purposely modified in such a way that no human eye can notice a difference, but the AI is unable to correctly classify them any more. Think at what could happen if, with some simple tiny bits of paint on STOP road signs, you could make all autonomous vehicles see a « Speed limit 80 » sign.
- Gender discrimination: It might seem a good idea to remove the gender from all personal data to avoid gender discrimination. But unfortunately, it’s not that simple. Your digital behavior (favorite purchased items, journeys, age …) is enough to « guess » your gender, and thus to possibly discriminate.
AI (artificial intelligence) is the simulation of human intelligence processes by machines, especially computer systems. These processes include : learning (the ability to assimilate information as well as the rules for using this information), reasoning (using rules to reach conclusions) and self-correction.