class: center, middle, inverse, title-slide # Current Research Trends ## Curso JAE ICMAT 2021 ### Roi Naveiro y David Ríos Insua ### 2021-06-28 --- <script type="text/x-mathjax-config"> MathJax.Hub.Config({ TeX: { Macros: { Xcal: "{\\mathcal{X}}", Xbf: "{\\mathbf{X}}", Zbf: "{\\mathbf{Z}}", Vbf: "{\\mathbf{V}}", Hbf: "{\\mathbf{H}}", Rbb: "{\\mathbb{R}}" }, extensions: ["AMSmath.js","AMSsymbols.js"] } }); </script> # Modern research (ML and Bayes) * NLP. * RL. * Interpretability and Explainability. * Fairness. * Causal Inference. * Scalable Bayesian Inference. * Adversarial ML. --- # NLP - Arms race * Pre-trained language models + fine tuning  * Wu Dao 1.75 **trillion parameters**!! --- # NLP - Research * Architecture design. * Applications. 1. Dialog systems. 2. Emotion recognition. 3. Computer vision + NLP.s * Theoretical results? * Non democratic... only accesible to people with data and computational power (google, facebook, ...) --- # Reinforcenment Learning * According to a lot of people, the way to AGI. * Multi-agent RL. * Many applications.  --- # Interpretability & Explainability * Black-box algorithms are not always applicable. * We need algorithms to tell us why the make the decisions. * **Interpretability**: use algorithms that are interpretable per se. * **Explainability**: use black box algorithms and ellaborate explanations a posteriori. * The do not explain reality!! * Rashomon sets Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5), 206-215. --- # Fairness - Bias in ML * Data is biased `\(\rightarrow\)` algorithms are biased  --- # Fairness - Bias in ML  --- # Research on Fairness * Definition. * How to **guarantee** that models are not using, e.g., racial information? * Uncovering biases. * Interpretability... --- # Causal Inference * Data will provide associations between variables... * ... to do science we need to measure **causation**! --- # Correlation is not causation  --- # Causal Inference * The model is **key**!  --- # Causal Inference * Can we determine cause-effect relations using data? * Under some circumstances, and some assumptions, yes. * Essentially, we assume that we observe every confounder. * Research on how to adjust for every confounder... * And on applications. --- # Scalable Bayesian Inference --- # Adversarial Machine Learning * Automation using ML increases... attempts to cheat ML systems increases.  --- # Adversarial Machine Learning  --- # Adversarial Machine Learning * Make ML algorithms **robust** against likely attacks. * Idea: formalize confrontation between algorithm and adversary as a **game**. * Compute **Nash equilibrium** and use it as robust solution. * Common Knowledge? * Bayesian games, adversarial risk analysis? ---