Publications

Statistical Evidence for Learnable Lexical Subclasses in Japanese
Takashi Morita
Timothy J. O’Donnell
It has been proposed that the Japanese lexicon can be divided into etymologically defined sublexica on phonotactic and other grounds. Howeve… (see more)r, the psychological reality of this sublexical analysis has been challenged by some authors, who have appealed to putative problems with the learnability of the system. In this study, we apply a commonly used clustering method to Japanese words and show that there is robust statistical evidence for the sublexica and, thereby, that such sublexica are learnable. The model is able to recover phonotactic properties of sublexica previously discussed in the literature, and also reveals hitherto unnoticed phonotactic properties that are characteristic of sublexical membership and can serve as a basis for future experimental investigations. The proposed approach is general and based purely on phonotactic information and thus can be applied to other languages.
2022-01-04
Linguistic Inquiry (published)
doi.org
AI-based modeling of brain and behavior: Combining neuroimaging, imitation learning and video games
Anirudha Kemtur
François Paugam
Basile Pinsard
Pravish Sainath
Yann Harel
Maximilien Le Clei
Julie Boyle
Karim Jerbi
Pierre Bellec
2021-12-31
Conference on Cognitive Computational Neuroscience (published)
doi.org
Appendix: On the Expressivity of Markov Reward
David Abel
Will Dabney
Anna Harutyunyan
Mark K. Ho
Michael L. Littman
Doina Precup
Satinder Singh
(Q1) What does it mean for Bob to *solve* one of these tasks? That is, if Alice chooses a SOAP, PO, or TO for Bob to learn to solve, when ca… (see more)n Alice determine Bob has solved the task? A: Bob can be said to be doing better on a given task if his behavior improves, as is typical in evaluating behavior under reward. The difference with SOAPs, POs, and TOs is that we measure improvement relative to the task rather than reward. For instance, given a SOAP, we might say that Bob has solved the task once he has found one of the good policies, and we might measure Bob’s progress on a task in terms of the distance of his greedy policy to one of the good policies (as done in our learning experiments). The same reasoning applies to POs and TOs: Bob is doing better on a task in so far as his greedy policy (or trajectories) is (are) higher up the ordering.
2021-12-31
(published)
www.semanticscholar.org
Approximate information state for approximate planning and reinforcement learning in partially observed systems
Jayakumar Subramanian
Amit Sinha
Raihan Seraj
Aditya Mahajan
We propose a theoretical framework for approximate planning and learning in partially observed systems. Our framework is based on the fundam… (see more)ental notion of information state. We provide two equivalent definitions of information state---i) a function of history which is sufficient to compute the expected reward and predict its next value; ii) equivalently, a function of the history which can be recursively updated and is sufficient to compute the expected reward and predict the next observation. An information state always leads to a dynamic programming decomposition. Our key result is to show that if a function of the history (called approximate information state (AIS)) approximately satisfies the properties of the information state, then there is a corresponding approximate dynamic program. We show that the policy computed using this is approximately optimal with bounded loss of optimality. We show that several approximations in state, observation and action spaces in literature can be viewed as instances of AIS. In some of these cases, we obtain tighter bounds. A salient feature of AIS is that it can be learnt from data. We present AIS based multi-time scale policy gradient algorithms. and detailed numerical experiments with low, moderate and high dimensional environments.
2021-12-31
Journal of Machine Learning Research (published)
arxiv.org
arxiv.org
Approximate minimization of weighted tree automata
Borja Balle
Guillaume Rabusseau
2021-12-31
Information and Computation (published)
doi.org
Aspirations and Practice of Model Documentation: Moving the Needle with Nudging and Traceability
Avinash Bhat
Austin Coursey
Grace Hu
Sixian Li
Nadia Nahar
Shurui Zhou
Christian Kästner
Jin L.C. Guo
2021-12-31
(published)
doi.org
Attention Is All You Need in Speech Separation
Cem Subakan
Mirco Ravanelli
Samuele Cornell
Mirko Bronzi
Jianyuan Zhong
Recurrent Neural Networks (RNNs) have long been the dominant architecture in sequence-to-sequence learning. RNNs, however, are inherently se… (see more)quential models that do not allow parallelization of their computations. Transformers are emerging as a natural alternative to standard RNNs, replacing recurrent computations with a multi-head attention mechanism. In this paper, we propose the SepFormer, a novel RNN-free Transformer-based neural network for speech separation. The SepFormer learns short and long-term dependencies with a multi-scale approach that employs transformers. The proposed model achieves state-of-the-art (SOTA) performance on the standard WSJ0-2/3mix datasets. It reaches an SI-SNRi of 22.3 dB on WSJ0-2mix and an SI-SNRi of 19.5 dB on WSJ0-3mix. The SepFormer inherits the parallelization advantages of Transformers and achieves a competitive performance even when downsampling the encoded representation by a factor of 8. It is thus significantly faster and it is less memory-demanding than the latest speech separation systems with comparable performance.
2021-12-31
IEEE International Conference on Acoustics, Speech and Signal Processing (unknown)
doi.org
arxiv.org
Augmenting Human Selves Through Artificial Agents – Lessons From the Brain
Georg Northoff
Maia Fraser
John Griffiths
Dimitris A. Pinotsis
Prakash Panangaden
Rosalyn Moran
Karl Friston
Much of current artificial intelligence (AI) and the drive toward artificial general intelligence (AGI) focuses on developing machines for f… (see more)unctional tasks that humans accomplish. These may be narrowly specified tasks as in AI, or more general tasks as in AGI – but typically these tasks do not target higher-level human cognitive abilities, such as consciousness or morality; these are left to the realm of so-called “strong AI” or “artificial consciousness.” In this paper, we focus on how a machine can augment humans rather than do what they do, and we extend this beyond AGI-style tasks to augmenting peculiarly personal human capacities, such as wellbeing and morality. We base this proposal on associating such capacities with the “self,” which we define as the “environment-agent nexus”; namely, a fine-tuned interaction of brain with environment in all its relevant variables. We consider richly adaptive architectures that have the potential to implement this interaction by taking lessons from the brain. In particular, we suggest conjoining the free energy principle (FEP) with the dynamic temporo-spatial (TSD) view of neuro-mental processes. Our proposed integration of FEP and TSD – in the implementation of artificial agents – offers a novel, expressive, and explainable way for artificial agents to adapt to different environmental contexts. The targeted applications are broad: from adaptive intelligence augmenting agents (IA’s) that assist psychiatric self-regulation to environmental disaster prediction and personal assistants. This reflects the central role of the mind and moral decision-making in most of what we do as humans.
2021-12-31
Frontiers in Computational Neuroscience (published)
doi.org
Author Correction: Gradient-based learning drives robust representations in recurrent neural networks by balancing compression and expansion
Matthew Farrell
Stefano Recanatesi
Timothy Moore
Guillaume Lajoie
Eric Shea-Brown
2021-12-31
Nature Machine Intelligence (published)
doi.org
Behind the Machine's Gaze: Biologically Constrained Neural Networks Exhibit Human-like Visual Attention
Leo Schwinn
Doina Precup
B. Eskofier
Dario Zanca
.
2021-12-31
arXiv.org (preprint)
doi.org
Biasly: a machine learning based platform for automatic racial discrimination detection in online texts
David Bamman
Chris Dyer
Noah A. Smith. 2014
Steven Bird
Ewan Klein
Edward Loper
Nat-527
Jacob Devlin
Ming-Wei Chang
Kenton Lee
Kristina Toutanova. 2019
Bert
Samuel Gehman
Suchin Gururangan
Maarten Sap
Dan Hendrycks
Kevin Gimpel. 2020
Gaussian
Alex Lamb
Di He … (see 22 more)
Anirudh Goyal
Guolin Ke
Feng-Ju Liao
Mirco Ravanaelli
Yoshua Bengio
Zhenzhong Lan
Mingda Chen
Sebastian Goodman
Yann Lecun
Bernhard E. Boser
J. Denker
Don-608 nie Henderson
Robin Howard
Wayne Hubbard
Yinhan Liu
Myle Ott
Naman Goyal
Jingfei Du
Mandar Joshi
Danqi Chen
Omer Levy
Mike Lewis
Warning : this paper contains content that may 001 be offensive or upsetting. 002 Detecting hateful, toxic, and otherwise racist 003 or sexi… (see more)st language in user-generated online con-004 tents has become an increasingly important task 005 in recent years. Indeed, the anonymity, the 006 transience, the size of messages, and the dif-007 ficulty of management, facilitate the diffusion 008 of racist or hateful messages across the Inter-009 net. The critical influence of this cyber-racism 010 is no longer limited to social media, but also 011 has a significant effect on our society : corpo-012 rate business operation, users’ health, crimes, 013 etc. Traditional racist speech reporting chan-014 nels have proven inadequate due to the enor-015 mous explosion of information, so there is an 016 urgent need for a method to automatically and 017 promptly detect texts with racial discrimination. 018 We propose in this work, a machine learning-019 based approach to enable automatic detection 020 of racist text content over the internet. State-of-021 the-art machine learning models that are able 022 to grasp language structures are adapted in this 023 study. Our main contribution include 1) a large 024 scale racial discrimination data set collected 025 from three distinct sources and annotated ac-026 cording to a guideline developed by specialists, 027 2) a set of machine learning models with vari-028 ous architectures for racial discrimination de-029 tection, and 3) a web-browser-based software 030 that assist users to debias their texts when us-031 ing the internet. All these resources are made 032 publicly available.
2021-12-31
(published)
www.semanticscholar.org
Bisimulation metrics and norms for real-weighted automata
Borja Balle
Pascale Gourdeau
Prakash Panangaden
2021-12-31
Information and Computation (published)
doi.org