Publications

How Do the Open Source Communities Address Usability and UX Issues?: An Exploratory Study

Jinghui Cheng

Jin L.C. Guo

Usability and user experience (UX) issues are often not well emphasized and addressed in open source software (OSS) development. There is an… (see more) imperative need for supporting OSS communities to collaboratively identify, understand, and fix UX design issues in a distributed environment. In this paper, we provide an initial step towards this effort and report on an exploratory study that investigated how the OSS communities currently reported, discussed, negotiated, and eventually addressed usability and UX issues. We conducted in-depth qualitative analysis of selected issue tracking threads from three OSS projects hosted on GitHub. Our findings indicated that discussions about usability and UX issues in OSS communities were largely influenced by the personal opinions and experiences of the participants. Moreover, the characteristics of the community may have greatly affected the focus of such discussion.

2018-04-19

Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (published)

Minimization of Graph Weighted Models over Circular Strings

Guillaume Rabusseau

2018-04-15

Foundations of Software Science and Computation Structure (published)

Dynamic Frame Skipping for Fast Speech Recognition in Recurrent Neural Network Based Acoustic Models

A recurrent neural network is a powerful tool for modeling sequential data such as text and speech. While recurrent neural networks have ach… (see more)ieved record-breaking results in speech recognition, one remaining challenge is their slow processing speed. The main cause comes from the nature of recurrent neural networks that read only one frame at each time step. Therefore, reducing the number of reads is an effective approach to reducing processing time. In this paper, we propose a novel recurrent neural network architecture called Skip-RNN, which dynamically skips speech frames that are less important. The Skip-RNN consists of an acoustic model network and skip-policy network that are jointly trained to classify speech frames and determine how many frames to skip. We evaluate our proposed approach on the Wall Street Journal corpus and show that it can accelerate acoustic model computation by up to 2.4 times without any noticeable degradation in transcription accuracy.

2018-04-14

2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (published)

Monaural Singing Voice Separation with Skip-Filtering Connections and Recurrent Inference of Time-Frequency Mask

Stylianos Ioannis Mimilakis

Konstantinos Drossos

Joao Felipe Santos

Gerald Schuller

Tuomas Virtanen

Singing voice separation based on deep learning relies on the usage of time-frequency masking. In many cases the masking process is not a le… (see more)arnable function or is not encapsulated into the deep learning optimization. Consequently, most of the existing methods rely on a post processing step using the generalized Wiener filtering. This work proposes a method that learns and optimizes (during training) a source-dependent mask and does not need the aforementioned post processing step. We introduce a recurrent inference algorithm, a sparse transformation step to improve the mask generation process, and a learned denoising filter. Obtained results show an increase of 0.49 dB for the signal to distortion ratio and 0.30 dB for the signal to interference ratio, compared to previous state-of-the-art approaches for monaural singing voice separation.

2018-04-14

2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (published)

Towards End-to-end Spoken Language Understanding

Dmitriy Serdyuk

Yongqiang Wang

Christian Fuegen

Anuj Kumar

Baiyang Liu

Spoken language understanding system is traditionally designed as a pipeline of a number of components. First, the audio signal is processed… (see more) by an automatic speech recognizer for transcription or n-best hypotheses. With the recognition results, a natural language understanding system classifies the text to structured data as domain, intent and slots for down-streaming consumers, such as dialog system, hands-free applications. These components are usually developed and optimized independently. In this paper, we present our study on an end-to-end learning system for spoken language understanding. With this unified approach, we can infer the semantic meaning directly from audio features without the intermediate text representation. This study showed that the trained model can achieve reasonable good result and demonstrated that the model can capture the semantic attention directly from the audio features.

2018-04-14

2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (published)

Iteratively unveiling new regions of interest in Deep Learning models

Florian Bordes

Tess Berthier

Lisa Di Jorio

P Vincent

Recent advance of deep learning has been transforming the landscape in many domains. However, understanding the predictions of a deep networ… (see more)k remains a challenge, which is especially sensitive in health care domains as interpretability is key. Techniques that rely on saliency maps -highlighting the region of an image that influence the classifier’s decision the mostare often used for that purpose. However, gradients fluctuation make saliency maps noisy and thus difficult to interpret at a human level. Moreover, models tend to focus on one particular influential region of interest (ROI) in the image, even though other regions might be relevant for the decision. We propose a new framework that refines those saliency maps to generate segmentation masks over the ROI on the initial image. In a second contribution, we propose to apply those masks over the original inputs, then evaluate our classifier on the masked inputs to identify previously overlooked ROI. This iterative procedure allows us to emphasize new region of interests by extracting meaningful information from the saliency maps.

2018-04-10

MIDL.amsterdam/2018/Abstract (accepted)

openreview.net

Fortified Networks: Improving the Robustness of Deep Networks by Modeling the Manifold of Hidden Representations

Deep networks have achieved impressive results across a variety of important tasks. However a known weakness is a failure to perform well wh… (see more)en evaluated on data which differ from the training distribution, even if these differences are very small, as is the case with adversarial examples. We propose Fortified Networks, a simple transformation of existing networks, which fortifies the hidden layers in a deep network by identifying when the hidden states are off of the data manifold, and maps these hidden states back to parts of the data manifold where the network performs well. Our principal contribution is to show that fortifying these hidden states improves the robustness of deep networks and our experiments (i) demonstrate improved robustness to standard adversarial attacks in both black-box and white-box threat models; (ii) suggest that our improvements are not primarily due to the gradient masking problem and (iii) show the advantage of doing this fortification in the hidden layers instead of the input space.

2018-04-06

ArXiv (preprint)

openreview.net

Dynamic Neural Turing Machine with Continuous and Discrete Addressing Schemes

We extend the neural Turing machine (NTM) model into a dynamic neural Turing machine (D-NTM) by introducing trainable address vectors. This … (see more)addressing scheme maintains for each memory cell two separate vectors, content and address vectors. This allows the D-NTM to learn a wide variety of location-based addressing strategies, including both linear and nonlinear ones. We implement the D-NTM with both continuous and discrete read and write mechanisms. We investigate the mechanisms and effects of learning to read and write into a memory through experiments on Facebook bAbI tasks using both a feedforward and GRU controller. We provide extensive analysis of our model and compare different variations of neural Turing machines on this task. We show that our model outperforms long short-term memory and NTM variants. We provide further experimental results on the sequential [Formula: see text]MNIST, Stanford Natural Language Inference, associative recall, and copy tasks.

2018-03-31

Neural Computation (published)

openreview.net

Fine-grained attention mechanism for neural machine translation

Heeyoul Choi

Kyunghyun Cho

2018-03-31

Neurocomputing (published)

Fisher Pruning of Deep Nets for Facial Trait Classification

Qing Tian

Tal Arbel

James J. Clark

Although deep nets have resulted in high accuracies for various visual tasks, their computational and space requirements are prohibitively h… (see more)igh for inclusion on devices without high-end GPUs. In this paper, we introduce a neuron/filter level pruning framework based on Fisher's LDA which leads to high accuracies for a wide array of facial trait classification tasks, while significantly reducing space/computational complexities. The approach is general and can be applied to convolutional, fully-connected, and module-based deep structures, in all cases leveraging the high decorrelation of neuron activations found in the pre-decision layer and cross-layer deconv dependency. Experimental results on binary and multi-category facial traits from the LFWA and Adience datasets illustrate the framework's comparable/better performance to state-of-the-art pruning approaches and compact structures (e.g. SqueezeNet, MobileNet). Ours successfully maintains comparable accuracies even after discarding most parameters (98%-99% for VGG-16, 82% for GoogLeNet) and with significant FLOP reductions (83% for VGG-16, 64% for GoogLeNet).

2018-03-20

ArXiv (preprint)