Publications

Analog-to-Stochastic Converter Using Magnetic Tunnel Junction Devices for Vision Chips
Naoya Onizawa
Daisaku Katagiri
Warren J. Gross
Takahiro Hanyu
This paper introduces an analog-to-stochastic converter using a magnetic tunnel junction (MTJ) device for vision chips based on stochastic c… (voir plus)omputation. Stochastic computation has been recently exploited for area-efficient hardware implementation, such as low-density parity-check (LDPC) decoders and image processors. However, power-and-area hungry two-step (analog-to-digital and digital-to-stochastic) converters are required for the analog to stochastic signal conversion. To realize a one-step conversion, an MTJ device is used as it inherently exhibits a probabilistic switching behavior between two resistance states. Exploiting the device-based probabilistic behavior, analog signals can be directly and area-efficiently converted to stochastic signals to mitigate the signal-conversion overhead. The analog-to-stochastic signal conversion is theoretically described and the conversion characteristic is evaluated using device and circuit parameters. In addition, the resistance variability of the MTJ device is considered in order to compensate the variability effect on the signal conversion. Based on the theoretical analysis, the analog-to-stochastic converter is designed in 90nm CMOS and 100nm MTJ technologies and is verified using a SPICE simulator (NS-SPICE) that handles both transistors and MTJ devices.
2015-12-21
IEEE Transactions on Nanotechnology (publié)
doi.org
arxiv.org
Describing Multimedia Content Using Attention-Based Encoder-Decoder Networks
Kyunghyun Cho
Aaron Courville
Yoshua Bengio
Whereas deep neural networks were first mostly used for classification tasks, they are rapidly expanding in the realm of structured output p… (voir plus)roblems, where the observed target is composed of multiple random variables that have a rich joint distribution, given the input. In this paper we focus on the case where the input also has a rich structure and the input and output structures are somehow related. We describe systems that learn to attend to different places in the input, for each element of the output, for a variety of tasks: machine translation, image caption generation, video clip description, and speech recognition. All these systems are based on a shared set of building blocks: gated recurrent neural networks and convolutional neural networks, along with trained attention mechanisms. We report on experimental results with these systems, showing impressively good performance and the advantage of the attention mechanism.
2015-10-31
IEEE Transactions on Multimedia (publié)
doi.org
arxiv.org
Poisson Group Testing: A Probabilistic Model for Boolean Compressed Sensing
Amin Emad
Olgica Milenkovic
We introduce a novel probabilistic group testing framework, termed Poisson group testing, in which the number of defectives follows a right-… (voir plus)truncated Poisson distribution. The Poisson model has a number of new applications, including dynamic testing with diminishing relative rates of defectives. We consider both nonadaptive and semi-adaptive identification methods. For nonadaptive methods, we derive a lower bound on the number of tests required to identify the defectives with a probability of error that asymptotically converges to zero; in addition, we propose test matrix constructions for which the number of tests closely matches the lower bound. For semiadaptive methods, we describe a lower bound on the expected number of tests required to identify the defectives with zero error probability. In addition, we propose a stage-wise reconstruction algorithm for which the expected number of tests is only a constant factor away from the lower bound. The methods rely only on an estimate of the average number of defectives, rather than on the individual probabilities of subjects being defective.
2015-08-14
IEEE Transactions on Signal Processing (publié)
doi.org
arxiv.org
Clinical Image-Based Procedures. Translational Research in Medical Imaging
Ian J. Gerard
Marta Kersten-Oertel
Simon Drouin
Jeffery Alan Hall
Kevin Petrecca
Dante De Nigris
Tal Arbel
D. Collins
2014-12-31
Lecture Notes in Computer Science (publié)
doi.org
ReSeg: A Recurrent Neural Network-based Model for Semantic Segmentation
Francesco Visin
Adriana Romero
Kyunghyun Cho
Matteo Matteucci
Marco Ciccone
Kyle Kastner
Yoshua Bengio
Aaron Courville
We propose a structured prediction architecture, which exploits the local generic features extracted by Convolutional Neural Networks and th… (voir plus)e capacity of Recurrent Neural Networks (RNN) to retrieve distant dependencies. The proposed architecture, called ReSeg, is based on the recently introduced ReNet model for image classification. We modify and extend it to perform the more challenging task of semantic segmentation. Each ReNet layer is composed of four RNN that sweep the image horizontally and vertically in both directions, encoding patches or activations, and providing relevant global information. Moreover, ReNet layers are stacked on top of pre-trained convolutional layers, benefiting from generic local features. Upsampling layers follow ReNet layers to recover the original image resolution in the final predictions. The proposed ReSeg architecture is efficient, flexible and suitable for a variety of semantic segmentation tasks. We evaluate ReSeg on several widely-used semantic segmentation datasets: Weizmann Horse, Oxford Flower, and CamVid; achieving state-of-the-art performance. Results show that ReSeg can act as a suitable architecture for semantic segmentation tasks, and may have further applications in other structured prediction problems. The source code and model hyperparameters are available on https://github.com/fvisin/reseg.
2014-12-31
CoRR (publié)
doi.org
arxiv.org
A Scalable Successive-Cancellation Decoder for Polar Codes
Alexandre J. Raymond
Warren J. Gross
Polar codes are the first error-correcting codes to provably achieve channel capacity, asymptotically in code length, with an explicit const… (voir plus)ruction. However, under successive-cancellation decoding, polar codes require very long code lengths to compete with existing modern codes. Nonetheless, the successive cancellation algorithm enables very-low-complexity implementations in hardware, due to the regular structure exhibited by polar codes. In this paper, we present an improved architecture for successive-cancellation decoding of polar codes, making use of a novel semi-parallel, encoder-based partial-sum computation module. We also provide quantization results for realistic code length N=215, and explore various optimization techniques such as a chained processing element and a variable quantization scheme. This design is shown to scale to code lengths of up to N=221, enabled by its low logic use, low register use and simple datapaths, limited almost exclusively by the amount of available SRAM. It also supports an overlapped loading of frames, allowing full-throughput decoding with a single set of input buffers.
2014-10-14
IEEE Transactions on Signal Processing (publié)
doi.org
arxiv.org
High-Throughput Energy-Efficient LDPC Decoders Using Differential Binary Message Passing
Kevin Cushon
Saied Hemati
Camille Leroux
Shie Mannor
Warren J. Gross
In this paper, we present energy-efficient architectures for decoders of low-density parity check (LDPC) codes using the differential decodi… (voir plus)ng with binary message passing (DD-BMP) algorithm and its modified variant (MDD-BMP). We also propose an improved differential binary (IDB) decoding algorithm. These algorithms offer significant intrinsic advantages in the energy domain: simple computations, low interconnect complexity, and very high throughput, while achieving error correction performance up to within 0.25 dB of the offset min-sum algorithm. We report on fully parallel decoder implementations of (273, 191), (1023, 781), and (4095, 3367) finite geometry-based LDPC codes in 65 nm CMOS. Using the MDD-BMP algorithm, these decoders achieve respective areas of 0.28 mm2, 1.38 mm2, and 15.37 mm2, average throughputs of 37 Gbps, 75 Gbps, and 141 Gbps, and energy efficiencies of 4.9 pJ/bit, 13.2 pJ/bit, and 37.9 pJ/bit with a 1.0 V supply voltage in post-layout simulations. At a reduced supply voltage of 0.8 V, these decoders achieve respective throughputs of 26 Gbps, 54 Gbps, and 94 Gbps, and energy efficiencies of 3.1 pJ/bit, 8.2 pJ/bit, and 23.5 pJ/bit. We also report on a fully parallel implementation of IDB for the (2048, 1723) LDPC code specified in the IEEE 802.3an (10GBASE-T) standard. This decoder achieves an area of 1.44 mm2, average throughput of 172 Gbps, and an energy efficiency of 2.8 pJ/bit with a 1.0 V supply voltage; at 0.8 V, it achieves throughput of 116 Gbps and energy efficiency of 1.7 pJ/bit.
2014-01-31
IEEE Transactions on Signal Processing (publié)
doi.org
Bayesian and grAphical Models for Biomedical Imaging
M. Jorge Cardoso
Ivor J. A. Simpson
Arbel, Tal
Doina Precup
Annemie Ribbens
2013-12-31
Lecture Notes in Computer Science (publié)
doi.org
Machine Learning and Interpretation in Neuroimaging
Irina Rish
Georg Langs
Leila Wehbe
Guillermo Cecchi
Kai-min Kevin Chang
Brian G Murphy
2013-12-31
Lecture Notes in Computer Science (publié)
doi.org
Experimental Algorithms
Samuel Rosat
Issmail ElHallaoui
François Soumis
Andrea Lodi
2013-10-31
Lecture Notes in Computer Science (publié)
doi.org
Adaptive Multiset Stochastic Decoding of Non-Binary LDPC Codes
Alexandru Ciobanu
Saied Hemati
Warren J. Gross
We propose a non-binary stochastic decoding algorithm for low-density parity-check (LDPC) codes over GF(q) with degree two variable nodes, c… (voir plus)alled Adaptive Multiset Stochastic Algorithm (AMSA). The algorithm uses multisets, an extension of sets that allows multiple occurrences of an element, to represent probability mass functions that simplifies the structure of the variable nodes. The run-time complexity of one decoding cycle using AMSA is O(q) for conventional memory architectures, and O(1) if a custom memory architecture is used. Two fully-parallel AMSA decoders are implemented on FPGA for two (192,96) (2,4)-regular codes over GF(64) and GF(256), both achieving a maximum clock frequency of 108 MHz. The GF(64) decoder has a coded throughput of 65 Mb/s at Eb/N0=2.4 dB when using conventional memory, while a decoder using the custom memory version can achieve 698 Mb/s at the same Eb/N0. At a frame error rate (FER) of 2×10-6 the GF(64) version of the algorithm is only 0.04 dB away from the floating-point SPA performance, and for the GF(256) code the difference is 0.2 dB. To the best of our knowledge, this is the first fully parallel non-binary LDPC decoder over GF(256) reported in the literature.
2013-08-14
IEEE Transactions on Signal Processing (publié)
doi.org
Multiscale Gossip for Efficient Decentralized Averaging in Wireless Packet Networks
Konstantinos I. Tsianos
Michael G. Rabbat
This paper describes and analyzes a hierarchical algorithm called Multiscale Gossip for solving the distributed average consensus problem in… (voir plus) wireless sensor networks. The algorithm proceeds by recursively partitioning a given network. Initially, nodes at the finest scale gossip to compute local averages. Then, using multi-hop communication and geographic routing to communicate between nodes that are not directly connected, these local averages are progressively fused up the hierarchy until the global average is computed. We show that the proposed hierarchical scheme with k=Θ(loglogn) levels of hierarchy is competitive with state-of-the-art randomized gossip algorithms in terms of message complexity, achieving ε-accuracy with high probability after O(n loglogn log[1/(ε)] ) single-hop messages. Key to our analysis is the way in which the network is recursively partitioned. We find that the above scaling law is achieved when subnetworks at scale j contain O(n(2/3)j) nodes; then the message complexity at any individual scale is O(n log[1/ε]). Another important consequence of the hierarchical construction is that the longest distance over which messages are exchanged is O(n1/3) hops (at the highest scale), and most messages (at lower scales) travel shorter distances. In networks that use link-level acknowledgements, this results in less congestion and resource usage by reducing message retransmissions. Simulations illustrate that the proposed scheme is more efficient than state-of-the-art randomized gossip algorithms based on averaging along paths.
2013-04-30
IEEE Transactions on Signal Processing (publié)
doi.org
arxiv.org