2017 · 43 citations · 22 references
ComputingComputer EngineeringComputer ScienceComputer ArchitectureMachine LearningParallel ComputingAccelerator ArchitectureDeep LearningHardware AccelerationParallel ArchitecturesParallel Computer ArchitectureScalable Machine LearningAcceleratorsConvolutional Neural NetworksDeep Learning AccelerationDeep Learning AcceleratorsError ResilienceImplementation ComplexityLayer-based CompositionScalable Computing
Stochastic Computing (SC) is an alternative design paradigm particularly useful for applications where cost is critical. SC has been applied to neural networks, as neural networks are known for their high computational complexity. However previous work in this area has critical limitations such as the fully-parallel architecture assumption, which prevent them from being applicable to recent ones such as convolutional neural networks, or ConvNets. This paper presents the first SC architecture for ConvNets, shows its feasibility, with detailed analyses of implementation overheads. Our SC-ConvNet is a hybrid between SC and conventional binary design, which is a marked difference from earlier SC-based neural networks. Though this might seem like a compromise, it is a novel feature driven by the need to support modern ConvNets at scale, which commonly have many, large layers. Our proposed architecture also features hybrid layer composition, which helps achieve very high recognition accuracy. Our detailed evaluation results involving functional simulation and RTL synthesis suggest that SC-ConvNets are indeed competitive with conventional binary designs, even without considering inherent error resilience of SC.
22
Yangqing Jia, Evan Shelhamer, Jeff Donahue et al. · 2014 · 11.2K citations
Tianshi Chen, Zidong Du, Ninghui Sun et al. · 2014 · 1.3K citations
Dynamic energy-accuracy trade-off using stochastic computing in deep neural networks
Kyounghoon Kim, Jungki Kim, Joonsang Yu et al. · 2016 · 193 citations