Towards Efficient and Scalable Acceleration of Online Decision Tree Learning on FPGA

by   Zhe Lin, et al.

Decision trees are machine learning models commonly used in various application scenarios. In the era of big data, traditional decision tree induction algorithms are not suitable for learning large-scale datasets due to their stringent data storage requirement. Online decision tree learning algorithms have been devised to tackle this problem by concurrently training with incoming samples and providing inference results. However, even the most up-to-date online tree learning algorithms still suffer from either high memory usage or high computational intensity with dependency and long latency, making them challenging to implement in hardware. To overcome these difficulties, we introduce a new quantile-based algorithm to improve the induction of the Hoeffding tree, one of the state-of-the-art online learning models. The proposed algorithm is light-weight in terms of both memory and computational demand, while still maintaining high generalization ability. A series of optimization techniques dedicated to the proposed algorithm have been investigated from the hardware perspective, including coarse-grained and fine-grained parallelism, dynamic and memory-based resource sharing, pipelining with data forwarding. We further present a high-performance, hardware-efficient and scalable online decision tree learning system on a field-programmable gate array (FPGA) with system-level optimization techniques. Experimental results show that our proposed algorithm outperforms the state-of-the-art Hoeffding tree learning method, leading to 0.05 accuracy. Real implementation of the complete learning system on the FPGA demonstrates a 384x to 1581x speedup in execution time over the state-of-the-art design.


page 1

page 4

page 5


Hard-ODT: Hardware-Friendly Online Decision Tree Learning Algorithm and System

Decision trees are machine learning models commonly used in various appl...

DT2CAM: A Decision Tree to Content Addressable Memory Framework

Decision trees are considered one of the most powerful tools for data cl...

Decision Tree Based Hardware Power Monitoring for Run Time Dynamic Power Management in FPGA

Fine-grained runtime power management techniques could be promising solu...

Online Multi-target regression trees with stacked leaf models

The amount of available data raises at large steps. Developing machine l...

An Ensemble Learning Approach for In-situ Monitoring of FPGA Dynamic Power

As field-programmable gate arrays become prevalent in critical applicati...

HMT: A Hardware-Centric Hybrid Bonsai Merkle Tree Algorithm for High-Performance Authentication

Bonsai Merkle tree (BMT) is a widely used data structure for authenticat...

A Hardware-Friendly Algorithm for Scalable Training and Deployment of Dimensionality Reduction Models on FPGA

With ever-increasing application of machine learning models in various d...

Please sign up or login with your details

Forgot password? Click here to reset