“You can use the straight through operator! During the forward pass you sample a discrete outcome given your NN weights to get an error for backprop. During the backward pass you directly propogate through the weights. This GradTree paper[1] does a good job covering how to do discrete gradient-based optimization (i.e. NNs w/ discrete representations). A…”
GradTree: Learning Axis-Aligned Decision Trees with Gradient Descent
Decision Trees (DTs) are commonly used for many machine learning tasks due to their high degree of interpretability. However, learning a DT from data is a difficult optimization problem, as it is non-convex and non-differentiable. Therefore, common approaches learn DTs using a greedy growth algorithm that minimizes the impurity locally at each internal node. Unfortunately, this greedy procedure can lead to inaccurate trees. In this paper, we present a novel approach for learning hard, axis-aligned DTs with gradient descent. The proposed method uses backpropagation with a straight-through operator on a dense DT representation, to jointly optimize all tree parameters. Our approach outperforms existing methods on binary classification benchmarks and achieves competitive results for multi-class tasks. The method is available under: https://github.com/s-marton/GradTree
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“There is some work to convert NNs to decision trees. https://towardsdatascience.com/neural-networks-as-decision-trees-89cd9fdcdf6a https://arxiv.org/abs/2210.05189 I haven't reviewed any of it, I only know of it tangentially. https://www.semanticscholar.org/paper/Converting-A-Trained-Neural-Network-To-a-Decision-Boz/fb6172737873a69bd8d0117c88301121b5c…”