Affiliation:
1. Stanford, Stanford, CA, USA
2. MIT, Cambridge, MA, USA
Abstract
We give an
n
O
(log log
n
)
-time membership query algorithm for properly and agnostically learning decision trees under the uniform distribution over { ± 1}
n
. Even in the realizable setting, the previous fastest runtime was
n
O
(log
n
)
, a consequence of a classic algorithm of Ehrenfeucht and Haussler.
Our algorithm shares similarities with practical heuristics for learning decision trees, which we augment with additional ideas to circumvent known lower bounds against these heuristics. To analyze our algorithm, we prove a new structural result for decision trees that strengthens a theorem of O’Donnell, Saks, Schramm, and Servedio. While the OSSS theorem says that every decision tree has an influential variable, we show how every decision tree can be “pruned” so that
every
variable in the resulting tree is influential.
Funder
NSF CAREER Award
DOE Award
ONR Young Investigator Award
NSF Award
Publisher
Association for Computing Machinery (ACM)
Subject
Artificial Intelligence,Hardware and Architecture,Information Systems,Control and Systems Engineering,Software
Reference30 articles.
1. Approximating Optimal Binary Decision Trees
2. Guy Blanc, Jane Lange, and Li-Yang Tan. 2020. Top-down induction of decision trees: Rigorous guarantees and inherent limitations. In Proceedings of the 11th Innovations in Theoretical Computer Science Conference (ITCS’20), Vol. 151. 1–44.
3. Rank-r decision trees are a subclass of r-decision lists
4. Weakly learning DNF and characterizing statistical query learning using Fourier analysis
Cited by
2 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Properly learning decision trees with queries is NP-hard;2023 IEEE 64th Annual Symposium on Foundations of Computer Science (FOCS);2023-11-06
2. Randomized versus Deterministic Decision Tree Size;Proceedings of the 55th Annual ACM Symposium on Theory of Computing;2023-06-02