6533b856fe1ef96bd12b26d4

RESEARCH PRODUCT

Extending the Tsetlin Machine With Integer-Weighted Clauses for Increased Interpretability

K. Darshana AbeyrathnaMorten GoodwinOle-christoffer Granmo

subject

FOS: Computer and information sciencesBoosting (machine learning)Theoretical computer scienceinteger-weighted Tsetlin machineGeneral Computer ScienceComputer scienceComputer Science - Artificial Intelligence0206 medical engineeringNatural language understandingInference02 engineering and technologycomputer.software_genre0202 electrical engineering electronic engineering information engineeringGeneral Materials ScienceTsetlin machineVDP::Teknologi: 500::Informasjons- og kommunikasjonsteknologi: 550InterpretabilityArtificial neural networkLearning automatabusiness.industryDeep learningGeneral Engineeringinterpretable machine learningrule-based learninginterpretable AIPropositional calculusSupport vector machineArtificial Intelligence (cs.AI)TheoryofComputation_MATHEMATICALLOGICANDFORMALLANGUAGESXAIPattern recognition (psychology)020201 artificial intelligence & image processinglcsh:Electrical engineering. Electronics. Nuclear engineeringArtificial intelligencebusinesslcsh:TK1-9971computer020602 bioinformaticsInteger (computer science)

description

Despite significant effort, building models that are both interpretable and accurate is an unresolved challenge for many pattern recognition problems. In general, rule-based and linear models lack accuracy, while deep learning interpretability is based on rough approximations of the underlying inference. Using a linear combination of conjunctive clauses in propositional logic, Tsetlin Machines (TMs) have shown competitive performance on diverse benchmarks. However, to do so, many clauses are needed, which impacts interpretability. Here, we address the accuracy-interpretability challenge in machine learning by equipping the TM clauses with integer weights. The resulting Integer Weighted TM (IWTM) deals with the problem of learning which clauses are inaccurate and thus must team up to obtain high accuracy as a team (low weight clauses), and which clauses are sufficiently accurate to operate more independently (high weight clauses). Since each TM clause is formed adaptively by a team of Tsetlin Automata, identifying effective weights becomes a challenging online learning problem. We address this problem by extending each team of Tsetlin Automata with a stochastic searching on the line (SSL) automaton. In our novel scheme, the SSL automaton learns the weight of its clause in interaction with the corresponding Tsetlin Automata team, which, in turn, adapts the composition of the clause by the adjusting weight. We evaluate IWTM empirically using five datasets, including a study of interpetability. On average, IWTM uses 6.5 times fewer literals than the vanilla TM and 120 times fewer literals than a TM with real-valued weights. Furthermore, in terms of average F1-Score, IWTM outperforms simple Multi-Layered Artificial Neural Networks, Decision Trees, Support Vector Machines, K-Nearest Neighbor, Random Forest, XGBoost, Explainable Boosting Machines, and standard and real-value weighted TMs.

yearjournalcountryeditionlanguage
2020-05-11
http://arxiv.org/abs/2005.05131