6533b852fe1ef96bd12aac38

RESEARCH PRODUCT

Online Web Bot Detection Using a Sequential Classification Approach

Grażyna SuchackaFrancesco MasulliAlberto CabriStefano Rovetta

subject

Web serverHTTP request analysis; Internet security; Machine learning; Neural networks; Sequential classification; Web bot detectionSettore INF/01 - InformaticaWeb bot detectionComputer sciencebusiness.industrySequential classification020206 networking & telecommunications02 engineering and technologyMachine learningcomputer.software_genreInternet securitySession (web analytics)Task (computing)Web trafficMachine learning0202 electrical engineering electronic engineering information engineeringHTTP request analysis020201 artificial intelligence & image processingArtificial intelligencebusinesscomputerNeural networksInternet security

description

A significant problem nowadays is detection of Web traffic generated by automatic software agents (Web bots). Some studies have dealt with this task by proposing various approaches to Web traffic classification in order to distinguish the traffic stemming from human users' visits from that generated by bots. Most of previous works addressed the problem of offline bot recognition, based on available information on user sessions completed on a Web server. Very few approaches, however, have been proposed to recognize bots online, before the session completes. This paper proposes a novel approach to binary classification of a multivariate data stream incoming on a Web server, in order to recognize ongoing user sessions as generated by bots or humans. The present approach uses deep neural networks combined with Wald's Sequential Probability Ratio Test to express the relationship between subsequent HTTP requests in an ongoing session and to assess the likelihood of each session being generated by a bot or human before it ends. Experimental results showed the ability of the proposed approach to detect Web bots online with high performance scores and a small number of false negatives, as evidenced by the Recall index, minimizing the impact on human visitors. Another valuable indicator is the speed of decision: the present method allows very quick classification of nearly all sessions, leaving only very few of them undecided.

yearjournalcountryeditionlanguage
2019-01-012018 IEEE 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference on Smart City; IEEE 4th International Conference on Data Science and Systems (HPCC/SmartCity/DSS)
https://doi.org/10.1109/hpcc/smartcity/dss.2018.00252