0000000000808007
AUTHOR
Mykola Harvat
Memory degradation induced by attention in recurrent neural architectures
This paper studies the memory mechanisms in recurrent neural architectures when attention models are included. Pure-attention models like Transformers are more and more popular as they tend to outperform models with recurrent connections in many different tasks. Our conjecture is that attention prevents the recurrent connections from transferring information properly between consecutive next steps. This conjecture is empirically tested using five different models, namely, a model without attention, a standard Loung attention model, a standard Bahdanau attention model, and our proposal to add attention to the inputs in order to fill the gap between recurrent and parallel architectures (for b…
Regression Modeling of the Antioxidant-to-Nephroprotective Relation Shows the Pivotal Role of Oxidative Stress in Cisplatin Nephrotoxicity
The clinical utility of the chemotherapeutic drug cisplatin is significantly limited by its nephrotoxicity, which is characterized by electrolytic disorders, glomerular filtration rate decline, and azotemia. These alterations are consequences of a primary tubulopathy causing injury to proximal and distal epithelial cells, and thus tubular dysfunction. Oxidative stress plays a role in cisplatin nephrotoxicity and cytotoxicity, but its relative contribution to overall toxicity remains unknown. We studied the relation between the degree of oxidative reduction (provided by antioxidant treatment) and the extent of nephrotoxicity amelioration (i.e., nephroprotection) by means of a regression anal…