Activities

Publications

2024

• Collapse of Self-trained Language Models.
  David Herel, Tomás Mikolov
  The Second Tiny Papers Track at ICLR 2024, Tiny Papers @ ICLR 2024, Vienna, Austria, May 11, 2024: (2024).
  
• Editorial Introduction to the 2022 Conference on Artificial Life Special Issue.
  Silvia Holler, Barbora Hudcová, Richard Löffler, Stuart Bartlett
  Artif. Life 30: 144-146 (2024).
  
• Large Language Models: A Survey.
  Shervin Minaee, Tomás Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, Jianfeng Gao
  arXiv CoRR abs/2402.06196 (2024).
  
• Collapse of Self-trained Language Models.
  David Herel, Tomás Mikolov
  arXiv CoRR abs/2404.02305 (2024).
  
• Thinking Tokens for Language Modeling.
  David Herel, Tomás Mikolov
  arXiv CoRR abs/2405.08644 (2024).
  
• Simulation limitations of affine cellular automata.
  Barbora Hudcová, Jakub Krásenský
  Theor. Comput. Sci. 1003: (2024).
  

2023

• Preserving Semantics in Textual Adversarial Attacks.
  David Herel, Hugo Cisneros, Tomás Mikolov
  ECAI 2023 - 26th European Conference on Artificial Intelligence, September 30 - October 4, 2023, Kraków, Poland - Including 12th Conference on Prestigious Applications of Intelligent Systems (PAIS 2023): 1036-1043 (2023).
  
• Simulation Limitations of Affine Cellular Automata.
  Barbora Hudcová, Jakub Krásenský
  arXiv CoRR abs/2311.14477 (2023).
  
• Advancing State of the Art in Language Modeling.
  David Herel, Tomás Mikolov
  arXiv CoRR abs/2312.03735 (2023).
  

2022

• Benchmarking Learning Efficiency in Deep Reservoir Computing.
  Hugo Cisneros, Tomás Mikolov, Josef Sivic
  Conference on Lifelong Learning Agents, CoLLAs 2022, 22-24 August 2022, McGill University, Montréal, Québec, Canada.: 532-547 (2022).
  
• Classification of Discrete Dynamical Systems Based on Transients.
  Barbora Hudcová, Tomás Mikolov
  Artif. Life 27: 220-245 (2022/03).
  
• Emergence of Self-Reproducing Metabolisms as Recursive Algorithms in an Artificial Chemistry.
  Germán Kruszewski, Tomás Mikolov
  Artif. Life 27: 277-299 (2022/03).
  
• Emergence of Novelty in Evolutionary Algorithms.
  David Herel, Dominika Zogatova, Matej Kripner, Tomás Mikolov
  arXiv CoRR abs/2207.04857 (2022).
  
• Benchmarking Learning Efficiency in Deep Reservoir Computing.
  Hugo Cisneros, Josef Sivic, Tomás Mikolov
  arXiv CoRR abs/2210.02549 (2022).
  
• Preserving Semantics in Textual Adversarial Attacks.
  David Herel, Hugo Cisneros, Tomás Mikolov
  arXiv CoRR abs/2211.04205 (2022).
  

2021

• Language Modeling and Artificial Intelligence.
  Tomás Mikolov
  22nd Annual Conference of the International Speech Communication Association, Interspeech 2021, Brno, Czechia, August 30 - September 3, 2021.: (2021).
  
• Computational Hierarchy of Elementary Cellular Automata.
  Barbora Hudcová, Tomás Mikolov
  2021 Conference on Artificial Life, ALIFE 2021, online, July 19-23, 2021: (2021).
  
• Emergence of self-reproducing metabolisms as recursive algorithms in an Artificial Chemistry.
  Germán Kruszewski, Tomás Mikolov
  arXiv CoRR abs/2103.08245 (2021).
  
• Visualizing computation in large-scale cellular automata.
  Hugo Cisneros, Josef Sivic, Tomás Mikolov
  arXiv CoRR abs/2104.01008 (2021).
  
• Computational Hierarchy of Elementary Cellular Automata.
  Barbora Hudcová, Tomás Mikolov
  arXiv CoRR abs/2108.00415 (2021).
  
• Classification of Discrete Dynamical Systems Based on Transients.
  Barbora Hudcová, Tomás Mikolov
  arXiv CoRR abs/2108.01573 (2021).
  

2020

• Visualizing computation in large-scale cellular automata.
  Hugo Cisneros, Josef Sivic, Tomás Mikolov
  2020 Conference on Artificial Life, ALIFE 2020, online, July 13-18, 2020: 239-247 (2020).
  
• Classification of Complex Systems Based on Transients.
  Barbora Hudcová, Tomás Mikolov
  2020 Conference on Artificial Life, ALIFE 2020, online, July 13-18, 2020: 367-375 (2020).
  
• Combinatory Chemistry: Towards a Simple Model of Emergent Evolution.
  Germán Kruszewski, Tomás Mikolov
  2020 Conference on Artificial Life, ALIFE 2020, online, July 13-18, 2020: 411-419 (2020).
  
• Combinatory Chemistry: Towards a Simple Model of Emergent Evolution.
  Germán Kruszewski, Tomás Mikolov
  arXiv CoRR abs/2003.07916 (2020).
  
• Class-Agnostic Continual Learning of Alternating Languages and Domains.
  Germán Kruszewski, Ionut-Teodor Sorodoc, Tomás Mikolov
  arXiv CoRR abs/2004.03340 (2020).
  
• Classification of Complex Systems Based on Transients.
  Barbora Hudcová, Tomás Mikolov
  arXiv CoRR abs/2008.13503 (2020).
  
• Special Issue "On Defining Artificial Intelligence" - Commentaries and Author's Response.
  Dagmar Monett, Colin W. P. Lewis, Kristinn R. Thórisson, Joscha Bach, Gianluca Baldassarre, Giovanni Granato, Istvan S. N. Berkeley, François Chollet, Matthew Crosby, Henry Shevlin, John F. Sowa, John E. Laird, Shane Legg, Peter Lindes, Tomás Mikolov, William J. Rapaport, Raúl Rojas, Marek Rosa, Peter Stone, Richard S. Sutton, Roman V. Yampolskiy, Pei Wang, Roger C. Schank, Aaron Sloman, Alan F. T. Winfield
  J. Artif. Gen. Intell. 11: 1-100 (2020).
  

2019

• Evolving Structures in Complex Systems.
  Hugo Cisneros, Josef Sivic, Tomás Mikolov
  IEEE Symposium Series on Computational Intelligence, SSCI 2019, Xiamen, China, December 6-9, 2019: 230-237 (2019).
  
• Place Deduplication with Embeddings.
  Carl Yang, Do Huy Hoang, Tomás Mikolov, Jiawei Han
  The World Wide Web Conference, WWW 2019, San Francisco, CA, USA, May 13-17, 2019: 3420-3426 (2019).
  
• Place Deduplication with Embeddings.
  Carl Yang, Do Huy Hoang, Tomás Mikolov, Jiawei Han
  arXiv CoRR abs/1910.04861 (2019).
  
• Updating Pre-trained Word Vectors and Text Classifiers using Monolingual Alignment.
  Piotr Bojanowski, Onur Celebi, Tomás Mikolov, Edouard Grave, Armand Joulin
  arXiv CoRR abs/1910.06241 (2019).
  
• Evolving Structures in Complex Systems.
  Hugo Cisneros, Josef Sivic, Tomás Mikolov
  arXiv CoRR abs/1911.01086 (2019).
  

2018

• Efficient Large-Scale Multi-Modal Classification.
  Douwe Kiela, Edouard Grave, Armand Joulin, Tomás Mikolov
  Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018: 5198-5204 (2018).
  
• Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterion.
  Armand Joulin, Piotr Bojanowski, Tomás Mikolov, Hervé Jégou, Edouard Grave
  Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018: 2979-2984 (2018).
  
• Learning Word Vectors for 157 Languages.
  Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, Tomás Mikolov
  Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC 2018, Miyazaki, Japan, May 7-12, 2018.: (2018).
  
• Advances in Pre-Training Distributed Word Representations.
  Tomás Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, Armand Joulin
  Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC 2018, Miyazaki, Japan, May 7-12, 2018.: (2018).
  
• Efficient Large-Scale Multi-Modal Classification.
  Douwe Kiela, Edouard Grave, Armand Joulin, Tomás Mikolov
  arXiv CoRR abs/1802.02892 (2018).
  
• Learning Word Vectors for 157 Languages.
  Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, Tomás Mikolov
  arXiv CoRR abs/1802.06893 (2018).
  
• Improving Supervised Bilingual Mapping of Word Embeddings.
  Armand Joulin, Piotr Bojanowski, Tomás Mikolov, Edouard Grave
  arXiv CoRR abs/1804.07745 (2018).
  

2017

• Fast Linear Model for Knowledge Graph Embeddings.
  Armand Joulin, Edouard Grave, Piotr Bojanowski, Maximilian Nickel, Tomás Mikolov
  6th Workshop on Automated Knowledge Base Construction, AKBC@NIPS 2017, Long Beach, California, USA, December 8, 2017: (2017).
  
• Bag of Tricks for Efficient Text Classification.
  Armand Joulin, Edouard Grave, Piotr Bojanowski, Tomás Mikolov
  Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, Valencia, Spain, April 3-7, 2017, Volume 2: Short Papers: 427-431 (2017).
  
• CommAI: Evaluating the first steps towards a useful general AI.
  Marco Baroni, Armand Joulin, Allan Jabri, Germán Kruszewski, Angeliki Lazaridou, Klemen Simonic, Tomás Mikolov
  5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Workshop Track Proceedings: (2017).
  
• Variable Computation in Recurrent Neural Networks.
  Yacine Jernite, Edouard Grave, Armand Joulin, Tomás Mikolov
  5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings: (2017).
  
• CommAI: Evaluating the first steps towards a useful general AI.
  Marco Baroni, Armand Joulin, Allan Jabri, Germán Kruszewski, Angeliki Lazaridou, Klemen Simonic, Tomás Mikolov
  arXiv CoRR abs/1701.08954 (2017).
  
• Learning Simpler Language Models with the Delta Recurrent Neural Network Framework.
  II Alexander G. Ororbia, Tomás Mikolov, David Reitter
  arXiv CoRR abs/1703.08864 (2017).
  
• Fast Linear Model for Knowledge Graph Embeddings.
  Armand Joulin, Edouard Grave, Piotr Bojanowski, Maximilian Nickel, Tomás Mikolov
  arXiv CoRR abs/1710.10881 (2017).
  
• Advances in Pre-Training Distributed Word Representations.
  Tomás Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, Armand Joulin
  arXiv CoRR abs/1712.09405 (2017).
  
• Learning Simpler Language Models with the Differential State Framework.
  II Alexander G. Ororbia, Tomás Mikolov, David Reitter
  Neural Comput. 29: (2017).
  
• Enriching Word Vectors with Subword Information.
  Piotr Bojanowski, Edouard Grave, Armand Joulin, Tomás Mikolov
  Trans. Assoc. Comput. Linguistics 5: 135-146 (2017).
  

2016

• A Roadmap Towards Machine Intelligence.
  Tomás Mikolov, Armand Joulin, Marco Baroni
  Computational Linguistics and Intelligent Text Processing - 17th International Conference, CICLing 2016, Konya, Turkey, April 3-9, 2016, Revised Selected Papers, Part I: 29-61 (2016).
  
• Learning Simple Algorithms from Examples.
  Wojciech Zaremba, Tomás Mikolov, Armand Joulin, Rob Fergus
  Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016: 421-429 (2016).
  
• Enriching Word Vectors with Subword Information.
  Piotr Bojanowski, Edouard Grave, Armand Joulin, Tomás Mikolov
  arXiv CoRR abs/1607.04606 (2016).
  
• Variable Computation in Recurrent Neural Networks.
  Yacine Jernite, Edouard Grave, Armand Joulin, Tomás Mikolov
  arXiv CoRR abs/1611.06188 (2016).
  
• FastText.zip: Compressing text classification models.
  Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Hervé Jégou, Tomás Mikolov
  arXiv CoRR abs/1612.03651 (2016).
  
• Bag of Tricks for Efficient Text Classification.
  Armand Joulin, Edouard Grave, Piotr Bojanowski, Tomás Mikolov
  arXiv CoRR abs/1607.01759 (2016).
  
• Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks.
  Jason Weston, Antoine Bordes, Sumit Chopra, Tomás Mikolov
  4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings: (2016).
  

2015

• Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets.
  Armand Joulin, Tomás Mikolov
  Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada: 190-198 (2015).
  
• Alternative structures for character-level RNNs.
  Piotr Bojanowski, Armand Joulin, Tomás Mikolov
  arXiv CoRR abs/1511.06303 (2015).
  
• Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets.
  Armand Joulin, Tomás Mikolov
  arXiv CoRR abs/1503.01007 (2015).
  
• Ensemble of Generative and Discriminative Techniques for Sentiment Analysis of Movie Reviews.
  Grégoire Mesnil, Tomás Mikolov, Marc'Aurelio Ranzato, Yoshua Bengio
  3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Workshop Track Proceedings: (2015).
  
• A Roadmap towards Machine Intelligence.
  Tomás Mikolov, Armand Joulin, Marco Baroni
  arXiv CoRR abs/1511.08130 (2015).
  
• Learning Longer Memory in Recurrent Neural Networks.
  Tomás Mikolov, Armand Joulin, Sumit Chopra, Michaël Mathieu, Marc'Aurelio Ranzato
  3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Workshop Track Proceedings: (2015).
  
• Learning Simple Algorithms from Examples.
  Wojciech Zaremba, Tomás Mikolov, Armand Joulin, Rob Fergus
  arXiv CoRR abs/1511.07275 (2015).