Arabert

Arabic Language Model based on Bert

Corpora

• OSCAR's CommonCrawl Dataset
• Arabic BERT Corpus
• Hindawi
• ArabicWeb16
• OPUS
• Wikimedia

Tutorials

• How to train a new language model from scratch using Transformers and Tokenizers
• Training BERT Language Model From Scratch On TPUs

Compute

• TensorFlow Research Cloud
• Google Colab
• Kaggle Kernels