Explanation of Pre-training of the transformers