 This proposed technique is an effective approach to develop localized medical BERT models and to enhance domain-specific models in the biomedical domain .
<clarity>  This proposed technique is an effective approach to develop localized medical BERT models and to enhance domain-specific models in the biomedical domain .
 Well-balanced pre-training by up-sampling instances derived from a corpus appropriate for the target task allows us to construct a high-performance BERT model .