---
language:
- {en}  # Example: fr
tags:
- {fill-mask}  # Example: audio
---


This model was pretrained from scratch on a custom vocabulary on Pubmed, Clinical trials corpus, and a small subset of Bookcorpus

It was used to do NER as is, with no fine-tuning as described [in this post](https://ajitrajasekharan.github.io/2021/01/02/my-first-post.html)

[Towards Data Science link to the same post](https://twitter.com/TDataScience/status/1486300137366466560?s=20)