This is a crowdsourced annotated bibliography of research and resources related to BERT-like models.

If you’d like to add to the bibliography, you can do so in this Dropbox document. We will update the bibliography on this web page periodically.

Technical Readings

Tutorials & Primers

Risks & Ethical Concerns

Applied Humanities

Critical Humanities

Tools

  • transformers from HuggingFace

    • An API for accessing and training ML models. Includes access to popular models such as BERT, T5, LLaMA, and much more.
  • Easy-Bert by Rob Rua

    • A simple API for accessing BERT
  • CLIP-as-Service from Jina AI

    • A service for easy image and text embedding

Educational Resources

  • Using BERT for next sentence prediction by Ted Underwood (adapted and used in Dan Sinykin’s Emory course “Practical Approaches to Data Science with Text” in 2020)

    • A notebook for teaching students about BERT