microsoft codebert base model

🤗 Huggingface microsoft/codebert-base

The model microsoft codebert base is a Natural Language Processing (NLP) Model implemented in Transformer library, generally using the Python programming language.

What is the microsoft codebert base model?

Pretrained weights for CodeBERT: A Pre-Trained Model for Programming and Natural Languages . The model is trained on bi-modal data (documents & code) of CodeSearchNet . It is trained with MLM+RTD objective (cf. the paper) Please see the official repository for scripts that support "code search" and "code-to-document generation" for "code to document generation" The author of the paper is Zhangyin Feng and Daya Guo and . Duyu Tang and . Nan Duan and Xiaocheng Feng and . Xiaoyng Feng . Ming Gong and Linjun Shou and . Bing Qin and Ting,

Fine-tune microsoft codebert-base models

Metatext is a powerful no-code tool for train, tune and integrate custom NLP models

➡️  Learn more

Model usage

You can find microsoft codebert base model easily in transformers python library. To download and use any of the pretrained models on your given task, you just need to use those a few lines of codes (PyTorch version). Here an example to download using pip (a package installer for Python)

Download and install using pip

$ pip install transformers


Usage in python

# Import generic wrappers
from transformers import AutoModel, AutoTokenizer 


# Define the model repo
model_name = "microsoft/codebert-base" 


# Download pytorch model
model = AutoModel.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)


# Transform input tokens 
inputs = tokenizer("Hello world!", return_tensors="pt")

# Model apply
outputs = model(**inputs)
    

More info about microsoft codebert-base

See the paper, download and more info