{"library":"pytorch-pretrained-bert","title":"PyTorch Pretrained BERT","description":"PyTorch version of Google AI's BERT model with a script to load Google pre-trained models. This library (version 0.6.2) was the predecessor to the 'transformers' library by Hugging Face, which now includes BERT and many other models. It is deprecated and no longer maintained; all users should migrate to the 'transformers' package for active development, bug fixes, and better performance.","language":"python","status":"deprecated","last_verified":"Fri May 01","install":{"commands":["pip install pytorch-pretrained-bert==0.6.2","pip install transformers"],"cli":null},"imports":["from pytorch_pretrained_bert import BertModel","from pytorch_pretrained_bert import BertTokenizer"],"auth":{"required":false,"env_vars":[]},"quickstart":{"code":"from pytorch_pretrained_bert import BertTokenizer, BertModel\nimport torch\n\n# Load pre-trained model tokenizer (vocabulary)\ntokenizer = BertTokenizer.from_pretrained('bert-base-uncased')\n\n# Encode text\ntext = \"Who was Jim Henson?\"\ntokenized_text = tokenizer.tokenize(text)\nindexed_tokens = tokenizer.convert_tokens_to_ids(tokenized_text)\n\n# Convert to PyTorch tensors\ntokens_tensor = torch.tensor([indexed_tokens])\n\n# Load pre-trained model\nmodel = BertModel.from_pretrained('bert-base-uncased')\nmodel.eval()\n\n# Predict hidden states features\nwith torch.no_grad():\n    outputs = model(tokens_tensor)\nprint(outputs[0].shape)  # (batch_size, seq_len, hidden_size)","lang":"python","description":"Minimal example: tokenize input, load BERT, and get encoder output.","tag":null,"tag_description":null,"last_tested":null,"results":[]},"compatibility":null}