Hugging Face's logo

yasmineee
/
NLLB_QLoRA

PEFT
Safetensors
Generated from Trainer
Model card Files Community
NLLB_QLoRA / README.md
yasmineee's picture
yasmineee
finetune-NLLB-600M-on-opus100-Ar2En-with-Qlora
dec0cc5 verified
preview code
|
raw
history blame contribute delete
1.93 kB
metadata
base_model: facebook/nllb-200-distilled-600M
library_name: peft
license: cc-by-nc-4.0
metrics:
  - bleu
  - rouge
tags:
  - generated_from_trainer
model-index:
  - name: NLLB_QLoRA
    results: []

Visualize in Weights & Biases

NLLB_QLoRA

This model is a fine-tuned version of facebook/nllb-200-distilled-600M on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.3340
  • Bleu: 31.5945
  • Rouge: 0.5906
  • Gen Len: 17.338

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 2
  • eval_batch_size: 2
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 8
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 3

Training results

Training Loss Epoch Step Validation Loss Bleu Rouge Gen Len
2.858 1.0 875 1.4023 30.5493 0.5771 17.3705
1.4649 2.0 1750 1.3447 31.343 0.5886 17.284
1.4247 3.0 2625 1.3340 31.5945 0.5906 17.338

Framework versions

  • PEFT 0.12.0
  • Transformers 4.42.3
  • Pytorch 2.1.2
  • Datasets 2.20.0
  • Tokenizers 0.19.1