owanr/ghc-google-t5-v1_1-large-inter_model-sorted-model_annots_str

ghc-google-t5-v1_1-large-inter_model-sorted

This model is a fine-tuned version of google/t5-v1_1-large on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2715

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 200

Training results

Training Loss	Epoch	Step	Validation Loss
19.7566	1.0	345	17.7436
13.8344	2.0	690	12.7846
11.4651	3.0	1035	12.1160
0.5573	4.0	1380	0.4277
0.4353	5.0	1725	0.2603
0.3576	6.0	2070	0.2659
0.2938	7.0	2415	0.2504
0.3004	8.0	2760	0.2596
0.2794	9.0	3105	0.2495
0.3628	10.0	3450	0.2468
0.2832	11.0	3795	0.2537
0.2646	12.0	4140	0.2437
0.2656	13.0	4485	0.2404
0.2782	14.0	4830	0.2395
0.2869	15.0	5175	0.2406
0.2706	16.0	5520	0.2366
0.2925	17.0	5865	0.2385
0.2979	18.0	6210	0.2359
0.2838	19.0	6555	0.2356
0.2515	20.0	6900	0.2358
0.2547	21.0	7245	0.2357
0.278	22.0	7590	0.2340
0.2633	23.0	7935	0.2384
0.2489	24.0	8280	0.2353
0.272	25.0	8625	0.2331
0.2607	26.0	8970	0.2331
0.2419	27.0	9315	0.2344
0.2888	28.0	9660	0.2335
0.2876	29.0	10005	0.2334
0.2677	30.0	10350	0.2334

Framework versions

• Transformers 4.34.0
• Pytorch 2.1.0+cu121
• Datasets 2.14.5
• Tokenizers 0.14.1