distilbert-base-multilingual-cased-2-classification-contract-sections-v1

This model is a fine-tuned version of distilbert-base-multilingual-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2701
• Accuracy: 0.9677

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.9622	1.0	1000	1.7976	0.5933
0.984	2.0	2000	0.8920	0.8725
0.4814	3.0	3000	0.4687	0.9205
0.2581	4.0	4000	0.3044	0.9313
0.1556	5.0	5000	0.2370	0.9343
0.0978	6.0	6000	0.1958	0.9455
0.0807	7.0	7000	0.1774	0.948
0.0609	8.0	8000	0.1741	0.9515
0.0481	9.0	9000	0.1567	0.9565
0.0399	10.0	10000	0.1618	0.959
0.0429	11.0	11000	0.1470	0.965
0.0324	12.0	12000	0.1480	0.9667
0.0381	13.0	13000	0.1595	0.9637
0.0202	14.0	14000	0.1697	0.9613
0.0234	15.0	15000	0.1570	0.9685
0.0175	16.0	16000	0.1503	0.9663
0.0192	17.0	17000	0.1647	0.9657
0.0152	18.0	18000	0.1551	0.9708
0.0183	19.0	19000	0.1663	0.9683
0.0126	20.0	20000	0.1751	0.9623
0.013	21.0	21000	0.1703	0.9683
0.0095	22.0	22000	0.1834	0.964
0.0094	23.0	23000	0.1937	0.9623
0.0115	24.0	24000	0.1956	0.961
0.011	25.0	25000	0.1722	0.966
0.0089	26.0	26000	0.1850	0.966
0.0077	27.0	27000	0.1823	0.9663
0.0068	28.0	28000	0.1757	0.966
0.0064	29.0	29000	0.1831	0.9653
0.0058	30.0	30000	0.1906	0.9692
0.0067	31.0	31000	0.1939	0.9655
0.0065	32.0	32000	0.1895	0.9677
0.0074	33.0	33000	0.2113	0.9655
0.0046	34.0	34000	0.1990	0.9695
0.0063	35.0	35000	0.2073	0.9633
0.0029	36.0	36000	0.2171	0.968
0.0026	37.0	37000	0.2089	0.9655
0.0039	38.0	38000	0.2030	0.9673
0.0006	39.0	39000	0.2188	0.9692
0.0039	40.0	40000	0.2385	0.9635
0.003	41.0	41000	0.1982	0.9702
0.0013	42.0	42000	0.2411	0.9623
0.002	43.0	43000	0.2111	0.9683
0.0024	44.0	44000	0.2229	0.966
0.003	45.0	45000	0.2298	0.9655
0.0019	46.0	46000	0.2333	0.9653
0.0015	47.0	47000	0.2370	0.968
0.002	48.0	48000	0.2474	0.9643
0.0013	49.0	49000	0.2424	0.9645
0.002	50.0	50000	0.2471	0.967
0.003	51.0	51000	0.2567	0.9627
0.0005	52.0	52000	0.2434	0.9688
0.0022	53.0	53000	0.2486	0.9645
0.002	54.0	54000	0.2422	0.9688
0.0009	55.0	55000	0.2434	0.9685
0.0002	56.0	56000	0.2408	0.9683
0.0019	57.0	57000	0.2510	0.965
0.0024	58.0	58000	0.2695	0.964
0.0001	59.0	59000	0.2514	0.9663
0.0017	60.0	60000	0.2619	0.9637
0.0013	61.0	61000	0.2756	0.9643
0.0013	62.0	62000	0.2531	0.9665
0.0003	63.0	63000	0.2546	0.9675
0.0014	64.0	64000	0.2571	0.9667
0.0009	65.0	65000	0.2920	0.9623
0.0012	66.0	66000	0.2739	0.9647
0.0005	67.0	67000	0.2551	0.967
0.0013	68.0	68000	0.2640	0.9653
0.0009	69.0	69000	0.2640	0.9647
0.0009	70.0	70000	0.2802	0.965
0.0001	71.0	71000	0.2696	0.9643
0.0018	72.0	72000	0.2783	0.9643
0.0012	73.0	73000	0.2718	0.9665
0.0003	74.0	74000	0.2534	0.9698
0.0018	75.0	75000	0.2565	0.9657
0.0021	76.0	76000	0.2813	0.9637
0.0011	77.0	77000	0.2779	0.9633
0.0009	78.0	78000	0.2627	0.9667
0.0005	79.0	79000	0.2655	0.9685
0.001	80.0	80000	0.2718	0.9657
0.0018	81.0	81000	0.2643	0.968
0.0018	82.0	82000	0.2727	0.9655
0.001	83.0	83000	0.2747	0.9655
0.002	84.0	84000	0.2844	0.9637
0.0	85.0	85000	0.2634	0.9675
0.0016	86.0	86000	0.2762	0.9663
0.0008	87.0	87000	0.2599	0.9683
0.0	88.0	88000	0.2575	0.9683
0.0011	89.0	89000	0.2636	0.9677
0.0021	90.0	90000	0.2725	0.9665
0.0009	91.0	91000	0.2672	0.967
0.0	92.0	92000	0.2689	0.9673
0.0014	93.0	93000	0.2779	0.9653
0.0008	94.0	94000	0.2776	0.9653
0.0005	95.0	95000	0.2765	0.9657
0.0011	96.0	96000	0.2721	0.967
0.0013	97.0	97000	0.2716	0.9667
0.0006	98.0	98000	0.2709	0.9673
0.0002	99.0	99000	0.2702	0.9675
0.0006	100.0	100000	0.2701	0.9677

Framework versions

• Transformers 4.42.4
• Pytorch 2.4.1+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1