layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.7008
• Answer: {'precision': 0.7050592034445641, 'recall': 0.8096415327564895, 'f1': 0.7537399309551209, 'number': 809}
• Header: {'precision': 0.2803030303030303, 'recall': 0.31092436974789917, 'f1': 0.29482071713147406, 'number': 119}
• Question: {'precision': 0.7809187279151943, 'recall': 0.8300469483568075, 'f1': 0.8047337278106509, 'number': 1065}
• Overall Precision: 0.7187
• Overall Recall: 0.7908
• Overall F1: 0.7530
• Overall Accuracy: 0.8087

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.787	1.0	10	1.5982	{'precision': 0.02607561929595828, 'recall': 0.024721878862793572, 'f1': 0.025380710659898473, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.2468354430379747, 'recall': 0.21971830985915494, 'f1': 0.23248882265275708, 'number': 1065}	0.1481	0.1274	0.1370	0.3555
1.4393	2.0	20	1.2504	{'precision': 0.10978520286396182, 'recall': 0.11372064276885044, 'f1': 0.1117182756527019, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.4381169324221716, 'recall': 0.5417840375586854, 'f1': 0.4844668345927792, 'number': 1065}	0.3104	0.3357	0.3226	0.5539
1.0904	3.0	30	0.9333	{'precision': 0.5273109243697479, 'recall': 0.6205191594561187, 'f1': 0.5701306076093129, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5661929693343306, 'recall': 0.7107981220657277, 'f1': 0.630308076602831, 'number': 1065}	0.5436	0.6317	0.5844	0.7201
0.8353	4.0	40	0.7609	{'precision': 0.6157786885245902, 'recall': 0.7428924598269468, 'f1': 0.673389355742297, 'number': 809}	{'precision': 0.10344827586206896, 'recall': 0.05042016806722689, 'f1': 0.06779661016949153, 'number': 119}	{'precision': 0.651414309484193, 'recall': 0.7352112676056338, 'f1': 0.6907807675341862, 'number': 1065}	0.6216	0.6974	0.6574	0.7679
0.6619	5.0	50	0.7136	{'precision': 0.6655879180151025, 'recall': 0.7626699629171817, 'f1': 0.7108294930875575, 'number': 809}	{'precision': 0.275, 'recall': 0.18487394957983194, 'f1': 0.22110552763819097, 'number': 119}	{'precision': 0.685214626391097, 'recall': 0.8093896713615023, 'f1': 0.7421437795953508, 'number': 1065}	0.6627	0.7531	0.7050	0.7866
0.5642	6.0	60	0.6861	{'precision': 0.6413373860182371, 'recall': 0.7824474660074165, 'f1': 0.7048997772828508, 'number': 809}	{'precision': 0.3382352941176471, 'recall': 0.19327731092436976, 'f1': 0.24598930481283424, 'number': 119}	{'precision': 0.7156357388316151, 'recall': 0.7821596244131456, 'f1': 0.7474203678779722, 'number': 1065}	0.6710	0.7471	0.7070	0.7846
0.4894	7.0	70	0.6645	{'precision': 0.6925601750547046, 'recall': 0.7824474660074165, 'f1': 0.73476494486361, 'number': 809}	{'precision': 0.3106796116504854, 'recall': 0.2689075630252101, 'f1': 0.28828828828828823, 'number': 119}	{'precision': 0.7319762510602206, 'recall': 0.8103286384976526, 'f1': 0.7691622103386809, 'number': 1065}	0.6958	0.7667	0.7295	0.7993
0.4396	8.0	80	0.6633	{'precision': 0.68580375782881, 'recall': 0.8121137206427689, 'f1': 0.7436332767402377, 'number': 809}	{'precision': 0.25210084033613445, 'recall': 0.25210084033613445, 'f1': 0.25210084033613445, 'number': 119}	{'precision': 0.7321131447587355, 'recall': 0.8262910798122066, 'f1': 0.776356418173798, 'number': 1065}	0.6876	0.7863	0.7336	0.8033
0.381	9.0	90	0.6612	{'precision': 0.7039473684210527, 'recall': 0.7935723114956736, 'f1': 0.7460778617083091, 'number': 809}	{'precision': 0.2920353982300885, 'recall': 0.2773109243697479, 'f1': 0.28448275862068967, 'number': 119}	{'precision': 0.7660869565217391, 'recall': 0.8272300469483568, 'f1': 0.7954853273137698, 'number': 1065}	0.7154	0.7807	0.7466	0.8040
0.3737	10.0	100	0.6725	{'precision': 0.6994652406417112, 'recall': 0.8084054388133498, 'f1': 0.7499999999999999, 'number': 809}	{'precision': 0.2818181818181818, 'recall': 0.2605042016806723, 'f1': 0.27074235807860264, 'number': 119}	{'precision': 0.7605512489233419, 'recall': 0.8291079812206573, 'f1': 0.7933513027852651, 'number': 1065}	0.7108	0.7868	0.7468	0.8067
0.3174	11.0	110	0.6862	{'precision': 0.7039827771797632, 'recall': 0.8084054388133498, 'f1': 0.7525891829689298, 'number': 809}	{'precision': 0.2713178294573643, 'recall': 0.29411764705882354, 'f1': 0.28225806451612906, 'number': 119}	{'precision': 0.7706342311033884, 'recall': 0.8328638497652582, 'f1': 0.8005415162454873, 'number': 1065}	0.7134	0.7908	0.7501	0.8033
0.2976	12.0	120	0.6907	{'precision': 0.7048648648648649, 'recall': 0.8059332509270705, 'f1': 0.7520184544405998, 'number': 809}	{'precision': 0.2926829268292683, 'recall': 0.3025210084033613, 'f1': 0.2975206611570248, 'number': 119}	{'precision': 0.7772887323943662, 'recall': 0.8291079812206573, 'f1': 0.8023625624716039, 'number': 1065}	0.7193	0.7883	0.7522	0.8081
0.2799	13.0	130	0.6973	{'precision': 0.7105549510337323, 'recall': 0.8071693448702101, 'f1': 0.755787037037037, 'number': 809}	{'precision': 0.31451612903225806, 'recall': 0.3277310924369748, 'f1': 0.32098765432098764, 'number': 119}	{'precision': 0.7857777777777778, 'recall': 0.8300469483568075, 'f1': 0.8073059360730593, 'number': 1065}	0.7269	0.7908	0.7575	0.8066
0.2597	14.0	140	0.7004	{'precision': 0.7083786724700761, 'recall': 0.8046971569839307, 'f1': 0.7534722222222221, 'number': 809}	{'precision': 0.2803030303030303, 'recall': 0.31092436974789917, 'f1': 0.29482071713147406, 'number': 119}	{'precision': 0.781195079086116, 'recall': 0.8347417840375587, 'f1': 0.8070812528370404, 'number': 1065}	0.7204	0.7913	0.7542	0.8073
0.2627	15.0	150	0.7008	{'precision': 0.7050592034445641, 'recall': 0.8096415327564895, 'f1': 0.7537399309551209, 'number': 809}	{'precision': 0.2803030303030303, 'recall': 0.31092436974789917, 'f1': 0.29482071713147406, 'number': 119}	{'precision': 0.7809187279151943, 'recall': 0.8300469483568075, 'f1': 0.8047337278106509, 'number': 1065}	0.7187	0.7908	0.7530	0.8087

Framework versions

• Transformers 4.42.4
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1