---
license: apache-2.0
language:
- vi
metrics:
- exact_match
- f1
base_model:
- google-bert/bert-base-multilingual-cased
pipeline_tag: question-answering
library_name: transformers
new_version: google-bert/bert-base-multilingual-cased
tags:
- legal
---

# BERT-Law: Information Extraction Model for Legal Texts

## Model Description

**BERT-Law** is a fine-tuned version of **BERT (Bidirectional Encoder Representations from Transformers)**, focusing on information extraction from legal documents. The model is specifically trained on a custom dataset called **UTE_LAW**, which consists of approximately 30,000 pairs of legal questions and related documents. The main goal of this model is to extract relevant information from legal text while reducing the costs associated with using third-party APIs.

Additionally, the model supports **Retrieval-Augmented Generation (RAG)**, which enhances its ability to handle smaller context windows, thereby optimizing API costs for third-party integrations. RAG is especially useful in scenarios where processing large legal documents in a single request might be inefficient or expensive.

### Key Features
- **Base Model**: The model is built on top of `google-bert/bert-base-multilingual-cased`, which is a pre-trained multilingual BERT model.
- **Fine-tuning**: It has been fine-tuned with the **UTE_LAW** dataset, focusing on extracting relevant information from legal texts.
- **Model Type**: BERT-based model for **question-answering** tasks.
- **Task**: The model is optimized for information extraction tasks, specifically designed to handle legal documents.
- **RAG Support**: Enhanced ability to process smaller context windows, improving cost-efficiency when using third-party APIs.

### Model Specifications
| Specification             | Description                                  |
|---------------------------|----------------------------------------------|
| **Maximum Sequence Length**| 512 tokens                                   |
| **Language**               | Primarily focused on **Vietnamese** legal texts|
| **Task**                   | Question-answering, Information extraction   |
| **RAG Support**            | Yes                                          |
| **LLMS Generate**            | Yes                                          |

### References
- Zaib, Munazza and Tran, Dai Hoang and Sagar, Subhash and Mahmood, Adnan and Zhang, Wei E. and Sheng, Quan Z. (2021). BERT-CoQAC: BERT-based Conversational Question Answering in Context. In *Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*.
  [Link](https://arxiv.org/abs/2104.11394)
- Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. [Link](https://arxiv.org/abs/1810.04805)

## Usage

This model is suitable for applications in legal domains, such as:
- **Legal document analysis**: Extracting relevant information from legal texts.
- **Question answering**: Providing answers to legal questions based on the content of legal documents.

The model aims to reduce reliance on third-party APIs, which can incur higher costs, by providing a locally deployable solution for legal document processing. With the integration of **RAG**, it further optimizes the extraction process by handling smaller context windows, improving efficiency and reducing costs when dealing with large or complex legal documents.