Model Card
This is my first fine tuned LLM project.
Usage
from transformers import GPT2LMHeadModel, GPT2Tokenizer
finetunedGPT = GPT2LMHeadModel.from_pretrained("rakeshkiriyath/gpt2Medium_text_to_sql")
finetunedTokenizer = GPT2Tokenizer.from_pretrained("rakeshkiriyath/gpt2Medium_text_to_sql")
def generate_text_to_sql(query, model, tokenizer, max_length=256):
prompt = f"Translate the following English question to SQL: {query}"
input_tensor = tokenizer.encode(prompt, return_tensors='pt').to('cuda')
output = model.generate(input_tensor, max_length=max_length, num_return_sequences=1, pad_token_id=tokenizer.eos_token_id)
decoded_output = tokenizer.decode(output[0], skip_special_tokens=True)
# Return only the SQL part (removing the input text)
sql_output = decoded_output[len(prompt):].strip()
return sql_output
queryList = ["I need a list of employees who joined in the company last 6 months with a salary hike of 30% ",
"Give me loginid,status,company of a user who is mapped to the organization XYZ "]
for query in queryList:
sql_result = generate_text_to_sql(query, finetunedGPT, finetunedTokenizer)
print(sql_result,"\n")
Output
SELECT COUNT(*) FROM employees WHERE last_6_months = "6 months" AND salary_hike = "30%"
SELECT loginid,status,company FROM user_mapped_to_organization WHERE mapping = "XYZ"
Training Hyperparameters
num_train_epochs=1
per_device_train_batch_size=3
gradient_accumulation_steps=9
learning_rate=5e-5
weight_decay=0.01
Evaluation
Step | Training Loss |
---|---|
500 | 0.337800 |
1000 | 0.262900 |
1500 | 0.253200 |
2000 | 0.246400 |
{'eval_loss': 0.23689331114292145, 'eval_runtime': 104.4102, 'eval_samples_per_second': 67.043, 'eval_steps_per_second': 8.38, 'epoch': 1.0}
- Downloads last month
- 1,112
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.