Fine-tuning with SQL coder
Hi @rishdotblog , since I have 370 tables in my database and as I know I won't be able to pass the entire 370 schema and question as prompt to the model to get the sql query.
So, I thought I can do domain based fine-tuning with SQLCoder model with my all my 370 schema and later I can convert it that as instruction finetuning. Because I assume after I fine tune the model with my 370 schema , the will have information about my database schema, so I can ask questions directly to the model and get SQL query as an output.
is it doable? or is my understanding is correct?
@Iamexperimenting any progress with the above?
Yup, that's correct @Iamexperimenting .
@rishdotblog @wongjingping but I have a little confusion, I assume SQLCoder base model would have been trained with other internet database schema. In that case, model would hallucinate right?
Also, have you tried the approach which mentioned above? from domain based finetuning to chat based fine tuning.
@rishdotblog could you please share your thoughts?
@Iamexperimenting we always train with the schema of interest inside the prompt, as it helps the model to learn that it needs to actually see the required column names and not hallucinate them. We have yet to try chat-based finetuning, please let us know how your results turn out if you do!