xavierwoon/cesterllama

Model Card for Model ID

Cesterllama is a fine-tuned Llama 3 8B model that is able to generate Libcester unit test cases in the correct format.

Model Details

Model Description

• Developed by: Xavier Woon

  Bias, Risks, and Limitations

  The model often regenerates the input prompt in the output. This can lead to limited test cases being printed due to truncations based on max_new_tokens.

  Recommendations

  Expanding the dataset will help increase the accuracy and robustness of the model, and improve code coverage based on real life scenarios.

  How to Get Started with the Model

  Use the code below to get started with the model.

  from transformers import AutoModelForCausalLM, AutoTokenizer
  
  model_name = "xavierwoon/cesterllama"
  model = AutoModelForCausalLM.from_pretrained(model_name)
  tokenizer = AutoTokenizer.from_pretrained(model_name)
  
  # Paste your own code inside
  code = """
  void add()
  {
     int a,b,c;
     printf("\nEnter The Two values:");
     scanf("%d%d",&a,&b);
     c=a+b;
     printf("Addition:%d",c);
  }
  """
  
  prompt = f"""### Instruction:
  create cester test cases for this function:
  {code}
  
  ### Input:
  
  ### Response:
  """
  
  inputs = tokenizer(prompt, return_tensors="pt").to("cpu")
  
  from transformers import TextStreamer
  text_streamer = TextStreamer(tokenizer)
  _ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 2048)
  

  Training Details

  Training Data

  Training Data was created based on Data Structures and Algorithm (DSA) codes created using ChatGPT. It would also create corresponding Cester test cases. After testing and ensuring a good code coverage, the prompt and corresponding test cases were added to the dataset.

  Training Procedure

  1. Prompt GPT for sample DSA C code
  2. Prompt GPT for Libcester unit test cases with 100% code coverage
  3. Test generated test cases for robustness and code coverage