metadata

license: mit
datasets:
  - teknium/openhermes

Conversational Language Model Interface using FASTTEXT

This project provides a Command Line Interface (CLI) for interacting with a FastText language model, enabling users to generate text sequences based on their input. The script allows customization of parameters such as temperature, input text, top-k predictions, and model file path.

Installation

Before running the script, ensure you have Python installed on your system. Additionally, you'll need to install the FastText library:

Colab

Google Colab Notebook

pip install fasttext

Usage

To use the script, you should first obtain or train a FastText model. Place the model file (usually with a .bin extension) in a known directory.

The script can be executed with various command-line arguments to specify the behavior:

import argparse
import fasttext
import numpy as np

def apply_repetition_penalty(labels, probabilities, used_labels, penalty_scale=1.9):
    """
    Applies a repetition penalty to reduce the probability of already used labels.

    :param labels: List of possible labels.
    :param probabilities: Corresponding list of probabilities.
    :param used_labels: Set of labels that have already been used.
    :param penalty_scale: Scale of the penalty to be applied.
    :return: Adjusted probabilities.
    """
    adjusted_probabilities = probabilities.copy()
    for i, label in enumerate(labels):
        if label in used_labels:
            adjusted_probabilities[i] /= penalty_scale
    # Normalize the probabilities to sum to 1 again
    adjusted_probabilities /= adjusted_probabilities.sum()
    return adjusted_probabilities

def predict_sequence(model, text, sequence_length=20, temperature=.5, penalty_scale=1.9):
    """
    Generates a sequence of labels using the FastText model with repetition penalty.

    :param model: Loaded FastText model.
    :param text: Initial text to start the prediction from.
    :param sequence_length: Desired length of the sequence.
    :param temperature: Temperature for sampling.
    :param penalty_scale: Scale of repetition penalty.
    :return: List of predicted labels.
    """
    used_labels = set()
    sequence = []

    for _ in range(sequence_length):
        # Predict the top k most probable labels
        labels, probabilities = model.predict(text, k=40)
        labels = [label.replace('__label__', '') for label in labels]
        probabilities = np.array(probabilities)

        # Adjust the probabilities with repetition penalty
        probabilities = apply_repetition_penalty(labels, probabilities, used_labels, penalty_scale)

        # Sampling according to the adjusted probabilities
        label_index = np.random.choice(range(len(labels)), p=probabilities)
        chosen_label = labels[label_index]

        # Add the chosen label to the sequence and to the set of used labels
        sequence.append(chosen_label)
        used_labels.add(chosen_label)

        # Update the text with the chosen label for the next prediction
        text += ' ' + chosen_label

    return sequence

def generate_response(model, input_text, sequence_length=512, temperature=.5, penalty_scale=1.9):
    generated_sequence = predict_sequence(model, input_text, sequence_length, temperature, penalty_scale)
    return ' '.join(generated_sequence)

def main():
    parser = argparse.ArgumentParser(description="Run the language model with specified parameters.")
    parser.add_argument('-t', '--temperature', type=float, default=0.5, help='Temperature for sampling.')
    parser.add_argument('-f', '--file', type=str, help='File containing input text.')
    parser.add_argument('-p', '--text', type=str, help='Direct input text.')
    parser.add_argument('-n', '--length', type=int, default=50, help='length predictions to consider.')
    parser.add_argument('-m', '--model', type=str, required=True, help='Address of the FastText model file.')

    args = parser.parse_args()

    # Load the model
    model = fasttext.load_model(args.model)

    input_text = ''
    if args.file:
        with open(args.file, 'r') as file:
            input_text = file.read()
    elif args.text:
        input_text = args.text
    else:
        print("No input text provided. Please use -f to specify a file or -p for direct text input.")
        return

    # Generate and print the response
    response = generate_response(model, input_text + " [RESPONSE]", sequence_length=args.length, temperature=args.temperature)
    print("\nResponse:")
    print(response)

if __name__ == "__main__":
    main()

python conversation_app.py -t TEMPERATURE -f FILE -p TEXT -k TOPK -m MODEL_PATH

-t TEMPERATURE or --temperature TEMPERATURE: Sets the temperature for predictions. A higher temperature results in more diverse results. Default is 0.5.
-f FILE or --file FILE: Specifies a path to a file containing input text. The script will read this file and use its contents as input.
-p TEXT or --text TEXT: Directly provide the input text as a string.
-n LENGTH or --length TOPK: Determines the number of top predictions to consider for the model's output. Default is 50.
-m MODEL_PATH or --model MODEL_PATH: The path to the FastText model file (required).

Example

python conversation_app.py -t 0.7 -p "What is the future of AI?" -n 40 -m /path/to/model.bin

This command sets the temperature to 0.7, uses the provided question as input, considers the top 40 predictions, and specifies the model file path.

Note

The script's output depends on the quality and training of the FastText model used.
Ensure the specified model file path and input file path (if used) are correct.