Skip to main content
This guide explains how to fine-tune a large language model using Runpod and Axolotl. You’ll learn how to select a base model, configure your training environment, and start the fine-tuning process.

​
Prerequisites

Before you begin fine-tuning, ensure you have:
  • A Runpod account with access to the Fine Tuning feature
  • (Optional) A Hugging Face access token for gated models

​
Select a base model

To start fine-tuning, you’ll need to choose a base model from Hugging Face:
  1. Navigate to the Fine Tuning section in the sidebar
  2. Enter the Hugging Face model ID in the Base Model field
    • Example: NousResearch/Meta-Llama-3-8B
  3. For gated models (requiring special access):
    1. Generate a Hugging Face token with appropriate permissions
    2. Add your token in the designated field

​
Select a dataset

You can choose a dataset from Hugging Face for fine-tuning:
  1. Browse available datasets on Hugging Face
  2. Enter your chosen dataset identifier in the Dataset field
    • Example: tatsu-lab/alpaca

​
Deploy the fine-tuning pod

Follow these steps to set up your training environment:
  1. Click Deploy the Fine Tuning Pod
  2. Select a GPU instance based on your model’s requirements:
    • Smaller models: Choose GPUs with less memory
    • Larger models/datasets: Choose GPUs with higher memory capacity
  3. Monitor the system logs for deployment progress
  4. Wait for the success message: "You've successfully configured your training environment!"

​
Connect to your training environment

After your pod is deployed and active, you can connect using any of these methods:
  1. Go to your Fine Tuning pod dashboard
  2. Click Connect and choose your preferred connection method:
    • Jupyter Notebook: Browser-based notebook interface
    • Web Terminal: Browser-based terminal
    • SSH: Local machine terminal connection
To use SSH, add your public SSH key in your account settings. The system automatically adds your key to the pod’s authorized_keys file.

​
Configure your environment

Your training environment includes this directory structure in /workspace/fine-tuning/:
  • examples/: Sample configurations and scripts
  • outputs/: Training results and model outputs
  • config.yaml: Training parameters for your model
The system generates an initial config.yaml based on your selected base model and dataset.

​
Review and modify the configuration

The config.yaml file controls your fine-tuning parameters. Here’s how to customize it:
  1. Open the configuration file:
  2. Review and adjust the parameters based on your specific use case
Here’s an example configuration with common parameters:
The config.yaml file contains all hyperparameters needed for fine-tuning. You may need to iterate on these settings to achieve optimal results.
For more configuration examples, visit the Axolotl examples repository.

​
Start the fine-tuning process

Once your configuration is ready, follow these steps:
  1. Start the training process:
  2. Monitor the training progress in your terminal

​
Push your model to Hugging Face

After completing the fine-tuning process, you can share your model:
  1. Log in to Hugging Face:
  2. Create a new repository on Hugging Face if needed
  3. Upload your model:
Replace <your-username> with your Hugging Face username and <model-name> with your desired model name.

​
Additional resources

For more information about fine-tuning with Axolotl, see: