Fine-Tuning a Minecraft Assistant on a Free GPU

Two assistants building together in Minecraft, both running a model I fine-tuned on a free GPU.

The point was never Minecraft. It was proving that a domain-specific fine-tune runs end to end on hardware anyone can reach: the free Colab tier with a T4.

What the run does

Qwen 7B loads with 4-bit quantization, which is what makes it fit at all. LoRA adapters go on top so only a small set of layers train instead of the whole model. The dataset is Andy-3.5, formatted with ChatML templates. Training runs on hyperparameters chosen to survive a free-tier session rather than to win a benchmark. Then the model gets evaluated on crafting and game mechanics questions, and saved locally or pushed to the Hugging Face Hub.

Seven steps, no cluster, no budget.

What it taught me

The interesting constraint is memory, not compute. Quantization and LoRA are both answers to the same question: how do you touch a 7B model when you cannot hold a 7B model. Once that clicked, the rest of the pipeline stopped feeling like magic and started feeling like plumbing.

The obvious improvements are all data problems, not training problems. Widen the dataset and the assistant knows more of the game. Extend the steps and the answers get less generic. Adjust the inference parameters and the tone changes. None of that requires a bigger machine.

The full walkthrough

Every cell below is the notebook, in order. The assistant is called Andy. It plays through Mindcraft, which handles the game side.

You need a Colab account (the free T4 is enough, an A100 or V100 is faster), working Python, and a Hugging Face account if you want to share the result.

Setup

Unsloth and its dependencies handle the memory-efficient training.

%%capture
# Installs Unsloth, Xformers (Flash Attention) and all other packages
!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
!pip install --no-deps xformers trl peft accelerate bitsandbytes

Load the base model

Qwen 7B, loaded in 4 bits. That single flag is the difference between fitting on a T4 and not.

from unsloth import FastLanguageModel
import torch

max_seq_length = 2048   # RoPE scaling is handled internally, pick what you need
dtype = None            # None for auto detection. float16 for T4 and V100, bfloat16 for Ampere+
load_in_4bit = True     # 4-bit quantization to reduce memory use

# 4-bit pre-quantized models: 4x faster download, no OOMs
fourbit_models = [
    "unsloth/mistral-7b-bnb-4bit",
    "unsloth/mistral-7b-instruct-v0.2-bnb-4bit",
    "unsloth/llama-2-7b-bnb-4bit",
    "unsloth/llama-2-13b-bnb-4bit",
    "unsloth/codellama-34b-bnb-4bit",
    "unsloth/tinyllama-bnb-4bit",
    "unsloth/gemma-7b-bnb-4bit",
    "unsloth/gemma-2b-bnb-4bit",
    "unsloth/Qwen2.5-7B-bnb-4bit",
]

dtype = torch.bfloat16

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "unsloth/Qwen2.5-7B-bnb-4bit",
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
    # token = "your-token", # for gated models like meta-llama/Llama-2-7b-hf
    trust_remote_code = True,
)

Add the LoRA adapters

Low-Rank Adaptation trains a small set of layers and leaves the rest frozen. That is what keeps the compute inside a free session.

model = FastLanguageModel.get_peft_model(
    model,
    r = 16, # any number > 0. 8, 16, 32, 64, 128 are the usual picks
    target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
                      "gate_proj", "up_proj", "down_proj", "embed_tokens", "lm_head"],
    lora_alpha = 16,
    lora_dropout = 0, # any value works, 0 is optimized
    bias = "none",    # any value works, "none" is optimized
    use_gradient_checkpointing = "unsloth",  # True or "unsloth" for very long context
    random_state = 3407,
    use_rslora = False,     # rank-stabilized LoRA is supported
    loftq_config = None,    # and LoftQ
)

A full tune can be done through LoRA, but it earns its keep on small adjustments like this one.

Prepare the dataset

Andy-3.5 was curated by Sweaterdog for Minecraft: crafting, exploration, survival. It arrives in ShareGPT style, so the ChatML template gets a mapping.

from unsloth.chat_templates import get_chat_template

tokenizer = get_chat_template(
    tokenizer,
    chat_template = "chatml", # zephyr, chatml, mistral, llama, alpaca, vicuna, vicuna_old, unsloth
    mapping = {"role" : "from", "content" : "value", "user" : "human", "assistant" : "gpt"}, # ShareGPT style
    map_eos_token = True, # maps <|im_end|> to </s>
)

def formatting_prompts_func(examples):
    convos = examples["conversations"]
    texts = [tokenizer.apply_chat_template(convo, tokenize = False, add_generation_prompt = False) for convo in convos]
    return { "text" : texts, }

from datasets import load_dataset
dataset = load_dataset("Sweaterdog/Andy-3.5", split = "train")
dataset = dataset.map(formatting_prompts_func, batched = True,)

Normally you have to train the <|im_start|> and <|im_end|> tokens. Mapping <|im_end|> to the EOS token sidesteps that, and <|im_start|> stays as it is with no extra training.

# ShareGPT style
{"from": "human", "value" : "Hi"}

Print one example to see the shape before and after the template.

dataset[5]["conversations"]
print(dataset[5]["text"])

If you want your own template instead of ChatML, this is the whole thing. It is left switched off here.

unsloth_template = \
    "{{ bos_token }}" \
    "{{ 'You are a helpful assistant to the user\n' }}" \
    "{% endif %}" \
    "{% for message in messages %}" \
        "{% if message['role'] == 'user' %}" \
            "{{ '>>> User: ' + message['content'] + '\n' }}" \
        "{% elif message['role'] == 'assistant' %}" \
            "{{ '>>> Assistant: ' + message['content'] + eos_token + '\n' }}" \
        "{% endif %}" \
    "{% endfor %}" \
    "{% if add_generation_prompt %}" \
        "{{ '>>> Assistant: ' }}" \
    "{% endif %}"

unsloth_eos_token = "eos_token"

if False:
    tokenizer = get_chat_template(
        tokenizer,
        chat_template = (unsloth_template, unsloth_eos_token,),
        mapping = {"role" : "from", "content" : "value", "user" : "human", "assistant" : "gpt"},
        map_eos_token = True,
    )

Configure training

These numbers are tuned to finish inside a Colab session, not to win anything. For a full run, set max_steps to 0 and let num_train_epochs = 1 carry it.

from trl import SFTTrainer
from transformers import TrainingArguments
from unsloth import is_bfloat16_supported

trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    train_dataset = dataset,
    dataset_text_field = "text",
    max_seq_length = max_seq_length,
    dataset_num_proc = 2,   # set to 1 to disable multiprocessing while debugging
    packing = False,        # packs short sequences into batches; can speed up short-sequence training

    args = TrainingArguments(
        per_device_train_batch_size = 16,  # drop to 1 if you run out of memory
        gradient_accumulation_steps = 1,
        warmup_steps = 500,
        max_steps = 1000,
        learning_rate = 2e-5,  # lower than the usual default to avoid overshooting

        fp16 = not is_bfloat16_supported(),
        bf16 = is_bfloat16_supported(),

        logging_steps = 1,
        optim = "adamw_8bit",  # 8-bit AdamW to save memory
        weight_decay = 0.01,
        lr_scheduler_type = "linear",
        seed = 3407,
        output_dir = "outputs",
        gradient_checkpointing = True,
        num_train_epochs = 1,
    ),
)

Weights and Biases will ask for a key when training starts if you want the curves.

Free unused memory

Call this before the training loop, and read the numbers so you know what you started with.

import torch
torch.cuda.empty_cache()

gpu_stats = torch.cuda.get_device_properties(0)
start_gpu_memory = round(torch.cuda.max_memory_reserved() / 1024 / 1024 / 1024, 3)
max_memory = round(gpu_stats.total_memory / 1024 / 1024 / 1024, 3)
print(f"GPU = {gpu_stats.name}. Max memory = {max_memory} GB.")
print(f"{start_gpu_memory} GB of memory reserved.")

Train

Minutes to hours, depending on the GPU and the steps.

trainer_stats = trainer.train()

Then the tally.

used_memory = round(torch.cuda.max_memory_reserved() / 1024 / 1024 / 1024, 3)
used_memory_for_lora = round(used_memory - start_gpu_memory, 3)
used_percentage = round(used_memory         /max_memory*100, 3)
lora_percentage = round(used_memory_for_lora/max_memory*100, 3)
print(f"{trainer_stats.metrics['train_runtime']} seconds used for training.")
print(f"{round(trainer_stats.metrics['train_runtime']/60, 2)} minutes used for training.")
print(f"Peak reserved memory = {used_memory} GB.")
print(f"Peak reserved memory for training = {used_memory_for_lora} GB.")
print(f"Peak reserved memory % of max memory = {used_percentage} %.")
print(f"Peak reserved memory for training % of max memory = {lora_percentage} %.")

Run it

ChatML again, this time with the generation prompt on.

from unsloth.chat_templates import get_chat_template

tokenizer = get_chat_template(
    tokenizer,
    chat_template = "chatml",
    mapping = {"role" : "from", "content" : "value", "user" : "human", "assistant" : "gpt"},
    map_eos_token = True,
)

FastLanguageModel.for_inference(model) # native 2x faster inference

messages = [
    {"from": "human", "value": "Build a large house with windows, doors, rooms, and beds. Place torches and other decorative materials around the house."},
]
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize = True,
    add_generation_prompt = True, # required for generation
    return_tensors = "pt",
).to("cuda")

outputs = model.generate(input_ids = inputs, max_new_tokens = 64, use_cache = True)
tokenizer.batch_decode(outputs)

A TextStreamer shows the tokens as they land.

FastLanguageModel.for_inference(model)

messages = [
    {"from": "human", "value": "Complete the achievments in order?"},
]
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize = True,
    add_generation_prompt = True,
    return_tensors = "pt",
).to("cuda")

from transformers import TextStreamer
text_streamer = TextStreamer(tokenizer)
_ = model.generate(input_ids = inputs, streamer = text_streamer, max_new_tokens = 128, use_cache = True)

Save the adapters

This saves the LoRA adapters only, not the full model. The merged and GGUF exports come next.

# Local saving
# model.save_pretrained("lora_model")
# tokenizer.save_pretrained("lora_model")

# Online saving
model.push_to_hub("your-hf-username/model-name", token = "your-token")
tokenizer.push_to_hub("your-hf-username/model-name", token = "your-token")

To load the adapters back for inference, flip the False to True. max_seq_length can go up for longer sequences; the model loses stability somewhere around 8192 at the low end, 16,000 is the safer number.

if False:
    from unsloth import FastLanguageModel
    model, tokenizer = FastLanguageModel.from_pretrained(
        model_name = "unsloth/Qwen2.5-7B-bnb-4bit", # the model you trained on
        max_seq_length = max_seq_length,
        dtype = dtype,
        load_in_4bit = load_in_4bit,
    )
    FastLanguageModel.for_inference(model)

messages = [
    {"from": "human", "value": "Find me 20 blocks of gold."},
]
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize = True,
    add_generation_prompt = True,
    return_tensors = "pt",
).to("cuda")

from transformers import TextStreamer
text_streamer = TextStreamer(tokenizer)
_ = model.generate(input_ids = inputs, streamer = text_streamer, max_new_tokens = 128, use_cache = True)

Merge to float16 for vLLM

Pick merged_16bit for float16, merged_4bit for int4, or keep the adapters. Tokens live at huggingface.co/settings/tokens. Replace the username, model name, and token before running.

# Save and merge to 16bit
if False: model.save_pretrained_merged("name-of-model", tokenizer, save_method = "merged_16bit",)
if False: model.push_to_hub_merged("your-hf-username/name-of-model", tokenizer, save_method = "merged_16bit", token = "your-hf-token")

# Save and merge to 4bit
if False: model.save_pretrained_merged("name-of-model", tokenizer, save_method = "merged_4bit",)
if False: model.push_to_hub_merged("your-hf-username/name-of-model", tokenizer, save_method = "merged_4bit", token = "your-hf-token")

# Save LoRA adapters
if False: model.save_pretrained_merged("name-of-model", tokenizer, save_method = "lora",)
if False: model.push_to_hub_merged("your-hf-username/name-of-model", tokenizer, save_method = "lora", token = "your-hf-token")

GGUF for llama.cpp

save_pretrained_gguf for a local file, push_to_hub_gguf for the Hub. Three quantizations worth knowing:

• q8_0: fast conversion, high resource use, generally fine.
• q4_k_m: recommended. Q6_K for half of the attention.wv and feed_forward.w2 tensors, Q4_K for the rest.
• q5_k_m: same split with Q5_K for the rest.

# Save to 8bit Q8_0
if False: model.save_pretrained_gguf("model-name", tokenizer,)
if True: model.push_to_hub_gguf("your-hf-username/model-name", tokenizer, token = "your-hf-token")

# Save to 16bit GGUF
if False: model.save_pretrained_gguf("model-name", tokenizer, quantization_method = "f16")
if True: model.push_to_hub_gguf("your-hf-username/model-name", tokenizer, quantization_method = "f16", token = "your-hf-token")

# Save to q4_k_m GGUF
if False: model.save_pretrained_gguf("model-name", tokenizer, quantization_method = "q4_k_m")
if True: model.push_to_hub_gguf("your-hf-username/model-name", tokenizer, quantization_method = "q4_k_m", token = "your-hf-token")

Drop the resulting model-name.gguf or model-name-Q4_K_M.gguf into llama.cpp or LM Studio and it runs.

What to change next

Data. Advanced crafting recipes, redstone tutorials, combat, survival. Every one of these makes Andy know more of the game.

Training. Start at 500 to 1000 steps. Lower the learning rate as the steps go up. Try batch sizes and schedulers against the hardware you have.

Inference. Temperature trades creativity for consistency. Repetition penalty keeps the answers natural. max_new_tokens sets how long it talks.

None of it is specific to Minecraft. Swap the dataset and the same notebook trains an assistant for whatever domain you have data for.

Resources

• Andy-3.5 dataset
• Mindcraft
• LoRA paper
• Unsloth documentation
• The original write-up on my old blog

Citation

@misc{celaya2025minecraft,
  author = {Christopher B. Celaya},
  title = {Efficient Fine-Tuning of Large Language Models - A Minecraft AI Assistant Tutorial},
  year = {2025},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/kolbytn/mindcraft}},
  note = {\url{https://chris-celaya-blog.vercel.app/articles/unsloth-training}}
}