问题
为段言的AI辅助编程模型进行微调训练duan:基于中文深层认知特性的编程语言项目 – AtomGit
将max-len的size 设为8k,结果爆显存了，后来试了试，就2048可以pass
第 3 步：配置 LoRA
============================================================
trainable params: 2,162,688 || all params: 496,195,456 || trainable%: 0.4359
============================================================
第 4 步：加载数据集
============================================================
训练数据: 978 条
max_len: 8192
类别分布:
复合: 135
列表: 131
字符串: 124
异常: 99
类: 82
字典: 73
段落: 54
函数: 51
循环: 48
变量: 48
条件: 48
暗坑: 46
文件: 30
导入: 9
============================================================
第 5 步：训练（GPU）
============================================================
GPU: Tesla T4
预计总步数: ~184
预计时间: ~291 秒 (4.9 分钟)
batch_size: 2 x grad_accum: 8 = 等效 batch 16
epochs: 3, lr: 0.0002, LoRA rank: 16
precision: bf16, QLoRA: False
warmup_ratio is deprecated and will be removed in v5.2. Use `warmup_steps` instead.
The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': None, 'pad_token_id': 151643}.
0%| | 0/93 [00:00<?, ?it/s]Traceback (most recent call last):
File "/kaggle/working/duan/tools/ai_copilot/train_gpu_lora.py", line 652, in <module>
main()
File "/kaggle/working/duan/tools/ai_copilot/train_gpu_lora.py", line 620, in main
final_dir = train(
^^^^^^
File "/kaggle/working/duan/tools/ai_copilot/train_gpu_lora.py", line 454, in train
trainer.train()
File "/usr/local/lib/python3.12/dist-packages/transformers/trainer.py", line 2174, in train
return inner_training_loop(
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/trainer.py", line 2536, in _inner_training_loop
tr_loss_step = self.training_step(model, inputs, num_items_in_batch)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/trainer.py", line 3809, in training_step
loss = self.compute_loss(model, inputs, num_items_in_batch=num_items_in_batch)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/trainer.py", line 3880, in compute_loss
outputs = model(**inputs)
^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1787, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/nn/parallel/data_parallel.py", line 197, in forward
outputs = self.parallel_apply(replicas, inputs, module_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/nn/parallel/data_parallel.py", line 214, in parallel_apply
return parallel_apply(
^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/nn/parallel/parallel_apply.py", line 133, in parallel_apply
output.reraise()
File "/usr/local/lib/python3.12/dist-packages/torch/_utils.py", line 775, in reraise
raise exception
torch.OutOfMemoryError: Caught OutOfMemoryError in replica 0 on device 0.
Original Traceback (most recent call last):
File "/usr/local/lib/python3.12/dist-packages/torch/nn/parallel/parallel_apply.py", line 103, in _worker
output = module(*input, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1787, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/peft/peft_model.py", line 1993, in forward
return self.base_model(
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1787, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/peft/tuners/tuners_utils.py", line 330, in forward
return self.model.forward(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/utils/generic.py", line 835, in wrapper
output = func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/models/qwen2/modeling_qwen2.py", line 475, in forward
outputs: BaseModelOutputWithPast = self.model(
^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1787, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/utils/generic.py", line 1002, in wrapper
outputs = func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/models/qwen2/modeling_qwen2.py", line 410, in forward
hidden_states = decoder_layer(
^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/modeling_layers.py", line 92, in __call__
return self._gradient_checkpointing_func(partial(super().__call__, **kwargs), *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/_compile.py", line 54, in inner
return disable_fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/_dynamo/eval_frame.py", line 1181, in _fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py", line 512, in checkpoint
ret = function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1787, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/models/qwen2/modeling_qwen2.py", line 298, in forward
hidden_states, _ = self.self_attn(
^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1776, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1787, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/models/qwen2/modeling_qwen2.py", line 234, in forward
attn_output, attn_weights = attention_interface(
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/integrations/sdpa_attention.py", line 92, in sdpa_attention_forward
attn_output = torch.nn.functional.scaled_dot_product_attention(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 7.00 GiB. GPU 0 has a total capacity of 14.56 GiB of which 5.61 GiB is free. Including non-PyTorch memory, this process has 8.95 GiB memory in use. Of the allocated memory 8.69 GiB is allocated by PyTorch, and 72.39 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
建议：
PYTORCH_ALLOC_CONF=expandable
还是不行
设为2048，解决问题
[预设] qwen3.5-2b: Qwen3.5-2B (2B, 多模态架构, LoRA ~5GB / QLoRA ~3GB, 需 transformers>=5.0)
model_path=/kaggle/working/duan/tools/ai_copilot/model_cache/qwen3.5-2b
output_dir=/kaggle/working/duan/tools/ai_copilot/output/qwen3.5_2b_duan_gpu
max_len=1024, batch_size=1, grad_accum=4
lora_rank=32, lora_alpha=64, lr=0.0001
[T4 检测] Tesla T4 (15.6GB, SM 7.5)
[Unsloth] 自动处理精度和显存优化
当前配置: max_len=1024, batch_size=1, QLoRA=True
[WARN] Qwen3.5 系列官方不建议使用 QLoRA（量化差异可能高于正常水平）
但显存不足时仍可使用，如效果不佳请换用 bf16 LoRA 模式
继续使用 QLoRA 训练…
============================================================
第 1 步：环境检查（GPU 模式）
============================================================
[OK] PyTorch 2.10.0+cu128
[OK] 检测到 2 块 GPU:
GPU 0: Tesla T4 (15.6 GB)
GPU 1: Tesla T4 (15.6 GB)
[OK] Kaggle 双 T4 环境检测到，将启用 DDP 分布式训练
[INFO] 此 GPU 不支持原生 bf16（需 SM >= 8.0），将自动使用 fp16
[OK] transformers 5.14.1
Skipping import of cpp extensions due to incompatible torch version. Please upgrade to torch >= 2.11.0 (found 2.10.0+cu128).
[OK] peft 0.19.1
[OK] bitsandbytes 0.49.2
[OK] torchao 0.17.0
[OK] unsloth 已安装（可用 –unsloth 启用）
[OK] 数据集: /kaggle/working/duan/tools/ai_copilot/sft_dataset.jsonl (978 条)
============================================================
第 2 步：加载模型
============================================================
模型路径: /kaggle/working/duan/tools/ai_copilot/model_cache/qwen3.5-2b
QLoRA 4bit: 是
[多GPU] 检测到 2 块 GPU，将启用 DDP 分布式训练
[Kaggle] 双 T4 环境已确认
[INFO] GPU 不支持 bf16（Turing 架构如 T4），使用 fp16 替代
[INFO] 使用 fp16 精度（GPU 不支持 bf16）
/kaggle/working/duan/tools/ai_copilot/train_gpu_lora.py:589: UserWarning: WARNING: Unsloth should be imported before transformers, peft to ensure all optimizations are applied. Your code may run slower or encounter memory issues without these optimizations.
Please restructure your imports with 'import unsloth' at the top of your file.
from unsloth import FastLanguageModel
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
[WARN] Unsloth 导入失败: name 'auto_docstring' is not defined
[WARN] 自动降级到标准 HF+PEFT 模式
Unsloth 可能与当前 Python/transformers 版本不兼容
可去掉 –unsloth 使用标准模式训练
[FIX] 已屏蔽 unsloth_zoo monkey-patch，恢复标准 HF 路径
[transformers] The fast path is not available because one of the required library is not installed. Falling back to torch implementation. To install follow https://github.com/fla-org/flash-linear-attention#installation and https://github.com/Dao-AILab/causal-conv1d
Loading weights: 100%|███████████████████████| 320/320 [00:01<00:00, 242.46it/s]
[QLoRA] 模型已 4bit 量化加载, SDPA attention
[WARN] QLoRA + 多 GPU: bitsandbytes 量化模型仅在 GPU 0 上运行
多 GPU 并行不生效，建议去掉 –qlora 以使用双 T4 DDP
设备: cuda
模型参数量: 1195.5M
[OK] gradient checkpointing 已启用
============================================================
第 3 步：配置 LoRA
============================================================
trainable params: 2,949,120 || all params: 1,884,774,208 || trainable%: 0.1565
============================================================
第 4 步：加载数据集
============================================================
训练数据: 978 条
max_len: 1024
类别分布:
复合: 135
列表: 131
字符串: 124
异常: 99
类: 82
字典: 73
段落: 54
函数: 51
循环: 48
变量: 48
条件: 48
暗坑: 46
文件: 30
导入: 9
============================================================
第 5 步：训练（GPU）
============================================================
GPU: Tesla T4 (15.6 GB)
预计总步数: ~734
预计时间: ~703 秒 (11.7 分钟)
batch_size: 1 x grad_accum: 4 = 等效 batch 4
epochs: 3, lr: 0.0001, LoRA rank: 32
precision: fp16 权重 + fp32 LoRA (无 AMP), QLoRA: True
[OPTIM] adamw_8bit (8-bit AdamW, 省 ~75% 优化器显存)
[transformers] The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'eos_token_id': 248046, 'pad_token_id': 248044}.
{'loss': '2.203', 'grad_norm': '10.14', 'learning_rate': '1.111e-05', 'epoch': '0.0409'}
{'loss': '1.938', 'grad_norm': '11.06', 'learning_rate': '2.5e-05', 'epoch': '0.0818'}
{'loss': '1.842', 'grad_norm': '8.35', 'learning_rate': '3.889e-05', 'epoch': '0.1227'}
{'loss': '1.314', 'grad_norm': '4.42', 'learning_rate': '5.278e-05', 'epoch': '0.1636'}
{'loss': '0.9932', 'grad_norm': '4.952', 'learning_rate': '6.667e-05', 'epoch': '0.2045'}
{'loss': '0.9426', 'grad_norm': '4.323', 'learning_rate': '8.056e-05', 'epoch': '0.2454'}
{'loss': '0.8306', 'grad_norm': '4.247', 'learning_rate': '9.444e-05', 'epoch': '0.2863'}
{'loss': '0.6309', 'grad_norm': '4.116', 'learning_rate': '9.998e-05', 'epoch': '0.3272'}
{'loss': '0.4392', 'grad_norm': '4.536', 'learning_rate': '9.986e-05', 'epoch': '0.3681'}
{'loss': '0.5706', 'grad_norm': '5.684', 'learning_rate': '9.962e-05', 'epoch': '0.409'}
{'loss': '0.4475', 'grad_norm': '3.037', 'learning_rate': '9.928e-05', 'epoch': '0.4499'}
{'loss': '0.335', 'grad_norm': '3.829', 'learning_rate': '9.883e-05', 'epoch': '0.4908'}
{'loss': '0.336', 'grad_norm': '4.08', 'learning_rate': '9.827e-05', 'epoch': '0.5317'}
{'loss': '0.3219', 'grad_norm': '3.651', 'learning_rate': '9.76e-05', 'epoch': '0.5726'}
{'loss': '0.3859', 'grad_norm': '5.835', 'learning_rate': '9.682e-05', 'epoch': '0.6135'}
{'loss': '0.2622', 'grad_norm': '3.23', 'learning_rate': '9.594e-05', 'epoch': '0.6544'}
{'loss': '0.453', 'grad_norm': '4.503', 'learning_rate': '9.496e-05', 'epoch': '0.6953'}
{'loss': '0.3871', 'grad_norm': '4.261', 'learning_rate': '9.388e-05', 'epoch': '0.7362'}
{'loss': '0.3932', 'grad_norm': '3.55', 'learning_rate': '9.27e-05', 'epoch': '0.7771'}
{'loss': '0.2394', 'grad_norm': '2.763', 'learning_rate': '9.143e-05', 'epoch': '0.818'}
{'loss': '0.3433', 'grad_norm': '2.859', 'learning_rate': '9.006e-05', 'epoch': '0.8589'}
{'loss': '0.3537', 'grad_norm': '3.831', 'learning_rate': '8.86e-05', 'epoch': '0.8998'}
{'loss': '0.2138', 'grad_norm': '3.705', 'learning_rate': '8.706e-05', 'epoch': '0.9407'}
{'loss': '0.2411', 'grad_norm': '4.92', 'learning_rate': '8.544e-05', 'epoch': '0.9816'}
{'loss': '0.1983', 'grad_norm': '3.793', 'learning_rate': '8.374e-05', 'epoch': '1.016'}
{'loss': '0.2174', 'grad_norm': '7.729', 'learning_rate': '8.196e-05', 'epoch': '1.057'}
{'loss': '0.1598', 'grad_norm': '3.7', 'learning_rate': '8.011e-05', 'epoch': '1.098'}
{'loss': '0.2643', 'grad_norm': '4.253', 'learning_rate': '7.819e-05', 'epoch': '1.139'}
{'loss': '0.2372', 'grad_norm': '3.132', 'learning_rate': '7.622e-05', 'epoch': '1.18'}
{'loss': '0.2047', 'grad_norm': '1.599', 'learning_rate': '7.418e-05', 'epoch': '1.221'}
{'loss': '0.1716', 'grad_norm': '2.777', 'learning_rate': '7.209e-05', 'epoch': '1.262'}
{'loss': '0.162', 'grad_norm': '5.104', 'learning_rate': '6.995e-05', 'epoch': '1.303'}
{'loss': '0.1407', 'grad_norm': '2.402', 'learning_rate': '6.776e-05', 'epoch': '1.344'}
{'loss': '0.1522', 'grad_norm': '3.454', 'learning_rate': '6.554e-05', 'epoch': '1.384'}
{'loss': '0.1693', 'grad_norm': '3.723', 'learning_rate': '6.328e-05', 'epoch': '1.425'}
{'loss': '0.1325', 'grad_norm': '3.864', 'learning_rate': '6.099e-05', 'epoch': '1.466'}
{'loss': '0.1998', 'grad_norm': '2.124', 'learning_rate': '5.868e-05', 'epoch': '1.507'}
{'loss': '0.1384', 'grad_norm': '3.873', 'learning_rate': '5.635e-05', 'epoch': '1.548'}
{'loss': '0.1281', 'grad_norm': '2.123', 'learning_rate': '5.401e-05', 'epoch': '1.589'}
{'loss': '0.2184', 'grad_norm': '5.875', 'learning_rate': '5.165e-05', 'epoch': '1.63'}
{'loss': '0.1393', 'grad_norm': '2.365', 'learning_rate': '4.929e-05', 'epoch': '1.671'}
{'loss': '0.1659', 'grad_norm': '3.061', 'learning_rate': '4.694e-05', 'epoch': '1.712'}
{'loss': '0.09851', 'grad_norm': '3.075', 'learning_rate': '4.459e-05', 'epoch': '1.753'}
{'loss': '0.1217', 'grad_norm': '2.83', 'learning_rate': '4.225e-05', 'epoch': '1.793'}
{'loss': '0.1022', 'grad_norm': '2.245', 'learning_rate': '3.993e-05', 'epoch': '1.834'}
{'loss': '0.1441', 'grad_norm': '4.338', 'learning_rate': '3.763e-05', 'epoch': '1.875'}
{'loss': '0.1768', 'grad_norm': '2.467', 'learning_rate': '3.536e-05', 'epoch': '1.916'}
{'loss': '0.09541', 'grad_norm': '1.974', 'learning_rate': '3.312e-05', 'epoch': '1.957'}
{'loss': '0.07662', 'grad_norm': '1.124', 'learning_rate': '3.092e-05', 'epoch': '1.998'}
{'loss': '0.05175', 'grad_norm': '0.7626', 'learning_rate': '2.876e-05', 'epoch': '2.033'}
{'loss': '0.07802', 'grad_norm': '0.5053', 'learning_rate': '2.665e-05', 'epoch': '2.074'}
{'loss': '0.1381', 'grad_norm': '2.901', 'learning_rate': '2.459e-05', 'epoch': '2.115'}
{'loss': '0.1125', 'grad_norm': '2.289', 'learning_rate': '2.259e-05', 'epoch': '2.155'}
{'loss': '0.08472', 'grad_norm': '2.452', 'learning_rate': '2.065e-05', 'epoch': '2.196'}
{'loss': '0.09751', 'grad_norm': '0.896', 'learning_rate': '1.877e-05', 'epoch': '2.237'}
{'loss': '0.1345', 'grad_norm': '3.427', 'learning_rate': '1.697e-05', 'epoch': '2.278'}
{'loss': '0.0417', 'grad_norm': '1.938', 'learning_rate': '1.523e-05', 'epoch': '2.319'}
{'loss': '0.08852', 'grad_norm': '1.324', 'learning_rate': '1.358e-05', 'epoch': '2.36'}
{'loss': '0.1007', 'grad_norm': '3.997', 'learning_rate': '1.2e-05', 'epoch': '2.401'}
{'loss': '0.09326', 'grad_norm': '0.5659', 'learning_rate': '1.051e-05', 'epoch': '2.442'}
{'loss': '0.09737', 'grad_norm': '1.744', 'learning_rate': '9.11e-06', 'epoch': '2.483'}
{'loss': '0.05748', 'grad_norm': '3.018', 'learning_rate': '7.799e-06', 'epoch': '2.524'}
{'loss': '0.08245', 'grad_norm': '2.699', 'learning_rate': '6.581e-06', 'epoch': '2.564'}
{'loss': '0.08583', 'grad_norm': '2.702', 'learning_rate': '5.46e-06', 'epoch': '2.605'}
{'loss': '0.05044', 'grad_norm': '1.507', 'learning_rate': '4.439e-06', 'epoch': '2.646'}
{'loss': '0.052', 'grad_norm': '1.309', 'learning_rate': '3.518e-06', 'epoch': '2.687'}
{'loss': '0.08038', 'grad_norm': '1.494', 'learning_rate': '2.701e-06', 'epoch': '2.728'}
{'loss': '0.08601', 'grad_norm': '2.069', 'learning_rate': '1.989e-06', 'epoch': '2.769'}
{'loss': '0.04808', 'grad_norm': '2.772', 'learning_rate': '1.384e-06', 'epoch': '2.81'}
{'loss': '0.05216', 'grad_norm': '2.606', 'learning_rate': '8.874e-07', 'epoch': '2.851'}
{'loss': '0.07321', 'grad_norm': '4.124', 'learning_rate': '4.998e-07', 'epoch': '2.892'}
{'loss': '0.03868', 'grad_norm': '1.86', 'learning_rate': '2.223e-07', 'epoch': '2.933'}
{'loss': '0.05519', 'grad_norm': '3.036', 'learning_rate': '5.562e-08', 'epoch': '2.973'}
{'train_runtime': '8815', 'train_samples_per_second': '0.333', 'train_steps_per_second': '0.042', 'train_loss': '0.3028', 'epoch': '3'}
100%|███████████████████████████████████████| 369/369 [2:26:55<00:00, 23.89s/it]
训练完成！耗时 8816 秒 (146.9 分钟)
LoRA 权重保存到: /kaggle/working/duan/tools/ai_copilot/output/qwen3.5_2b_duan_gpu/final
训练信息保存到: /kaggle/working/duan/tools/ai_copilot/output/qwen3.5_2b_duan_gpu/training_info.json
============================================================
全部完成！
============================================================
训练完毕，合并模型
Writing model shards: 100%|███████████████████████| 1/1 [00:09<00:00, 9.72s/it]
合并后模型保存到: /kaggle/working/duan/tools/ai_copilot/output/duan_translator_merged_3.5_2b
Modelfile 保存到: /kaggle/working/duan/tools/ai_copilot/output/duan_translator_merged_3.5_2b/Modelfile
============================================================
完成！
============================================================
合并后模型: /kaggle/working/duan/tools/ai_copilot/output/duan_translator_merged_3.5_2b
下一步:
推理测试: python local_infer.py –fine-tuned
模型合并后转为gguf格式
!python llama.cpp/convert_hf_to_gguf.py \\
/kaggle/working/duan/tools/ai_copilot/output/duan_translator_merged_3.5_2b \\
–outfile /kaggle/working/duan/tools/ai_copilot/output/duan_translator_merged_3.5_2b/duan_translator.gguf \\
–outtype f16
输出：
INFO:gguf.gguf_writer:Writing the following files:
INFO:gguf.gguf_writer:/kaggle/working/duan/tools/ai_copilot/output/duan_translator_merged_3.5_2b/duan_translator.gguf: n_tensors = 320, total_size = 3.8G
Writing: 100%|███████████████████████████| 3.76G/3.76G [00:32<00:00, 115Mbyte/s]
INFO:hf-to-gguf:Model successfully exported to /kaggle/working/duan/tools/ai_copilot/output/duan_translator_merged_3.5_2b/duan_translator.gguf
量化
再编译llama.cpp
%cd /kaggle/working/llama.cpp
!mkdir build
%cd build
!cmake ..
!make -j4
量化
%cd /kaggle/working/llama.cpp/build/bin/
# 执行量化 (例如转为 Q4_K_M)
!./llama-quantize \\
/kaggle/working/duan/tools/ai_copilot/output/duan_translator_merged_3.5_2b/duan_translator.gguf \\
/kaggle/working/duan/tools/ai_copilot/output/duan_translator_merged_3.5_2b/duan_translator_q4_k_m.gguf \\
Q4_K_M
然后打包成tar包，下载到本地！
文件大小
skywalk@fb98:~/Downloads $ ls -ls /home/skywalk/Downloads/kaggle/working/duan/tool
total 1239105
1239101 -rw-r–r– 1 skywalk skywalk 1274396096 7月 23 23:14 duan_translator_q4_k_m.gguf
5 -rw-r–r– 1 skywalk skywalk 610 7月 23 22:40 Modelfile
