欢迎光临
我们一直在努力

怎么让 transformers 和 vLLM 从 modelscope 下载资源而不是从 huggingface 下载资源?

怎么让 qwen_asr 、transformers 后端 和 vLLM 后端从 modelscope 下载资源而不是从 huggingface 下载资源?


看了 https://modelscope.cn/models/Qwen/Qwen3-ASR-0.6B 这个教程, 运行下面的命令报错了

qwen-asr-demo /
  --asr-checkpoint Qwen/Qwen3-ASR-1.7B /
  --aligner-checkpoint Qwen/Qwen3-ForcedAligner-0.6B /
  --backend vllm /
  --cuda-visible-devices 0 /
  --backend-kwargs '{"gpu_memory_utilization":0.7,"max_inference_batch_size":8,"max_new_tokens":2048}' /
  --aligner-kwargs '{"device_map":"cuda:0","dtype":"bfloat16"}' /
  --ip 0.0.0.0 --port 8000

报错如下:

    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pon/.local/share/virtualenvs/modelscope_example-DACykz4b/lib/python3.11/site-packages/transformers/configuration_utils.py", line 721, in _get_config_dict
    resolved_config_file = cached_file(
                           ^^^^^^^^^^^^
  File "/home/pon/.local/share/virtualenvs/modelscope_example-DACykz4b/lib/python3.11/site-packages/transformers/utils/hub.py", line 322, in cached_file
    file = cached_files(path_or_repo_id=path_or_repo_id, filenames=[filename], **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pon/.local/share/virtualenvs/modelscope_example-DACykz4b/lib/python3.11/site-packages/transformers/utils/hub.py", line 553, in cached_files
    raise OSError(
OSError: We couldn't connect to 'https://huggingface.co' to load the files, and couldn't find them in the cached files.
Check your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.

所以怎么办?

https://segmentfault.com/a/1190000047583141

未经允许不得转载:IT极限技术分享汇 » 怎么让 transformers 和 vLLM 从 modelscope 下载资源而不是从 huggingface 下载资源?

评论 抢沙发

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址