AndroidLab

LLM小模型综述，模型一览

Dockerfile环境：

```bash
FROM dockerpull.org/pytorch/pytorch:2.4.0-cuda12.1-cudnn9-devel
RUN pip install diffusers -i https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple
RUN pip install gradio uvicorn
RUN pip install transformers -i https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple
RUN pip install accelerate sentencepiece ftfy openai -i https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple
WORKDIR /app
COPY ./scripts /app/scripts
RUN chmod +x /app/scripts/entrypoint.sh
EXPOSE 7860
ENTRYPOINT ["/app/scripts/entrypoint.sh"]
```



推理代码：

```python
import torch
from diffusers import PixArtAlphaPipeline

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

model_path = "/ssd/xiedong/PixArt-alpha/output/model_diffusers_caidao"
pipe = PixArtAlphaPipeline.from_pretrained(model_path, torch_dtype=torch.float16, use_safetensors=True)
pipe.to(device)

prompt = "Spoiling things through excessive enthusiasm"
image = pipe(prompt=prompt,
             negative_prompt="paintings, sketches, (worst quality, low quality, normal quality:1.7), \nlowres, blurry, text, logo, monochrome, grayscale, skin spots, acnes, skin blemishes, \nage spot, strabismus, wrong finger, bad anatomy, bad hands, error, missing fingers, cropped, \njpeg artifacts, signature, watermark, username, (dark skin:1.1), fused girls, fushion, bad-hands-5, \nugly, pregnant, vore, duplicate, morbid, mutilated, tran nsexual, hermaphrodite, long neck, \nmutated hands, poorly drawn face, mutation, deformed, bad proportions, malformed limbs, extra limbs,\ncloned face, disfigured, gross proportions, (missing arms:1.331), (missing legs:1.331), \n(extra arms:1.331), (extra legs:1.331), plump, bad legs, fewer digits, (open month:2),(tooth:2),(teeth:2), \nNegro, black people, black race",
             num_inference_steps=20
             ).images[0]
rgb_image = image.convert("RGB")
rgb_image.save("./acat.jpg")

```

出问题：


Setting `clean_caption=True` requires the ftfy library but it was not found in your environment. Checkout the instructions on the
installation section: https://github.com/rspeer/python-ftfy/tree/master#installing and follow the ones
that match your environment. Please note that you may need to restart your runtime after installation.


这样超过tokens限制：


```bash
            pil_image = pipe(prompt=prompt,
                             negative_prompt=negative_prompt,
                             num_inference_steps=20,
                             max_sequence_length=200
                             ).images[0]
```

pixart Docker部署推理

在Ubuntu中卸载现有的NVIDIA驱动并安装指定的驱动文件，可以按照以下步骤操作：

### 1. 卸载现有NVIDIA驱动
首先，确保没有程序正在使用NVIDIA显卡驱动，推荐在纯文本模式下进行操作：



1. 按`Ctrl + Alt + F3`进入命令行模式，然后登录。
   
2. 停止显示管理器服务（适用于Ubuntu系统）：
   ```bash
   sudo systemctl stop gdm  # 适用于GNOME
   # 如果使用其他桌面管理器，请将gdm替换为sddm、lightdm等
   ```

3. 卸载NVIDIA驱动：
   ```bash
   sudo apt-get purge '^nvidia-.*'
   ```

4. 清理残留的依赖包：
   ```bash
   sudo apt-get autoremove
   ```

### 2. 安装指定版本的NVIDIA驱动
确保目标安装文件`NVIDIA-Linux-x86_64-550.90.07.run`已下载到本地目录（例如`~/Downloads`）。

1. 为安装文件添加执行权限：
   ```bash
   chmod +x ~/Downloads/NVIDIA-Linux-x86_64-550.90.07.run
   ```

2. 禁用`nouveau`驱动（这是NVIDIA开源驱动，可能会冲突）。创建或编辑文件`/etc/modprobe.d/blacklist-nouveau.conf`，并添加以下内容：
   ```bash
   blacklist nouveau
   options nouveau modeset=0
   ```
   
3. 更新`initramfs`，使更改生效：
   ```bash
   sudo update-initramfs -u
   ```

4. 重启系统，并再次进入命令行模式（`Ctrl + Alt + F3`）。

5. 安装新的NVIDIA驱动：
   ```bash
   sudo ~/Downloads/NVIDIA-Linux-x86_64-550.90.07.run
   ```

6. 按照安装程序提示完成安装。

### 3. 重启并验证驱动安装
安装完成后，重新启动系统：
```bash
sudo reboot
```

重启后，可以通过以下命令验证驱动是否正确安装：
```bash
nvidia-smi
```

如果一切正常，应该可以看到NVIDIA驱动的版本信息和显卡状态。

Ubuntu卸载并安装新的Nvidia驱动

```bash
docker run -d --gpus device=3 \
    -v /data/xiedong/LLaMA-Factory/output/checkpoint-20600:/data/xiedong/LLaMA-Factory/output/checkpoint-20600 \
    -p 8005:8000 \
    --ipc=host \
    dockerpull.org/vllm/vllm-openai:latest \
    --model /data/xiedong/LLaMA-Factory/output/checkpoint-20600 --gpu_memory_utilization=0.4 

```




离线推理：

```python
from transformers import AutoTokenizer
from vllm import LLM, SamplingParams

# 定义模型的路径
model_path = r"/data/xiedong/LLaMA-Factory/output/checkpoint-20600"

# 初始化分词器
tokenizer = AutoTokenizer.from_pretrained(model_path)

# 设置模型的默认解码超参数
sampling_params = SamplingParams(
    temperature=0.3,
    top_p=0.3,
    repetition_penalty=1.05,
    max_tokens=512
)

# 初始化LLM，传入模型路径
llm = LLM(model=model_path,
          enforce_eager=True,
          )

# 准备聊天消息
messages = [
    {"role": "system", "content": "你是Prompt提示词生成器，擅长将用户提供的提示词优化为更合适的表达。你只能输出英文。"},
    {"role": "user", "content": "现在请你开始优化提示词，用户输入为：一个女人，动漫风格"}
]

# 对聊天消息进行分词
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

# 使用LLM生成输出
outputs = llm.generate([text], sampling_params)

# 打印生成的输出
for output in outputs:
    prompt = output.prompt
    generated_text = output.outputs[0].text
    print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")

```

api访问：


```python
from openai import OpenAI

client = OpenAI(
    api_key="xxx",
    base_url="http://101.136.8.66:8005/v1/"
)

chat_completion = client.chat.completions.create(
    messages=[
        {"role": "system",
         "content": "你是Prompt提示词生成器，擅长将用户提供的提示词优化为更合适的表达。你只能输出英文。"},
        {"role": "user", "content": "现在请你开始优化提示词，用户输入为：一个女人，动漫风格"}
    ],
    model="/data/xiedong/LLaMA-Factory/output/checkpoint-20600",
)
print(chat_completion)

generated_text = chat_completion.choices[0].message.content
print(generated_text)

```

vllm 部署 Qwen/Qwen2.5 ，访问代码

我是一个专注技术分享的博主,尤其爱搞深度学习和单片机电子。你现在看到的是我的个人博客网站。

目的

模型一览

1. 卸载现有NVIDIA驱动