PaddleOCR-VL-0.9B生产部署

paddleocr genai_server --model_name PaddleOCR-VL-0.9B --backend vllm --port 8118

from paddleocr import PaddleOCRVLpipeline = PaddleOCRVL(vl_rec_backend="vllm-server", vl_rec_server_url="http://127.0.0.1:8118/v1")output = pipeline.predict("webwxgetmsgimg.jpeg")#print(output)for res in output:    #res.print()    res.save_to_json(save_path="output")    res.save_to_markdown(save_path="output")    print(res._to_json())    print(dir(res))

def get_config(backend):    if backend == "fastdeploy":        return {            "gpu-memory-utilization": 0.3,            "max-model-len": 16384,            "max-num-batched-tokens": 131072,            "max-num-seqs": 256,        }    elif backend == "vllm":        return {            "trust-remote-code": True,            "gpu-memory-utilization": 0.3,            "max-model-len": 16384,            "max-num-batched-tokens": 131072,            "api-server-count": 4,            "max-num-seqs":4,        }    elif backend == "sglang":        return {            "trust-remote-code": True,            "mem-fraction-static": 0.5,            "context-length": 16384,            "max-prefill-tokens": 131072,        }    else:        raise ValueError(f"Unsupported backend: {backend}")

paddleocr genai_server --model_name PaddleOCR-VL-0.9B --backend sglang --port 8119

[2025-11-22 15:02:40] INFO:     Application startup complete.[2025-11-22 15:02:40] INFO:     Uvicorn running on http://localhost:8119 (Press CTRL+C to quit)[2025-11-22 15:02:40] INFO:     127.0.0.1:53766 - "GET /health HTTP/1.1" 503 Service Unavailable[2025-11-22 15:02:41] INFO:     127.0.0.1:53774 - "GET /get_model_info HTTP/1.1" 200 OK[2025-11-22 15:02:41] Prefill batch, #new-seq: 1, #new-token: 6, #cached-token: 0, token usage: 0.00, #running-req: 0, #queue-req: 0, [2025-11-22 15:02:41] INFO:     127.0.0.1:53790 - "GET /health HTTP/1.1" 503 Service Unavailable[2025-11-22 15:02:42] INFO:     127.0.0.1:53802 - "GET /health HTTP/1.1" 503 Service Unavailable[2025-11-22 15:02:43] INFO:     127.0.0.1:53784 - "POST /generate HTTP/1.1" 200 OK[2025-11-22 15:02:43] The server is fired up and ready to roll![2025-11-22 15:02:43] Prefill batch, #new-seq: 1, #new-token: 1, #cached-token: 0, token usage: 0.00, #running-req: 0, #queue-req: 0, [2025-11-22 15:02:44] INFO:     127.0.0.1:53808 - "GET /health HTTP/1.1" 200 OK[2025/11/22 15:02:44] paddleocr INFO: The PaddleOCR GenAI server has been started. You can either:    1. Set the server URL in the module or pipeline configuration and call the PaddleOCR CLI or Python API. For example:        paddleocr doc_parser --input demo.png --vl_rec_backend sglang-server --vl_rec_server_url http://localhost:8119/v1    2. Make HTTP requests directly, or using the OpenAI client library.

from paddleocr import PaddleOCRVLpipeline = PaddleOCRVL(vl_rec_backend="sglang-server", vl_rec_server_url="http://127.0.0.1:8119/v1")output = pipeline.predict("webwxgetmsgimg.jpeg")#print(output)for res in output:    #res.print()    res.save_to_json(save_path="output")    res.save_to_markdown(save_path="output")    print(res._to_json())    print(dir(res))

paddlex --install serving

paddlex --serve --pipeline PaddleOCR-VL

import base64import requestsimport pathlib
API_URL = "http://localhost:8080/layout-parsing" # 服务URL
image_path = "./demo.jpg"
# 对本地图像进行Base64编码with open(image_path, "rb") as file:    image_bytes = file.read()    image_data = base64.b64encode(image_bytes).decode("ascii")
payload = {    "file": image_data, # Base64编码的文件内容或者文件URL    "fileType": 1, # 文件类型，1表示图像文件}
# 调用APIresponse = requests.post(API_URL, json=payload)
# 处理接口返回数据assert response.status_code == 200result = response.json()["result"]for i, res in enumerate(result["layoutParsingResults"]):    print(res["prunedResult"])    md_dir = pathlib.Path(f"markdown_{i}")    md_dir.mkdir(exist_ok=True)    (md_dir / "doc.md").write_text(res["markdown"]["text"])    for img_path, img in res["markdown"]["images"].items():        img_path = md_dir / img_path        img_path.parent.mkdir(parents=True, exist_ok=True)        img_path.write_bytes(base64.b64decode(img))    print(f"Markdown document saved at {md_dir / 'doc.md'}")    for img_name, img in res["outputImages"].items():        img_path = f"{img_name}_{i}.jpg"        pathlib.Path(img_path).parent.mkdir(exist_ok=True)        with open(img_path, "wb") as f:            f.write(base64.b64decode(img))        print(f"Output image saved at {img_path}")

paddlex --serve --pipeline PaddleOCR-VL.yaml

Creating model: ('PP-DocLayoutV2', None)Model files already exist. Using cached files. To redownload, please delete the directory manually: `/home/zhds/.paddlex/official_models/PP-DocLayoutV2`./home/zhds/.local/lib/python3.10/site-packages/paddle/utils/cpp_extension/extension_utils.py:718: UserWarning: No ccache found. Please be aware that recompiling all source files may be required. You can download and install ccache from: https://github.com/ccache/ccache/blob/master/doc/INSTALL.md  warnings.warn(warning_message)Creating model: ('PaddleOCR-VL-0.9B', None)INFO:     Started server process [1938346]INFO:     Waiting for application startup.INFO:     Application startup complete.INFO:     Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)INFO:     127.0.0.1:49176 - "POST /layout-parsing HTTP/1.1" 200 OK