我要投稿
微信搜一搜，关注“NLP工程化”
Dify 1.4.0版本重磅推出多模态大语言模型输出功能，让AI同时处理文本和图像，开启更丰富的数据交互体验。 核心内容： 1. Gemini 2.0 Flash Exp模型的多模态处理能力 2. API key验证的常见问题与解决方案 3. 实际工作流程中的输出结果分析
本文使用dify 1.4.0版本，该版本除了浅色和深色模式之间切换功能外，还有一个重要功能，即多模态大语言模型输出，允许大语言模型（LLMs）在 LLMNode 中同时处理文本和图像。这一功能提升了数据交互的丰富性，并增强了模型的能力。
现在可使用 Gemini 2.0 Flash Exp 模型尝试这些新功能，这一集成确保了 AI 能够无缝处理和响应文本与图像数据。其中，Gemini 是 Google 推出的一系列多模态 AI 模型，能够处理和生成多种类型的数据，包括文本、图像、音频和视频。
由于Dify在做Gemini的API key验证的时候，使用的google库，所以基本上直接是网络原因导致验证失败，可以使用https://cloud.dify.ai/或者其它代理设置方式进行操作。
data: {"event": "workflow_started", "conversation_id": "30a1c40b-536d-4bf8-954f-afb27a29040a", "message_id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "created_at": 1748335614, "task_id": "4cc59763-acb2-47e3-bce4-0700c0c2753a", "workflow_run_id": "0e2f2237-05dc-4272-b536-3e97bf02691c", "data": {"id": "0e2f2237-05dc-4272-b536-3e97bf02691c", "workflow_id": "c600204e-1bdd-4f28-86db-e46777890c46", "sequence_number": 2, "inputs": {"sys.query": "\u63cf\u8ff0\u56fe\u50cf", "sys.files": [{"dify_model_identity": "__dify__file__", "id": null, "tenant_id": "796610e5-ff95-42ed-8f0e-f5d3fa8c40ae", "type": "image", "transfer_method": "local_file", "remote_url": "https://upload.dify.ai/files/3e063508-298c-4188-ad53-1b7cdb9421b1/file-preview?timestamp=1748335608&nonce=c3a2116a1019aeceeab65bae841da65a&sign=4nJFZvggRLT73fGLnWskMOlMe3qQbe374LbZwrSwrSM=", "related_id": "3e063508-298c-4188-ad53-1b7cdb9421b1", "filename": "dog.png", "extension": ".png", "mime_type": "image/png", "size": 368313, "url": "https://upload.dify.ai/files/3e063508-298c-4188-ad53-1b7cdb9421b1/file-preview?timestamp=1748335614&nonce=82184cad92dcb7aea431765ce35105d3&sign=K2C69sZxG4Z1U-WSe41xbLKpz1JHbh-JAjv2rAl4NiI="}], "sys.conversation_id": "30a1c40b-536d-4bf8-954f-afb27a29040a", "sys.user_id": "7d8864c3-c456-4588-9b0a-9368c94ca377", "sys.dialogue_count": 0, "sys.app_id": "22d976e3-4842-4769-a5b0-73ca84894e8d", "sys.workflow_id": "c600204e-1bdd-4f28-86db-e46777890c46", "sys.workflow_run_id": "0e2f2237-05dc-4272-b536-3e97bf02691c"}, "created_at": 1748335614}}
data: {"event": "node_started", "conversation_id": "30a1c40b-536d-4bf8-954f-afb27a29040a", "message_id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "created_at": 1748335614, "task_id": "4cc59763-acb2-47e3-bce4-0700c0c2753a", "workflow_run_id": "0e2f2237-05dc-4272-b536-3e97bf02691c", "data": {"id": "2800f85c-907b-4bcc-aab0-273a4b87779e", "node_id": "1748333062401", "node_type": "start", "title": "\u5f00\u59cb", "index": 1, "predecessor_node_id": null, "inputs": null, "created_at": 1748335614, "extras": {}, "parallel_id": null, "parallel_start_node_id": null, "parent_parallel_id": null, "parent_parallel_start_node_id": null, "iteration_id": null, "loop_id": null, "parallel_run_id": null, "Agent_strategy": null}}
data: {"event": "node_finished", "conversation_id": "30a1c40b-536d-4bf8-954f-afb27a29040a", "message_id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "created_at": 1748335614, "task_id": "4cc59763-acb2-47e3-bce4-0700c0c2753a", "workflow_run_id": "0e2f2237-05dc-4272-b536-3e97bf02691c", "data": {"id": "2800f85c-907b-4bcc-aab0-273a4b87779e", "node_id": "1748333062401", "node_type": "start", "title": "\u5f00\u59cb", "index": 1, "predecessor_node_id": null, "inputs": {"sys.query": "\u63cf\u8ff0\u56fe\u50cf", "sys.files": [{"dify_model_identity": "__dify__file__", "id": null, "tenant_id": "796610e5-ff95-42ed-8f0e-f5d3fa8c40ae", "type": "image", "transfer_method": "local_file", "remote_url": "https://upload.dify.ai/files/3e063508-298c-4188-ad53-1b7cdb9421b1/file-preview?timestamp=1748335608&nonce=c3a2116a1019aeceeab65bae841da65a&sign=4nJFZvggRLT73fGLnWskMOlMe3qQbe374LbZwrSwrSM=", "related_id": "3e063508-298c-4188-ad53-1b7cdb9421b1", "filename": "dog.png", "extension": ".png", "mime_type": "image/png", "size": 368313, "url": "https://upload.dify.ai/files/3e063508-298c-4188-ad53-1b7cdb9421b1/file-preview?timestamp=1748335614&nonce=67dfea243007a69609a2b16e6b14bbcd&sign=AA4HmgJKtcdPkztPsadtLzr8LypyaNUAeWmEiuqpugo="}], "sys.conversation_id": "30a1c40b-536d-4bf8-954f-afb27a29040a", "sys.user_id": "7d8864c3-c456-4588-9b0a-9368c94ca377", "sys.dialogue_count": 0, "sys.app_id": "22d976e3-4842-4769-a5b0-73ca84894e8d", "sys.workflow_id": "c600204e-1bdd-4f28-86db-e46777890c46", "sys.workflow_run_id": "0e2f2237-05dc-4272-b536-3e97bf02691c"}, "process_data": null, "outputs": {"sys.query": "\u63cf\u8ff0\u56fe\u50cf", "sys.files": [{"dify_model_identity": "__dify__file__", "id": null, "tenant_id": "796610e5-ff95-42ed-8f0e-f5d3fa8c40ae", "type": "image", "transfer_method": "local_file", "remote_url": "https://upload.dify.ai/files/3e063508-298c-4188-ad53-1b7cdb9421b1/file-preview?timestamp=1748335608&nonce=c3a2116a1019aeceeab65bae841da65a&sign=4nJFZvggRLT73fGLnWskMOlMe3qQbe374LbZwrSwrSM=", "related_id": "3e063508-298c-4188-ad53-1b7cdb9421b1", "filename": "dog.png", "extension": ".png", "mime_type": "image/png", "size": 368313, "url": "https://upload.dify.ai/files/3e063508-298c-4188-ad53-1b7cdb9421b1/file-preview?timestamp=1748335614&nonce=a83024f2e70dd080937d147963747f64&sign=9bhuGRAnH4YKORSG5l8I9FWhtKaJqiE9gUJ-vYBxwa0="}], "sys.conversation_id": "30a1c40b-536d-4bf8-954f-afb27a29040a", "sys.user_id": "7d8864c3-c456-4588-9b0a-9368c94ca377", "sys.dialogue_count": 0, "sys.app_id": "22d976e3-4842-4769-a5b0-73ca84894e8d", "sys.workflow_id": "c600204e-1bdd-4f28-86db-e46777890c46", "sys.workflow_run_id": "0e2f2237-05dc-4272-b536-3e97bf02691c"}, "status": "succeeded", "error": null, "elapsed_time": 0.031084, "execution_metadata": null, "created_at": 1748335614, "finished_at": 1748335614, "files": [{"dify_model_identity": "__dify__file__", "id": null, "tenant_id": "796610e5-ff95-42ed-8f0e-f5d3fa8c40ae", "type": "image", "transfer_method": "local_file", "remote_url": "https://upload.dify.ai/files/3e063508-298c-4188-ad53-1b7cdb9421b1/file-preview?timestamp=1748335608&nonce=c3a2116a1019aeceeab65bae841da65a&sign=4nJFZvggRLT73fGLnWskMOlMe3qQbe374LbZwrSwrSM=", "related_id": "3e063508-298c-4188-ad53-1b7cdb9421b1", "filename": "dog.png", "extension": ".png", "mime_type": "image/png", "size": 368313, "url": "https://upload.dify.ai/files/3e063508-298c-4188-ad53-1b7cdb9421b1/file-preview?timestamp=1748335614&nonce=a83024f2e70dd080937d147963747f64&sign=9bhuGRAnH4YKORSG5l8I9FWhtKaJqiE9gUJ-vYBxwa0="}], "parallel_id": null, "parallel_start_node_id": null, "parent_parallel_id": null, "parent_parallel_start_node_id": null, "iteration_id": null, "loop_id": null}}
data: {"event": "node_started", "conversation_id": "30a1c40b-536d-4bf8-954f-afb27a29040a", "message_id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "created_at": 1748335614, "task_id": "4cc59763-acb2-47e3-bce4-0700c0c2753a", "workflow_run_id": "0e2f2237-05dc-4272-b536-3e97bf02691c", "data": {"id": "ed004c58-4c89-4432-ab91-29888e10f3d6", "node_id": "llm", "node_type": "llm", "title": "LLM", "index": 2, "predecessor_node_id": "1748333062401", "inputs": null, "created_at": 1748335614, "extras": {}, "parallel_id": null, "parallel_start_node_id": null, "parent_parallel_id": null, "parent_parallel_start_node_id": null, "iteration_id": null, "loop_id": null, "parallel_run_id": null, "agent_strategy": null}}
data: {"event": "message", "conversation_id": "30a1c40b-536d-4bf8-954f-afb27a29040a", "message_id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "created_at": 1748335614, "task_id": "4cc59763-acb2-47e3-bce4-0700c0c2753a", "id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "answer": "\u8fd9", "from_variable_selector": ["llm", "text"]}
data: {"event": "message", "conversation_id": "30a1c40b-536d-4bf8-954f-afb27a29040a", "message_id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "created_at": 1748335614, "task_id": "4cc59763-acb2-47e3-bce4-0700c0c2753a", "id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "answer": "\u5e45\u7167\u7247\u662f\u4e00\u4e2a\u7279\u5199\u955c\u5934\uff0c\u62cd\u6444\u4e86\u4e00\u53ea\u5fb7\u56fd\u7267\u7f8a\u72ac\u3002\u8fd9", "from_variable_selector": ["llm", "text"]}
data: {"event": "message", "conversation_id": "30a1c40b-536d-4bf8-954f-afb27a29040a", "message_id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "created_at": 1748335614, "task_id": "4cc59763-acb2-47e3-bce4-0700c0c2753a", "id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "answer": "\u53ea\u72d7\u6709\u7740\u4f20\u7edf\u7684\u9ed1\u8910\u8272\u6bdb\u76ae\uff0c\u7ad6\u7acb\u7740\u4e00\u5bf9\u8b66\u60d5\u7684\u8033\u6735\u3002\u5b83\u7684\u773c\u775b\u662f\u6e29\u6696\u7684\u68d5\u8272\uff0c\u76f4\u89c6\u524d\u65b9\u3002\u5b83\u7684\u5634\u5df4\u5fae\u5fae\u5f20\u5f00\uff0c\u9732\u51fa\u7c89\u7ea2\u8272\u7684\u820c\u5934\uff0c\u663e\u793a\u51fa\u8f7b\u5fae", "from_variable_selector": ["llm", "text"]}
data: {"event": "message", "conversation_id": "30a1c40b-536d-4bf8-954f-afb27a29040a", "message_id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "created_at": 1748335614, "task_id": "4cc59763-acb2-47e3-bce4-0700c0c2753a", "id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "answer": "\u7684\u5598\u606f\u3002\u5b83\u6234\u7740\u4e00\u4e2a\u91d1\u5c5e\u94fe\u6761\u9879\u5708\uff0c\u4e0a\u9762\u6302\u7740\u4e00\u4e2a\u8eab\u4efd\u724c\u3002\u80cc\u666f\u662f\u67d4\u548c\u6a21\u7cca\u7684\u7eff\u8272\u548c\u68d5\u8272\uff0c\u6697\u793a\u7740\u6237\u5916\u73af\u5883\uff0c\u53ef\u80fd\u662f\u8349\u5730\u6216\u6811\u6797\uff0c\u9633\u5149\u4f3c\u4e4e\u4ece\u540e", "from_variable_selector": ["llm", "text"]}
data: {"event": "message", "conversation_id": "30a1c40b-536d-4bf8-954f-afb27a29040a", "message_id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "created_at": 1748335614, "task_id": "4cc59763-acb2-47e3-bce4-0700c0c2753a", "id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "answer": "\u65b9\u7167\u5c04\uff0c\u7167\u4eae\u4e86\u72d7\u72d7\u7684\u8f6e\u5ed3\u3002", "from_variable_selector": ["llm", "text"]}
data: {"event": "node_finished", "conversation_id": "30a1c40b-536d-4bf8-954f-afb27a29040a", "message_id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "created_at": 1748335614, "task_id": "4cc59763-acb2-47e3-bce4-0700c0c2753a", "workflow_run_id": "0e2f2237-05dc-4272-b536-3e97bf02691c", "data": {"id": "ed004c58-4c89-4432-ab91-29888e10f3d6", "node_id": "llm", "node_type": "llm", "title": "LLM", "index": 2, "predecessor_node_id": "1748333062401", "inputs": null, "process_data": {"model_mode": "chat", "prompts": [{"role": "user", "text": "\u63cf\u8ff0\u56fe\u50cf\n", "files": []}, {"role": "user", "text": "", "files": [{"type": "image", "data": "data:image...[TRUNCATED]...lFTkSuQmCC", "detail": "high"}]}], "model_provider": "langgenius/gemini/google", "model_name": "gemini-2.0-flash-exp"}, "outputs": {"text": "\u8fd9\u5e45\u7167\u7247\u662f\u4e00\u4e2a\u7279\u5199\u955c\u5934\uff0c\u62cd\u6444\u4e86\u4e00\u53ea\u5fb7\u56fd\u7267\u7f8a\u72ac\u3002\u8fd9\u53ea\u72d7\u6709\u7740\u4f20\u7edf\u7684\u9ed1\u8910\u8272\u6bdb\u76ae\uff0c\u7ad6\u7acb\u7740\u4e00\u5bf9\u8b66\u60d5\u7684\u8033\u6735\u3002\u5b83\u7684\u773c\u775b\u662f\u6e29\u6696\u7684\u68d5\u8272\uff0c\u76f4\u89c6\u524d\u65b9\u3002\u5b83\u7684\u5634\u5df4\u5fae\u5fae\u5f20\u5f00\uff0c\u9732\u51fa\u7c89\u7ea2\u8272\u7684\u820c\u5934\uff0c\u663e\u793a\u51fa\u8f7b\u5fae\u7684\u5598\u606f\u3002\u5b83\u6234\u7740\u4e00\u4e2a\u91d1\u5c5e\u94fe\u6761\u9879\u5708\uff0c\u4e0a\u9762\u6302\u7740\u4e00\u4e2a\u8eab\u4efd\u724c\u3002\u80cc\u666f\u662f\u67d4\u548c\u6a21\u7cca\u7684\u7eff\u8272\u548c\u68d5\u8272\uff0c\u6697\u793a\u7740\u6237\u5916\u73af\u5883\uff0c\u53ef\u80fd\u662f\u8349\u5730\u6216\u6811\u6797\uff0c\u9633\u5149\u4f3c\u4e4e\u4ece\u540e\u65b9\u7167\u5c04\uff0c\u7167\u4eae\u4e86\u72d7\u72d7\u7684\u8f6e\u5ed3\u3002", "usage": {"prompt_tokens": 0, "prompt_unit_price": "0.0", "prompt_price_unit": "0.0", "prompt_price": "0.0", "completion_tokens": 0, "completion_unit_price": "0.0", "completion_price_unit": "0.0", "completion_price": "0.0", "total_tokens": 0, "total_price": "0.0", "currency": "USD", "latency": 0.0}, "finish_reason": null, "files": []}, "status": "succeeded", "error": null, "elapsed_time": 1.894678, "execution_metadata": {"total_tokens": 0, "total_price": "0.0", "currency": "USD"}, "created_at": 1748335614, "finished_at": 1748335616, "files": [], "parallel_id": null, "parallel_start_node_id": null, "parent_parallel_id": null, "parent_parallel_start_node_id": null, "iteration_id": null, "loop_id": null}}
data: {"event": "node_started", "conversation_id": "30a1c40b-536d-4bf8-954f-afb27a29040a", "message_id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "created_at": 1748335614, "task_id": "4cc59763-acb2-47e3-bce4-0700c0c2753a", "workflow_run_id": "0e2f2237-05dc-4272-b536-3e97bf02691c", "data": {"id": "ca84c578-56b7-4f9b-be57-299f9ffc5f08", "node_id": "answer", "node_type": "answer", "title": "\u76f4\u63a5\u56de\u590d", "index": 3, "predecessor_node_id": "llm", "inputs": null, "created_at": 1748335616, "extras": {}, "parallel_id": null, "parallel_start_node_id": null, "parent_parallel_id": null, "parent_parallel_start_node_id": null, "iteration_id": null, "loop_id": null, "parallel_run_id": null, "agent_strategy": null}}
data: {"event": "node_finished", "conversation_id": "30a1c40b-536d-4bf8-954f-afb27a29040a", "message_id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "created_at": 1748335614, "task_id": "4cc59763-acb2-47e3-bce4-0700c0c2753a", "workflow_run_id": "0e2f2237-05dc-4272-b536-3e97bf02691c", "data": {"id": "ca84c578-56b7-4f9b-be57-299f9ffc5f08", "node_id": "answer", "node_type": "answer", "title": "\u76f4\u63a5\u56de\u590d", "index": 3, "predecessor_node_id": "llm", "inputs": null, "process_data": null, "outputs": {"answer": "\u8fd9\u5e45\u7167\u7247\u662f\u4e00\u4e2a\u7279\u5199\u955c\u5934\uff0c\u62cd\u6444\u4e86\u4e00\u53ea\u5fb7\u56fd\u7267\u7f8a\u72ac\u3002\u8fd9\u53ea\u72d7\u6709\u7740\u4f20\u7edf\u7684\u9ed1\u8910\u8272\u6bdb\u76ae\uff0c\u7ad6\u7acb\u7740\u4e00\u5bf9\u8b66\u60d5\u7684\u8033\u6735\u3002\u5b83\u7684\u773c\u775b\u662f\u6e29\u6696\u7684\u68d5\u8272\uff0c\u76f4\u89c6\u524d\u65b9\u3002\u5b83\u7684\u5634\u5df4\u5fae\u5fae\u5f20\u5f00\uff0c\u9732\u51fa\u7c89\u7ea2\u8272\u7684\u820c\u5934\uff0c\u663e\u793a\u51fa\u8f7b\u5fae\u7684\u5598\u606f\u3002\u5b83\u6234\u7740\u4e00\u4e2a\u91d1\u5c5e\u94fe\u6761\u9879\u5708\uff0c\u4e0a\u9762\u6302\u7740\u4e00\u4e2a\u8eab\u4efd\u724c\u3002\u80cc\u666f\u662f\u67d4\u548c\u6a21\u7cca\u7684\u7eff\u8272\u548c\u68d5\u8272\uff0c\u6697\u793a\u7740\u6237\u5916\u73af\u5883\uff0c\u53ef\u80fd\u662f\u8349\u5730\u6216\u6811\u6797\uff0c\u9633\u5149\u4f3c\u4e4e\u4ece\u540e\u65b9\u7167\u5c04\uff0c\u7167\u4eae\u4e86\u72d7\u72d7\u7684\u8f6e\u5ed3\u3002", "files": []}, "status": "succeeded", "error": null, "elapsed_time": 0.026377, "execution_metadata": null, "created_at": 1748335616, "finished_at": 1748335616, "files": [], "parallel_id": null, "parallel_start_node_id": null, "parent_parallel_id": null, "parent_parallel_start_node_id": null, "iteration_id": null, "loop_id": null}}
data: {"event": "workflow_finished", "conversation_id": "30a1c40b-536d-4bf8-954f-afb27a29040a", "message_id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "created_at": 1748335614, "task_id": "4cc59763-acb2-47e3-bce4-0700c0c2753a", "workflow_run_id": "0e2f2237-05dc-4272-b536-3e97bf02691c", "data": {"id": "0e2f2237-05dc-4272-b536-3e97bf02691c", "workflow_id": "c600204e-1bdd-4f28-86db-e46777890c46", "sequence_number": 2, "status": "succeeded", "outputs": {"answer": "\u8fd9\u5e45\u7167\u7247\u662f\u4e00\u4e2a\u7279\u5199\u955c\u5934\uff0c\u62cd\u6444\u4e86\u4e00\u53ea\u5fb7\u56fd\u7267\u7f8a\u72ac\u3002\u8fd9\u53ea\u72d7\u6709\u7740\u4f20\u7edf\u7684\u9ed1\u8910\u8272\u6bdb\u76ae\uff0c\u7ad6\u7acb\u7740\u4e00\u5bf9\u8b66\u60d5\u7684\u8033\u6735\u3002\u5b83\u7684\u773c\u775b\u662f\u6e29\u6696\u7684\u68d5\u8272\uff0c\u76f4\u89c6\u524d\u65b9\u3002\u5b83\u7684\u5634\u5df4\u5fae\u5fae\u5f20\u5f00\uff0c\u9732\u51fa\u7c89\u7ea2\u8272\u7684\u820c\u5934\uff0c\u663e\u793a\u51fa\u8f7b\u5fae\u7684\u5598\u606f\u3002\u5b83\u6234\u7740\u4e00\u4e2a\u91d1\u5c5e\u94fe\u6761\u9879\u5708\uff0c\u4e0a\u9762\u6302\u7740\u4e00\u4e2a\u8eab\u4efd\u724c\u3002\u80cc\u666f\u662f\u67d4\u548c\u6a21\u7cca\u7684\u7eff\u8272\u548c\u68d5\u8272\uff0c\u6697\u793a\u7740\u6237\u5916\u73af\u5883\uff0c\u53ef\u80fd\u662f\u8349\u5730\u6216\u6811\u6797\uff0c\u9633\u5149\u4f3c\u4e4e\u4ece\u540e\u65b9\u7167\u5c04\uff0c\u7167\u4eae\u4e86\u72d7\u72d7\u7684\u8f6e\u5ed3\u3002"}, "error": null, "elapsed_time": 1.9400834180414677, "total_tokens": 0, "total_steps": 3, "created_by": {"id": "7d8864c3-c456-4588-9b0a-9368c94ca377", "name": "HIS\u7cfb\u7edf", "email": "913292836@qq.com"}, "created_at": 1748335614, "finished_at": 1748335616, "exceptions_count": 0, "files": []}}
data: {"event": "message_end", "conversation_id": "30a1c40b-536d-4bf8-954f-afb27a29040a", "message_id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "created_at": 1748335614, "task_id": "4cc59763-acb2-47e3-bce4-0700c0c2753a", "id": "af0c19aa-5cc6-4002-bc5a-517d4cc83922", "metadata": {"usage": {"prompt_tokens": 0, "prompt_unit_price": "0.0", "prompt_price_unit": "0.0", "prompt_price": "0.0", "completion_tokens": 0, "completion_unit_price": "0.0", "completion_price_unit": "0.0", "completion_price": "0.0", "total_tokens": 0, "total_price": "0.0", "currency": "USD", "latency": 0.0}}, "files": []}
（1）node_started
{
"event" : "node_started",
"conversation_id" : "30a1c40b-536d-4bf8-954f-afb27a29040a",
"message_id" : "af0c19aa-5cc6-4002-bc5a-517d4cc83922",
"created_at" : 1748335614,
"task_id" : "4cc59763-acb2-47e3-bce4-0700c0c2753a",
"workflow_run_id" : "0e2f2237-05dc-4272-b536-3e97bf02691c",
"data" : {
"id" : "ed004c58-4c89-4432-ab91-29888e10f3d6",
"node_id" : "llm",
"node_type" : "llm",
"title" : "LLM",
"index" : 2,
"predecessor_node_id" : "1748333062401",
"inputs" : null,
"created_at" : 1748335614,
"extras" : {
},
"parallel_id" : null,
"parallel_start_node_id" : null,
"parent_parallel_id" : null,
"parent_parallel_start_node_id" : null,
"iteration_id" : null,
"loop_id" : null,
"parallel_run_id" : null,
"agent_strategy" : null
}
}
（2）message1
{
"event" : "message",
"conversation_id" : "30a1c40b-536d-4bf8-954f-afb27a29040a",
"message_id" : "af0c19aa-5cc6-4002-bc5a-517d4cc83922",
"created_at" : 1748335614,
"task_id" : "4cc59763-acb2-47e3-bce4-0700c0c2753a",
"id" : "af0c19aa-5cc6-4002-bc5a-517d4cc83922",
"answer" : "这",
"from_variable_selector" : [
"llm",
"text"
]
}
（3）message2
{
"event" : "message",
"conversation_id" : "30a1c40b-536d-4bf8-954f-afb27a29040a",
"message_id" : "af0c19aa-5cc6-4002-bc5a-517d4cc83922",
"created_at" : 1748335614,
"task_id" : "4cc59763-acb2-47e3-bce4-0700c0c2753a",
"id" : "af0c19aa-5cc6-4002-bc5a-517d4cc83922",
"answer" : "幅照片是一个特写镜头，拍摄了一只德国牧羊犬。这",
"from_variable_selector" : [
"llm",
"text"
]
}
（4）message3
{
"event" : "message",
"conversation_id" : "30a1c40b-536d-4bf8-954f-afb27a29040a",
"message_id" : "af0c19aa-5cc6-4002-bc5a-517d4cc83922",
"created_at" : 1748335614,
"task_id" : "4cc59763-acb2-47e3-bce4-0700c0c2753a",
"id" : "af0c19aa-5cc6-4002-bc5a-517d4cc83922",
"answer" : "只狗有着传统的黑褐色毛皮，竖立着一对警惕的耳朵。它的眼睛是温暖的棕色，直视前方。它的嘴巴微微张开，露出粉红色的舌头，显示出轻微",
"from_variable_selector" : [
"llm",
"text"
]
}
（5）message4
{
"event" : "message",
"conversation_id" : "30a1c40b-536d-4bf8-954f-afb27a29040a",
"message_id" : "af0c19aa-5cc6-4002-bc5a-517d4cc83922",
"created_at" : 1748335614,
"task_id" : "4cc59763-acb2-47e3-bce4-0700c0c2753a",
"id" : "af0c19aa-5cc6-4002-bc5a-517d4cc83922",
"answer" : "的喘息。它戴着一个金属链条项圈，上面挂着一个身份牌。背景是柔和模糊的绿色和棕色，暗示着户外环境，可能是草地或树林，阳光似乎从后",
"from_variable_selector" : [
"llm",
"text"
]
}
（6）message5
{
"event" : "message",
"conversation_id" : "30a1c40b-536d-4bf8-954f-afb27a29040a",
"message_id" : "af0c19aa-5cc6-4002-bc5a-517d4cc83922",
"created_at" : 1748335614,
"task_id" : "4cc59763-acb2-47e3-bce4-0700c0c2753a",
"id" : "af0c19aa-5cc6-4002-bc5a-517d4cc83922",
"answer" : "方照射，照亮了狗狗的轮廓。",
"from_variable_selector" : [
"llm",
"text"
]
}
（7）node_finished
{
"event" : "node_finished",
"conversation_id" : "30a1c40b-536d-4bf8-954f-afb27a29040a",
"message_id" : "af0c19aa-5cc6-4002-bc5a-517d4cc83922",
"created_at" : 1748335614,
"task_id" : "4cc59763-acb2-47e3-bce4-0700c0c2753a",
"workflow_run_id" : "0e2f2237-05dc-4272-b536-3e97bf02691c",
"data" : {
"id" : "ed004c58-4c89-4432-ab91-29888e10f3d6",
"node_id" : "llm",
"node_type" : "llm",
"title" : "LLM",
"index" : 2,
"predecessor_node_id" : "1748333062401",
"inputs" : null,
"process_data" : {
"model_mode" : "chat",
"prompts" : [
{
"role" : "user",
"text" : "描述图像
",
"files" : [
]
},
{
"role" : "user",
"text" : "",
"files" : [
{
"type" : "image",
"data" : "data:image...[TRUNCATED]...lFTkSuQmCC",
"detail" : "high"
}
]
}
],
"model_provider" : "langgenius/gemini/google",
"model_name" : "gemini-2.0-flash-exp"
},
"outputs" : {
"text" : "这幅照片是一个特写镜头，拍摄了一只德国牧羊犬。这只狗有着传统的黑褐色毛皮，竖立着一对警惕的耳朵。它的眼睛是温暖的棕色，直视前方。它的嘴巴微微张开，露出粉红色的舌头，显示出轻微的喘息。它戴着一个金属链条项圈，上面挂着一个身份牌。背景是柔和模糊的绿色和棕色，暗示着户外环境，可能是草地或树林，阳光似乎从后方照射，照亮了狗狗的轮廓。",
"usage" : {
"prompt_tokens" : 0,
"prompt_unit_price" : "0.0",
"prompt_price_unit" : "0.0",
"prompt_price" : "0.0",
"completion_tokens" : 0,
"completion_unit_price" : "0.0",
"completion_price_unit" : "0.0",
"completion_price" : "0.0",
"total_tokens" : 0,
"total_price" : "0.0",
"currency" : "USD",
"latency" : 0.0
},
"finish_reason" : null,
"files" : [
]
},
"status" : "succeeded",
"error" : null,
"elapsed_time" : 1.894678,
"execution_metadata" : {
"total_tokens" : 0,
"total_price" : "0.0",
"currency" : "USD"
},
"created_at" : 1748335614,
"finished_at" : 1748335616,
"files" : [
],
"parallel_id" : null,
"parallel_start_node_id" : null,
"parent_parallel_id" : null,
"parent_parallel_start_node_id" : null,
"iteration_id" : null,
"loop_id" : null
}
}
{
"sys.query": "描述图像",
"sys.files": [
{
"dify_model_identity": "__dify__file__",
"id": null,
"tenant_id": "796610e5-ff95-42ed-8f0e-f5d3fa8c40ae",
"type": "image",
"transfer_method": "local_file",
"remote_url": "https://upload.dify.ai/files/3e063508-298c-4188-ad53-1b7cdb9421b1/file-preview?timestamp=1748335608&nonce=c3a2116a1019aeceeab65bae841da65a&sign=4nJFZvggRLT73fGLnWskMOlMe3qQbe374LbZwrSwrSM=",
"related_id": "3e063508-298c-4188-ad53-1b7cdb9421b1",
"filename": "dog.png",
"extension": ".png",
"mime_type": "image/png",
"size": 368313,
"url": "https://upload.dify.ai/files/3e063508-298c-4188-ad53-1b7cdb9421b1/file-preview?timestamp=1748335614&nonce=67dfea243007a69609a2b16e6b14bbcd&sign=AA4HmgJKtcdPkztPsadtLzr8LypyaNUAeWmEiuqpugo="
}
],
"sys.conversation_id": "30a1c40b-536d-4bf8-954f-afb27a29040a",
"sys.user_id": "7d8864c3-c456-4588-9b0a-9368c94ca377",
"sys.dialogue_count": 0,
"sys.app_id": "22d976e3-4842-4769-a5b0-73ca84894e8d",
"sys.workflow_id": "c600204e-1bdd-4f28-86db-e46777890c46",
"sys.workflow_run_id": "0e2f2237-05dc-4272-b536-3e97bf02691c"
}
{
"sys.query": "描述图像",
"sys.files": [
{
"dify_model_identity": "__dify__file__",
"id": null,
"tenant_id": "796610e5-ff95-42ed-8f0e-f5d3fa8c40ae",
"type": "image",
"transfer_method": "local_file",
"remote_url": "https://upload.dify.ai/files/3e063508-298c-4188-ad53-1b7cdb9421b1/file-preview?timestamp=1748335608&nonce=c3a2116a1019aeceeab65bae841da65a&sign=4nJFZvggRLT73fGLnWskMOlMe3qQbe374LbZwrSwrSM=",
"related_id": "3e063508-298c-4188-ad53-1b7cdb9421b1",
"filename": "dog.png",
"extension": ".png",
"mime_type": "image/png",
"size": 368313,
"url": "https://upload.dify.ai/files/3e063508-298c-4188-ad53-1b7cdb9421b1/file-preview?timestamp=1748335614&nonce=a83024f2e70dd080937d147963747f64&sign=9bhuGRAnH4YKORSG5l8I9FWhtKaJqiE9gUJ-vYBxwa0="
}
],
"sys.conversation_id": "30a1c40b-536d-4bf8-954f-afb27a29040a",
"sys.user_id": "7d8864c3-c456-4588-9b0a-9368c94ca377",
"sys.dialogue_count": 0,
"sys.app_id": "22d976e3-4842-4769-a5b0-73ca84894e8d",
"sys.workflow_id": "c600204e-1bdd-4f28-86db-e46777890c46",
"sys.workflow_run_id": "0e2f2237-05dc-4272-b536-3e97bf02691c"
}
{
"model_mode": "chat",
"prompts": [
{
"role": "user",
"text": "描述图像\n",
"files": []
},
{
"role": "user",
"text": "",
"files": [
{
"type": "image",
"data": "data:image...[TRUNCATED]...lFTkSuQmCC",
"detail": "high"
}
]
}
],
"model_provider": "langgenius/gemini/google",
"model_name": "gemini-2.0-flash-exp"
}
{
"text": "这幅照片是一个特写镜头，拍摄了一只德国牧羊犬。这只狗有着传统的黑褐色毛皮，竖立着一对警惕的耳朵。它的眼睛是温暖的棕色，直视前方。它的嘴巴微微张开，露出粉红色的舌头，显示出轻微的喘息。它戴着一个金属链条项圈，上面挂着一个身份牌。背景是柔和模糊的绿色和棕色，暗示着户外环境，可能是草地或树林，阳光似乎从后方照射，照亮了狗狗的轮廓。",
"usage": {
"prompt_tokens": 0,
"prompt_unit_price": "0.0",
"prompt_price_unit": "0.0",
"prompt_price": "0.0",
"completion_tokens": 0,
"completion_unit_price": "0.0",
"completion_price_unit": "0.0",
"completion_price": "0.0",
"total_tokens": 0,
"total_price": "0.0",
"currency": "USD",
"latency": 0
},
"finish_reason": null,
"files": []
}
{
"answer": "这幅照片是一个特写镜头，拍摄了一只德国牧羊犬。这只狗有着传统的黑褐色毛皮，竖立着一对警惕的耳朵。它的眼睛是温暖的棕色，直视前方。它的嘴巴微微张开，露出粉红色的舌头，显示出轻微的喘息。它戴着一个金属链条项圈，上面挂着一个身份牌。背景是柔和模糊的绿色和棕色，暗示着户外环境，可能是草地或树林，阳光似乎从后方照射，照亮了狗狗的轮廓。",
"files": []
}
这个数据是LLM节点处理过程中生成的process_data对象，描述了一个多模态AI模型调用的请求配置。它是这样得到的：
model_mode: "chat"表示使用聊天模式，这来自model_config.mode
model_provider: "langgenius/gemini/google"表示使用Google的Gemini模型
model_name: "gemini-2.0-flash-exp"是具体使用的模型版本
通过_fetch_prompt_messages方法生成，包含两个用户消息：（1）第一个消息包含文本提示"描述图像"（2）第二个消息包含图像文件，使用base64编码存储
图像通过_fetch_files方法获取
detail: "high"表示要求模型进行高分辨率的图像分析，对应代码中的vision_detail_config参数
当工作流执行到LLM节点时，系统提取用户配置的提示和上传的图像
组装成规范化的请求格式
记录在process_data中，用于后续调用AI模型和记录执行过程
这个数据随后会被传递给_invoke_llm方法，向Gemini模型发送请求来获取对图像的描述结果。
53AI，企业落地大模型首选服务商
产品：场景落地咨询+大模型应用平台+行业解决方案
承诺：免费场景POC验证，效果验证后签署服务协议。零风险落地应用大模型，已交付160+中大型企业
2025-06-19
搜索 ≠ 简单匹配！0代码实现语义级图文互搜
2025-06-11
AI提效99.5%！英国政府联手 Gemini，破解城市规划审批困局
2025-06-10
多模态 RAG VS 传统文本 RAG ，到底效果如何，从应用视角来测试下
2025-06-10
实战复盘 | 基于视觉模型的多模态 RAG 系统，我们踩过的坑与收获 (项目已开源)
2025-06-05
多模态模型在RagFlow中的应用
2025-06-04
清华首创多模态+知识图谱+RAG，问答精准度超 94%
2025-05-30
Deepseek 多模态来解析图片，结合上下文分析pdf文档
2025-05-28
Lovart再次证明：AI不是卖工具而是卖成果
2025-05-14
2025-04-27
2025-03-26
2025-04-28
2025-05-16
2025-04-05
2025-05-08
2025-05-16
2025-05-13
2025-05-15
回到顶部
粤ICP备17114055号