{"id":10020406,"date":"2024-11-21T20:10:03","date_gmt":"2024-11-21T12:10:03","guid":{"rendered":"https:\/\/itmax.cn\/?p=10020406"},"modified":"2024-11-21T20:10:03","modified_gmt":"2024-11-21T12:10:03","slug":"rag-%e7%a4%ba%e4%be%8b%ef%bc%9a%e4%bd%bf%e7%94%a8-langchain%e3%80%81redis%e3%80%81llama-cpp-%e6%9e%84%e5%bb%ba%e4%b8%80%e4%b8%aa-kubernetes-%e7%9f%a5%e8%af%86%e5%ba%93%e9%97%ae%e7%ad%94","status":"publish","type":"post","link":"https:\/\/itmax.cn\/?p=10020406","title":{"rendered":"RAG \u793a\u4f8b\uff1a\u4f7f\u7528 langchain\u3001Redis\u3001llama.cpp \u6784\u5efa\u4e00\u4e2a kubernetes \u77e5\u8bc6\u5e93\u95ee\u7b54"},"content":{"rendered":"<p><code>RAG<\/code>\uff08<code>Retrieval Augmented Generation<\/code> \u68c0\u7d22\u589e\u5f3a\u751f\u6210\uff09\u662f\u76ee\u524d\u4e1a\u754c\u4e2d\u7684\u4e00\u79cd\u4e3b\u6d41\u65b9\u6cd5\uff0c\u901a\u8fc7\u589e\u52a0\u989d\u5916\u77e5\u8bc6\u7684\u65b9\u5f0f\u6765\u51cf\u5c11\u5927\u8bed\u8a00\u6a21\u578b\uff08<code>LLM<\/code>\uff09\u7684\u5e7b\u89c9\u95ee\u9898\uff08\u4e00\u672c\u6b63\u7ecf\u7684\u80e1\u8bf4\u516b\u9053\uff09\u3002<\/p>\n<h2>RAG \u7cfb\u7edf\u6982\u89c8<\/h2>\n<p><img decoding=\"async\"   referrerpolicy=\"no-referrer\" src=\"https:\/\/raw.githubusercontent.com\/RifeWang\/images\/master\/ai\/arg\/RAG.drawio.png\" alt=\"\" title=\"\"><\/p>\n<p>\u5982\u4e0a\u56fe\u6240\u793a\uff0c<code>RAG<\/code> \u7cfb\u7edf\u53ef\u4ee5\u5206\u4e3a\u4e24\u4e2a\u90e8\u5206\uff1a<\/p>\n<ul>\n<li><code>Indexing<\/code>\uff1a\u6784\u5efa\u77e5\u8bc6\u5e93\u3002<\/li>\n<li><code>Retrieval &amp; Generation<\/code>\uff1a\u4ece\u77e5\u8bc6\u5e93\u4e2d\u83b7\u53d6\u76f8\u5173\u4fe1\u606f\uff0c\u7136\u540e\u751f\u6210\u7ed3\u679c\u3002<\/li>\n<\/ul>\n<p><code>Indexing<\/code> \u6784\u5efa\u77e5\u8bc6\u5e93\u7684\u8fc7\u7a0b\u53ef\u4ee5\u5206\u4e3a\u56db\u6b65\uff1a<\/p>\n<ol>\n<li><code>Load<\/code>\uff1a\u52a0\u8f7d PDF\u3001doc\u3001markdown\u3001web \u7b49\u7b49\u5f62\u5f0f\u7684\u77e5\u8bc6\u6570\u636e\u3002<\/li>\n<li><code>Split<\/code>\uff1a\u7531\u4e8e <code>LLM<\/code> \u4e0a\u4e0b\u6587\u5927\u5c0f\u7684\u9650\u5236\uff0c\u9700\u8981\u5c06\u6587\u6863\u8fdb\u884c\u5207\u5272\u3002<\/li>\n<li><code>Embedding<\/code>\uff1a\u5c06\u6587\u672c\u8f6c\u6362\u4e3a\u5411\u91cf\u3002<\/li>\n<li><code>store to VectorDB<\/code>\uff1a\u5c06\u6587\u672c\u5185\u5bb9\u548c\u5411\u91cf\u5b58\u50a8\u81f3\u5411\u91cf\u6570\u636e\u5e93\uff0c\u5373\u77e5\u8bc6\u5e93\u3002<\/li>\n<\/ol>\n<p><code>Retrieval &amp; Generation<\/code> \u7684\u8fc7\u7a0b\u4e5f\u662f\u56db\u6b65\uff1a<\/p>\n<ol>\n<li><code>Embedding<\/code>\uff1a\u5c06\u7528\u6237\u63d0\u51fa\u7684\u95ee\u9898\u8f6c\u6362\u4e3a\u5411\u91cf\u3002<\/li>\n<li><code>search VectorDB<\/code>\uff1a\u4ece\u77e5\u8bc6\u5e93\u4e2d\u67e5\u8be2\u4e0e\u95ee\u9898\u8bed\u4e49\u76f8\u8fd1\u7684\u6587\u672c\u6bb5\u843d\u3002<\/li>\n<li><code>prompt<\/code>\uff1a\u5c06\u68c0\u7d22\u51fa\u6765\u7684\u6587\u672c\u6bb5\u843d\u4e0e\u7528\u6237\u95ee\u9898\u5408\u5e76\uff0c\u751f\u6210 prompt\u3002<\/li>\n<li><code>LLM<\/code>\uff1a\u5c06 prompt \u63d0\u4ea4\u7ed9\u5927\u8bed\u8a00\u6a21\u578b\uff0c\u5f97\u5230\u6700\u7ec8\u7b54\u6848\u3002<\/li>\n<\/ol>\n<p>\u4ece\u4e0a\u8ff0\u8fc7\u7a0b\u4e2d\u53ef\u4ee5\u770b\u5230\uff0c\u76f8\u8f83\u4e8e\u76f4\u63a5\u628a\u95ee\u9898\u63d0\u4ea4\u7ed9 <code>LLM<\/code> \u5f97\u5230\u7b54\u6848\uff0c<code>RAG<\/code> \u7cfb\u7edf\u989d\u5916\u6784\u5efa\u4e86\u4e00\u4e2a\u77e5\u8bc6\u5e93\uff0c\u5e76\u4e14\u4f1a\u628a\u95ee\u9898\u8ddf\u5df2\u6709\u77e5\u8bc6\u76f8\u7ed3\u5408\uff0c\u751f\u6210\u65b0\u7684 <code>prompt<\/code> \u540e\u518d\u63d0\u4ea4\u7ed9 <code>LLM<\/code> \u5f97\u5230\u7b54\u6848\u3002<br \/>\u6362\u53e5\u8bdd\u8bf4\uff0c<code>RAG<\/code> \u7cfb\u7edf\u5c31\u662f\u5728\u7528\u6237\u63d0\u51fa\u7684\u95ee\u9898\u4e4b\u5916\uff0c\u989d\u5916\u589e\u52a0\u4e86\u4e00\u4e9b\u4e0a\u4e0b\u6587\/\u80cc\u666f\u4fe1\u606f\uff0c\u8fd9\u4e9b\u4fe1\u606f\u53ef\u4ee5\u662f\u5b9e\u65f6\u4fe1\u606f\u3001\u4e5f\u53ef\u4ee5\u662f\u4e13\u4e1a\u9886\u57df\u4fe1\u606f\uff0c\u4ee5\u6b64\u4ece <code>LLM<\/code> \u5f97\u5230\u66f4\u597d\u7684\u56de\u7b54\u3002<\/p>\n<h2>RAG \u7cfb\u7edf\u793a\u4f8b<\/h2>\n<p>\u5728\u672c\u793a\u4f8b\u4e2d\uff0c\u6211\u5c06\u4f7f\u7528 <code>langchain<\/code>\u3001<code>Redis<\/code>\u3001<code>llama.cpp<\/code> \u6784\u5efa\u4e00\u4e2a <code>kubernetes<\/code> \u77e5\u8bc6\u5e93\u95ee\u7b54\u3002<\/p>\n<p><code>langchain<\/code> \u662f\u4e00\u4e2a\u5de5\u5177\u5305\uff0c<code>Redis<\/code> \u88ab\u6211\u7528\u4f5c\u5411\u91cf\u6570\u636e\u5e93\uff0c<code>llama.cpp<\/code> \u662f\u4e00\u4e2a\u5f00\u6e90\u7684\u52a0\u8f7d\u5927\u8bed\u8a00\u6a21\u578b\u7684\u8fd0\u884c\u65f6\u7a0b\u5e8f\u3002<\/p>\n<p>\u6211\u5728\u672c\u5730\u4f7f\u7528\u4e86 Docker \u5bb9\u5668\u73af\u5883\u8fdb\u884c\u5f00\u53d1\uff0c\u901a\u8fc7\u4ee5\u4e0b docker-compose \u6587\u4ef6\u62c9\u53d6\u4f9d\u8d56\u7684\u670d\u52a1\uff1a<\/p>\n<pre><code class=\"yaml\">version: &quot;3.9&quot;\n\nservices:\n  redis:\n    image: redis\/redis-stack:7.4.0-v1\n    container_name: redis\n    ports:\n      - &quot;6379:6379&quot;   # \u6620\u5c04 Redis \u9ed8\u8ba4\u7aef\u53e3\n      - &quot;8001:8001&quot;   # \u6620\u5c04 RedisInsight \u9ed8\u8ba4\u7aef\u53e3\n\n  llama_cpp_server:\n    image: ghcr.io\/ggerganov\/llama.cpp:server\n    container_name: llama_cpp_server\n    ports:\n      - &quot;8080:8080&quot;\n    volumes:\n      - ~\/ai-models:\/models  # \u6620\u5c04\u4e3b\u673a\u8def\u5f84\u5230\u5bb9\u5668\n    environment:\n      LLAMA_ARG_MODEL: \/models\/llama3.2-1B.gguf\n      LLAMA_ARG_CTX_SIZE: 4096\n      LLAMA_ARG_HOST: &quot;0.0.0.0&quot;\n      LLAMA_ARG_PORT: 8080<\/code><\/pre>\n<p>\u4ee3\u7801\u793a\u4f8b\u5982\u4e0b\uff1a<\/p>\n<p><img decoding=\"async\"   referrerpolicy=\"no-referrer\" src=\"https:\/\/raw.githubusercontent.com\/RifeWang\/images\/master\/ai\/arg\/RAG-code.drawio.png\" alt=\"\" title=\"\" loading=\"lazy\"><\/p>\n<p>\u4ee3\u7801\u91ca\u4e49\uff1a<\/p>\n<ul>\n<li>\u52a0\u8f7d\u6587\u6863\uff1a\u4f7f\u7528 <code>langchain<\/code> \u6293\u53d6 <code>kubernetes<\/code> \u5b98\u65b9\u6587\u6863\u9875\u9762\u5185\u5bb9\u3002<\/li>\n<li>\u5207\u5272\u6587\u6863\uff1a\u4f7f\u7528 <code>langchain<\/code> \u5207\u5272\u6587\u6863\u3002<\/li>\n<li>\n<p>\u5411\u91cf\u5316\uff1a\u4f7f\u7528 <a href=\"https:\/\/link.segmentfault.com\/?enc=%2Fx0neP2%2BVNMHfh4fabuaVw%3D%3D.%2FCoyC6NpI2zNI7fIN7uQ4MQ4TADYhvVk225FJM8w%2F%2FsmiFJWz1czLIaqN2%2FmfxXXx%2FDiqb9QZIWWG%2BM60zUL5g%3D%3D\" rel=\"nofollow noopener noreferrer\" target=\"_blank\"><code>sentence_transformers<\/code><\/a> \u5c06\u6587\u672c\u8f6c\u6362\u4e3a\u5411\u91cf\uff1a<\/p>\n<pre><code class=\"Python\">from sentence_transformers import SentenceTransformer\nsentences = [&quot;This is an example sentence&quot;]\n\nmodel = SentenceTransformer('sentence-transformers\/all-mpnet-base-v2')\nembeddings = model.encode(sentences)\nprint(embeddings)<\/code><\/pre>\n<\/li>\n<li>\u5b58\u50a8\u81f3\u5411\u91cf\u6570\u636e\u5e93\uff1a\u5c06\u6587\u672c\u548c\u5411\u91cf\u5b58\u50a8\u81f3 <code>Redis<\/code>\uff0c\u4f60\u4e5f\u53ef\u4ee5\u4f7f\u7528\u5176\u5b83\u5411\u91cf\u6570\u636e\u5e93\u3002<\/li>\n<li>\u7528\u6237\u63d0\u95ee\u5411\u91cf\u5316\uff1a\u4f7f\u7528\u540c\u6837\u7684\u6a21\u578b\u5c06\u7528\u6237\u63d0\u51fa\u7684\u95ee\u9898\u8f6c\u6362\u4e3a\u5411\u91cf\u3002<\/li>\n<li>\u5411\u91cf\u76f8\u4f3c\u6027\u68c0\u7d22\uff1a\u4ece\u5411\u91cf\u6570\u636e\u5e93\u4e2d\u68c0\u7d22\u51fa\u4e0e\u63d0\u95ee\u76f8\u4f3c\u7684\u6587\u672c\u6bb5\u843d\u3002<\/li>\n<li>\u7ec4\u5408\u751f\u6210 <code>prompt<\/code>\uff1a\u5c06\u68c0\u7d22\u51fa\u6765\u7684\u4fe1\u606f\u4e0e\u7528\u6237\u63d0\u95ee\u4e00\u8d77\u5408\u5e76\u751f\u6210\u65b0\u7684 <code>prompt<\/code>\u3002<\/li>\n<li>\u5c06 <code>prompt<\/code> \u63d0\u4ea4\u7ed9 <code>LLM<\/code> \u5f97\u5230\u7b54\u6848\u3002<\/li>\n<\/ul>\n<p>\u6700\u7ec8\u7684\u6d4b\u8bd5\u7ed3\u679c\u5982\u4e0b\uff1a<\/p>\n<p><img decoding=\"async\"   referrerpolicy=\"no-referrer\" src=\"https:\/\/raw.githubusercontent.com\/RifeWang\/images\/master\/ai\/arg\/rag-answers.png\" alt=\"\" title=\"\" loading=\"lazy\"><\/p>\n<p>\u53ef\u4ee5\u770b\u5230\uff0c\u52a0\u4e86\u4e0a\u4e0b\u6587\u4e4b\u540e\uff0c<code>LLM<\/code> \u7ed9\u51fa\u7684\u7b54\u6848\u66f4\u597d\u4e86\u3002<\/p>\n<h2>RAG \u9762\u4e34\u7684\u6311\u6218<\/h2>\n<p><code>RAG<\/code> \u7684\u6bcf\u4e00\u6b65\u51e0\u4e4e\u90fd\u9762\u4e34\u6311\u6218\uff1a<\/p>\n<ul>\n<li>\u5982\u4f55\u52a0\u8f7d\u4e0d\u540c\u5f62\u5f0f\u7684\u6587\u6863\u6570\u636e\uff1f\u8fd9\u4e00\u6b65\u95ee\u9898\u5012\u4e0d\u5927\u3002<\/li>\n<li>\u5982\u4f55\u5207\u5272\u6587\u6863\uff1f\u5207\u5272\u7684\u6548\u679c\u5f71\u54cd\u4e86 <code>prompt<\/code> \u7684\u4e0a\u4e0b\u6587\u4fe1\u606f\uff0c\u56e0\u6b64\u4e5f\u4f1a\u5f71\u54cd <code>LLM<\/code> \u751f\u6210\u7684\u7ed3\u679c\u3002<\/li>\n<li>\u5982\u4f55\u505a <code>embedding<\/code>\uff0c\u9009\u62e9\u54ea\u79cd\u6a21\u578b\uff1f<\/li>\n<li>\u9009\u62e9\u54ea\u4e2a\u5411\u91cf\u6570\u636e\u5e93\uff1f\u5e38\u89c1\u7684\u6280\u672f\u9009\u578b\u95ee\u9898\u3002<\/li>\n<li>\u5982\u4f55\u5c06\u68c0\u7d22\u51fa\u6765\u7684\u4fe1\u606f\u4e0e\u7528\u6237\u63d0\u95ee\u4e00\u8d77\u5408\u5e76\u6210\u65b0\u7684 <code>prompt<\/code>\uff1f<code>prompt<\/code> \u672c\u6765\u5c31\u53ef\u4ee5\u4e94\u82b1\u516b\u95e8\u3002<\/li>\n<li>\u9009\u62e9\u54ea\u4e2a <code>LLM<\/code> \u4ee5\u53ca\u8fd0\u884c\u65f6\uff1f\u5c31\u6a21\u578b\u800c\u8a00\uff0c<code>Llama<\/code> \u7cfb\u5217\u7b97\u662f\u6700\u706b\u7684\u5f00\u6e90\u6a21\u578b\u4e86\uff1b\u800c\u8fd0\u884c\u65f6\u5219\u6709 <code>llama.cpp<\/code>\uff08\u6216\u8005\u5728\u5176\u4e4b\u4e0a\u5c01\u88c5\u7684 <code>Ollama<\/code>\uff09\u3001<code>HuggingFace\/transformers<\/code>\u3001<code>vLLM<\/code> \u7b49\u7b49\u3002<\/li>\n<\/ul>\n<h2>\u603b\u7ed3<\/h2>\n<p><code>RAG<\/code> \u7cfb\u7edf\u901a\u8fc7\u5f15\u5165\u77e5\u8bc6\u5e93\uff0c\u6269\u5c55\u4e86 <code>LLM<\/code> \u5bf9\u4e13\u4e1a\u9886\u57df\u548c\u5b9e\u65f6\u4fe1\u606f\u7684\u652f\u6301\u80fd\u529b\uff0c\u4f7f\u5176\u5728\u56de\u7b54\u4e13\u4e1a\u95ee\u9898\u65f6\u66f4\u52a0\u51c6\u786e\u9ad8\u6548\u3002<\/p>\n<p>(\u6211\u662f\u51cc\u865a\uff0c\u5173\u6ce8\u6211\uff0c\u65e0\u5e7f\u544a\uff0c\u4e13\u6ce8\u6280\u672f\uff0c\u4e0d\u717d\u52a8\u60c5\u7eea\uff0c\u6b22\u8fce\u4e0e\u6211\u4ea4\u6d41)<\/p>\n<hr>\n<p>\u53c2\u8003\u8d44\u6599\uff1a<\/p>\n<ul>\n<li><em><a href=\"https:\/\/link.segmentfault.com\/?enc=G5jqGnGixNnrZ%2FNhrRgDJw%3D%3D.QePiQ3GzYxst7z5%2FCcPP8CIHXqZMI%2BtpRV3JkTOQXzX0ZpPWf8VYyfNP4bxVrQgqwMfQh7hZLxHX4AGMyC6LIQ%3D%3D\" rel=\"nofollow noopener noreferrer\" target=\"_blank\">https:\/\/python.langchain.com\/docs\/tutorials\/rag\/<\/a><\/em><\/li>\n<li><em><a href=\"https:\/\/link.segmentfault.com\/?enc=Qatk7%2Fgzco39fXhS6UYpcg%3D%3D.JXfOZikpEsj3oLSV3dTBbWhbSaIg67Kj7%2B57eHrHOmUw8VTGxR5DKwEBU4iNBs5i5zs0ql%2B9hOfJr45lQLw3bA%3D%3D\" rel=\"nofollow noopener noreferrer\" target=\"_blank\">https:\/\/huggingface.co\/sentence-transformers\/all-mpnet-base-v2<\/a><\/em><\/li>\n<li><em><a href=\"https:\/\/link.segmentfault.com\/?enc=2Wa2GsG4i%2F%2B2NUiqz8wI%2BA%3D%3D.gkdJSLnRixXdshpquPbaJi69QYCgcuv9qa0d0qTYKzJlpuDaCF1EHMnaeFWshlTK7SE1QEC9RjdWiZzf7vWPmbLzjYD4ynOjjVgWhA6yOcs%3D\" rel=\"nofollow noopener noreferrer\" target=\"_blank\">https:\/\/github.com\/ggerganov\/llama.cpp\/blob\/master\/examples\/server\/README.md<\/a><\/em><\/li>\n<li><em><a href=\"https:\/\/link.segmentfault.com\/?enc=WrfB7AbaMUwq1FCGl5%2F1Kg%3D%3D.toYiMBbLS%2BTDvwyX%2BYyqBA%2Bf0UEHgtyKfEt8IFWIPKqrmfUCDnkNCy7PD%2Fzc%2F0VzwA3jUbs6NyBtI%2BqAoxDBTxmx%2BXMhsEgRgSSw%2Btjv36Qi%2FuYhWiLMYEcFA6Ge0FtK\" rel=\"nofollow noopener noreferrer\" target=\"_blank\">https:\/\/redis.io\/docs\/latest\/develop\/interact\/search-and-quer&#8230;<\/a><\/em><\/li>\n<\/ul>\n<p><a href=\"https:\/\/segmentfault.com\/a\/1190000045487806\">https:\/\/segmentfault.com\/a\/1190000045487806<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>RAG\uff08Retrieval Augmented Generation \u68c0\u7d22\u589e\u5f3a\u751f\u6210\uff09\u662f\u76ee\u524d\u4e1a\u754c\u4e2d\u7684\u4e00\u79cd\u4e3b\u6d41\u65b9\u6cd5 [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[4],"tags":[],"class_list":["post-10020406","post","type-post","status-publish","format-standard","hentry","category-share"],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/itmax.cn\/index.php?rest_route=\/wp\/v2\/posts\/10020406","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/itmax.cn\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/itmax.cn\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/itmax.cn\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/itmax.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=10020406"}],"version-history":[{"count":0,"href":"https:\/\/itmax.cn\/index.php?rest_route=\/wp\/v2\/posts\/10020406\/revisions"}],"wp:attachment":[{"href":"https:\/\/itmax.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=10020406"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/itmax.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=10020406"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/itmax.cn\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=10020406"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}