Vulnhuntr 源码分析
无图无真相 这年头 AI 都有 5 个 CVE 编号了 你呢？
居然比我厉害不行我要解刨你
其实很久前就准备读了一直拖到现在
step 01 解析命令行参数 项目支持 4 个命令行参数
分别是
- root 要审计的项目根路径
- analyze 指定分析项目中的某一个文件或者文件夹
- llm 指定 llm 模型 项目支持 claude gpt 以及 ollama 默认是 claude
- verbosity 啰嗦模式 -v 打印 INFO 日志 -vv 答应 debug 日志
step 02 加载代码库
加载代码仓库 默认会排除一些非项目代码文件
repo = RepoOps(args.root)
class RepoOps:
def __init__(self, repo_path: Path | str ) -> None:
self.repo_path = Path(repo_path)
self.to_exclude = {'/setup.py', '/test', '/example', '/docs', '/site-packages', '.venv', 'virtualenv', '/dist'} # 排除的文件夹
self.file_names_to_exclude = ['test_', 'conftest', '_test.py'] # 排除的文件
patterns = [
# Flask
r'@app\.route\(.*?\)',
r'@blueprint\.route\(.*?\)',
r'class\s+\w+\(MethodView\):',
r'@(?:app|blueprint)\.add_url_rule\(.*?\)',
# FastAPI
r'@app\.(?:get|post|put|delete|patch|options|head|trace)\(.*?\)',
r'@router\.(?:get|post|put|delete|patch|options|head|trace)\(.*?\)',
*****
] # python Web框架接口匹配正则 (这里省略了大量 摘录了常见的web框架)
# Compile the patterns for efficiency
self.compiled_patterns = [re.compile(pattern) for pattern in patterns] # 预编译正则
这里使用了大量常见的 python Web 框架正则，所以该项目只支持 常见的 Python Web 项目的审计
step 03 使用代码仓库 构建符号提取器
class SymbolExtractor:
def __init__(self, repo_path: str | pathlib.Path) -> None:
self.repo_path = pathlib.Path(repo_path)
self.project = jedi.Project(self.repo_path)
self.parsed_symbols = None
self.ignore = ['/test', '_test/', '/docs', '/example'] # 排除的文件夹
这里使用了 jedi 加载了代码仓库 Jedi 是一个用于 Python 的静态分析工具 所以这个项目都是针对 Python 的不支持其它语言
step 04 获取所有的 py 代码文件 并排除非项目代码文件
获取项目中所有 Python 文件 并排除 RepoOps.to_exclude 中的文件夹中的文件 以及 RepoOps.file_names_to_exclude 中的文件
files = repo.get_relevant_py_files()
def get_relevant_py_files(self) -> Generator[Path, None, None]:
"""Gets all Python files in a repo minus the ones in the exclude list (test, example, doc, docs)"""
files = []
for f in self.repo_path.rglob("*.py"):
# Convert the path to a string with forward slashes
f_str = str(f).replace('\\', '/')
# Lowercase the string for case-insensitive matching
f_str = f_str.lower()
# Check if any exclusion pattern matches a substring of the full path
if any(exclude in f_str for exclude in self.to_exclude):
continue
# Check if the file name should be excluded
if any(fn in f.name for fn in self.file_names_to_exclude):
continue
files.append(f)
return files
step 05 获取要分析的指定文件
# 如果指定了分析文件夹路径或文件路径 会使用 指定的文件或目录中的文件 同时排除非项目代码文件
# User specified --analyze flag
if args.analyze:
# Determine the path to analyze
analyze_path = Path(args.analyze)
# If the path is absolute, use it as is, otherwise join it with the root path so user can specify relative paths
if analyze_path.is_absolute():
files_to_analyze = repo.get_files_to_analyze(analyze_path)
else:
files_to_analyze = repo.get_files_to_analyze(Path(args.root) / analyze_path)
# 否则提取项目中Python Web 框架接口相关的Python文件
# Analyze the entire project for network-related files
else:
files_to_analyze = repo.get_network_related_files(files)
使用预编译好的 Python Web 框架正则 清单 获取网络相关的 Python 文件
files_to_analyze = repo.get_network_related_files(files)
def get_network_related_files(self, files: List) -> Generator[Path, None, None]:
for py_f in files:
with py_f.open(encoding='utf-8') as f:
content = f.read()
if any(re.search(pattern, content) for pattern in self.compiled_patterns):
yield py_f
step 06 初始化 llm 对象 并使用项目 Readme 文件 构建 System Prompt
llm 模型 是通过 命令行参数 指定
llm = initialize_llm(args.llm)
readme_content = repo.get_readme_content() # 获取readme 文件内容 如果没有则为空
if readme_content:
log.info("Summarizing project README")
summary = llm.chat(
(ReadmeContent(content=readme_content).to_xml() + b'\n' +
Instructions(instructions=README_SUMMARY_PROMPT_TEMPLATE).to_xml()
).decode()
)
summary = extract_between_tags("summary", summary)[0]
log.info("README summary complete", summary=summary)
else:
log.warning("No README summary found")
summary = ''
# Initialize the system prompt with the README summary
system_prompt = (Instructions(instructions=SYS_PROMPT_TEMPLATE).to_xml() + b'\n' +
ReadmeSummary(readme_summary=summary).to_xml()
).decode()
llm = initialize_llm(args.llm, system_prompt)
使用项目 Readme 文件 构建 System Prompt
摘要 readme 文件
summary = llm.chat(
(ReadmeContent(content=readme_content).to_xml() + b'\n' +
Instructions(instructions=README_SUMMARY_PROMPT_TEMPLATE).to_xml()
).decode()
)
summary = extract_between_tags("summary", summary)[0]
这里使用了 pydantic_xml 库
将 readme 文件和摘要提示词分别使用 <readme_content> </readme_content> 和 <readme_summary></readme_summary> XML 标签 进行包裹 并使用 \n 连接 构建后的 Prompt 大概是
<readme_content> README 内容</readme_content>
<readme_summary>
Provide a very concise summary of the README.md content in <readme_content></readme_content> tags from a security researcher's perspective, focusing specifically on:
1. The project's main purpose
2. Any networking capabilities, such as web interfaces or remote API calls that constitute remote attack surfaces
3. Key features that involve network communications
Please keep the summary brief and to the point, highlighting only the most relevant networking-related functionality as it relates to attack surface.
Output in <summary></summary> XML tags.
</readme_summary>
然后将使用正则将 <summary></summary> 中的内容提取了出来
这段提取内容的注释中 贴一个 anthropics 如何处理 LLM 输出的不规则 json 的 Code book 感兴趣的可以看看
https://github.com/anthropics/anthropic-cookbook/blob/main/misc/how_to_enable_json_mode.ipynb
构建 System 系统提示词
# Initialize the system prompt with the README summary
system_prompt = (Instructions(instructions=SYS_PROMPT_TEMPLATE).to_xml() + b'\n' +
ReadmeSummary(readme_summary=summary).to_xml()
).decode()
同样的方式 构造了提示词使用 LLM 生成 System Prompt 提示词大概为
<instructions>
You are the world's foremost expert in Python security analysis, renowned for uncovering novel and complex vulnerabilities in web applications. Your task is to perform an exhaustive static code analysis, focusing on remotely exploitable vulnerabilities including but not limited to:
1. Local File Inclusion (LFI)
2. Remote Code Execution (RCE)
3. Server-Side Request Forgery (SSRF)
4. Arbitrary File Overwrite (AFO)
5. SQL Injection (SQLI)
6. Cross-Site Scripting (XSS)
7. Insecure Direct Object References (IDOR)
Your analysis must:
- Meticulously track user input from remote sources to high-risk function sinks.
- Uncover complex, multi-step vulnerabilities that may bypass multiple security controls.
- Consider non-obvious attack vectors and chained vulnerabilities.
- Identify vulnerabilities that could arise from the interaction of multiple code components.
If you don't have the complete code chain from user input to high-risk function, strategically request the necessary context to fill in the gaps in the <context_code> tags of your response.
The project's README summary is provided in <readme_summary> tags. Use this to understand the application's purpose and potential attack surfaces.
Remember, you have many opportunities to respond and request additional context. Use them wisely to build a comprehensive understanding of the application's security posture.
Output your findings in JSON format, conforming to the schema in <response_format> tags.
</instructions>
<readme_summary> READEM 摘要 </readme_summary>
支持的模型
这里支持了 claude gpt ollama 三种类型的 LLM 的 API 从 env 中读取 API 配置
def initialize_llm(llm_arg: str, system_prompt: str = "") -> Claude | ChatGPT | Ollama:
llm_arg = llm_arg.lower()
if llm_arg == 'claude':
anth_model = os.getenv("ANTHROPIC_MODEL", "claude-3-5-sonnet-latest")
anth_base_url = os.getenv("ANTHROPIC_BASE_URL", "https://api.anthropic.com")
llm = Claude(anth_model, anth_base_url, system_prompt)
elif llm_arg == 'gpt':
openai_model = os.getenv("OPENAI_MODEL", "chatgpt-4o-latest")
openai_base_url = os.getenv("OPENAI_BASE_URL", "https://api.openai.com/v1")
llm = ChatGPT(openai_model, openai_base_url, system_prompt)
elif llm_arg == 'ollama':
ollama_model = os.getenv("OLLAMA_MODEL", "llama3")
ollama_base_url = os.getenv("OLLAMA_BASE_URL", "http://127.0.0.1:11434/api/generate")
llm = Ollama(ollama_model, ollama_base_url, system_prompt)
else:
raise ValueError(f"Invalid LLM argument: {llm_arg}\nValid options are: claude, gpt, ollama")
return llm
step 06 逐个代码文件分析
构造代码文件分析提示词进行分析
user_prompt =(
FileCode(file_path=str(py_f), file_source=content).to_xml() + b'\n' +
Instructions(instructions=INITIAL_ANALYSIS_PROMPT_TEMPLATE).to_xml() + b'\n' +
AnalysisApproach(analysis_approach=ANALYSIS_APPROACH_TEMPLATE).to_xml() + b'\n' +
PreviousAnalysis(previous_analysis='').to_xml() + b'\n' +
Guidelines(guidelines=GUIDELINES_TEMPLATE).to_xml() + b'\n' +
ResponseFormat(response_format=json.dumps(Response.model_json_schema(), indent=4
)
).to_xml()
).decode()
initial_analysis_report: Response = llm.chat(user_prompt, response_model=Response)
log.info("Initial analysis complete", report=initial_analysis_report.model_dump())
print_readable(initial_analysis_report)
构造的分析提示词大概是
<file_code>
<file_path>源代码路径</file_path>
<file_source>源代码内容</file_source>
</file_code>
<instructions>
Analyze the code in <file_code> tags for potential remotely exploitable vulnerabilities: 1. Identify all remote user input entry points (e.g., API endpoints, form submissions) and if you can't find that, request the necessary classes or functions in the <context_code> tags. 2. Locate potential vulnerability sinks for: - Local File Inclusion (LFI) - Arbitrary File Overwrite (AFO) - Server-Side Request Forgery (SSRF) - Remote Code Execution (RCE) - Cross-Site Scripting (XSS) - SQL Injection (SQLI) - Insecure Direct Object Reference (IDOR) 3. Note any security controls or sanitization measures encountered along the way so you can craft bypass techniques for the proof of concept (PoC). 4. Highlight areas where more context is needed to complete the analysis.
Be generous and thorough in identifying potential vulnerabilities as you'll analyze more code in subsequent steps so if there's just a possibility of a vulnerability, include it the <vulnerability_types> tags.
</instructions>
<analysis_approach>Analysis Instructions:
1. Comprehensive Review:
- Thoroughly examine the content in <file_code>, <context_code> tags (if provided) with a focus on remotely exploitable vulnerabilities.
2. Vulnerability Scanning:
- You only care about remotely exploitable network related components and remote user input handlers.
- Identify potential entry points for vulnerabilities.
- Consider non-obvious attack vectors and edge cases.
3. Code Path Analysis:
- Very important: trace the flow of user input from remote request source to function sink.
- Examine input validation, sanitization, and encoding practices.
- Analyze how data is processed, stored, and output.
4. Security Control Analysis:
- Evaluate each security measure's implementation and effectiveness.
- Formulate potential bypass techniques, considering latest exploit methods.
5. Context-Aware Analysis:
- If this is a follow-up analysis, build upon previous findings in <previous_analysis> using the new information provided in the <context_code>.
- Request additional context code as needed to complete the analysis and you will be provided with the necessary code.
- Confirm that the requested context class or function is not already in the <context_code> tags from the user's message.
6. Final Review:
- Confirm your proof of concept (PoC) exploits bypass any security controls.
- Double-check that your JSON response is well-formed and complete.
</analysis_approach>
<previous_analysis></previous_analysis>
<guidelines>Reporting Guidelines:
1. JSON Format:
- Provide a single, well-formed JSON report combining all findings.
- Use 'None' for any aspect of the report that you lack the necessary information for.
- Place your step-by-step analysis in the scratchpad field, before doing a final analysis in the analysis field.
2. Context Requests:
- Classes: Use ClassName1,ClassName2
- Functions: Use func_name,ClassName.method_name
- If you request ClassName, do not also request ClassName.method_name as that code will already be fetched with the ClassName request.
- Important: Do not request code from standard libraries or third-party packages. Simply use what you know about them in your analysis.
3. Vulnerability Reporting:
- Report only remotely exploitable vulnerabilities (no local access/CLI args).
- Always include at least one vulnerability_type field when requesting context.
- Provide a confidence score (0-10) and detailed justification for each vulnerability.
- If your proof of concept (PoC) exploit does not start with remote user input via remote networking calls such as remote HTTP, API, or RPC calls, set the confidence score to 6 or below.
4. Proof of Concept:
- Include a PoC exploit or detailed exploitation steps for each vulnerability.
- Ensure PoCs are specific to the analyzed code, not generic examples.
- Review the code path ofthe potential vulnerability and be sure that the PoC bypasses any security controls in the code path.
</guidelines>
<response_format>
{
"$defs": {
"ContextCode": {
"properties": {
"name": {
"description": "Function or Class name",
"title": "Name",
"type": "string"
},
"reason": {
"description": "Brief reason why this function's code is needed for analysis",
"title": "Reason",
"type": "string"
},
"code_line": {
"description": "The single line of code where where this context object is referenced.",
"title": "Code Line",
"type": "string"
}
},
"required": [
"name",
"reason",
"code_line"
],
"title": "ContextCode",
"type": "object"
},
"VulnType": {
"enum": [
"LFI",
"RCE",
"SSRF",
"AFO",
"SQLI",
"XSS",
"IDOR"
],
"title": "VulnType",
"type": "string"
}
},
"properties": {
"scratchpad": {
"description": "Your step-by-step analysis process. Output in plaintext with no line breaks.",
"title": "Scratchpad",
"type": "string"
},
"analysis": {
"description": "Your final analysis. Output in plaintext with no line breaks.",
"title": "Analysis",
"type": "string"
},
"poc": {
"description": "Proof-of-concept exploit, if applicable.",
"title": "Poc",
"type": "string"
},
"confidence_score": {
"description": "0-10, where 0 is no confidence and 10 is absolute certainty because you have the entire user input to server output code path.",
"title": "Confidence Score",
"type": "integer"
},
"vulnerability_types": {
"description": "The types of identified vulnerabilities",
"items": {
"$ref": "#/$defs/VulnType"
},
"title": "Vulnerability Types",
"type": "array"
},
"context_code": {
"description": "List of context code items requested for analysis, one function or class name per item. No standard library or third-party package code.",
"items": {
"$ref": "#/\$defs/ContextCode"
},
"title": "Context Code",
"type": "array"
}
},
"required": [
"scratchpad",
"analysis",
"poc",
"confidence_score",
"vulnerability_types",
"context_code"
],
"title": "Response",
"type": "object"
}
</response_format>
这段提示词一共是 6 部分 分别是
- FileCode：源代码信息 路径加上内容
- Instructions：分析指令提示词
- AnalysisApproach：分析方法提示词
- PreviousAnalysis：上一次分析得到的信息
- Guidelines: 分析指南提示词
- ResponseFormat： 响应格式
然后得到初步的分析报告
class Response(BaseModel):
scratchpad: str = Field(description="Your step-by-step analysis process. Output in plaintext with no line breaks.")
analysis: str = Field(description="Your final analysis. Output in plaintext with no line breaks.")
poc: str = Field(description="Proof-of-concept exploit, if applicable.")
confidence_score: int = Field(description="0-10, where 0 is no confidence and 10 is absolute certainty because you have the entire user input to server output code path.")
vulnerability_types: List[VulnType] = Field(description="The types of identified vulnerabilities")
context_code: List[ContextCode] = Field(description="List of context code items requested for analysis, one function or class name per item. No standard library or third-party package code.")
分析报告分为 6 部分分别是
- scratchpad 分析过程
- analysis 最终分析
- poc 概念利用
- confidence_score 可信度 INT 0-10
- vulnerability_types 弱点类型 List 这里枚举值有 LFI RCE SSRF AFO SQLI XSS IDOR
- context_code 上下文代码 List
- name 方法名或者类名
- reason 解释为什么分析该部分代码
- code_line 命中的代码
step 07 二次分析
# Secondary analysis
if initial_analysis_report.confidence_score > 0 and len(initial_analysis_report.vulnerability_types):
for vuln_type in initial_analysis_report.vulnerability_types:
当初步分析报告中出现可信度>0 且存在弱点类型枚举时 逐个对发现的弱点类型进行二次分析
初始化迭代分析 上下文
在二次分析前 会初始化一些上下文进行迭代分析
# Do not fetch the context code on the first pass of the secondary analysis because the context will be from the general analysis
stored_code_definitions = {}
definitions = CodeDefinitions(definitions=[])
same_context = False
# Don't include the initial analysis or the first iteration of the secondary analysis in the user_prompt
previous_analysis = ''
previous_context_amount = 0
迭代分析
迭代分析会进行 7 轮 第一次会生成上下文 之后的分析会根据上下文再进行迭代分析
for i in range(7):
log.info(f"Performing vuln-specific analysis", iteration=i, vuln_type=vuln_type, file=py_f)
# Only lookup context code and previous analysis on second pass and onwards
if i > 0:
#之后的分析
#第一次分析
第一次分析(不带上下文的分析)
构造分析提示词
vuln_specific_user_prompt = (
FileCode(file_path=str(py_f), file_source=content).to_xml() + b'\n' +
definitions.to_xml() + b'\n' + # These are all the requested context functions and classes
ExampleBypasses(
example_bypasses='\n'.join(VULN_SPECIFIC_BYPASSES_AND_PROMPTS[vuln_type]['bypasses'])
).to_xml() + b'\n' +
Instructions(instructions=VULN_SPECIFIC_BYPASSES_AND_PROMPTS[vuln_type]['prompt']).to_xml() + b'\n' +
AnalysisApproach(analysis_approach=ANALYSIS_APPROACH_TEMPLATE).to_xml() + b'\n' +
PreviousAnalysis(previous_analysis=previous_analysis).to_xml() + b'\n' +
Guidelines(guidelines=GUIDELINES_TEMPLATE).to_xml() + b'\n' +
ResponseFormat(
response_format=json.dumps(
Response.model_json_schema(), indent=4
)
).to_xml()
).decode()
secondary_analysis_report: Response = llm.chat(vuln_specific_user_prompt, response_model=Response)
log.info("Secondary analysis complete", secondary_analysis_report=secondary_analysis_report.model_dump())
构造后的分析提示词大概是
<file_code>
<file_path>源代码路径</file_path>
<file_source>源代码内容</file_source>
</file_code>
<example_bypasses>
"../../../../etc/passwd",
"/proc/self/environ",
"data://text/plain;base64,PD9waHAgc3lzdGVtKCRfR0VUWydjbWQnXSk7Pz4=",
"file:///etc/passwd",
"C:\\win.ini"
"/?../../../../../../../etc/passwd"
</example_bypasses>
<instructions>
Combine the code in <file_code> and <context_code> then analyze the code for remotely-exploitable Local File Inclusion (LFI) vulnerabilities by following the remote user-input call chain of code.
LFI-Specific Focus Areas:
1. High-Risk Functions and Methods:
- open(), file(), io.open()
- os.path.join() for file paths
- Custom file reading functions
2. Path Traversal Opportunities:
- User-controlled file paths or names
- Dynamic inclusion of files or modules
3. File Operation Wrappers:
- Template engines with file inclusion features
- Custom file management classes
4. Indirect File Inclusion:
- Configuration file parsing
- Plugin or extension loading systems
- Log file viewers
5. Example LFI-Specific Bypass Techniques are provided in <example_bypasses></example_bypasses> tags
When analyzing, consider:
- How user input influences file paths or names
- Effectiveness of path sanitization and validation
- Potential for null byte injection or encoding tricks
- Interaction with file system access controls
</instructions>
<analysis_approach>Analysis Instructions:
1. Comprehensive Review:
- Thoroughly examine the content in <file_code>, <context_code> tags (if provided) with a focus on remotely exploitable vulnerabilities.
2. Vulnerability Scanning:
- You only care about remotely exploitable network related components and remote user input handlers.
- Identify potential entry points for vulnerabilities.
- Consider non-obvious attack vectors and edge cases.
3. Code Path Analysis:
- Very important: trace the flow of user input from remote request source to function sink.
- Examine input validation, sanitization, and encoding practices.
- Analyze how data is processed, stored, and output.
4. Security Control Analysis:
- Evaluate each security measure's implementation and effectiveness.
- Formulate potential bypass techniques, considering latest exploit methods.
5. Context-Aware Analysis:
- If this is a follow-up analysis, build upon previous findings in <previous_analysis> using the new information provided in the <context_code>.
- Request additional context code as needed to complete the analysis and you will be provided with the necessary code.
- Confirm that the requested context class or function is not already in the <context_code> tags from the user's message.
6. Final Review:
- Confirm your proof of concept (PoC) exploits bypass any security controls.
- Double-check that your JSON response is well-formed and complete.
</analysis_approach>
<previous_analysis> </previous_analysis>
<guidelines>Reporting Guidelines:
1. JSON Format:
- Provide a single, well-formed JSON report combining all findings.
- Use 'None' for any aspect of the report that you lack the necessary information for.
- Place your step-by-step analysis in the scratchpad field, before doing a final analysis in the analysis field.
2. Context Requests:
- Classes: Use ClassName1,ClassName2
- Functions: Use func_name,ClassName.method_name
- If you request ClassName, do not also request ClassName.method_name as that code will already be fetched with the ClassName request.
- Important: Do not request code from standard libraries or third-party packages. Simply use what you know about them in your analysis.
3. Vulnerability Reporting:
- Report only remotely exploitable vulnerabilities (no local access/CLI args).
- Always include at least one vulnerability_type field when requesting context.
- Provide a confidence score (0-10) and detailed justification for each vulnerability.
- If your proof of concept (PoC) exploit does not start with remote user input via remote networking calls such as remote HTTP, API, or RPC calls, set the confidence score to 6 or below.
4. Proof of Concept:
- Include a PoC exploit or detailed exploitation steps for each vulnerability.
- Ensure PoCs are specific to the analyzed code, not generic examples.
- Review the code path ofthe potential vulnerability and be sure that the PoC bypasses any security controls in the code path.
</guidelines>
<response_format>{
"$defs": {
"ContextCode": {
"properties": {
"name": {
"description": "Function or Class name",
"title": "Name",
"type": "string"
},
"reason": {
"description": "Brief reason why this function's code is needed for analysis",
"title": "Reason",
"type": "string"
},
"code_line": {
"description": "The single line of code where where this context object is referenced.",
"title": "Code Line",
"type": "string"
}
},
"required": [
"name",
"reason",
"code_line"
],
"title": "ContextCode",
"type": "object"
},
"VulnType": {
"enum": [
"LFI",
"RCE",
"SSRF",
"AFO",
"SQLI",
"XSS",
"IDOR"
],
"title": "VulnType",
"type": "string"
}
},
"properties": {
"scratchpad": {
"description": "Your step-by-step analysis process. Output in plaintext with no line breaks.",
"title": "Scratchpad",
"type": "string"
},
"analysis": {
"description": "Your final analysis. Output in plaintext with no line breaks.",
"title": "Analysis",
"type": "string"
},
"poc": {
"description": "Proof-of-concept exploit, if applicable.",
"title": "Poc",
"type": "string"
},
"confidence_score": {
"description": "0-10, where 0 is no confidence and 10 is absolute certainty because you have the entire user input to server output code path.",
"title": "Confidence Score",
"type": "integer"
},
"vulnerability_types": {
"description": "The types of identified vulnerabilities",
"items": {
"$ref": "#/$defs/VulnType"
},
"title": "Vulnerability Types",
"type": "array"
},
"context_code": {
"description": "List of context code items requested for analysis, one function or class name per item. No standard library or third-party package code.",
"items": {
"$ref": "#/\$defs/ContextCode"
},
"title": "Context Code",
"type": "array"
}
},
"required": [
"scratchpad",
"analysis",
"poc",
"confidence_score",
"vulnerability_types",
"context_code"
],
"title": "Response",
"type": "object"
}</response_format>
这段提示词一共是 6 部分 分别是
- FileCode：源代码信息 路径加上内容
- ExampleBypasses：Bypass Payload 例子
- Instructions：分析指令提示词
- AnalysisApproach：分析方法提示词
- PreviousAnalysis：上一次分析得到的信息
- Guidelines: 分析指南提示词
- ResponseFormat： 响应格式
与代码文件分析的区别为多了一个 ExampleBypasses Bypass Payload 例子 然后分析指令提示词变成了指定的漏洞分析指令提示词
之后的迭代分析
if i > 0:
previous_context_amount = len(stored_code_definitions) #统计一共存储的 弱点代码
previous_analysis = secondary_analysis_report.analysis # 存储上一次的迭代分析得到的分析结果到previous_analysis
for context_item in secondary_analysis_report.context_code:
# Make sure bot isn't requesting the same code multiple times
if context_item.name not in stored_code_definitions: # 将新命中的代码存储到 stored_code_definitions
name = context_item.name
code_line = context_item.code_line
match = code_extractor.extract(name, code_line, files) # 使用code提取器从项目中提取
if match:
stored_code_definitions[name] = match
code_definitions = list(stored_code_definitions.values())
definitions = CodeDefinitions(definitions=code_definitions)
之后的迭代分析 会
统计一共存储的 弱点代码
存储第一次的迭代分析得到的分析结果到 previous_analysis
并将新命中的代码存储到 stored_code_definitions
然后会把上一次的迭代分析得到的分析结果 作为上下文填充到 分析提示词的 previous_analysis 中 继续和第一次一样的分析
一共会分析 7 轮
当 分析报告中命中的代码是否存在
结束跳出
每次分析完后会检查 分析报告中命中的代码是否存在 不存在则跳过该弱点的二次分析
if not len(secondary_analysis_report.context_code):
log.debug("No new context functions or classes found")
if args.verbosity == 0:
print_readable(secondary_analysis_report)
break
第二次迭代分析之后的每次迭代分析会检查 是否出现了新的命中 或是没有出现新的命中 当出现两次(出现了新的命中 或是没有出现新的命中) 则跳出
# Check if any new context code is requested
if previous_context_amount >= len(stored_code_definitions) and i > 0:
# Let it request the same context once, then on the second time it requests the same context, break
if same_context:
log.debug("No new context functions or classes requested")
if args.verbosity == 0:
print_readable(secondary_analysis_report)
break
same_context = True
log.debug("No new context functions or classes requested")
结语
有很多prompt 没有贴出来推荐大家去看原项目的prompt 文件 该项目支持 常见 Python Web 框架的代码审查
- Async
- Gradio
- Flask
- FastAPI
- Django
- Pyramid
- Bottle
- Tornado
- WebSockets
- aiohttp
- Sanic
- Falcon
- CherryPy
- web2py
- Quart (ASGI version of Flask)
- Starlette (which FastAPI is based on)
- Responder
- Hug
- Dash (for analytical web applications)
- GraphQL entry points
- Generic decorators that might indicate custom routing
- AWS Lambda handlers (which could be used with API Gateway)
- Azure Functions
- Google Cloud Functions
- Server startup code 漏洞仅支持 LFI RCE SSRF AFO SQLI XSS IDOR
刚开始读的时候就觉得一开始 就让 LLM 一步检查整个文件中存在哪些漏洞 不太合适很容易出现错误 导致真正存在的问题未被发现 应该是每个漏洞检查一遍每个问题这样效果更好 可能是为了节省 token 提升效率吧
只支持指定的语言和 web 框架 和 部分类型的漏洞感觉还是太有限了 不过这个项目已经发现了一些主流框架的漏洞了(不知道真的假了) 已经很厉害了
本来准备在dify 上复刻一下这个项目的 弄了好久 llm输出不能直接解析成json 每次还有误差 算了 能写代码一定不用这种低代码工具
附录
参考
https://not-only-security.pages.dev/posts/llm-detection-proxy/
版权信息
本文原载于 not only security，复制请保留原文出处。