一款漏洞库批量下载更新工具，便于在离线情况下漏洞的访问检索

工具介绍

本项目是一款漏洞库知识库的批量下载更新脚本，主要收集了GitHub上的一些优质漏洞库知识库，并提供一键下载及更新功能，便于在离线情况下的访问检索，适用于Windows平台。

目前收录漏洞库

• • eeeeeeeeee-code/POC(https://github.com/eeeeeeeeee-code/POC)

• • vulhub/vulhub(https://github.com/vulhub/vulhub)

• • Threekiii/Awesome-POC(https://github.com/Threekiii/Awesome-POC)

• • Mr-xn/Penetration_Testing_POC(https://github.com/Mr-xn/Penetration_Testing_POC)

• • ax1sX/SecurityList(https://github.com/ax1sX/SecurityList)

• • XiaomingX/data-cve-poc(https://github.com/XiaomingX/data-cve-poc)

• • trickest/cve(https://github.com/trickest/cve)

• • adysec/nuclei_poc(https://github.com/adysec/nuclei_poc)

• • coffeehb/Some-PoC-oR-ExP(https://github.com/coffeehb/Some-PoC-oR-ExP)

• • ycdxsb/PocOrExp_in_Github(https://github.com/ycdxsb/PocOrExp_in_Github)

• • zhzyker/exphub(https://github.com/zhzyker/exphub)

使用说明

usage: run.py -h --proxy PROXY --mode {ssh,git} --urlpath URLPATH {init,upgrade}

Bulk pull and update vulnerability databases

positional arguments:
  {init,upgrade}        需要进行的操作，'init'或'upgrade'
                          The operation that needs to be done, init or upgrade

options:
  -h, --help            show this help message and exit
  --proxy PROXY, -P PROXY
                        代理地址，默认为空
                        Proxy address, default is empty
  --mode {ssh,git}, -m {ssh,git}
                        拉取模式，'ssh'或'git'，默认为'git'
                        Pull mode, 'ssh' or 'git', defaults to 'git'
  --urlpath URLPATH, -p URLPATH
                        url文件路径，默认为'./urls.txt'
                        URL file path, which defaults to './urls.txt'

​ 首次使用该项目，请先执行以下命令拉取项目进行初始化：

python run.py init

​ 此后更新项目只需执行以下命令即可：

python run.py upgrade

​ 为避免网络不稳定或项目过大导致的无法正常下载，建议使用’ssh’的模式拉取项目的，使用’ssh’模式拉取项目请先完成’ssh’配置。

​ GitHub通过ssh方法下载详细配置过程(https://blog.csdn.net/boybs/article/details/124222148)

​ 本项目预置默认库全部拉取下来预计需要15+GB左右的空间。

run.py

import subprocess
import threading
import platform
import argparse

def check_platform():
    if platform.system() == "Windows":
        return True
    else:
        print(f"Sorry, {platform.system()} platform is not supported at the moment...")
        return False

def threaded_function(command, thread_name):
    print(f"Thread {thread_name} start...")
    subprocess.Popen(f'start cmd.exe /K "{command}"', shell=True)
    print(f"Thread {thread_name} end...")

def add_suffix(s) -> str:
    if s[-4:] != ".git":
        s += ".git"        
    return s

def ssh_mode(s):
    s = add_suffix(s)
    s = s.replace("https://github.com/", "git@github.com:")
    return s

def git_mode(s) -> str:
    s = add_suffix(s)
    s = s.replace("git@github.com:", "https://github.com/")
    return s

def get_repo_name(url) -> str:
    last_part = url.split('/')[-1]
    
    if last_part[-4:] == ".git":
        last_part = last_part[:-4]
    return last_part

def import_urls(path, mode) -> list:
    urls = []

    with open(path, "r") as f:
        data = f.readlines()
        for url in data:
            url = url.strip()
            if mode == "ssh":
                url = ssh_mode(url)
            elif mode == "git":
                url = git_mode(url)
            else:
                print(f"\033[31m[!] Error Mode: {mode} !\033[0m")
                return []
            
            urls.append(url)

    return urls

def run(commands):
    threads = []
    for i, cmd in enumerate(commands):
        thread = threading.Thread(target=threaded_function, args=(cmd, f'Thread-{i+1}'))
        threads.append(thread)
        thread.start()

    for thread in threads:
        thread.join()


if not check_platform():
    exit(1)

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Bulk pull and update vulnerability databases")
    parser.add_argument('operate', choices=['init', 'upgrade'], type=str, help="The operation that needs to be done, init or upgrade")
    parser.add_argument('--proxy', '-P', type=str, default="", help="Proxy address, default is empty")
    parser.add_argument('--mode', '-m', type=str, choices=['ssh', 'git'], default="git", help="Pull mode, 'ssh' or 'git', defaults to 'git'")
    parser.add_argument('--urlpath', '-p', type=str, default="./urls.txt", help="URL file path, which defaults to './urls.txt'")

    args = parser.parse_args()

    commands = []

    urls = import_urls(args.urlpath, args.mode)
    for url in urls:
        command = "git config --global core.autocrlf input && "

        if args.operate == "init":
            command += f"git {'' if args.proxy == '' else f'-c http.proxy={args.proxy}'} clone {url} {get_repo_name(url)}"
        elif args.operate == "upgrade":
            command += f"cd {get_repo_name(url)} && git {'' if args.proxy == '' else f'-c http.proxy={args.proxy}'} pull"
        
        commands.append(command)

    run(commands)

其他（Other）

​ Q：如何增加或删除下载库

​ A：在urls.txt文件中添加项目链接即可