Lian
文章目录 站点概览

Lian

make thing better and simpler.

103 文章
63 标签

网络管理 - 抓包/梯子

发布于 2025-08-07 | 标签: network | 20分钟 | 4164字数

img

一. 前言

本文作为 小型工控机-软路由配置拾遗 | Lian 部分内容的补充.

二. 抓包

当然这里的抓包主要满足的是简单需求, 如手机端app, 各平台浏览器等请求分析, 不需要用到Wireshark这种大杀器.

2.1 Fiddler

  • 免费
  • 易用

Web Debugging Proxy Tool | Fiddler Classic

一直用的是fiddler classic这个版本, 但是这个软件被出售后, 已经停止开发, 对于新的特性支持不会再有, 很多问题, 主要是这两个问题.

  • 解析https流量各种报错, 尽管已经安装了根证书.
  • 不支持http2协议

2.1.1 Fiddler Anywhere

目前替代产品为: Network Traffic Debugging Tool for macOS, Linux and Windows | Fiddler Everywhere, 需要付费, 52有大佬给出了破解工具(针对的6.4版本), 对于商用还是建议支持一下正版.

The Fiddler Everywhere 3.0 release is one we made a big splash about, mainly with the news about it now supporting HTTP/2. By supporting HTTP/2, Fiddler Everywhere allows developers to debug applications across Windows, macOS and Linux without the need for a complicated setup.

How to Inspect HTTP/2 Traffic in Fiddler Everywhere

img

img

顺带修改版本号, 防止自动升级, 导致破解失效.

img

打开软件, 特意引导安装ca

img

默认端口, 从之前的8888改成8866, 默认也没有修改系统代理.

img

操作界面相比于之前的版本更为细腻了.

img

https/http2的支持.

需要注意检索

img

默认状态下, 是对内容不进行解码, 需要点击这个小点, 才能全部解码内容, 对于试图通过明文检索来查找内容, 需要这个操作后才能进行检索.

2.2 Mitproxy

mitmproxy is a free and open source interactive HTTPS proxy.

mitmproxy - an interactive HTTPS proxy, 顾名思义: mitm + prxoy, Man-in-the-MiddleAttack

支持python, 只意味着其可玩性非常强, 例如, 直接劫持网络流量, 修改请求或者返回的内容.

2.2.1 安装

img

假如没有python环境的, 可以下载安装包, 就是将python的包打包成了一个exe程序, 安装包的交互和直接pip安装的一样.

pip install mitmproxy

mitmweb

将会启动一个web, 注意监听的端口, 默认还是8080, 不要同时启动命令行.

结束运行和其他的python web服务类似, ctrl + c.

img

mitmproxy

命令行下, 在powershell下的界面同样具有交互性

img

当点击相应的内容

img

会进入更为详细的模式, esc返回主界面, 连续按下, 将退出程序.

至于python下如何调用, 这里不展开讲了, 需要的可以查看文档.

2.2.2 拦截修改相应内容

例如, 拦截修改B站首页的api数据

img

from mitmproxy import http
import json

target_url = "https://api.bilibili.com/x/web-interface/wbi/index/top/feed/rcmd"

def response(flow: http.HTTPFlow) -> None:
    if target_url in flow.request.url:
        try:
            original_data = json.loads(flow.response.text)

            if original_data.get("code") == 0 and "data" in original_data:

                if "item" in original_data["data"]:
                    for item in original_data["data"]["item"]:
                        if "title" in item:
                            item["title"] = f"data_changed_{item['title']}"

                # 修改返回的视频数量( 只保留前3个) 
                original_data["data"]["item"] = original_data["data"]["item"][:3]

                original_data["data"]["modified_by_mitmproxy"] = True
                original_data["message"] = "response has been changed by mitmproxy"

            flow.response.text = json.dumps(original_data, ensure_ascii=False)
            print("data have been changed")
        except json.JSONDecodeError:
            print("fail to decode")
        except Exception as e:
            print(f"some error: {str(e)}")

使用mitmdump将脚本加载进去

mitmdump -s 21.py
#   --scripts, -s SCRIPT  Execute a script. May be passed multiple times.

# 只显示bili的内容
mitmdump -s script.py "~u bilibili.com"

当然拦截修改请求内容也是类似的.

假如共享网络, 那么就可以对局域网内的内容进行精确的控制...可玩性非常强.

(proxy_env) PS D:\Code\python_workspace\test> mitmdump --help
usage: mitmdump [options] [filter]

positional arguments:
  filter_args           Filter expression, equivalent to setting both the view_filter and
                        save_stream_filter options.

options:
  -h, --help            show this help message and exit
  --version             show version number and exit
  --options             Show all options and their default values
  --commands            Show all commands and their signatures
  --set option[=value]  Set an option. When the value is omitted, booleans are set to true,
                        strings and integers are set to None (if permitted), and sequences are
                        emptied. Boolean values can be true, false or toggle. Sequences are set
                        using multiple invocations to set for the same option.
  -q, --quiet           Quiet.
  -v, --verbose         Increase log verbosity.
  --mode, -m MODE       The proxy server type(s) to spawn. Can be passed multiple times.
                        Mitmproxy supports "regular" (HTTP), "transparent", "socks5",
                        "reverse:SPEC", "upstream:SPEC", and "wireguard[:PATH]" proxy servers.
                        For reverse and upstream proxy modes, SPEC is host specification in the
                        form of "http[s]://host[:port]". For WireGuard mode, PATH may point to a
                        file containing key material. If no such file exists, it will be created
                        on startup. You may append `@listen_port` or `@listen_host:listen_port`
                        to override `listen_host` or `listen_port` for a specific proxy mode.
                        Features such as client playback will use the first mode to determine
                        which upstream server to use. May be passed multiple times.
  --no-anticache
  --anticache           Strip out request headers that might cause the server to return 304-not-
                        modified.
  --no-showhost
  --showhost            Use the Host header to construct URLs for display. This option is
                        disabled by default because malicious apps may send misleading host
                        headers to evade your analysis. If this is not a concern, enable this
                        options for better flow display.
  --no-show-ignored-hosts
  --show-ignored-hosts  Record ignored flows in the UI even if we do not perform TLS
                        interception. This option will keep ignored flows' contents in memory,
                        which can greatly increase memory usage. A future release will fix this
                        issue, record ignored flows by default, and remove this option.
  --rfile, -r PATH      Read flows from file.
  --scripts, -s SCRIPT  Execute a script. May be passed multiple times.
  --stickycookie FILTER
                        Set sticky cookie filter. Matched against requests.
  --stickyauth FILTER   Set sticky auth filter. Matched against requests.
  --save-stream-file, -w PATH
                        Stream flows to file as they arrive. Prefix path with + to append. The
                        full path can use python strftime() formating, missing directories are
                        created as needed. A new file is opened every time the formatted string
                        changes.
  --no-anticomp
  --anticomp            Try to convince servers to send us un-compressed data.
  --flow-detail LEVEL   The display detail level for flows in mitmdump: 0 (quiet) to 4 (very
                        verbose). 0: no output 1: shortened request URL with response status code
                        2: full request URL with response status code and HTTP headers 3: 2 +
                        truncated response content, content of WebSocket and TCP messages
                        (content_view_lines_cutoff: 512) 4: 3 + nothing is truncated

Proxy Options:
  --listen-host HOST    Address to bind proxy server(s) to (may be overridden for individual
                        modes, see `mode`).
  --listen-port, -p PORT
                        Port to bind proxy server(s) to (may be overridden for individual modes,
                        see `mode`). By default, the port is mode-specific. The default regular
                        HTTP proxy spawns on port 8080.
  --no-server, -n
  --server              Start a proxy server. Enabled by default.
  --ignore-hosts HOST   Ignore host and forward all traffic without processing it. In transparent
                        mode, it is recommended to use an IP address (range), not the hostname.
                        In regular mode, only SSL traffic is ignored and the hostname should be
                        used. The supplied value is interpreted as a regular expression and
                        matched on the ip or the hostname. May be passed multiple times.
  --allow-hosts HOST    Opposite of --ignore-hosts. May be passed multiple times.
  --tcp-hosts HOST      Generic TCP SSL proxy mode for all hosts that match the pattern. Similar
                        to --ignore-hosts, but SSL connections are intercepted. The communication
                        contents are printed to the log in verbose mode. May be passed multiple
                        times.
  --upstream-auth USER:PASS
                        Add HTTP Basic authentication to upstream proxy and reverse proxy
                        requests. Format: username:password.
  --proxyauth SPEC      Require proxy authentication. Format: "username:pass", "any" to accept
                        any user/pass combination, "@path" to use an Apache htpasswd file, or "ld
                        ap[s]:url_server_ldap[:port]:dn_auth:password:dn_subtree[?search_filter_k
                        ey=...]" for LDAP authentication.
  --no-store-streamed-bodies
  --store-streamed-bodies
                        Store HTTP request and response bodies when streamed (see
                        `stream_large_bodies`). This increases memory consumption, but makes it
                        possible to inspect streamed bodies.
  --no-rawtcp
  --rawtcp              Enable/disable raw TCP connections. TCP connections are enabled by
                        default.
  --no-http2
  --http2               Enable/disable HTTP/2 support. HTTP/2 support is enabled by default.

SSL:
  --certs SPEC          SSL certificates of the form "[domain=]path". The domain may include a
                        wildcard, and is equal to "*" if not specified. The file at path is a
                        certificate in PEM format. If a private key is included in the PEM, it is
                        used, else the default key in the conf dir is used. The PEM file should
                        contain the full certificate chain, with the leaf certificate as the
                        first entry. May be passed multiple times.
  --cert-passphrase PASS
                        Passphrase for decrypting the private key provided in the --cert option.
                        Note that passing cert_passphrase on the command line makes your
                        passphrase visible in your system's process list. Specify it in
                        config.yaml to avoid this.
  --no-ssl-insecure
  --ssl-insecure, -k    Do not verify upstream server SSL/TLS certificates. If this option is
                        enabled, certificate validation is skipped and mitmproxy itself will be
                        vulnerable to TLS interception.

Client Replay:
  --client-replay, -C PATH
                        Replay client requests from a saved file. May be passed multiple times.

Server Replay:
  --server-replay, -S PATH
                        Replay server responses from a saved file. May be passed multiple times.
  --no-server-replay-kill-extra
  --server-replay-kill-extra
                        Kill extra requests during replay (for which no replayable response was
                        found).[Deprecated, prefer to use server_replay_extra='kill']
  --server-replay-extra {forward,kill,204,400,404,500}
                        Behaviour for extra requests during replay for which no replayable
                        response was found. Setting a numeric string value will return an empty
                        HTTP response with the respective status code.
  --no-server-replay-reuse
  --server-replay-reuse
                        Don't remove flows from server replay state after use. This makes it
                        possible to replay same response multiple times.
  --no-server-replay-refresh
  --server-replay-refresh
                        Refresh server replay responses by adjusting date, expires and last-
                        modified headers, as well as adjusting cookie expiration.

Map Remote:
  --map-remote, -M PATTERN
                        Map remote resources to another remote URL using a pattern of the form
                        "[/flow-filter]/url-regex/replacement", where the separator can be any
                        character. May be passed multiple times.

Map Local:
  --map-local PATTERN   Map remote resources to a local file using a pattern of the form "[/flow-
                        filter]/url-regex/file-or-directory-path", where the separator can be any
                        character. May be passed multiple times.

Modify Body:
  --modify-body, -B PATTERN
                        Replacement pattern of the form "[/flow-filter]/regex/[@]replacement",
                        where the separator can be any character. The @ allows to provide a file
                        path that is used to read the replacement string. May be passed multiple
                        times.

Modify Headers:
  --modify-headers, -H PATTERN
                        Header modify pattern of the form "[/flow-filter]/header-name/[@]header-
                        value", where the separator can be any character. The @ allows to provide
                        a file path that is used to read the header value string. An empty
                        header-value removes existing header-name headers. May be passed multiple
                        times.

2.2.3 小结

对于习惯了UI交互的, mitmproxy使用起来可能不会那么习惯, 但是由于支持python脚本, 使用起来很是方便.

需要注意, mitmproxy并不会直接修改系统代理, 需要手动接入需要监听的软件, 如浏览器需要手动设置代理.

2.3 外部设备接入

外部设备接入和简单, 和正常的网络设置一般, 将代理设置为当前抓包软件所在主机的IP和端口即可, 其问题在于解析https流量, 以iPhone为例:

在iPhone上安装ca, 将iPhone上的网络代理设置好:

  • Fiddler, 访问对应的连接, 如192.168.1.108:8866, 即出现一个证书的界面, 点击证书会跳转到设置,
  • mitmproxy, 则是在设置好代理网络后, 访问http://mitm.it/, 必须是在代理网络下访问, 否则不会出现ca下载的界面.

img

安装好之后, 需要注意高版本的ios需要进入设置-关于中启用ca信任, 才会真正启用生效.

三. 梯子

可以说当下, 没有梯子寸步难行, 内外封锁, 简中已经沦为各个企业网, 外部CN IP狗嫌弃.

Clash与V2RayN全面对比: 选择适合你的网络代理工具

3.1 clash

img

如果说这个软件的最大特色, 纯傻瓜化, 可以自动切换节点, 自动ping检查延迟等, 但目前很多机场现在对这个软件的支持不好.

3.2 V2rayN

目前主力使用, 稍微记录一下使用细节, 一些简单的操作就略过.

总体操作也很简单, 但没有clash的操作界面这么友好, 换言之就是菜单栏挺多的, 菜单多的好处在于可以更为的精细控制.

img

甚至需要手动测试服务器是否可用

支持三种设置代理的方式

  • 系统代理
  • pac模式
  • 手动接入代理(和上述的抓包软件一样)

img

上述两种, 在系统中对应如下:

/pac?t=638901829927855034

数字后缀, 每次都会发生变化.

img

上述两种设置后, 均会直接接管网络接入, 区别在于系统设置是全局的, pac设置后, 浏览器不需要做其他设置即可正常使用梯子.

需要注意的是pac模式

代理自动配置文件( PAC) 文件 - HTTP | MDN

代理自动配置( PAC) 文件是一个 JavaScript 脚本, 其核心是一个 JavaScript 函数, 用来决定网页浏览请求( HTTP, HTTPS, 和 FTP) 应当直连目标地址, 还是被转发给一个网页代理服务器并通过代理连接

img

实际上就是个js脚本, 即根据定义的规则细化对于网络的请求

img

例如这个bilibili.tv在预置的规则名单之上, 走梯子流量.

这和路由支持的特性类似:

【以下内容更新于2024年2月】2024年目前的软件版本已经到了6.X版本, 增加了" 路由" 选项, 包括" 绕过大陆( Whitelist) " , " 黑名单( Blacklist) " 和" 全局( Global) " 三个选项. 这3个选项的功能如下:

  • 绕过大陆( Whitelist) : 只是白名单内的网站通过 V2Ray 代理上网
  • 黑名单( Blacklist) : 除了黑名单内的网站, 其余网站都通过 V2Ray 代理上网
  • 全局( Global) : 所有网站通过 V2Ray 代理上网

6.x版本" PAC模式" 和" 自动配置系统代理" 有什么区别

但是需要注意的是pac模式不支持直接在其它设备上直接使用这个脚本

img

原有的端口依然保持监听状态, 并不会因为切换到pac模式而关闭, 其他设备依然可以正常使用这个端口连接梯子.

3.2.1 小结

上述的各种设置, 其目的旨在区分请求的网站, 哪些需要梯子, 哪些需要直连.

需要注意, v2rayN Tun模式详解 | RULTR

img

该设置直接对Windows的hyperV虚拟机造成影响

img

会新增一个虚拟网卡

img

新网卡的属性

3.3 浏览器代理设置

很多时候, 并不希望开启全局的梯子, 一般都是需要用到梯子的软件单独设置, 在其中浏览器是最常需要用到梯子的软件.

对于chromium系的设置很简单, 在浏览器的快捷键上添加一个后缀参数即可

img

--proxy-server="http://127.0.0.1:10808"

3.3.1 edge的坑

Microsoft Edge 代理设置 | Microsoft Learn

但是需要注意, 这里大概率会踩进一个坑, 就是在设置edge的代理

img

假如检索chrome相关设置, 这是很常见的设置方法.

img

但是edge不然, 大部分的结果都是对其文档的转载, 也许很多人都尝试过这样设置, 结果不起效, 所以很少将这样的信息刊载出来.

edge, 假如不进行特定设置, 在关闭edge后是无法彻底关闭程序的, 还有好几个后台进程

img

如果不注意这个细节, 会发现设置死活不生效.

设置好之后, 必须将浏览器彻底关闭, 重新打开, 代理设置才会生效.

3.3.2 扩展

由于全局设置有时候还是不满足需求, 同时, 使用上也不是很灵活, 浏览器扩展可以进一步解决实际的需求.

img

img

canbjhbbhfggbdfgpddpnckdjgfcbnpb, 以这个扩展为例, 类似的扩展在有很多.

3.4 python中的使用

需要注意的是在python中调用代理.

img

在支持的协议中, 需要稍微注意一下: sock5

SOCKS5 是一个代理协议, 它在使用TCP/IP协议通讯的前端机器和服务器机器之间扮演一个中介角色, 使得内部网中的前端机器变得能够访问Internet网中的服务器, 或者使通讯更加安全. SOCKS5 服务器通过将前端发来的请求转发给真正的目标服务器, 模拟了一个前端的行为. 在这里, 前端和SOCKS5之间也是通过TCP/IP协议进行通讯, 前端将原本要发送给真正服务器的请求发送给SOCKS5服务器, 然后SOCKS5服务器将请求转发给真正的服务器.

SOCKS5 协议原理详解与应用场景分析 - chris599 - 博客园

一口气搞明白有点奇怪的 Socks 5 协议以及 HTTP 代理

SOCKS vs HTTP Proxy: What Is the Difference?

维度 HTTP 代理( http://) HTTPS 代理( https://) SOCKS5 代理( socks5://)
工作层面 应用层( 仅理解 HTTP) 应用层( 优化 HTTPS 处理) 传输层( 不依赖应用协议)
支持的流量类型 仅 HTTP HTTP + HTTPS( 加密流量) 几乎所有 TCP/UDP 协议( 通用)
速度 中等( 需解析 HTTP 内容) 中等( 需处理加密逻辑) 较快( 仅转发数据, 不解析)
典型用途 老旧 HTTP 网站代理 现代 HTTPS 网站代理 多协议代理( 网页, 游戏, SSH 等)

在浏览器

"C:\Program Files\Google\Chrome\Application\chrome.exe" --proxy-server="socks5://127.0.0.1:10808"

"C:\Program Files\Google\Chrome\Application\chrome.exe" --proxy-server="http://127.0.0.1:10808"

在代码中

import requests

proxies = {
    "http": "socks5://127.0.0.1:10808",
    "https": "socks5://127.0.0.1:10808"
}
# or
proxies = {
#     "http": "http://127.0.0.1:10808",
#     "https": "http://127.0.0.1:10808"
# }
response = requests.get(
    'https://www.google.com/search',
    params=params,
    cookies=cookies,
    headers=headers,
    proxies=proxies,
)

四. one more thing

img

当梯子和本地抓包同时启动时, 该如何处理?

img

# network
广告漏网之鱼的背后 上一篇
FastAPI使用基础-性能篇 下一篇
已运行:
浏览数: 次 | 访客数: 人
Powered by Kyouichirou