Compare commits

...
Author SHA1 Message Date
patternihaandClaude Opus 5.5 f5cc086d6c TUN inbound: README.md says "ipv4:allexceptlan" includes "ipv4:dns" only
Add "dns" to block DNS over IPv6 as well.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 18:10:11 +03:30
patternihaandClaude Opus 5.5 6b45eadf45 TUN inbound: Recommend at least "dns" for autoSystemWfpBlockLeak in README.md, and shorten it
README.md now recommends at least "dns" to block DNS leaks, and lists the
values of autoSystemWfpBlockLeak more briefly, "dns" first. It also lists
FreeBSD among the systems, and mentions that on Linux, as on Android, the
TUN can be a file descriptor from xray.tun.fd (XRAY_TUN_FD).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 17:53:27 +03:30
patternihaandClaude Opus 5.5 c684201cd8 TUN inbound's autoSystemWfpBlockLeak: "ipv4:" or "ipv6:" in front of any value instead of "tcpv4" and the like
"ipv4:" or "ipv6:" in front of any value limits it to that IP version,
e.g. "ipv4:udp/53", "ipv6:all", "ipv4:dns" or "ipv6:misconfigtun".
"tcpv4", "udpv6", "icmpv4" and the like are gone. Values that only differ
in the IP version become one: "ipv4:udp/53" with "ipv6:udp/53" is
"udp/53".

"all" overrides "allexceptlan" per IP version, and "all" over one IP
version needs dns, as port 53 over one does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 17:52:23 +03:30
patternihaandClaude Opus 5.5 2218fec173 TUN inbound's autoSystemWfpBlockLeak: Add "tcpv4", "tcpv6", "udpv4", "udpv6", "icmpv4" and "icmpv6"
"v4" or "v6" after the protocol limits a value to one IP version, ports
written the same way, e.g. "tcpv4/853" or "udpv6/10-20"; "icmpv4" is ICMP
and "icmpv6" is ICMPv6. Their filters go to the ALE layers of that version
only. "tcp", "udp" and "icmp" keep covering both.

Anything covering TCP or UDP port 53 in either IP version, like
"udpv4/53", still needs dns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 13:09:55 +03:30
patternihaandClaude Opus 5.5 9ffa0f0552 TUN inbound's autoSystemWfpBlockLeak: Drop "/0"; Ports have to be 1-65535
"tcp/0" and "udp/0" no longer stand for all ports, which "tcp" and "udp"
already say: no syntactic sugar. Port 0 is rejected like any port outside
1-65535. Ranges keep being checked strictly: both ends within 1-65535, the
left one not greater than the right one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 12:49:56 +03:30
patternihaandClaude Opus 5.5 2933b0c8d5 TUN inbound: Don't warn that loopback DNS servers are blocked
With autoSystemWfpBlockLeak, loopback now always stays allowed, so a DNS
server in dns like 127.0.0.1, a local resolver, stays reachable although
it lies in neither gateway nor autoSystemRoutingTable. Leave loopback
servers out of the warning that queries to them cannot go through the TUN
and are blocked, and say so in the README.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 12:27:26 +03:30
patternihaandClaude Opus 5.5 bb4cd4b6d3 TUN inbound's autoSystemWfpBlockLeak: Add "allexceptlan"
"allexceptlan" is "all" but for the local network: traffic to and from
private (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, fc00::/7), link-local
(169.254.0.0/16, fe80::/10), multicast (224.0.0.0/4, ff00::/8) and broadcast
(255.255.255.255) addresses stays allowed, so that printers, the router's
settings page, or casting to a TV keep working while nothing else leaves
outside the TUN.

It includes "dns": a resolver on the local network, like the router's,
would forward the queries outside the TUN, so DNS stays blocked there. The
LAN permit sits below the other values, like the permit through the TUN, so
they still apply on the local network, e.g. "tcp/445" blocks Windows file
sharing with it. The DNS Client service keeps mDNS and LLMNR, to the local
network's addresses only. "all" overrides it.

The addresses are matched with FWP_V4_ADDR_MASK and FWP_V6_ADDR_MASK
conditions on the remote address, which match when any of them does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 11:50:18 +03:30
patternihaandClaude Opus 5.5 468f745fc2 TUN inbound's autoSystemWfpBlockLeak: Add "all", "tcp", "udp", "icmp" and port ranges; Always allow loopback, DHCP and IPv6 neighbor discovery
autoSystemWfpBlockLeak now also takes:

- "all": everything outside the TUN, IPv4 and IPv6, in both directions, so
  nothing but Xray leaves except through the TUN.
- "tcp", "udp": all ports of that protocol, also written "tcp/0", "udp/0".
- "tcp/<low>-<high>", "udp/<low>-<high>": a range of ports, matched with
  FWP_MATCH_RANGE, as two conditions on the same field would match when
  either does.
- "icmp": ICMP and ICMPv6, e.g. ping. It takes no port: in ICMP filters the
  local port holds the type, and "icmp/0" would read as the echo reply.

Values are normalized: "tcp/0" and "tcp/1-65535" become "tcp", and
"tcp/53-53" becomes "tcp/53".

Whatever is blocked, loopback, which cannot leave the system, and what
Windows itself needs on the local link now always stay allowed: DHCP, DHCPv6,
and IPv6 neighbor and multicast listener discovery. These used to be allowed
only along with "misconfigtun", so "udp" or "icmp" would otherwise have cut
DHCP or IPv6 neighbor discovery, and "tcp" any local proxy, Xray's own
inbounds included. Loopback being allowed means that with "dns", a local
resolver on 127.0.0.1:53 now works for other programs (its own queries to
port 53 or 853 outside the TUN stay blocked), and the DNS Client service
may use it.

"all" reuses the filters of "misconfigtun", which now also let everything
through the TUN, for both IP versions whatever the routes.

On other TUN options: keeping the system's DNS inside the TUN, with "all",
"dns", "windnsclient", or anything covering TCP or UDP port 53 ("udp",
"tcp/50-60", ...), needs dns, as Windows can no longer reach the DNS
servers of the other interfaces; the config is rejected otherwise. Like the
other values, the new ones need autoSystemRoutingTable. The filters do not
need autoOutboundsInterface, as Xray is allowed by its app ID; it still
keeps Xray's own connections out of the TUN's routes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 11:13:10 +03:30
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
7c8dbcb4bb Bump google.golang.org/grpc from 1.84.0 to 1.85.0 (#7311)
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.84.0 to 1.85.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.84.0...v1.85.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.85.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-02 02:03:49 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
8ab161a2b0 Bump golang.org/x/net from 0.61.0 to 0.62.0 (#7312)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.61.0 to 0.62.0.
- [Commits](https://github.com/golang/net/compare/v0.61.0...v0.62.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.62.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-02 02:03:35 +00:00
RPRXandGitHub a4a3dff5b2 Xray-core v26.10.31
Sponsor & Donation & NFTs: https://github.com/XTLS/Xray-core/issues/3668
Project X Channel: https://t.me/projectXtls

Announcement of NFTs by Project X: https://github.com/XTLS/Xray-core/discussions/3633
Project X NFT: https://opensea.io/assets/ethereum/0x5ee362866001613093361eb8569d59c4141b76d1/1

VLESS Post-Quantum Encryption: https://github.com/XTLS/Xray-core/pull/5067
VLESS NFT: https://opensea.io/collection/vless

XHTTP: Beyond REALITY: https://github.com/XTLS/Xray-core/discussions/4113
REALITY NFT: https://opensea.io/assets/ethereum/0x5ee362866001613093361eb8569d59c4141b76d1/2
2026-10-31 13:20:19 +00:00
6cdd323ad8 TUN inbound's autoSystemWfpBlockLeak: Support "tcp/port" and "udp/port"; "dns" covers "tcp/853" and "udp/853" as well (#7288)
https://github.com/XTLS/Xray-core/pull/6853#issuecomment-6492838227
https://github.com/XTLS/Xray-core/pull/7288#issuecomment-6507229727
https://github.com/XTLS/Xray-core/pull/7288#issuecomment-6516762634

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 13:16:31 +00:00
316e07a538 XHTTP server: Add h2Params (#7295)
https://github.com/XTLS/Xray-core/pull/7282#issuecomment-6523157944
https://github.com/XTLS/Xray-core/pull/7295#issuecomment-6543501751

Closes https://github.com/XTLS/Xray-core/issues/6268

---------

Co-authored-by: pqhaz3925 <195911022+pqhaz3925@users.noreply.github.com>
2026-10-31 12:52:14 +00:00
MeowandRPRX 0c3402d778 Geodata: Support streaming gzip decompression when downloading assets (#7294)
https://github.com/XTLS/Xray-core/pull/7294#issuecomment-6542074150
2026-10-31 11:56:59 +00:00
bc7c895278 HTTP inbound: Forward short 100 Continue responses promptly (#7298)
https://github.com/XTLS/Xray-core/pull/7298#issuecomment-6542585254

---------

Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-10-31 11:29:02 +00:00
konstpicandRPRX e7a69fa85a XHTTP server: Accept OPTIONS as uplinkHTTPMethod (#7290)
https://github.com/XTLS/Xray-core/pull/7290#issuecomment-6505845186
2026-10-31 08:43:57 +00:00
9b8190f52c QUIC sniffer: Don't accept ClientHello with gaps in CRYPTO data (#7292)
https://github.com/XTLS/Xray-core/pull/7292#pullrequestreview-5712194488

---------

Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-10-31 08:31:30 +00:00
6ca6d8a693 Commands: Fix xray api bi balancer1 balancer2 (#7299)
https://github.com/XTLS/Xray-core/pull/7279#issuecomment-6468250523

---------

Co-authored-by: RPRX <63339210+RPRX@users.noreply.github.com>
2026-10-31 07:21:32 +00:00
风扇滑翔翼andGitHub d1140de273 Chore: Regenerate *.pb.go files (#7287)
https://github.com/XTLS/Xray-core/pull/7263#issuecomment-6464669635
2026-10-31 07:13:55 +00:00
风扇滑翔翼andGitHub 6ea0ab5d3a DefaultListener: Disable MPTCP by default (#7291)
Fixes https://github.com/XTLS/Xray-core/pull/7291#issuecomment-6540309985
2026-10-31 07:05:32 +00:00
RPRXandGitHub 0afec66085 Xray-core v26.10.29
Sponsor & Donation & NFTs: https://github.com/XTLS/Xray-core/issues/3668
Project X Channel: https://t.me/projectXtls

Announcement of NFTs by Project X: https://github.com/XTLS/Xray-core/discussions/3633
Project X NFT: https://opensea.io/assets/ethereum/0x5ee362866001613093361eb8569d59c4141b76d1/1

VLESS Post-Quantum Encryption: https://github.com/XTLS/Xray-core/pull/5067
VLESS NFT: https://opensea.io/collection/vless

XHTTP: Beyond REALITY: https://github.com/XTLS/Xray-core/discussions/4113
REALITY NFT: https://opensea.io/assets/ethereum/0x5ee362866001613093361eb8569d59c4141b76d1/2
2026-10-29 08:38:50 +00:00
55eecaed16 Config: Move inbound module initialization to the end (#7284)
Fixes https://github.com/XTLS/Xray-core/pull/7284#issuecomment-6491545028

---------

Co-authored-by: Copilot <copilot@github.com>
2026-10-29 08:34:16 +00:00
2dfee4b842 XDRIVE transport: Carry a Google Drive session in a ring of slots (#7217)
https://github.com/XTLS/Xray-core/pull/7217#issuecomment-6489554617

---------

Co-authored-by: Ahmad-2213 <106820058+Ahmad-2213@users.noreply.github.com>
2026-10-29 06:47:42 +00:00
Жора ЗмейкинandGitHub b4123b5748 XDRIVE transport: Add "s3" storage backend (#7231)
https://github.com/XTLS/Xray-core/pull/7231#issuecomment-6488077212
2026-10-29 04:44:41 +00:00
ec124ff9e0 WebSocket transport: Treat normal close frames as io.EOF (#7236)
Fixes https://github.com/XTLS/Xray-core/issues/7235#issuecomment-6353389899

---------

Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-10-29 03:42:11 +00:00
SolomonandGitHub 8a3c0cbf1d Sockopt: Fix swapped TCP keepalive idle and interval on Darwin (#7219)
https://github.com/XTLS/Xray-core/pull/7219#pullrequestreview-5691960668
2026-10-29 02:54:53 +00:00
78b5a6ec5c GitHub Action CI: Enforce stricter checks on generated protobuf files (#7263)
https://github.com/XTLS/Xray-core/pull/7263#issuecomment-6447862934
https://github.com/XTLS/Xray-core/pull/7263#issuecomment-6484719865
https://github.com/XTLS/Xray-core/pull/7263#issuecomment-6485805671

---------

Co-authored-by: Fangliding <Fangliding.fshxy@outlook.com>
2026-10-29 02:34:47 +00:00
5865e3ac3d DNS outbound: Keep hijacked A/AAAA replies within the buffer (#7225)
https://github.com/XTLS/Xray-core/pull/7225#pullrequestreview-5613859264

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-10-29 00:42:19 +00:00
48f62b801a Geodata: Limit download size, file extensions, and update frequency (#7267)
https://github.com/XTLS/Xray-core/pull/7265#issuecomment-6464588139

---------

Co-authored-by: xqzr <34030394+xqzr@users.noreply.github.com>
2026-10-29 00:22:51 +00:00
MeowandGitHub 43c29f8eb1 Geodata: Update built-in generalized "private" IP ranges (#7249)
And https://github.com/XTLS/Xray-core/pull/7249#issuecomment-6483478811

https://github.com/XTLS/Xray-core/pull/7249#issuecomment-6391494484
https://github.com/XTLS/Xray-core/pull/7249#issuecomment-6483591550
2026-10-28 23:45:22 +00:00
MeowandGitHub 613d8147bb Direct/Freedom outbound: Warn about migrating fragment and noises to Finalmask (#7242)
https://github.com/XTLS/Xray-core/issues/7240#issuecomment-6361862358
2026-10-28 22:49:30 +00:00
38f090255c Commands: Fix xray api adrules c1.json c2.json (#7276)
https://github.com/XTLS/Xray-core/pull/7276#pullrequestreview-5680949792

---------

Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-10-28 21:59:27 +00:00
35a3121a83 File loader: Check file extensions without resolving symlinks (#7255)
https://github.com/XTLS/Xray-core/pull/7255#issuecomment-6481194940
https://github.com/XTLS/Xray-core/pull/7250#issuecomment-6398816842

Fixes https://github.com/XTLS/Xray-core/issues/7239#issuecomment-6356122816

---------

Co-authored-by: Eljees <3.14hell@gmail.com>
2026-10-28 21:59:03 +00:00
风扇滑翔翼andGitHub bca08ac080 Routing: Fix Linux IPv6 process (#7232)
Fixes https://github.com/XTLS/Xray-core/issues/7227
2026-10-28 20:49:08 +00:00
6a4274ab75 TUN inbound: autoOutboundsInterface should not set IP_BOUND_IF on IPv6 sockets on Darwin (mainly macOS) (#7230)
Fixes https://github.com/XTLS/Xray-core/issues/7228

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 20:34:40 +00:00
solodecodeandGitHub 229799a7a3 Direct/Freedom outbound: Fix blockDelay for DispatchLink() inbounds such as VLESS, Hysteria, etc. (#7238)
https://github.com/XTLS/Xray-core/pull/7238#pullrequestreview-5627787937
https://github.com/XTLS/Xray-core/pull/7238#issuecomment-6467092152
https://github.com/XTLS/Xray-core/pull/6060#issuecomment-4384511163
2026-10-28 11:02:55 +00:00
3071fbfa25 Sockopt: Handle single-family interface binding on Windows, including TUN inbound's autoOutboundsInterface (#7214)
And https://github.com/XTLS/Xray-core/pull/7214#issuecomment-6329118637

https://github.com/XTLS/Xray-core/pull/7201#issuecomment-6314453444

Fixes https://github.com/XTLS/Xray-core/issues/7160#issuecomment-6246944012

---------

Co-authored-by: patterniha <71074308+patterniha@users.noreply.github.com>
2026-10-28 10:15:58 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1576e79563 Bump github.com/pires/go-proxyproto from 0.15.0 to 0.16.0 (#7237)
Bumps [github.com/pires/go-proxyproto](https://github.com/pires/go-proxyproto) from 0.15.0 to 0.16.0.
- [Release notes](https://github.com/pires/go-proxyproto/releases)
- [Commits](https://github.com/pires/go-proxyproto/compare/v0.15.0...v0.16.0)

---
updated-dependencies:
- dependency-name: github.com/pires/go-proxyproto
  dependency-version: 0.16.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-27 04:31:46 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
67e7f82ca1 Bump docker/login-action from 4.6.0 to 4.7.0 (#7257)
Bumps [docker/login-action](https://github.com/docker/login-action) from 4.6.0 to 4.7.0.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v4.6.0...v4.7.0)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: 4.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-27 04:31:31 +00:00
RPRXandGitHub 1b1647cd9d Xray-core v26.10.21
Sponsor & Donation & NFTs: https://github.com/XTLS/Xray-core/issues/3668
Project X Channel: https://t.me/projectXtls

Announcement of NFTs by Project X: https://github.com/XTLS/Xray-core/discussions/3633
Project X NFT: https://opensea.io/assets/ethereum/0x5ee362866001613093361eb8569d59c4141b76d1/1

VLESS Post-Quantum Encryption: https://github.com/XTLS/Xray-core/pull/5067
VLESS NFT: https://opensea.io/collection/vless

XHTTP: Beyond REALITY: https://github.com/XTLS/Xray-core/discussions/4113
REALITY NFT: https://opensea.io/assets/ethereum/0x5ee362866001613093361eb8569d59c4141b76d1/2
2026-10-21 11:59:30 +00:00
Artem LytkinandGitHub 557657b576 TUN inbound: Read large TCP segments straight into Xray-core's buffers on Linux (#7179)
https://github.com/XTLS/Xray-core/pull/7130#issuecomment-6119897360
https://github.com/XTLS/Xray-core/pull/7156#issuecomment-6205035216
https://github.com/XTLS/Xray-core/pull/7179#issuecomment-6250145934
2026-10-21 11:49:36 +00:00
Artem LytkinandGitHub fc5ba8e6a2 XDRIVE transport: Support h3 (Chrome CH by default) and XMUX as well; Send Chrome HTTP headers (User-Agent, etc.) by default (#7187)
https://github.com/XTLS/Xray-core/pull/7181#issuecomment-6229694351
https://github.com/XTLS/Xray-core/pull/7187#issuecomment-6250200781
https://github.com/XTLS/Xray-core/pull/7187#issuecomment-6314784068
2026-10-21 11:33:55 +00:00
genkauzenandRPRX 00c0277787 XDRIVE transport: Support two-step "template" operations (hrefPath) (#7188)
https://github.com/XTLS/Xray-core/pull/7188#issuecomment-6250146420
2026-10-21 10:55:36 +00:00
MeowandGitHub 3653dcb4fc Finalmask: Add filter for inbound (TCP & UDP, source IP filtering) (#7206)
https://github.com/XTLS/Xray-core/pull/7206#issuecomment-6312634417
2026-10-21 10:48:15 +00:00
061c2df291 HTTP outbound: Fix hanging on server-first traffic (#7193)
https://github.com/XTLS/Xray-core/pull/7193#issuecomment-6312885274

Fixes https://github.com/XTLS/Xray-core/issues/3459

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-10-21 10:33:27 +00:00
23e2292344 Transport: Pass UDP destination to dialer controllers (#7201)
https://github.com/XTLS/Xray-core/pull/7201#issuecomment-6275732621

---------

Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-10-21 10:20:35 +00:00
41797c234d Socks inbound: Initialize TempUDPConn's Timer at the beginning (#7203)
Fixes https://github.com/XTLS/Xray-core/pull/7200

---------

Co-authored-by: Eliot the Cougar <1861169+eliotcougar@users.noreply.github.com>
2026-10-21 09:17:34 +00:00
MeowandGitHub 9424810830 Docker: Restrict log & conf permissions (#7204)
https://github.com/XTLS/Xray-core/pull/7204#issuecomment-6292218518
2026-10-21 08:55:32 +00:00
12b7f85de3 Routing: Fix Linux UDP process (#7202)
Fixes https://github.com/XTLS/Xray-core/issues/7192 and https://github.com/XTLS/Xray-core/pull/7195

---------

Co-authored-by: Faraquic! <141701604+faraquic@users.noreply.github.com>
2026-10-21 08:35:52 +00:00
𐲓𐳛𐳪𐳂𐳐 𐲀𐳢𐳦𐳫𐳢 𐲥𐳔𐳛𐳪𐳌𐳑𐳖𐳇andGitHub de1bbbff88 GitHub Actions: Switch geodata to XTLS/Xray-rules-dat (#7185)
https://github.com/XTLS/Xray-core/pull/7185#issuecomment-6247177356
2026-10-21 08:18:32 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
ec73b73f5e Bump github.com/klauspost/cpuid/v2 from 2.4.0 to 2.4.1 (#7197)
Bumps [github.com/klauspost/cpuid/v2](https://github.com/klauspost/cpuid) from 2.4.0 to 2.4.1.
- [Release notes](https://github.com/klauspost/cpuid/releases)
- [Commits](https://github.com/klauspost/cpuid/compare/v2.4.0...v2.4.1)

---
updated-dependencies:
- dependency-name: github.com/klauspost/cpuid/v2
  dependency-version: 2.4.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-21 08:07:31 +00:00
风扇滑翔翼andRPRX 6307fa1265 TLS client: Update built-in RootCAs to remove newly low-credit certs (#7212)
Fixes https://github.com/XTLS/Xray-core/pull/7212#issuecomment-6310150214

... https://github.com/XTLS/Xray-core/pull/7105#issuecomment-6056053599

---------

Co-authored-by: RPRX <63339210+RPRX@users.noreply.github.com>
2026-10-21 07:55:00 +00:00
RPRXandGitHub 92ccc939aa Xray-core v26.10.17
Sponsor & Donation & NFTs: https://github.com/XTLS/Xray-core/issues/3668
Project X Channel: https://t.me/projectXtls

Announcement of NFTs by Project X: https://github.com/XTLS/Xray-core/discussions/3633
Project X NFT: https://opensea.io/assets/ethereum/0x5ee362866001613093361eb8569d59c4141b76d1/1

VLESS Post-Quantum Encryption: https://github.com/XTLS/Xray-core/pull/5067
VLESS NFT: https://opensea.io/collection/vless

XHTTP: Beyond REALITY: https://github.com/XTLS/Xray-core/discussions/4113
REALITY NFT: https://opensea.io/assets/ethereum/0x5ee362866001613093361eb8569d59c4141b76d1/2
2026-10-17 15:23:56 +00:00
Artem LytkinandGitHub 95461cc148 XDRIVE transport: Require VLESS Multiplex+Encryption; "Google Drive"->"google-drive" (#7182)
https://github.com/XTLS/Xray-core/pull/6748#issuecomment-5752780799
2026-10-17 14:13:57 +00:00
Artem LytkinandGitHub 3834e504b8 XDRIVE transport: Prefer h2 (Chrome CH by default); End stalled requests; Other small changes (#7181)
https://github.com/XTLS/Xray-core/pull/7170#issuecomment-6204133548
2026-10-17 09:24:53 +00:00
0f127f934c XTLS Vision: Compatible with Go 1.27.3+ (#7176)
https://github.com/XTLS/Xray-core/pull/7176#issuecomment-6229087350

Fixes https://github.com/XTLS/Xray-core/issues/7162#issuecomment-6210430747

---------

Co-authored-by: RPRX <63339210+RPRX@users.noreply.github.com>
2026-10-17 03:37:14 +00:00
0f3d884786 WireGuard outbound: Keep the device when its setup fails (#7151)
https://github.com/XTLS/Xray-core/pull/7151#issuecomment-6186150131

Fixes https://github.com/XTLS/Xray-core/pull/7151#issuecomment-6190189547

---------

Co-authored-by: LjhAUMEM <llnu14702@gmail.com>
2026-10-16 09:50:47 +00:00
da2248c516 Hysteria client: Reset stalled writes when the connection is closed (#7172)
https://github.com/XTLS/Xray-core/pull/7172#issuecomment-6208589708

Fixes https://github.com/XTLS/Xray-core/issues/7169#issuecomment-6204450876

---------

Co-authored-by: LjhAUMEM <llnu14702@gmail.com>
2026-10-16 15:25:17 +08:00
fe2ae23f42 Sockopt: Cancel dialing after failed to bind interface, external controllers included such as TUN inbound's (#7173)
https://github.com/XTLS/Xray-core/pull/7161#issuecomment-6185101741

Fixes https://github.com/XTLS/Xray-core/issues/7160#issuecomment-6200788453

---------

Co-authored-by: Gleb <a1212gleb@gmail.com>
2026-10-16 06:37:52 +00:00
d15966c76c Fixes: Regard int() as int32() by default, which should be a coding standard (#7155)
https://github.com/XTLS/Xray-core/pull/7155#issuecomment-6208455069

Fixes https://github.com/XTLS/Xray-core/pull/7148#issuecomment-6173633056

---------

Co-authored-by: Denis Orlov <freshcrowd@gmail.com>
Co-authored-by: Milomilo777 <117558067+Milomilo777@users.noreply.github.com>
2026-10-16 05:12:52 +00:00
206a24a9f0 Noise finalmask: Support Lua script as well (on main branch: inline scripts were disabled for now; sandbox-only for now) (#7124)
https://github.com/XTLS/Xray-core/pull/7124#issuecomment-6108165942
https://github.com/XTLS/Xray-core/pull/7124#issuecomment-6149363945
https://github.com/XTLS/Xray-core/pull/7124#issuecomment-6207934753

---------

Co-authored-by: Meow <197331664+Meo597@users.noreply.github.com>
2026-10-16 04:43:55 +00:00
MeowandRPRX ac5d8a83ba Lua: Sandbox by default (whitelisted functions); Disable inline scripts for now; Add more checks for security reasons (#7136)
https://github.com/XTLS/Xray-core/pull/7136#issuecomment-6158632073
https://github.com/XTLS/Xray-core/pull/7136#issuecomment-6170831601
https://github.com/XTLS/Xray-core/pull/7136#issuecomment-6184362904
2026-10-16 01:55:35 +00:00
Artem LytkinandGitHub 8703fdb101 WireGuard, MASQUE proxies: Use Xray-core's built-in lightweight TCP/IP stack instead of gVisor as well (#7156)
https://github.com/XTLS/Xray-core/pull/7130#issuecomment-6116258188
https://github.com/XTLS/Xray-core/pull/7130#issuecomment-6120742739
https://github.com/XTLS/Xray-core/pull/7130#issuecomment-6137264458
https://github.com/XTLS/Xray-core/pull/7156#issuecomment-6202933664
https://github.com/XTLS/Xray-core/pull/7156#issuecomment-6203239736
2026-10-15 23:38:14 +00:00
LjhAUMEMandGitHub 10879a4a8b XDNS finalmask: Support DoH ("https://", tlsConfig, domain fronting, etc., quicConfig, Chrome Client Hello by default) with empty HTTP User-Agent (#7140)
https://github.com/XTLS/Xray-core/pull/7103#issuecomment-6118333973
https://github.com/XTLS/Xray-core/pull/7140#issuecomment-6161808702
https://github.com/XTLS/Xray-core/pull/7140#issuecomment-6167254749
https://github.com/XTLS/Xray-core/pull/7140#issuecomment-6185965142
https://github.com/XTLS/Xray-core/pull/7140#issuecomment-6200775375
2026-10-15 22:41:19 +00:00
0888beb38a SS2022 proxy: Correct interrupt behavior and UDP conn change (#7143)
Fixes https://github.com/XTLS/Xray-core/pull/7142 and https://github.com/XTLS/Xray-core/pull/7143#issuecomment-6163926468

---------

Co-authored-by: ChownYourLife <260039878+ChownYourLife@users.noreply.github.com>
2026-10-14 11:05:45 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
34acded003 Bump golang.org/x/net from 0.60.0 to 0.61.0 (#7147)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.60.0 to 0.61.0.
- [Commits](https://github.com/golang/net/compare/v0.60.0...v0.61.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.61.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-14 10:57:32 +00:00
RPRXandGitHub 676a600edb Update github.com/xtls/reality to 20261014101318
https://github.com/XTLS/REALITY/commit/83f621788ca88b83179810b12baa7ae518aacb83
https://github.com/XTLS/REALITY/commit/14cefac9bd7a724a811b02759fa97aa86a1ec894
https://github.com/XTLS/REALITY/commit/3c98159dee388c2914a2b9e73ea57bc10023f9d3
2026-10-14 10:44:41 +00:00
c77a4d0e49 MASQUE client: Support XMUX ("maxConnections": 1 by default); Use Chrome's User-Agent by default (#7121)
https://github.com/XTLS/Xray-core/pull/6810#issuecomment-5842441136
https://github.com/XTLS/Xray-core/pull/6878#issuecomment-6095112915
https://github.com/XTLS/Xray-core/pull/7121#issuecomment-6117636043

---------

Co-authored-by: SVLAVR <skugorov.s@gmail.com>
2026-10-12 19:40:04 +00:00
MeowandGitHub 5a64db009b Lua: Support inline scripts; Add ./xray base64 encode/decode [-gzip] command (#7132)
https://github.com/XTLS/Xray-core/pull/7124#issuecomment-6110557806
https://github.com/XTLS/Xray-core/pull/7132#issuecomment-6118169645
https://github.com/XTLS/Xray-core/pull/7132#issuecomment-6126428359
2026-10-12 16:58:20 +00:00
SVLAVRandGitHub 2bea2df6a1 Hysteria client: Release client map lock before cleanup (#7122)
https://github.com/XTLS/Xray-core/pull/7122#issuecomment-6119280343
2026-10-12 06:56:24 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
fb2dcedbb7 Bump golang.org/x/net from 0.59.0 to 0.60.0 (#7133)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.59.0 to 0.60.0.
- [Commits](https://github.com/golang/net/compare/v0.59.0...v0.60.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.60.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-12 05:16:45 +00:00
7c2e05dc47 Hysteria client: Honor cancellation during dialing (#7119)
Fixes https://github.com/XTLS/Xray-core/issues/7120

---------

Co-authored-by: LjhAUMEM <llnu14702@gmail.com>
2026-10-12 05:01:23 +00:00
brookwkoandRPRX 1dd2a0bd3c TUN inbound: Support autoSystemDnsToGateway on FreeBSD as well (#7123)
https://github.com/XTLS/Xray-core/issues/6454#issuecomment-5859521287
https://github.com/XTLS/Xray-core/pull/6691#issuecomment-5445818753
https://github.com/XTLS/Xray-core/pull/6773 & https://github.com/XTLS/Xray-core/pull/6853
2026-10-12 02:48:59 +00:00
Artem LytkinandRPRX 70ef97d564 TUN inbound: Use Xray-core's built-in lightweight TCP/IP stack instead of gVisor (#7130)
https://github.com/XTLS/Xray-core/pull/6771#issuecomment-5752154561
https://github.com/XTLS/Xray-core/pull/6795#issuecomment-6095740152
https://github.com/XTLS/Xray-core/pull/7130#issuecomment-6116258188
2026-10-12 02:18:56 +00:00
RPRXandGitHub 701af60772 Xray-core v26.10.10
Sponsor & Donation & NFTs: https://github.com/XTLS/Xray-core/issues/3668
Project X Channel: https://t.me/projectXtls

Announcement of NFTs by Project X: https://github.com/XTLS/Xray-core/discussions/3633
Project X NFT: https://opensea.io/assets/ethereum/0x5ee362866001613093361eb8569d59c4141b76d1/1

VLESS Post-Quantum Encryption: https://github.com/XTLS/Xray-core/pull/5067
VLESS NFT: https://opensea.io/collection/vless

XHTTP: Beyond REALITY: https://github.com/XTLS/Xray-core/discussions/4113
REALITY NFT: https://opensea.io/assets/ethereum/0x5ee362866001613093361eb8569d59c4141b76d1/2
2026-10-10 09:21:07 +00:00
MeowandGitHub 8855145e58 Xray-core: Add Lua script for dns and routing (#6823)
https://github.com/XTLS/Xray-core/pull/6823#issuecomment-5843754759
https://github.com/XTLS/Xray-core/pull/6823#issuecomment-5861456450
https://github.com/XTLS/Xray-core/pull/6823#issuecomment-6093596069
2026-10-10 09:01:16 +00:00
c7dbfd5e1e Hysteria client: Remove closed instance conn (#7107)
https://github.com/XTLS/Xray-core/pull/7102#issuecomment-6054105250

Fixes https://github.com/XTLS/Xray-core/issues/7100

---------

Co-authored-by: LjhAUMEM <llnu14702@gmail.com>
2026-10-10 08:08:06 +00:00
RPRX aa7aa9ad0a MASQUE client: Ignore publicKey in warp when pcs/vcn exists (domain fronting for WARP)
Closes https://github.com/XTLS/Xray-core/issues/7115#issuecomment-6088708005
2026-10-10 06:56:57 +00:00
2cc08769a5 TLS ECH DoH h2c: Support verifyPeerCertByName's "fromMitm" (after dialerProxy, in RAW outbound's tlsSettings) (#6246)
https://github.com/XTLS/Xray-core/pull/6246#issuecomment-5850758711

Like https://github.com/XTLS/Xray-core/commit/613c63b165829a0b3e5bfc3a72eff21d96ab4874

---------

Co-authored-by: patterniha <71074308+patterniha@users.noreply.github.com>
2026-10-10 06:55:46 +00:00
风扇滑翔翼andRPRX 5af2e9c228 TLS client: Use Xray-core's built-in RootCAs by default (avoids maliciously injected CA in the OS) (#7105)
https://github.com/XTLS/Xray-core/pull/5532#issuecomment-3745598515
https://github.com/XTLS/Xray-core/pull/7098#issuecomment-6053012480
https://github.com/XTLS/Xray-core/pull/7105#issuecomment-6093006914
2026-10-10 06:53:39 +00:00
Artem LytkinandGitHub 836a6fed38 Mux client: Fix b.UDP being overwritten by the next frame (#6822)
https://github.com/XTLS/Xray-core/pull/6822#issuecomment-5993165787
2026-10-08 09:24:11 +00:00
2a14c50775 Socks inbound: Fix stats for UDP Associate traffic (#7101)
https://github.com/XTLS/Xray-core/pull/7101#pullrequestreview-5452526703

---------

Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-10-08 09:21:21 +00:00
8989adfd3f TUN inbound: Handle weak host send and forwarding on the outbound interface on Windows (#7092)
https://github.com/XTLS/Xray-core/issues/6776#issuecomment-5994046421
https://github.com/XTLS/Xray-core/issues/6776#issuecomment-5994796106

Closes https://github.com/XTLS/Xray-core/issues/6776
Closes https://github.com/XTLS/Xray-core/issues/6872

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 05:49:28 +00:00
Hossin AsaadiandGitHub 1a60fc78ff TUN inbound: Duplicate the iOS tunnel file descriptor (#6883)
https://github.com/XTLS/Xray-core/pull/6883#issuecomment-6008368575
2026-10-08 05:14:14 +00:00
Artem LytkinandGitHub 2eedef51a2 DNS: Fix pubsub cleanup sizing every topic's slice by the topic count (#7091)
https://github.com/XTLS/Xray-core/pull/7091#pullrequestreview-5414371870
2026-10-08 05:08:43 +00:00
LjhAUMEMandGitHub 37184948d4 XDNS finalmask: Refine upload (#7095)
Fixes https://github.com/XTLS/Xray-core/pull/7090#issuecomment-6009651970
2026-10-08 04:58:49 +00:00
CluvexandGitHub 9e55a6ed18 MASQUE client: Support WARP (#6878)
https://github.com/XTLS/Xray-core/pull/6844#issuecomment-5859092964
https://github.com/XTLS/Xray-core/pull/6862#issuecomment-5897508127
https://github.com/XTLS/Xray-core/pull/6878#issuecomment-6052448898
2026-10-08 04:53:07 +00:00
7da5dae650 Mux client: Fix a race condition between Dispatch and SessionManager.Close (#6871)
Fixes https://github.com/XTLS/Xray-core/issues/6870

---------

Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-10-05 09:52:18 +00:00
LjhAUMEMandGitHub a3c6cc6aa8 XDNS finalmask: Default values for types; name->names; type+addr->addrs (#7090)
https://github.com/XTLS/Xray-core/pull/7090#issuecomment-5989883544
https://github.com/XTLS/Xray-core/pull/7090#issuecomment-5988755551
https://github.com/XTLS/Xray-core/pull/7089#issuecomment-5987266008
2026-10-05 08:07:19 +00:00
nSQUICKandGitHub 1e08d48d88 TUN inbound: Start on Windows when IPv6 is disabled system-wide (#6874)
https://github.com/XTLS/Xray-core/pull/6874#issuecomment-5989986544
2026-10-05 07:26:00 +00:00
5121c28b85 QUIC sniffer: Handle zero-filled datagram tails (#6882)
https://github.com/XTLS/Xray-core/pull/6882#pullrequestreview-5395639793

---------

Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-10-05 03:53:22 +00:00
92f7e72490 WireGuard kernelTun: Fix IPv6 table index allocation when the first table is occupied (#6881)
https://github.com/XTLS/Xray-core/pull/6881#issuecomment-5927334896

---------

Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-10-05 03:50:22 +00:00
Жора ЗмейкинandGitHub eddad37c69 XDNS finalmask: Fix resolver close deadlock and socket leaks (#7089)
https://github.com/XTLS/Xray-core/pull/7089#issuecomment-5987521972
2026-10-05 03:31:46 +00:00
SVLAVRandGitHub 00cde72a9b Test: Fix process leaks when retrying scenario tests (#6863)
https://github.com/XTLS/Xray-core/pull/6863#pullrequestreview-5362117967
2026-10-05 02:13:03 +00:00
DenisandGitHub 08775afd65 WireGuard inbound: Reduce memory held by idle UDP flows (#6856)
https://github.com/XTLS/Xray-core/pull/6856#issuecomment-5870639238
2026-10-05 01:56:36 +00:00
DenisandGitHub e0bae21201 WireGuard inbound: Reply fake pong to ICMP ping (#6855)
https://github.com/XTLS/Xray-core/pull/6015#issuecomment-4321525342
2026-10-05 01:36:54 +00:00
9d9a7a1c00 gRPC client: Fix redial with the first request's canceled context (#6877)
https://github.com/XTLS/Xray-core/pull/6877#issuecomment-5986476726

Fixes https://github.com/XTLS/Xray-core/issues/6876

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 01:26:12 +00:00
RPRXandGitHub b26a91de4f Xray-core v26.9.30
Sponsor & Donation & NFTs: https://github.com/XTLS/Xray-core/issues/3668
Project X Channel: https://t.me/projectXtls

Announcement of NFTs by Project X: https://github.com/XTLS/Xray-core/discussions/3633
Project X NFT: https://opensea.io/assets/ethereum/0x5ee362866001613093361eb8569d59c4141b76d1/1

VLESS Post-Quantum Encryption: https://github.com/XTLS/Xray-core/pull/5067
VLESS NFT: https://opensea.io/collection/vless

XHTTP: Beyond REALITY: https://github.com/XTLS/Xray-core/discussions/4113
REALITY NFT: https://opensea.io/assets/ethereum/0x5ee362866001613093361eb8569d59c4141b76d1/2
2026-09-30 07:40:04 +00:00
1f304916bd TUN inbound: Add autoSystemWfpBlockLeak on Windows (blocks "dns" and "misconfigtun" IPv4/IPv6 traffic leaks outside the TUN); Rename autoSystemDNS to autoSystemDnsToGateway on Linux (and change some behaviors) (#6853)
https://github.com/XTLS/Xray-core/pull/6853#issuecomment-5899791359
https://github.com/XTLS/Xray-core/pull/6853#issuecomment-5901287980
https://github.com/XTLS/Xray-core/pull/6853#issuecomment-5903680113
https://github.com/XTLS/Xray-core/pull/6853#issuecomment-5904123488
https://github.com/XTLS/Xray-core/pull/6853#issuecomment-5904647772
https://github.com/XTLS/Xray-core/pull/6853#issuecomment-5905047424

Fixes https://github.com/XTLS/Xray-core/issues/6454#issuecomment-5863800676

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 06:26:16 +00:00
风扇滑翔翼 0086362663 SS2022 proxy: Refactor to fix some issues (#6866)
https://github.com/XTLS/Xray-core/pull/6866#issuecomment-5904374529

Fixes https://github.com/XTLS/Xray-core/pull/6831#issuecomment-5884408501 and https://github.com/XTLS/Xray-core/pull/6866#issuecomment-5889568266
2026-09-30 13:16:27 +08:00
CluvexandGitHub e51b3c3621 Noise finalmask: type supports "exp" (#6862)
https://github.com/XTLS/Xray-core/pull/6844#issuecomment-5859092964
https://github.com/XTLS/Xray-core/pull/6844#issuecomment-5859855674
https://github.com/XTLS/Xray-core/pull/6862#issuecomment-5897471145
2026-09-30 03:11:12 +00:00
Artem LytkinandGitHub 6243d2a26e Geodata: Reduce matcher memory on mobile and desktop, skip regexes that cannot match (#6867)
https://github.com/XTLS/Xray-core/pull/6867#issuecomment-5895171934
2026-09-29 19:25:53 +00:00
风扇滑翔翼 35e616d3b9 Xray-core: Move PacketConnWrapper to common/net (#6854)
https://github.com/XTLS/Xray-core/pull/6854#issuecomment-5895541650

Fixes https://github.com/XTLS/Xray-core/issues/6849
2026-09-30 02:33:53 +08:00
v2rayandRPRX 08cb6e6bca WireGuard proxy: Fix potential startup races (#6852)
https://github.com/XTLS/Xray-core/pull/6852#issuecomment-5871816550

Fixes https://github.com/XTLS/Xray-core/issues/6850
2026-09-29 18:13:43 +00:00
wewon-backdownandRPRX 48ad0300ea API: xray api adu supports Hysteria (#6847)
https://github.com/XTLS/Xray-core/pull/6847#issuecomment-5895325181

---------

Co-authored-by: RPRX <63339210+RPRX@users.noreply.github.com>
2026-09-29 18:11:26 +00:00
0fc379203f HTTPUpgrade transport: Send some Sec-WebSocket-* headers (#6835)
https://github.com/XTLS/Xray-core/pull/6835#issuecomment-5853986428

---------

Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-09-29 17:25:26 +00:00
LjhAUMEMandGitHub fc8f8a451d XDNS finalmask: Refactor and new parameters (#6718)
https://github.com/XTLS/Xray-core/pull/6718#issuecomment-5894987590

Fixes https://github.com/XTLS/Xray-core/issues/6692
2026-09-29 17:16:59 +00:00
e5e85ca9da XTLS Vision: Suppress outer CloseNotify after switching to direct copy (#6834)
https://github.com/XTLS/Xray-core/pull/6816#issuecomment-5828082031
https://github.com/XTLS/Xray-core/issues/6794#issuecomment-5847200576
https://github.com/XTLS/Xray-core/pull/6834#issuecomment-5860728689

Fixes https://github.com/XTLS/Xray-core/issues/6794#issuecomment-5754894283
Fixes https://github.com/XTLS/Xray-core/issues/6124#issuecomment-4439918073
Fixes https://github.com/XTLS/Xray-core/issues/4878#issuecomment-5754638401

---------

Co-authored-by: Artem Lytkin <146867384+4RH1T3CT0R7@users.noreply.github.com>
2026-09-27 23:39:21 +00:00
7780db9bbe Finalmask: Fix panic when udpHop or xicmp fails; Restore udpHop's default interval (#6808)
https://github.com/XTLS/Xray-core/pull/6808#pullrequestreview-5324529175

---------

Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-09-27 22:17:47 +00:00
Artem LytkinandGitHub 2953d44734 Geodata: Reduce MPH matcher runtime memory usage (#6821)
https://github.com/XTLS/Xray-core/pull/6821#issuecomment-5860086534
2026-09-27 21:52:37 +00:00
7b8ade3ec5 TUN inbound: Only close the UDP connection that actually finished (#6814)
https://github.com/XTLS/Xray-core/pull/6814#pullrequestreview-5324536423

---------

Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-09-27 21:06:19 +00:00
Artem LytkinandGitHub 5dda894e29 TUN inbound: Reuse existing Wintun adapter by name on Windows again (#6811)
Fixes https://github.com/XTLS/Xray-core/issues/6780
2026-09-27 20:39:17 +00:00
youugiuhiuhandGitHub 47a2c2ffdc TUN inbound: Add autoSystemDNS on Linux (to TUN's gateway) (#6773)
https://github.com/XTLS/Xray-core/issues/6454#issuecomment-4931311976
https://github.com/XTLS/Xray-core/pull/6773#issuecomment-5755516423
https://github.com/XTLS/Xray-core/pull/6807#issuecomment-5807844210
2026-09-27 20:07:53 +00:00
CluvexandGitHub 7a018833ec Proxy: Add MASQUE inbound (IETF CONNECT-IP server, RFC 9484) (#6844)
Completes https://github.com/XTLS/Xray-core/pull/6807 and https://github.com/XTLS/Xray-core/pull/6810
2026-09-27 19:16:54 +00:00
风扇滑翔翼andGitHub 65e853ed84 SS2022 proxy: Refactor to remove sing* dependencies (#6831)
https://github.com/XTLS/Xray-core/pull/4356#issuecomment-2639571931
2026-09-27 18:42:53 +00:00
HuskyDGandGitHub 3519dfecbd FakeDNS: Change default FakeIPv6Pool to 2001:2::/48 (#6815)
https://github.com/XTLS/Xray-core/discussions/5458#discussioncomment-18555021

https://github.com/XTLS/Xray-core/pull/6815#issuecomment-5842712128
2026-09-26 03:52:03 +00:00
CluvexandRPRX df261e4479 MASQUE client: Support HTTP/2 (Extended CONNECT, RFC 8441) (#6810)
https://github.com/XTLS/Xray-core/pull/6807#issuecomment-5808933074

https://github.com/XTLS/Xray-core/pull/6810#issuecomment-5842441136
2026-09-26 03:49:37 +00:00
风扇滑翔翼andRPRX 61cad5ec8b Xray-core: Reduce error log usage for performance (#6796)
https://github.com/XTLS/Xray-core/pull/6796#issuecomment-5807809588
2026-09-25 17:59:49 +00:00
Artem LytkinandGitHub a642a190ed Geodata: Prefilter regexp rules by their required literals (#6818)
https://github.com/XTLS/Xray-core/pull/6818#issuecomment-5833512067
2026-09-25 17:25:52 +00:00
Hossin AsaadiandGitHub 60e2a0c502 WireGuard proxy: Release packet views after use (#6801)
https://github.com/XTLS/Xray-core/pull/6801#issuecomment-5807228464
2026-09-24 05:46:26 +00:00
Esko MobiusandGitHub 7d3e44fee2 Proxy: Add MASQUE outbound & transport (IETF CONNECT-IP, RFC 9484) (#6807)
Closes https://github.com/XTLS/Xray-core/issues/5495#issuecomment-3710683679
2026-09-24 02:13:37 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
9927942aaa Bump google.golang.org/grpc from 1.83.2 to 1.84.0 (#6793)
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.83.2 to 1.84.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.83.2...v1.84.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.84.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-24 02:04:39 +00:00
LjhAUMEMandGitHub a308ded2e6 WireGuard outbound: Fix endpoint IP address (#6804)
Fixes https://github.com/XTLS/Xray-core/issues/6803
2026-09-24 01:59:28 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
7741e9e77e Bump golang.zx2c4.com/wireguard/windows from 1.0.1 to 1.1.1 (#6809)
Bumps golang.zx2c4.com/wireguard/windows from 1.0.1 to 1.1.1.

---
updated-dependencies:
- dependency-name: golang.zx2c4.com/wireguard/windows
  dependency-version: 1.1.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-24 01:57:16 +00:00
d562d8947d Hysteria outbound: Fix UDP DATAGRAM truncation with ChromeParrot (#6788)
https://github.com/XTLS/Xray-core/pull/6788#issuecomment-5751428127

---------

Co-authored-by: LjhAUMEM <llnu14702@gmail.com>
2026-09-20 22:51:03 +00:00
Hossin AsaadiandGitHub dbb1ea30ba WireGuard proxy: Prevent panic when closing netTun with a write in flight (#6778)
https://github.com/XTLS/Xray-core/pull/6778#pullrequestreview-5251957308
2026-09-20 20:07:54 +00:00
LjhAUMEMandRPRX efc9e6da62 WireGuard outbound: Remove remoteDNS' "local" mode and domainStrategy (#6771)
https://github.com/XTLS/Xray-core/issues/6567#issuecomment-5660953757
https://github.com/XTLS/Xray-core/pull/6771#issuecomment-5751826982
https://github.com/XTLS/Xray-core/pull/6771#issuecomment-5751831637
2026-09-20 19:07:51 +00:00
LjhAUMEMandGitHub 8267cf953a Transport: Refactor to be based on Finalmask's dialer & listener (#6754)
https://github.com/XTLS/Xray-core/pull/6327#issuecomment-5645958010
https://github.com/XTLS/Xray-core/pull/6754#issuecomment-5720818254
https://github.com/XTLS/Xray-core/pull/6754#issuecomment-5751585906
2026-09-20 18:12:50 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
24e6f6d551 Bump golang.org/x/net from 0.58.0 to 0.59.0 (#6756)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.58.0 to 0.59.0.
- [Commits](https://github.com/golang/net/compare/v0.58.0...v0.59.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.59.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-20 17:46:51 +00:00
dcdfc57ccd XDRIVE transport: Add the Google Drive and "template" backend (#6748)
https://github.com/XTLS/Xray-core/pull/5645#issuecomment-3849778103
https://github.com/XTLS/Xray-core/pull/5645#issuecomment-3851839033
https://github.com/XTLS/Xray-core/pull/6745#issuecomment-5627294204
https://github.com/XTLS/Xray-core/pull/6748#issuecomment-5660444946
https://github.com/XTLS/Xray-core/pull/6748#issuecomment-5719209642

---------

Co-authored-by: RPRX <63339210+RPRX@users.noreply.github.com>
2026-09-19 09:03:11 +00:00
3461c511aa XDRIVE transport: The universal remote-storage/any-service-based proxy, ignoring IP whitelists (#5645)
https://github.com/XTLS/Xray-core/pull/5414#issuecomment-3796734827
https://github.com/XTLS/Xray-core/pull/5581#issuecomment-3797134147
https://github.com/XTLS/Xray-core/pull/5645#issuecomment-3899873945
https://github.com/XTLS/Xray-core/pull/6745#issuecomment-5627420177
https://github.com/XTLS/Xray-core/pull/6748#issuecomment-5740443122

---------

Co-authored-by: Risaro <62798663+Risaro@users.noreply.github.com>
2026-09-19 08:23:38 +00:00
c412e77a9b TUN inbound: Preserve UDP packet destinations with traffic stats (#6747)
https://github.com/XTLS/Xray-core/pull/6747#issuecomment-5647964551

---------

Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-09-12 20:09:35 +00:00
ccb69ea5e2 common/buf/readv_windows.go: Fix raw WSARecv blocking thread and preventing connection close (#6743)
https://github.com/XTLS/Xray-core/pull/6743#issuecomment-5647971870

---------

Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-09-12 19:51:59 +00:00
RPRXandGitHub 52a412d9e2 Xray-core v26.9.9
Sponsor & Donation & NFTs: https://github.com/XTLS/Xray-core/issues/3668
Project X Channel: https://t.me/projectXtls

Announcement of NFTs by Project X: https://github.com/XTLS/Xray-core/discussions/3633
Project X NFT: https://opensea.io/assets/ethereum/0x5ee362866001613093361eb8569d59c4141b76d1/1

VLESS Post-Quantum Encryption: https://github.com/XTLS/Xray-core/pull/5067
VLESS NFT: https://opensea.io/collection/vless

XHTTP: Beyond REALITY: https://github.com/XTLS/Xray-core/discussions/4113
REALITY NFT: https://opensea.io/assets/ethereum/0x5ee362866001613093361eb8569d59c4141b76d1/2
2026-09-08 22:25:43 +00:00
18a1b5042a Finalmask: Make "udpHop" a new UDP mask (#6327)
https://github.com/XTLS/Xray-core/pull/6327#issuecomment-5592421364

---------

Co-authored-by: RPRX <63339210+RPRX@users.noreply.github.com>
2026-09-08 22:15:02 +00:00
patternihaandGitHub c26d2eda24 Direct/Freedom outbound: Skip domain resolution and finalRules when using sockopt.dialerProxy (#6742)
https://github.com/XTLS/Xray-core/pull/6058#issuecomment-5578800461
2026-09-08 18:45:38 +00:00
506 changed files with 52175 additions and 8908 deletions
+5 -11
View File
@@ -9,8 +9,8 @@ ARG TARGETARCH
RUN GOOS=$TARGETOS GOARCH=$TARGETARCH CGO_ENABLED=0 go build -o xray -trimpath -buildvcs=false -gcflags="all=-l=4" -ldflags "-s -w -buildid=" ./main
# Download geodat into a staging directory
ADD https://raw.githubusercontent.com/Loyalsoldier/v2ray-rules-dat/release/geoip.dat /tmp/geodat/geoip.dat
ADD https://raw.githubusercontent.com/Loyalsoldier/v2ray-rules-dat/release/geosite.dat /tmp/geodat/geosite.dat
ADD https://raw.githubusercontent.com/XTLS/Xray-rules-dat/release/geoip/geoip-full.dat /tmp/geodat/geoip.dat
ADD https://raw.githubusercontent.com/XTLS/Xray-rules-dat/release/geosite/geosite-expand.dat /tmp/geodat/geosite.dat
RUN mkdir -p /tmp/empty
@@ -40,21 +40,15 @@ RUN echo '{}' >/tmp/usr/local/etc/xray/11_geodata.json
RUN echo '{}' >/tmp/usr/local/etc/xray/12_env.json
RUN echo '{}' >/tmp/usr/local/etc/xray/99_version.json
# Create log files
RUN mkdir -p /tmp/var/log/xray && touch \
/tmp/var/log/xray/access.log \
/tmp/var/log/xray/error.log
# Build finally image
FROM gcr.io/distroless/static:nonroot
COPY --from=build --chown=0:0 --chmod=755 /src/xray /usr/local/bin/xray
COPY --from=build --chown=65532:65532 --chmod=755 /tmp/empty /usr/local/share/xray
COPY --from=build --chown=65532:65532 --chmod=644 /tmp/geodat/*.dat /usr/local/share/xray/
COPY --from=build --chown=0:0 --chmod=755 /tmp/empty /usr/local/etc/xray
COPY --from=build --chown=0:0 --chmod=644 /tmp/usr/local/etc/xray/*.json /usr/local/etc/xray/
COPY --from=build --chown=0:0 --chmod=755 /tmp/empty /var/log/xray
COPY --from=build --chown=65532:65532 --chmod=600 /tmp/var/log/xray/*.log /var/log/xray/
COPY --from=build --chown=0:65532 --chmod=750 /tmp/empty /usr/local/etc/xray
COPY --from=build --chown=0:65532 --chmod=640 /tmp/usr/local/etc/xray/*.json /usr/local/etc/xray/
COPY --from=build --chown=65532:65532 --chmod=700 /tmp/empty /var/log/xray
VOLUME /usr/local/etc/xray
VOLUME /var/log/xray
+5 -11
View File
@@ -9,8 +9,8 @@ ARG TARGETARCH
RUN GOOS=$TARGETOS GOARCH=$TARGETARCH CGO_ENABLED=0 go build -o xray -trimpath -buildvcs=false -gcflags="all=-l=4" -ldflags "-s -w -buildid=" ./main
# Download geodat into a staging directory
ADD https://raw.githubusercontent.com/Loyalsoldier/v2ray-rules-dat/release/geoip.dat /tmp/geodat/geoip.dat
ADD https://raw.githubusercontent.com/Loyalsoldier/v2ray-rules-dat/release/geosite.dat /tmp/geodat/geosite.dat
ADD https://raw.githubusercontent.com/XTLS/Xray-rules-dat/release/geoip/geoip-full.dat /tmp/geodat/geoip.dat
ADD https://raw.githubusercontent.com/XTLS/Xray-rules-dat/release/geosite/geosite-expand.dat /tmp/geodat/geosite.dat
RUN mkdir -p /tmp/empty
@@ -40,11 +40,6 @@ RUN echo '{}' >/tmp/usr/local/etc/xray/11_geodata.json
RUN echo '{}' >/tmp/usr/local/etc/xray/12_env.json
RUN echo '{}' >/tmp/usr/local/etc/xray/99_version.json
# Create log files
RUN mkdir -p /tmp/var/log/xray && touch \
/tmp/var/log/xray/access.log \
/tmp/var/log/xray/error.log
# Build finally image
# Note on Distroless Base Image and Architecture Support:
# - The official 'gcr.io/distroless/static' image provided by Google only supports a limited set of architectures for Linux:
@@ -60,10 +55,9 @@ FROM --platform=linux/amd64 gcr.io/distroless/static:nonroot
COPY --from=build --chown=0:0 --chmod=755 /src/xray /usr/local/bin/xray
COPY --from=build --chown=65532:65532 --chmod=755 /tmp/empty /usr/local/share/xray
COPY --from=build --chown=65532:65532 --chmod=644 /tmp/geodat/*.dat /usr/local/share/xray/
COPY --from=build --chown=0:0 --chmod=755 /tmp/empty /usr/local/etc/xray
COPY --from=build --chown=0:0 --chmod=644 /tmp/usr/local/etc/xray/*.json /usr/local/etc/xray/
COPY --from=build --chown=0:0 --chmod=755 /tmp/empty /var/log/xray
COPY --from=build --chown=65532:65532 --chmod=600 /tmp/var/log/xray/*.log /var/log/xray/
COPY --from=build --chown=0:65532 --chmod=750 /tmp/empty /usr/local/etc/xray
COPY --from=build --chown=0:65532 --chmod=640 /tmp/usr/local/etc/xray/*.json /usr/local/etc/xray/
COPY --from=build --chown=65532:65532 --chmod=700 /tmp/empty /var/log/xray
VOLUME /usr/local/etc/xray
VOLUME /var/log/xray
+1 -1
View File
@@ -82,7 +82,7 @@ jobs:
uses: docker/setup-buildx-action@v4
- name: Login to GitHub Container Registry
uses: docker/login-action@v4.6.0
uses: docker/login-action@v4.7.0
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
@@ -40,7 +40,7 @@ jobs:
max_attempts: 60
command: |
[ -d 'resources' ] || mkdir resources
LIST=('Loyalsoldier v2ray-rules-dat geoip geoip' 'Loyalsoldier v2ray-rules-dat geosite geosite')
LIST=('XTLS Xray-rules-dat geoip/geoip-lite geoip' 'XTLS Xray-rules-dat geosite/geosite geosite')
for i in "${LIST[@]}"
do
INFO=($(echo $i | awk 'BEGIN{FS=" ";OFS=" "} {print $1,$2,$3,$4}'))
+3 -18
View File
@@ -35,24 +35,7 @@ jobs:
if: steps.check-assets.outputs.missing == 'true'
run: sleep 90
check-proto:
runs-on: ubuntu-latest
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name != github.event.pull_request.base.repo.full_name
steps:
- name: Checkout codebase
uses: actions/checkout@v7
- name: Check Proto Version Header
run: |
head -n 4 core/config.pb.go > ref.txt
find . -name "*.pb.go" ! -name "*_grpc.pb.go" -print0 | while IFS= read -r -d '' file; do
if ! cmp -s ref.txt <(head -n 4 "$file"); then
echo "Error: Header mismatch in $file"
head -n 4 "$file"
exit 1
fi
done
check-format:
check-format-and-proto:
runs-on: ubuntu-latest
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name != github.event.pull_request.base.repo.full_name
permissions:
@@ -68,6 +51,8 @@ jobs:
cache: false
- name: Check Format
run: go run ./infra/vformat/main.go -mode check -pwd ./
- name: Check Protobuf
run: go run ./infra/vprotogen/main.go -mode check -pwd ./
test:
needs: check-assets
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/commander/config.proto
package commander
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/dispatcher/config.proto
package dispatcher
+3
View File
@@ -470,6 +470,9 @@ func (d *DefaultDispatcher) routedDispatch(ctx context.Context, link *transport.
return // DO NOT CHANGE: the traffic shouldn't be processed by default outbound if the specified outbound tag doesn't exist (yet), e.g., VLESS Reverse Proxy
}
} else {
if err != common.ErrNoClue {
errors.LogErrorInner(ctx, err, "failed to pick route for ", destination)
}
errors.LogInfo(ctx, "default route for ", destination)
}
}
+1 -1
View File
@@ -23,7 +23,7 @@ func newFakeDNSSniffer(ctx context.Context) (protocolSnifferWithMetadata, error)
}
if fakeDNSEngine == nil {
errNotInit := errors.New("FakeDNSEngine is not initialized, but such a sniffer is used").AtError()
errNotInit := errors.New("FakeDNSEngine is not initialized, but such a sniffer is used")
return protocolSnifferWithMetadata{}, errNotInit
}
return protocolSnifferWithMetadata{protocolSniffer: func(ctx context.Context, bytes []byte) (SniffResult, error) {
+1 -1
View File
@@ -28,7 +28,7 @@ func toNetIP(addrs []net.Address) ([]net.IP, error) {
if addr.Family().IsIP() {
ips = append(ips, addr.IP())
} else {
return nil, errors.New("Failed to convert address", addr, "to Net IP.").AtWarning()
return nil, errors.New("Failed to convert address", addr, "to Net IP.")
}
}
return ips, nil
+24 -6
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/dns/config.proto
package dns
@@ -93,6 +93,7 @@ type NameServer struct {
UnexpectedIp []*geodata.IPRule `protobuf:"bytes,13,rep,name=unexpected_ip,json=unexpectedIp,proto3" json:"unexpected_ip,omitempty"`
ActUnprior bool `protobuf:"varint,14,opt,name=actUnprior,proto3" json:"actUnprior,omitempty"`
PolicyID uint32 `protobuf:"varint,17,opt,name=policyID,proto3" json:"policyID,omitempty"`
Id string `protobuf:"bytes,18,opt,name=id,proto3" json:"id,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
@@ -239,6 +240,13 @@ func (x *NameServer) GetPolicyID() uint32 {
return 0
}
func (x *NameServer) GetId() string {
if x != nil {
return x.Id
}
return ""
}
type Config struct {
state protoimpl.MessageState `protogen:"open.v1"`
// NameServer list used by this DNS client.
@@ -258,6 +266,7 @@ type Config struct {
DisableFallback bool `protobuf:"varint,10,opt,name=disableFallback,proto3" json:"disableFallback,omitempty"`
DisableFallbackIfMatch bool `protobuf:"varint,11,opt,name=disableFallbackIfMatch,proto3" json:"disableFallbackIfMatch,omitempty"`
EnableParallelQuery bool `protobuf:"varint,14,opt,name=enableParallelQuery,proto3" json:"enableParallelQuery,omitempty"`
Script string `protobuf:"bytes,15,opt,name=script,proto3" json:"script,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
@@ -369,6 +378,13 @@ func (x *Config) GetEnableParallelQuery() bool {
return false
}
func (x *Config) GetScript() string {
if x != nil {
return x.Script
}
return ""
}
type Config_HostMapping struct {
state protoimpl.MessageState `protogen:"open.v1"`
Domain *geodata.DomainRule `protobuf:"bytes,2,opt,name=domain,proto3" json:"domain,omitempty"`
@@ -435,7 +451,7 @@ var File_app_dns_config_proto protoreflect.FileDescriptor
const file_app_dns_config_proto_rawDesc = "" +
"\n" +
"\x14app/dns/config.proto\x12\fxray.app.dns\x1a\x1ccommon/net/destination.proto\x1a\x1bcommon/geodata/geodat.proto\"\xde\x05\n" +
"\x14app/dns/config.proto\x12\fxray.app.dns\x1a\x1ccommon/net/destination.proto\x1a\x1bcommon/geodata/geodat.proto\"\xee\x05\n" +
"\n" +
"NameServer\x123\n" +
"\aaddress\x18\x01 \x01(\v2\x19.xray.common.net.EndpointR\aaddress\x12\x1b\n" +
@@ -461,10 +477,11 @@ const file_app_dns_config_proto_rawDesc = "" +
"\n" +
"actUnprior\x18\x0e \x01(\bR\n" +
"actUnprior\x12\x1a\n" +
"\bpolicyID\x18\x11 \x01(\rR\bpolicyIDB\x0f\n" +
"\bpolicyID\x18\x11 \x01(\rR\bpolicyID\x12\x0e\n" +
"\x02id\x18\x12 \x01(\tR\x02idB\x0f\n" +
"\r_disableCacheB\r\n" +
"\v_serveStaleB\x12\n" +
"\x10_serveExpiredTTLJ\x04\b\x04\x10\x05\"\x82\x05\n" +
"\x10_serveExpiredTTLJ\x04\b\x04\x10\x05\"\x9a\x05\n" +
"\x06Config\x129\n" +
"\vname_server\x18\x05 \x03(\v2\x18.xray.app.dns.NameServerR\n" +
"nameServer\x12\x1b\n" +
@@ -480,7 +497,8 @@ const file_app_dns_config_proto_rawDesc = "" +
"\x0fdisableFallback\x18\n" +
" \x01(\bR\x0fdisableFallback\x126\n" +
"\x16disableFallbackIfMatch\x18\v \x01(\bR\x16disableFallbackIfMatch\x120\n" +
"\x13enableParallelQuery\x18\x0e \x01(\bR\x13enableParallelQuery\x1a}\n" +
"\x13enableParallelQuery\x18\x0e \x01(\bR\x13enableParallelQuery\x12\x16\n" +
"\x06script\x18\x0f \x01(\tR\x06script\x1a}\n" +
"\vHostMapping\x127\n" +
"\x06domain\x18\x02 \x01(\v2\x1f.xray.common.geodata.DomainRuleR\x06domain\x12\x0e\n" +
"\x02ip\x18\x03 \x03(\fR\x02ip\x12%\n" +
+3
View File
@@ -27,6 +27,7 @@ message NameServer {
repeated xray.common.geodata.IPRule unexpected_ip = 13;
bool actUnprior = 14;
uint32 policyID = 17;
string id = 18;
}
enum QueryStrategy {
@@ -73,4 +74,6 @@ message Config {
bool disableFallbackIfMatch = 11;
bool enableParallelQuery = 14;
string script = 15;
}
+42
View File
@@ -31,6 +31,7 @@ type DNS struct {
domainMatcher geodata.DomainMatcher
matcherInfos []*DomainMatcherInfo
checkSystem bool
script *scriptEngine
}
// DomainMatcherInfo contains information attached to index returned by Server.domainMatcher.
@@ -169,6 +170,13 @@ func New(ctx context.Context, config *Config) (*DNS, error) {
clients = append(clients, NewLocalDNSClient(ipOption))
}
var script *scriptEngine
if config.Script != "" {
if script, err = newScriptEngine(config.Script); err != nil {
return nil, errors.New("failed to prepare DNS script").Base(err)
}
}
return &DNS{
hosts: hosts,
ipOption: &ipOption,
@@ -180,6 +188,7 @@ func New(ctx context.Context, config *Config) (*DNS, error) {
disableFallbackIfMatch: config.DisableFallbackIfMatch,
enableParallelQuery: config.EnableParallelQuery,
checkSystem: checkSystem,
script: script,
}, nil
}
@@ -190,11 +199,19 @@ func (*DNS) Type() interface{} {
// Start implements common.Runnable.
func (s *DNS) Start() error {
if s.script != nil {
if err := s.script.start(s); err != nil {
return errors.New("failed to initialize DNS script").Base(err)
}
}
return nil
}
// Close implements common.Closable.
func (s *DNS) Close() error {
if s.script != nil {
s.script.close()
}
return nil
}
@@ -212,6 +229,28 @@ func (s *DNS) IsOwnLink(ctx context.Context) bool {
return false
}
// MayUseSystemResolver reports whether any name server configured here could
// still resolve through the system resolver. That is what happens when no name
// server is configured at all, and it is also what a name server pointed at
// "localhost" does. Callers that are about to redirect the system resolver need
// to know, because a resolution path that reaches it would then loop back to
// them.
//
// Any such server is enough: name servers can be selected per domain, so a
// single local one makes some query reach the system resolver even when
// independent upstreams are configured alongside it.
func (s *DNS) MayUseSystemResolver() bool {
if len(s.clients) == 0 {
return true
}
for _, client := range s.clients {
if _, isLocal := client.server.(*LocalNameServer); isLocal {
return true
}
}
return false
}
// LookupIP implements dns.Client.
func (s *DNS) LookupIP(domain string, option dns.IPOption) ([]net.IP, uint32, error) {
// Normalize the FQDN form query
@@ -257,6 +296,9 @@ func (s *DNS) LookupIP(domain string, option dns.IPOption) ([]net.IP, uint32, er
}
// Name servers lookup
if s.script != nil {
return s.script.query(domain, option)
}
if s.enableParallelQuery {
return s.parallelQuery(domain, option)
} else {
+59
View File
@@ -0,0 +1,59 @@
package dns
import (
"context"
"testing"
"github.com/xtls/xray-core/common/net"
feature_dns "github.com/xtls/xray-core/features/dns"
)
// fakeServer stands in for any name server that is not the system resolver.
type fakeServer struct{}
func (fakeServer) Name() string { return "fake" }
func (fakeServer) IsDisableCache() bool { return false }
func (fakeServer) QueryIP(context.Context, string, feature_dns.IPOption) ([]net.IP, uint32, error) {
return nil, 0, nil
}
// Callers that are about to redirect the system resolver rely on this to tell
// whether any resolution path could still reach the system resolver, so the
// mixed shape has to be reported as reachable: a domain-specific rule can
// select the system resolver even when an independent upstream also exists.
func TestMayUseSystemResolver(t *testing.T) {
tests := []struct {
name string
clients []*Client
want bool
}{
{
name: "no clients at all",
want: true,
},
{
name: "only the system resolver",
clients: []*Client{{server: NewLocalNameServer()}},
want: true,
},
{
name: "the system resolver alongside an independent name server",
clients: []*Client{{server: fakeServer{}}, {server: NewLocalNameServer()}},
want: true,
},
{
name: "only independent name servers",
clients: []*Client{{server: fakeServer{}}},
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
server := &DNS{clients: tt.clients}
if got := server.MayUseSystemResolver(); got != tt.want {
t.Errorf("MayUseSystemResolver() = %v, want %v", got, tt.want)
}
})
}
}
+2 -2
View File
@@ -188,10 +188,10 @@ func parseResponse(payload []byte) (*IPRecord, error) {
var parser dnsmessage.Parser
h, err := parser.Start(payload)
if err != nil {
return nil, errors.New("failed to parse DNS response").Base(err).AtWarning()
return nil, errors.New("failed to parse DNS response").Base(err)
}
if err := parser.SkipAllQuestions(); err != nil {
return nil, errors.New("failed to skip questions in DNS response").Base(err).AtWarning()
return nil, errors.New("failed to skip questions in DNS response").Base(err)
}
now := time.Now()
+3 -3
View File
@@ -58,7 +58,7 @@ func NewFakeDNSHolder() (*Holder, error) {
var err error
if fkdns, err = NewFakeDNSHolderConfigOnly(nil); err != nil {
return nil, errors.New("Unable to create Fake Dns Engine").Base(err).AtError()
return nil, errors.New("Unable to create Fake Dns Engine").Base(err)
}
err = fkdns.initialize(dns.FakeIPv4Pool, 65535)
if err != nil {
@@ -80,13 +80,13 @@ func (fkdns *Holder) initialize(ipPoolCidr string, lruSize int) error {
var err error
if _, ipRange, err = net.ParseCIDR(ipPoolCidr); err != nil {
return errors.New("Unable to parse CIDR for Fake DNS IP assignment").Base(err).AtError()
return errors.New("Unable to parse CIDR for Fake DNS IP assignment").Base(err)
}
ones, bits := ipRange.Mask.Size()
rooms := bits - ones
if math.Log2(float64(lruSize)) >= float64(rooms) {
return errors.New("LRU size is bigger than subnet size").AtError()
return errors.New("LRU size is bigger than subnet size")
}
fkdns.domainToIP = cache.NewLru(lruSize)
fkdns.ipRange = ipRange
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/dns/fakedns/fakedns.proto
package fakedns
+169
View File
@@ -0,0 +1,169 @@
package dns
import (
"context"
"strings"
"github.com/xtls/xray-core/common/errors"
xlua "github.com/xtls/xray-core/common/lua"
"github.com/xtls/xray-core/common/net"
featureDNS "github.com/xtls/xray-core/features/dns"
"github.com/xtls/xray-core/features/dns/localdns"
lua "github.com/yuin/gopher-lua"
)
// luaDNSServer adapts configured and local DNS to the same Lua API.
type luaDNSServer struct {
id string
name string
query func(context.Context, string, featureDNS.IPOption) ([]net.IP, uint32, error)
}
// RegisterLua makes xray.dns available to scripts backed by client.
func RegisterLua(L *lua.LState, client featureDNS.Client) {
var servers []luaDNSServer
switch client := client.(type) {
case *DNS:
servers = luaServers(client)
case *localdns.Client:
servers = []luaDNSServer{{
id: "localhost",
name: "localhost",
query: func(_ context.Context, domain string, option featureDNS.IPOption) ([]net.IP, uint32, error) {
return client.LookupIP(domain, option)
},
}}
}
registerLua(L, servers, client)
}
// registerLua makes xray.dns available to DNS scripts.
func (s *DNS) registerLua(L *lua.LState) {
registerLua(L, luaServers(s), nil)
}
func luaServers(s *DNS) []luaDNSServer {
servers := make([]luaDNSServer, len(s.clients))
for i, client := range s.clients {
servers[i] = luaDNSServer{id: client.id, name: client.Name(), query: client.QueryIP}
}
return servers
}
func registerLua(L *lua.LState, servers []luaDNSServer, client featureDNS.Client) {
L.PreloadModule("xray.dns", func(L *lua.LState) int {
pushIPs := xlua.NewReadOnlySlicePusher[net.IP](L)
serverList := L.CreateTable(len(servers), 0)
for i, client := range servers {
server := L.CreateTable(0, 2)
server.RawSetString("ID", lua.LString(client.id))
server.RawSetString("Query", L.NewFunction(func(L *lua.LState) int {
domain, ok := L.Get(2).(lua.LString)
if !ok {
L.RaiseError("server:Query requires a domain")
return 0
}
option := featureDNS.IPOption{
IPv4Enable: L.CheckBool(3),
IPv6Enable: L.CheckBool(4),
FakeEnable: L.CheckBool(5),
}
ctx := L.Context()
if ctx == nil {
L.RaiseError("server:Query requires an active DNS query")
return 0
}
var ips []net.IP
var ttl uint32
var err error
if !option.FakeEnable && strings.EqualFold(client.name, "FakeDNS") {
err = featureDNS.ErrEmptyResponse
} else {
ips, ttl, err = client.query(ctx, string(domain), option)
}
pushIPs(L, ips)
xlua.PushNumber(L, ttl)
xlua.PushError(L, err)
return 3
}))
serverList.RawSetInt(i+1, server)
}
module := L.CreateTable(0, 2)
if servers != nil {
module.RawSetString("Servers", serverList)
}
if client != nil {
module.RawSetString("Query", newLuaClientQuery(L, client, pushIPs))
}
L.Push(module)
return 1
})
}
func newLuaClientQuery(L *lua.LState, client featureDNS.Client, pushIPs func(*lua.LState, []net.IP)) *lua.LFunction {
return L.NewFunction(func(L *lua.LState) int {
domain, ok := L.Get(1).(lua.LString)
if !ok {
L.RaiseError("dns.Query requires a domain")
return 0
}
option := featureDNS.IPOption{
IPv4Enable: L.CheckBool(2),
IPv6Enable: L.CheckBool(3),
FakeEnable: L.CheckBool(4),
}
if L.Context() == nil {
L.RaiseError("dns.Query requires an active DNS query")
return 0
}
ips, ttl, err := client.LookupIP(string(domain), option)
pushIPs(L, ips)
xlua.PushNumber(L, ttl)
xlua.PushError(L, err)
return 3
})
}
// callLuaQuery runs HandleDNSQuery and leaves (ips, ttl, err) on the stack.
func callLuaQuery(L *lua.LState, domain string, option featureDNS.IPOption) error {
fn := L.GetGlobal("HandleDNSQuery")
if fn.Type() != lua.LTFunction {
return errors.New("DNS script must define HandleDNSQuery(...)")
}
return L.CallByParam(lua.P{Fn: fn, NRet: 3, Protect: true},
lua.LString(strings.ToLower(domain)),
lua.LBool(option.IPv4Enable),
lua.LBool(option.IPv6Enable),
lua.LBool(option.FakeEnable))
}
// readLuaQueryResult reads (ips, ttl, err) from the stack without copying the IPs.
func readLuaQueryResult(L *lua.LState) ([]net.IP, uint32, error) {
if err := xlua.ReadError(L.Get(-1), "DNS script error must be an error or string"); err != nil {
return nil, 0, err
}
ttl, err := xlua.ReadUint32(L.Get(-2), "DNS script returned invalid TTL")
if err != nil {
return nil, 0, err
}
addresses := L.Get(-3)
if addresses == lua.LNil {
return nil, 0, featureDNS.ErrEmptyResponse
}
ips, err := xlua.ReadUserData[[]net.IP](addresses, "DNS script IPs must be native IP slice userdata")
if err != nil {
return nil, 0, err
}
if len(ips) == 0 {
return nil, 0, featureDNS.ErrEmptyResponse
}
return ips, ttl, nil
}
+121
View File
@@ -0,0 +1,121 @@
package dns
import (
"context"
"os"
"path/filepath"
"testing"
"time"
"github.com/xtls/xray-core/common/net"
featureDNS "github.com/xtls/xray-core/features/dns"
lua "github.com/yuin/gopher-lua"
)
// BenchmarkLuaDNSHook isolates scalar argument bridging and a fixed return.
// It excludes upstream queries, result decoding, and state pool management.
func BenchmarkLuaDNSHook(b *testing.B) {
L := lua.NewState()
b.Cleanup(L.Close)
if err := L.DoString(`
function HandleDNSQuery(domain, ipv4, ipv6, fake)
return true
end
`); err != nil {
b.Fatal(err)
}
L.SetContext(context.Background())
option := featureDNS.IPOption{IPv4Enable: true}
if err := callLuaQuery(L, "example.com", option); err != nil {
b.Fatal(err)
}
if L.Get(-3) != lua.LTrue {
b.Fatal("hook did not return true")
}
L.Pop(3)
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
if err := callLuaQuery(L, "example.com", option); err != nil {
b.Fatal(err)
}
L.Pop(3)
}
}
// BenchmarkLuaDNSQuery queries the same preselected, in-memory upstream.
// client_query compares Client.QueryIP to a preloaded server:Query hook.
// script_query additionally measures production pool and timeout management.
// These cases do not measure DNS.LookupIP server selection or network latency.
func BenchmarkLuaDNSQuery(b *testing.B) {
ctx := context.Background()
option := featureDNS.IPOption{IPv4Enable: true}
ip := net.ParseIP("127.0.0.1")
upstream := &benchmarkLuaNameServer{ips: []net.IP{ip}}
client := &Client{server: upstream, ipOption: &option, timeoutMs: time.Second}
server := &DNS{ctx: ctx, clients: []*Client{client}}
const script = `
local server = require("xray.dns").Servers[1]
function HandleDNSQuery(domain, ipv4, ipv6, fake)
return server:Query(domain, ipv4, ipv6, fake)
end
`
L := lua.NewState()
b.Cleanup(L.Close)
server.registerLua(L)
if err := L.DoString(script); err != nil {
b.Fatal(err)
}
L.SetContext(ctx)
path := filepath.Join(b.TempDir(), "query.lua")
if err := os.WriteFile(path, []byte(script), 0o600); err != nil {
b.Fatal(err)
}
engine, err := newScriptEngine(path)
if err != nil {
b.Fatal(err)
}
if err := engine.start(server); err != nil {
b.Fatal(err)
}
b.Cleanup(engine.close)
for _, bench := range []struct {
name string
query func() ([]net.IP, uint32, error)
}{
{"client_query/native", func() ([]net.IP, uint32, error) {
return client.QueryIP(ctx, "example.com", option)
}},
{"client_query/lua", func() ([]net.IP, uint32, error) {
if err := callLuaQuery(L, "example.com", option); err != nil {
return nil, 0, err
}
ips, ttl, err := readLuaQueryResult(L)
L.Pop(3)
return ips, ttl, err
}},
{"script_query/lua", func() ([]net.IP, uint32, error) {
return engine.query("example.com", option)
}},
} {
b.Run(bench.name, func(b *testing.B) {
ips, ttl, err := bench.query()
if err != nil || ttl != 60 || len(ips) != 1 || !ips[0].Equal(ip) {
b.Fatalf("query() = %v, TTL %d, %v; want %v, TTL 60", ips, ttl, err, ip)
}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
ips, ttl, err = bench.query()
if err != nil {
b.Fatal(err)
}
}
b.StopTimer()
if ttl != 60 || len(ips) != 1 || !ips[0].Equal(ip) {
b.Fatalf("query() = %v, TTL %d; want %v, TTL 60", ips, ttl, ip)
}
})
}
}
+272
View File
@@ -0,0 +1,272 @@
package dns
import (
"context"
go_errors "errors"
"strings"
"testing"
"time"
"github.com/xtls/xray-core/common/geodata"
"github.com/xtls/xray-core/common/net"
featureDNS "github.com/xtls/xray-core/features/dns"
"github.com/xtls/xray-core/features/dns/localdns"
lua "github.com/yuin/gopher-lua"
)
func TestReadLuaQueryResult(t *testing.T) {
wantIPs := []net.IP{net.ParseIP("8.8.8.8"), {127, 0, 0, 1}, net.ParseIP("::1")}
nativeErr := go_errors.New("upstream failed")
for _, tc := range []struct {
name, values string
wantIPs []net.IP
wantTTL uint32
wantErr error
wantMessage string
}{
{name: "IPs", values: `ips, 45`, wantIPs: wantIPs, wantTTL: 45},
{name: "nil IPs", values: `nil, 0`, wantErr: featureDNS.ErrEmptyResponse},
{name: "empty IPs", values: `emptyIPs, 0`, wantErr: featureDNS.ErrEmptyResponse},
{name: "native error", values: `nil, nil, nativeError`, wantErr: nativeErr},
{name: "string error", values: `nil, nil, "blocked"`, wantMessage: "blocked"},
{name: "fractional TTL", values: `ips, 1.5`, wantMessage: "invalid TTL"},
{name: "oversized TTL", values: `ips, 4294967296`, wantMessage: "invalid TTL"},
{name: "negative TTL", values: `ips, -1`, wantMessage: "invalid TTL"},
{name: "NaN TTL", values: `ips, 0/0`, wantMessage: "invalid TTL"},
{name: "missing TTL", values: `ips`, wantMessage: "invalid TTL"},
{name: "string IPs", values: `"127.0.0.1", 60`, wantMessage: "native IP slice"},
{name: "wrong userdata", values: `ip, 60`, wantMessage: "native IP slice"},
{name: "invalid error", values: `ips, 60, false`, wantMessage: "error or string"},
} {
t.Run(tc.name, func(t *testing.T) {
L := lua.NewState()
defer L.Close()
for name, value := range map[string]any{"ips": wantIPs, "ip": wantIPs[0], "emptyIPs": []net.IP(nil), "nativeError": nativeErr} {
ud := L.NewUserData()
ud.Value = value
L.SetGlobal(name, ud)
}
fn, err := L.LoadString("return " + tc.values)
if err != nil {
t.Fatal(err)
}
if err := L.CallByParam(lua.P{Fn: fn, NRet: 3, Protect: true}); err != nil {
t.Fatal(err)
}
ips, ttl, err := readLuaQueryResult(L)
switch {
case tc.wantErr != nil:
if err != tc.wantErr {
t.Fatalf("error = %v, want original error %v", err, tc.wantErr)
}
case tc.wantMessage != "":
if err == nil || !strings.Contains(err.Error(), tc.wantMessage) {
t.Fatalf("error = %v, want %q", err, tc.wantMessage)
}
case err != nil:
t.Fatal(err)
}
if ttl != tc.wantTTL || len(ips) != len(tc.wantIPs) {
t.Fatalf("result = %v, TTL %d; want %v, TTL %d", ips, ttl, tc.wantIPs, tc.wantTTL)
}
for i := range ips {
if !ips[i].Equal(tc.wantIPs[i]) {
t.Fatalf("IP %d = %v, want %v", i, ips[i], tc.wantIPs[i])
}
}
if len(ips) != 0 && &ips[0] != &tc.wantIPs[0] {
t.Fatal("result copied the IP slice")
}
})
}
}
func TestCallLuaQueryCancellation(t *testing.T) {
L := lua.NewState()
defer L.Close()
if err := L.DoString(`function HandleDNSQuery(domain, ipv4, ipv6, fake) while true do end end`); err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
L.SetContext(ctx)
err := callLuaQuery(L, "example.com", featureDNS.IPOption{IPv4Enable: true})
if err == nil {
t.Fatal("callLuaQuery did not stop after context cancellation")
}
if L.Context() != ctx {
t.Fatal("callLuaQuery changed the Lua state's context")
}
}
func TestCallLuaQuery(t *testing.T) {
L := lua.NewState()
defer L.Close()
addresses := L.NewUserData()
addresses.Value = []net.IP{net.ParseIP("127.0.0.1")}
L.SetGlobal("ips", addresses)
if err := L.DoString(`
function HandleDNSQuery(domain, ipv4, ipv6, fake)
assert(domain == "example.com")
assert(ipv4 and not ipv6 and not fake)
return ips, 60, nil
end
`); err != nil {
t.Fatal(err)
}
if err := callLuaQuery(L, "ExAmPlE.CoM", featureDNS.IPOption{IPv4Enable: true}); err != nil {
t.Fatal(err)
}
if L.GetTop() != 3 || L.Get(1) != addresses || L.Get(2) != lua.LNumber(60) || L.Get(3) != lua.LNil {
t.Fatal("callLuaQuery did not leave the three query results on the stack")
}
}
func TestLuaDNSServerQuery(t *testing.T) {
L := lua.NewState()
defer L.Close()
geodata.RegisterLua(L)
option := featureDNS.IPOption{IPv4Enable: true}
ips := []net.IP{net.ParseIP("127.0.0.1"), net.ParseIP("8.8.8.8")}
server := &DNS{clients: []*Client{{server: &benchmarkLuaNameServer{ips: ips}, ipOption: &option, timeoutMs: time.Second}}}
server.registerLua(L)
if err := L.DoString(`
local server = require("xray.dns").Servers[1]
local matcher = require("xray.geodata").BuildIPMatcher("127.0.0.0/8")
function HandleDNSQuery(domain, ipv4, ipv6, fake)
local ips, ttl, err = server:Query(domain, ipv4, ipv6, fake)
assert(type(ips) == "userdata" and not err)
assert(#ips == 2 and ips[1]:String() == "127.0.0.1" and ips[2]:String() == "8.8.8.8")
assert(matcher:Match(ips[1]) and not matcher:Match(ips[2]))
assert(matcher:AnyMatch(ips))
local matched, unmatched = matcher:FilterIPs(ips)
assert(#matched == 1 and #unmatched == 1)
assert(matched[1]:Equal(ips[1]) and unmatched[1]:Equal(ips[2]))
return matched, ttl, err
end
`); err != nil {
t.Fatal(err)
}
L.SetContext(context.Background())
if err := callLuaQuery(L, "example.com", option); err != nil {
t.Fatal(err)
}
got, ttl, err := readLuaQueryResult(L)
if err != nil || ttl != 60 || len(got) != 1 || !got[0].Equal(ips[0]) {
t.Fatalf("server query = %v, TTL %d, %v", got, ttl, err)
}
}
type luaDNSClient struct {
featureDNS.Client
lookup func(string, featureDNS.IPOption) ([]net.IP, uint32, error)
}
func (c *luaDNSClient) LookupIP(domain string, option featureDNS.IPOption) ([]net.IP, uint32, error) {
return c.lookup(domain, option)
}
func TestLuaDNSClientQuery(t *testing.T) {
L := lua.NewState()
defer L.Close()
L.SetContext(context.Background())
geodata.RegisterLua(L)
want := []net.IP{{127, 0, 0, 1}, net.ParseIP("::1")}
client := &luaDNSClient{lookup: func(domain string, option featureDNS.IPOption) ([]net.IP, uint32, error) {
if domain != "MiXeD.Example." || !option.IPv4Enable || option.IPv6Enable || !option.FakeEnable {
t.Fatalf("dns.Query arguments = %q, %+v", domain, option)
}
return want, 42, nil
}}
RegisterLua(L, client)
if err := L.DoString(`
local dns = require("xray.dns")
local matcher = require("xray.geodata").BuildIPMatcher("127.0.0.1")
assert(dns.Servers == nil)
ips, ttl, err = dns.Query("MiXeD.Example.", true, false, true)
assert(not err and ttl == 42 and matcher:AnyMatch(ips))
assert(#ips == 2 and ips[1]:String() == "127.0.0.1" and ips[2]:String() == "::1")
assert(matcher:Match(ips[1]) and not matcher:Match(ips[2]))
`); err != nil {
t.Fatal(err)
}
got := L.GetGlobal("ips").(*lua.LUserData).Value.([]net.IP)
if &got[0] != &want[0] {
t.Fatal("dns.Query copied the IP slice")
}
}
func TestLuaDNSLocalClient(t *testing.T) {
L := lua.NewState()
defer L.Close()
L.SetContext(context.Background())
RegisterLua(L, localdns.New())
if err := L.DoString(`
local dns = require("xray.dns")
assert(dns.Servers[1].ID == "localhost")
serverIPs, _, serverErr = dns.Servers[1]:Query("127.0.0.1", true, false, false)
clientIPs, _, clientErr = dns.Query("127.0.0.1", true, false, false)
assert(not serverErr and not clientErr)
assert(#serverIPs == 1 and #clientIPs == 1)
assert(serverIPs[1]:String() == "127.0.0.1" and serverIPs[1]:Equal(clientIPs[1]))
`); err != nil {
t.Fatal(err)
}
for _, name := range []string{"serverIPs", "clientIPs"} {
ips := L.GetGlobal(name).(*lua.LUserData).Value.([]net.IP)
if len(ips) != 1 || !ips[0].Equal(net.ParseIP("127.0.0.1")) {
t.Fatalf("%s = %v", name, ips)
}
}
}
func TestLuaDNSQueryEmptyIPs(t *testing.T) {
for _, tc := range []struct {
name string
ips []net.IP
}{
{"nil", nil},
{"empty", []net.IP{}},
} {
t.Run(tc.name, func(t *testing.T) {
L := lua.NewState()
defer L.Close()
L.SetContext(context.Background())
L.SetGlobal("expectNil", lua.LBool(tc.ips == nil))
client := &luaDNSClient{lookup: func(string, featureDNS.IPOption) ([]net.IP, uint32, error) {
return tc.ips, 0, featureDNS.ErrEmptyResponse
}}
registerLua(L, []luaDNSServer{{query: func(_ context.Context, domain string, option featureDNS.IPOption) ([]net.IP, uint32, error) {
return client.LookupIP(domain, option)
}}}, client)
if err := L.DoString(`
local dns = require("xray.dns")
for _, query in ipairs({
function() return dns.Servers[1]:Query("empty.example", true, false, false) end,
function() return dns.Query("empty.example", true, false, false) end,
}) do
local ips, ttl, err = query()
assert(ttl == 0 and err)
if expectNil then
assert(ips == nil)
else
assert(type(ips) == "userdata" and #ips == 0)
assert(not pcall(function() return ips[1] end))
end
end
`); err != nil {
t.Fatal(err)
}
})
}
}
type benchmarkLuaNameServer struct {
ips []net.IP
}
func (*benchmarkLuaNameServer) Name() string { return "benchmark" }
func (*benchmarkLuaNameServer) IsDisableCache() bool { return true }
func (s *benchmarkLuaNameServer) QueryIP(context.Context, string, featureDNS.IPOption) ([]net.IP, uint32, error) {
return s.ips, 60, nil
}
+6 -5
View File
@@ -29,6 +29,7 @@ type Server interface {
// Client is the interface for DNS client.
type Client struct {
id string
server Server
skipFallback bool
expectedIPs geodata.IPMatcher
@@ -84,7 +85,7 @@ func NewServer(ctx context.Context, dest net.Destination, dispatcher routing.Dis
if dest.Network == net.Network_UDP { // UDP classic DNS mode
return NewClassicNameServer(dest, dispatcher, disableCache, serveStale, serveExpiredTTL, clientIP), nil
}
return nil, errors.New("No available name server could be created from ", dest).AtWarning()
return nil, errors.New("No available name server could be created from ", dest)
}
// NewClient creates a DNS client managing a name server with client IP, domain rules and expected IPs.
@@ -97,12 +98,12 @@ func NewClient(
ipOption dns.IPOption,
updateRules func(bool),
) (*Client, error) {
client := &Client{}
client := &Client{id: ns.Id}
err := core.RequireFeatures(ctx, func(dispatcher routing.Dispatcher) error {
// Create a new server for each client for now
server, err := NewServer(ctx, ns.Address.AsDestination(), dispatcher, disableCache, serveStale, serveExpiredTTL, clientIP)
if err != nil {
return errors.New("failed to create nameserver").Base(err).AtWarning()
return errors.New("failed to create nameserver").Base(err)
}
_, isLocalDNS := server.(*LocalNameServer)
@@ -113,7 +114,7 @@ func NewClient(
if len(ns.ExpectedIp) > 0 {
expectedMatcher, err = geodata.IPReg.BuildIPMatcher(ns.ExpectedIp)
if err != nil {
return errors.New("failed to create expected ip matcher").Base(err).AtWarning()
return errors.New("failed to create expected ip matcher").Base(err)
}
}
@@ -122,7 +123,7 @@ func NewClient(
if len(ns.UnexpectedIp) > 0 {
unexpectedMatcher, err = geodata.IPReg.BuildIPMatcher(ns.UnexpectedIp)
if err != nil {
return errors.New("failed to create unexpected ip matcher").Base(err).AtWarning()
return errors.New("failed to create unexpected ip matcher").Base(err)
}
}
+2 -2
View File
@@ -27,7 +27,7 @@ func (s *FakeDNSServer) IsDisableCache() bool {
func (f *FakeDNSServer) QueryIP(ctx context.Context, domain string, opt dns.IPOption) ([]net.IP, uint32, error) {
if f.fakeDNSEngine == nil {
return nil, 0, errors.New("Unable to locate a fake DNS Engine").AtError()
return nil, 0, errors.New("Unable to locate a fake DNS Engine")
}
var ips []net.Address
@@ -39,7 +39,7 @@ func (f *FakeDNSServer) QueryIP(ctx context.Context, domain string, opt dns.IPOp
netIP, err := toNetIP(ips)
if err != nil {
return nil, 0, errors.New("Unable to convert IP to net ip").Base(err).AtError()
return nil, 0, errors.New("Unable to convert IP to net ip").Base(err)
}
errors.LogInfo(ctx, f.Name(), " got answer: ", domain, " -> ", ips)
+1 -1
View File
@@ -49,5 +49,5 @@ func NewLocalNameServer() *LocalNameServer {
// NewLocalDNSClient creates localdns client object for directly lookup in system DNS.
func NewLocalDNSClient(ipOption dns.IPOption) *Client {
return &Client{server: NewLocalNameServer(), ipOption: &ipOption}
return &Client{id: "localhost", server: NewLocalNameServer(), ipOption: &ipOption}
}
+71
View File
@@ -0,0 +1,71 @@
package dns
import (
"time"
"github.com/xtls/xray-core/common/errors"
"github.com/xtls/xray-core/common/geodata"
"github.com/xtls/xray-core/common/log"
xlua "github.com/xtls/xray-core/common/lua"
"github.com/xtls/xray-core/common/net"
"github.com/xtls/xray-core/features/dns"
lua "github.com/yuin/gopher-lua"
)
const scriptExecutionTimeout = 6 * time.Second
type scriptEngine struct {
program *xlua.Program
pool *xlua.Pool
}
func newScriptEngine(script string) (*scriptEngine, error) {
program, err := xlua.Compile(script)
if err != nil {
return nil, err
}
return &scriptEngine{program: program}, nil
}
func (e *scriptEngine) start(server *DNS) error {
pool, err := xlua.NewPool(server.ctx, scriptExecutionTimeout, e.program.NewStateFactory(
scriptExecutionTimeout*20,
256,
func(L *lua.LState) {
geodata.RegisterLua(L)
log.RegisterLua(L)
server.registerLua(L)
},
func(L *lua.LState) error {
if L.GetGlobal("HandleDNSQuery").Type() != lua.LTFunction {
return errors.New("DNS script must define HandleDNSQuery(...)")
}
return nil
}))
if err != nil {
return err
}
e.pool = pool
errors.LogInfo(server.ctx, "DNS script initialized")
return nil
}
func (e *scriptEngine) close() {
if e.pool != nil {
e.pool.Close()
}
}
func (e *scriptEngine) query(domain string, option dns.IPOption) (ips []net.IP, ttl uint32, queryErr error) {
if err := e.pool.WithState(nil, 0, func(L *lua.LState) error {
if err := callLuaQuery(L, domain, option); err != nil {
return err
}
ips, ttl, queryErr = readLuaQueryResult(L)
return nil
}); err != nil {
return nil, 0, err
}
return ips, ttl, queryErr
}
+329
View File
@@ -0,0 +1,329 @@
package dns
import (
"context"
go_errors "errors"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/xtls/xray-core/common/net"
featureDNS "github.com/xtls/xray-core/features/dns"
)
type scriptNameServer struct {
name string
answers map[string]net.IP
errors map[string]error
ttl uint32
calls int
}
func (s *scriptNameServer) Name() string { return s.name }
func (s *scriptNameServer) IsDisableCache() bool { return true }
func (s *scriptNameServer) QueryIP(ctx context.Context, domain string, _ featureDNS.IPOption) ([]net.IP, uint32, error) {
if err := ctx.Err(); err != nil {
return nil, 0, err
}
s.calls++
if err := s.errors[domain]; err != nil {
return nil, 0, err
}
ip, ok := s.answers[domain]
if !ok {
return nil, 0, featureDNS.ErrEmptyResponse
}
return []net.IP{ip}, s.ttl, nil
}
func TestDNSScriptQuery(t *testing.T) {
wantIP := net.ParseIP("127.0.0.1")
upstreamErr := go_errors.New("upstream failed")
for _, tc := range []struct {
name, body string
wantIPs []net.IP
wantTTL uint32
wantErr error
wantMessage string
wantCalls uint32
}{
{name: "IPs", body: `return server:Query(domain, ipv4, ipv6, fake)`, wantIPs: []net.IP{wantIP}, wantTTL: 60, wantCalls: 2},
{name: "empty result", body: `return nil, 0`, wantErr: featureDNS.ErrEmptyResponse, wantCalls: 2},
{name: "upstream error", body: `return server:Query("failed.example", ipv4, ipv6, fake)`, wantErr: upstreamErr, wantCalls: 2},
{name: "string error", body: `return nil, nil, "blocked"`, wantMessage: "blocked", wantCalls: 2},
{name: "invalid result", body: `return false, 0`, wantMessage: "native IP slice", wantCalls: 2},
{name: "execution error", body: `error("execution failed")`, wantMessage: "execution failed", wantCalls: 1},
} {
t.Run(tc.name, func(t *testing.T) {
script := `
local server = require("xray.dns").Servers[1]
local calls = 0
function HandleDNSQuery(domain, ipv4, ipv6, fake)
calls = calls + 1
if domain == "count.example" then
local ips, _, err = server:Query("good.example", ipv4, ipv6, fake)
return ips, calls, err
end
` + tc.body + `
end
`
path := filepath.Join(t.TempDir(), "query.lua")
if err := os.WriteFile(path, []byte(script), 0o600); err != nil {
t.Fatal(err)
}
option := featureDNS.IPOption{IPv4Enable: true}
upstream := &scriptNameServer{
name: "test",
answers: map[string]net.IP{"good.example": wantIP},
errors: map[string]error{"failed.example": upstreamErr},
ttl: 60,
}
server := &DNS{
ctx: context.Background(),
clients: []*Client{{server: upstream, ipOption: &option, timeoutMs: time.Second}},
}
engine, err := newScriptEngine(path)
if err != nil {
t.Fatal(err)
}
if err := engine.start(server); err != nil {
t.Fatal(err)
}
defer engine.close()
ips, ttl, err := engine.query("good.example", option)
switch {
case tc.wantErr != nil:
if err != tc.wantErr {
t.Fatalf("query error = %v, want original error %v", err, tc.wantErr)
}
case tc.wantMessage != "":
if err == nil || !strings.Contains(err.Error(), tc.wantMessage) {
t.Fatalf("query error = %v, want %q", err, tc.wantMessage)
}
case err != nil:
t.Fatal(err)
}
if ttl != tc.wantTTL || len(ips) != len(tc.wantIPs) {
t.Fatalf("query = %v, TTL %d; want %v, TTL %d", ips, ttl, tc.wantIPs, tc.wantTTL)
}
for i := range ips {
if !ips[i].Equal(tc.wantIPs[i]) {
t.Fatalf("IP %d = %v, want %v", i, ips[i], tc.wantIPs[i])
}
}
ips, calls, err := engine.query("count.example", option)
if err != nil || calls != tc.wantCalls || len(ips) != 1 || !ips[0].Equal(wantIP) {
t.Fatalf("next query = %v, calls %d, %v; want %v, calls %d", ips, calls, err, wantIP, tc.wantCalls)
}
})
}
}
func TestDNSScriptGeoIPFallback(t *testing.T) {
t.Setenv("xray.location.asset", filepath.Join("..", "..", "resources"))
script := `
local servers = require("xray.dns").Servers
local ru_ips = require("xray.geodata").BuildIPMatcher("geoip:ru")
local by_id = {}
for _, server in ipairs(servers) do
by_id[server.ID] = server
end
assert(by_id.primary and by_id.fallback, "primary and fallback DNS servers are required")
function HandleDNSQuery(domain, ipv4, ipv6, fake)
local ips, ttl, err = by_id.primary:Query(domain, ipv4, ipv6, fake)
if not err and ru_ips:AnyMatch(ips) then
return ips, ttl, nil
end
return by_id.fallback:Query(domain, ipv4, ipv6, fake)
end
`
scriptPath := filepath.Join(t.TempDir(), "geoip_fallback.lua")
if err := os.WriteFile(scriptPath, []byte(script), 0o600); err != nil {
t.Fatal(err)
}
primary := &scriptNameServer{
name: "primary",
answers: map[string]net.IP{
"ru.example": net.ParseIP("2a02:6b8::feed:0ff"),
"other.example": net.ParseIP("127.0.0.1"),
},
ttl: 30,
}
fallback := &scriptNameServer{
name: "fallback",
answers: map[string]net.IP{"other.example": net.ParseIP("9.9.9.9")},
ttl: 60,
}
option := featureDNS.IPOption{IPv4Enable: true, IPv6Enable: true}
hosts, err := NewStaticHosts(nil)
if err != nil {
t.Fatal(err)
}
engine, err := newScriptEngine(scriptPath)
if err != nil {
t.Fatal(err)
}
server := &DNS{
ctx: context.Background(),
hosts: hosts,
ipOption: &option,
script: engine,
clients: []*Client{
{id: "primary", server: primary, ipOption: &option, timeoutMs: 2 * time.Second},
{id: "fallback", server: fallback, ipOption: &option, timeoutMs: 2 * time.Second},
},
}
if err := server.Start(); err != nil {
t.Fatal(err)
}
defer server.Close()
for _, tc := range []struct {
domain string
ip net.IP
ttl uint32
}{
{"Ru.Example.", net.ParseIP("2a02:6b8::feed:0ff"), 30},
{"other.example", net.ParseIP("9.9.9.9"), 60},
} {
ips, ttl, err := server.LookupIP(tc.domain, option)
if err != nil {
t.Fatalf("LookupIP(%q): %v", tc.domain, err)
}
if ttl != tc.ttl || len(ips) != 1 || !ips[0].Equal(tc.ip) {
t.Fatalf("LookupIP(%q) = %v, TTL %d; want %v, TTL %d", tc.domain, ips, ttl, tc.ip, tc.ttl)
}
}
if primary.calls != 2 || fallback.calls != 1 {
t.Fatalf("upstream calls: primary %d, fallback %d; want 2 and 1", primary.calls, fallback.calls)
}
}
func TestDNSScriptRejectsInvalidStartup(t *testing.T) {
for _, tc := range []struct {
name, script string
compileError bool
}{
{"syntax", "function HandleDNSQuery(", true},
{"missing hook", "value = 1", false},
{"invalid hook", "HandleDNSQuery = true", false},
{"top-level error", `error("setup failed")`, false},
} {
t.Run(tc.name, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "script.lua")
if err := os.WriteFile(path, []byte(tc.script), 0o600); err != nil {
t.Fatal(err)
}
server, err := New(context.Background(), &Config{Script: path})
if tc.compileError {
if server != nil || err == nil || !strings.Contains(err.Error(), "failed to prepare DNS script") {
t.Fatalf("New = %v, %v; want nil server and script preparation error", server, err)
}
return
}
if err != nil {
t.Fatal(err)
}
defer server.Close()
if server.script.program == nil || server.script.pool != nil {
t.Fatal("New did not compile the script without starting its pool")
}
if err := server.Close(); err != nil {
t.Fatal(err)
}
if err := server.Start(); err == nil {
t.Fatal("Start accepted an invalid DNS script")
}
if server.script.pool != nil {
t.Fatal("Start retained a script pool after failure")
}
})
}
}
func TestDNSScriptProgramSnapshot(t *testing.T) {
path := filepath.Join(t.TempDir(), "script.lua")
if err := os.WriteFile(path, []byte(`function HandleDNSQuery() return nil, nil, "compiled" end`), 0o600); err != nil {
t.Fatal(err)
}
engine, err := newScriptEngine(path)
if err != nil {
t.Fatal(err)
}
defer engine.close()
if err := os.Remove(path); err != nil {
t.Fatal(err)
}
if err := engine.start(&DNS{ctx: context.Background()}); err != nil {
t.Fatal(err)
}
if _, _, err := engine.query("example.com", featureDNS.IPOption{}); err == nil || !strings.Contains(err.Error(), "compiled") {
t.Fatalf("query error = %v, want compiled", err)
}
}
func TestDNSScriptFakeDNSOption(t *testing.T) {
path := filepath.Join(t.TempDir(), "script.lua")
script := `
local server = require("xray.dns").Servers[1]
local log = require("xray.log")
log.Info("DNS script loaded")
function HandleDNSQuery(domain, ipv4, ipv6, fake)
log.Debug("DNS query: ", domain)
local ips, ttl, err = server:Query(domain, ipv4, ipv6, fake)
if err then log.Error("DNS failed: ", err) end
return ips, ttl, err
end
`
if err := os.WriteFile(path, []byte(script), 0o600); err != nil {
t.Fatal(err)
}
option := featureDNS.IPOption{IPv4Enable: true}
hosts, err := NewStaticHosts(nil)
if err != nil {
t.Fatal(err)
}
upstream := &scriptNameServer{
name: "FakeDNS",
answers: map[string]net.IP{"good.example": net.ParseIP("198.18.0.1")},
ttl: 30,
}
engine, err := newScriptEngine(path)
if err != nil {
t.Fatal(err)
}
server := &DNS{
ctx: context.Background(),
hosts: hosts,
ipOption: &option,
script: engine,
clients: []*Client{{id: "fake", server: upstream, ipOption: &option, timeoutMs: time.Second}},
}
if err := server.Start(); err != nil {
t.Fatal(err)
}
defer server.Close()
if _, _, err := server.LookupIP("good.example", option); err != featureDNS.ErrEmptyResponse {
t.Fatalf("FakeDNS without FakeEnable = %v, want ErrEmptyResponse", err)
}
if upstream.calls != 0 {
t.Fatalf("FakeDNS was queried without FakeEnable: %d calls", upstream.calls)
}
withFake := featureDNS.IPOption{IPv4Enable: true, FakeEnable: true}
ips, ttl, err := server.LookupIP("good.example", withFake)
if err != nil || ttl != 30 || len(ips) != 1 || !ips[0].Equal(net.ParseIP("198.18.0.1")) {
t.Fatalf("FakeDNS with FakeEnable = %v, TTL %d, %v", ips, ttl, err)
}
if upstream.calls != 1 {
t.Fatalf("FakeDNS query count = %d, want 1", upstream.calls)
}
}
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/geodata/config.proto
package geodata
+62 -58
View File
@@ -1,6 +1,8 @@
package geodata
import (
"bufio"
"compress/gzip"
"context"
"crypto/tls"
go_errors "errors"
@@ -21,7 +23,13 @@ import (
"golang.org/x/net/http2"
)
const idleTimeout = 30 * time.Second
const (
idleTimeout = 30 * time.Second
maxDownloadSize int64 = 256 * 1024 * 1024
)
// DO NOT EDIT. Ignored by geodata, rejected by Lua.
const luaGuardHeader = "\x12\x01\x00"
type stage struct {
target string
@@ -30,7 +38,6 @@ type stage struct {
type downloader struct {
ctx context.Context
httpClient *http.Client
httpsClient *http.Client
}
@@ -58,12 +65,11 @@ func (c *idleConn) Write(b []byte) (int, error) {
func newDownloader(ctx context.Context, dispatcher routing.Dispatcher, outbound string) *downloader {
return &downloader{
ctx: ctx,
httpClient: newClient(ctx, dispatcher, outbound, false),
httpsClient: newClient(ctx, dispatcher, outbound, true),
httpsClient: newClient(ctx, dispatcher, outbound),
}
}
func newClient(baseCtx context.Context, dispatcher routing.Dispatcher, outbound string, isHTTPS bool) *http.Client {
func newClient(baseCtx context.Context, dispatcher routing.Dispatcher, outbound string) *http.Client {
dial := func(ctx context.Context, network, address string) (net.Conn, error) {
var conn net.Conn
err := task.Run(ctx, func() error {
@@ -88,53 +94,33 @@ func newClient(baseCtx context.Context, dispatcher routing.Dispatcher, outbound
Conn: conn,
}, nil
}
if isHTTPS {
return &http.Client{
Transport: &http2.Transport{
DialTLSContext: func(ctx context.Context, network string, address string, cfg *tls.Config) (net.Conn, error) {
conn, err := dial(ctx, network, address)
if err != nil {
return nil, err
}
host, _, _ := net.SplitHostPort(address)
tlsConn := utls.UClient(conn, &utls.Config{ServerName: host}, utls.HelloChrome_Auto)
handshakeCtx, cancel := context.WithTimeout(ctx, idleTimeout)
defer cancel()
if err := tlsConn.HandshakeContext(handshakeCtx); err != nil {
conn.Close()
return nil, err
}
return tlsConn, nil
},
},
CheckRedirect: func(req *http.Request, via []*http.Request) error {
if req.URL.Scheme != "https" {
return errors.New("redirected to non-https URL: ", req.URL.String())
return &http.Client{
Transport: &http2.Transport{
DialTLSContext: func(ctx context.Context, network string, address string, cfg *tls.Config) (net.Conn, error) {
conn, err := dial(ctx, network, address)
if err != nil {
return nil, err
}
if len(via) >= 10 {
return errors.New("stopped after 10 redirects")
host, _, _ := net.SplitHostPort(address)
tlsConn := utls.UClient(conn, &utls.Config{ServerName: host}, utls.HelloChrome_Auto)
handshakeCtx, cancel := context.WithTimeout(ctx, idleTimeout)
defer cancel()
if err := tlsConn.HandshakeContext(handshakeCtx); err != nil {
conn.Close()
return nil, err
}
return nil
return tlsConn, nil
},
}
} else {
return &http.Client{
Transport: &http.Transport{
Proxy: nil,
DisableKeepAlives: true,
DialContext: dial,
ResponseHeaderTimeout: idleTimeout,
},
CheckRedirect: func(req *http.Request, via []*http.Request) error {
if req.URL.Scheme != "https" {
return errors.New("redirected to non-https URL: ", req.URL.String())
}
if len(via) >= 10 {
return errors.New("stopped after 10 redirects")
}
return nil
},
}
},
CheckRedirect: func(req *http.Request, via []*http.Request) error {
if req.URL.Scheme != "https" {
return errors.New("redirected to non-https URL: ", req.URL.String())
}
if len(via) >= 20 {
return errors.New("stopped after 20 redirects")
}
return nil
},
}
}
@@ -170,6 +156,10 @@ func (d *downloader) downloadOne(asset *Asset) (stage, error) {
}
}()
if _, err := temp.WriteString(luaGuardHeader); err != nil {
temp.Close()
return stage{}, err
}
if err := d.fetch(asset.Url, temp); err != nil {
temp.Close()
return stage{}, err
@@ -196,27 +186,41 @@ func (d *downloader) fetch(rawURL string, writer io.Writer) error {
}
utils.TryDefaultHeadersWith(req.Header, "nav")
var client *http.Client
if req.URL.Scheme == "https" {
client = d.httpsClient
} else {
client = d.httpClient
}
resp, err := client.Do(req)
resp, err := d.httpsClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode < http.StatusOK || resp.StatusCode >= http.StatusMultipleChoices {
io.Copy(io.Discard, resp.Body)
return errors.New("unexpected status code: ", resp.StatusCode)
}
if resp.ContentLength > maxDownloadSize {
return errors.New("geodata download exceeds 256 MiB")
}
n, err := io.Copy(writer, resp.Body)
br := bufio.NewReader(resp.Body)
header, err := br.Peek(2)
if err != nil && err != io.EOF {
return err
}
var body io.Reader = br
if len(header) == 2 && header[0] == 0x1f && header[1] == 0x8b {
zr, err := gzip.NewReader(br)
if err != nil {
return err
}
defer zr.Close()
body = zr
}
n, err := io.Copy(writer, io.LimitReader(body, maxDownloadSize+1))
if err != nil {
return err
}
if n > maxDownloadSize {
return errors.New("geodata download exceeds 256 MiB")
}
if n == 0 {
return errors.New("empty response body")
}
+126
View File
@@ -0,0 +1,126 @@
package geodata
import (
"context"
"errors"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
commongeodata "github.com/xtls/xray-core/common/geodata"
xlua "github.com/xtls/xray-core/common/lua"
"github.com/xtls/xray-core/common/serial"
"github.com/xtls/xray-core/core"
"github.com/yuin/gopher-lua/parse"
"google.golang.org/protobuf/proto"
)
func TestDownloadOneLuaGuard(t *testing.T) {
for _, tc := range []struct {
name string
body string
code string
}{
{"geodata", "\x0a\x03\x0a\x01A", "A"},
{"prefixed geodata", luaGuardHeader + "\x0a\x03\x0a\x01A", "A"},
{"Lua", "function HandleRoute() end", ""},
} {
t.Run(tc.name, func(t *testing.T) {
dir := t.TempDir()
t.Setenv("xray.location.asset", dir)
if err := os.WriteFile(filepath.Join(dir, "asset.lua"), nil, 0o600); err != nil {
t.Fatal(err)
}
server := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(tc.body))
}))
defer server.Close()
d := &downloader{ctx: context.Background(), httpsClient: server.Client()}
staged, err := d.downloadOne(&Asset{Url: server.URL, File: "asset.lua"})
if err != nil {
t.Fatal(err)
}
tx, err := swapAll([]stage{staged})
if err != nil {
t.Fatal(err)
}
if err := tx.commit(); err != nil {
t.Fatal(err)
}
data, err := os.ReadFile(staged.target)
if err != nil || string(data) != luaGuardHeader+tc.body {
t.Fatalf("downloaded data = % x, %v", data, err)
}
program, err := xlua.Compile(staged.target)
var syntaxErr *parse.Error
if program != nil || !errors.As(err, &syntaxErr) || syntaxErr.Pos.Line != 1 || syntaxErr.Pos.Column != 1 {
t.Fatalf("want Lua rejection at the first byte, got %v, %v", program, err)
}
if tc.code != "" {
var list commongeodata.GeoSiteList
if err := proto.Unmarshal(data, &list); err != nil {
t.Fatal(err)
}
if len(list.Entry) != 1 || list.Entry[0].Code != tc.code {
t.Fatalf("header changed geodata: %v", &list)
}
if _, err := commongeodata.ParseDomainRules([]string{"ext:" + filepath.Base(staged.target) + ":" + tc.code}, commongeodata.Domain_Full); err != nil {
t.Fatalf("geodata loader rejected the header: %v", err)
}
}
})
}
}
func TestProtobufConfigValidation(t *testing.T) {
dir := t.TempDir()
t.Setenv("xray.location.asset", dir)
for _, name := range []string{"asset.dat", "asset.DAT"} {
if err := os.WriteFile(filepath.Join(dir, name), nil, 0o600); err != nil {
t.Fatal(err)
}
}
if err := os.Mkdir(filepath.Join(dir, "directory.dat"), 0o700); err != nil {
t.Fatal(err)
}
server, err := core.New(&core.Config{})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = server.Close() })
for _, tc := range []struct {
name string
config *Config
wantError string
}{
{"minutely cron", &Config{Cron: "* * * * *"}, ""},
{"minimum interval", &Config{Cron: "@every 1m"}, ""},
{"short interval", &Config{Cron: "@every 59s"}, "interval must be at least 1 minute"},
{"fractional interval", &Config{Cron: "@every 59.999s"}, "interval must be at least 1 minute"},
{"invalid cron", &Config{Cron: "invalid"}, "invalid geodata cron"},
{"dat extension", &Config{Cron: "@daily", Assets: []*Asset{{Url: "https://example.com/asset.dat", File: "asset.dat"}}}, ""},
{"uppercase extension", &Config{Cron: "@daily", Assets: []*Asset{{Url: "https://example.com/asset.dat", File: "asset.DAT"}}}, ""},
{"invalid extension", &Config{Cron: "@daily", Assets: []*Asset{{Url: "https://example.com/asset.dat", File: "asset.lua"}}}, "extension must be .dat"},
{"missing extension", &Config{Cron: "@daily", Assets: []*Asset{{Url: "https://example.com/asset.dat", File: "asset"}}}, "extension must be .dat"},
{"missing file", &Config{Cron: "@daily", Assets: []*Asset{{Url: "https://example.com/asset.dat", File: "missing.dat"}}}, "invalid geodata asset file"},
{"directory", &Config{Cron: "@daily", Assets: []*Asset{{Url: "https://example.com/asset.dat", File: "directory.dat"}}}, "asset is not a regular file"},
} {
t.Run(tc.name, func(t *testing.T) {
config, err := serial.ToTypedMessage(tc.config).GetInstance()
if err != nil {
t.Fatal(err)
}
_, err = core.CreateObject(server, config)
if tc.wantError == "" {
if err != nil {
t.Fatal(err)
}
} else if err == nil || !strings.Contains(err.Error(), tc.wantError) {
t.Fatalf("CreateObject() error = %v; want %q", err, tc.wantError)
}
})
}
}
+23 -3
View File
@@ -2,12 +2,16 @@ package geodata
import (
"context"
"path/filepath"
"strings"
"sync"
"time"
"github.com/robfig/cron/v3"
"github.com/xtls/xray-core/common"
"github.com/xtls/xray-core/common/errors"
commongeodata "github.com/xtls/xray-core/common/geodata"
"github.com/xtls/xray-core/common/platform/filesystem"
"github.com/xtls/xray-core/core"
"github.com/xtls/xray-core/features/routing"
)
@@ -25,6 +29,24 @@ func New(ctx context.Context, config *Config) (*Instance, error) {
if config.Cron == "" {
return &Instance{}, nil
}
if len(config.Assets) > 100 {
return nil, errors.New("geodata supports at most 100 assets")
}
for _, asset := range config.Assets {
if !strings.EqualFold(filepath.Ext(asset.GetFile()), ".dat") {
return nil, errors.New("geodata asset file extension must be .dat")
}
if _, err := filesystem.ResolveAsset(asset.GetFile()); err != nil {
return nil, errors.New("invalid geodata asset file: ", asset.GetFile()).Base(err)
}
}
schedule, err := cron.ParseStandard(config.Cron)
if err != nil {
return nil, errors.New("invalid geodata cron").Base(err)
}
if interval, ok := schedule.(cron.ConstantDelaySchedule); ok && interval.Delay < time.Minute {
return nil, errors.New("geodata cron interval must be at least 1 minute")
}
g := &Instance{
assets: config.Assets,
@@ -44,9 +66,7 @@ func New(ctx context.Context, config *Config) (*Instance, error) {
cron.WithChain(cron.SkipIfStillRunning(cron.DiscardLogger)),
cron.WithLogger(cron.DiscardLogger),
)
if _, err := g.tasker.AddFunc(config.Cron, g.execute); err != nil {
return nil, errors.New("invalid geodata cron").Base(err)
}
g.tasker.Schedule(schedule, cron.FuncJob(g.execute))
errors.LogInfo(ctx, "scheduled geodata reload with cron: ", config.Cron)
return g, nil
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/log/command/config.proto
package command
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go-grpc. DO NOT EDIT.
// versions:
// - protoc-gen-go-grpc v1.6.0
// - protoc v6.33.5
// - protoc-gen-go-grpc v1.6.2
// - protoc (unknown)
// source: app/log/command/config.proto
package command
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/log/config.proto
package log
+6 -2
View File
@@ -89,10 +89,10 @@ func (g *Instance) startInternal() error {
g.active = true
if err := g.initAccessLogger(); err != nil {
return errors.New("failed to initialize access logger").Base(err).AtWarning()
return errors.New("failed to initialize access logger").Base(err)
}
if err := g.initErrorLogger(); err != nil {
return errors.New("failed to initialize error logger").Base(err).AtWarning()
return errors.New("failed to initialize error logger").Base(err)
}
return nil
@@ -141,6 +141,10 @@ func (g *Instance) Handle(msg log.Message) {
}
}
func (g *Instance) Severity() log.Severity {
return g.config.ErrorLogLevel
}
// Close implements common.Closable.Close().
func (g *Instance) Close() error {
errors.LogDebug(context.Background(), "Logger closing")
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/metrics/config.proto
package metrics
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/observatory/burst/config.proto
package burst
+1 -1
View File
@@ -109,7 +109,7 @@ func (h *HealthPingRTTS) getStatistics() *HealthPingStats {
stats.Min = 0
return stats
}
stats.Average = time.Duration(int(sum) / cnt)
stats.Average = sum / time.Duration(cnt)
var std float64
if cnt < 2 {
// no enough data for standard deviation, we assume it's half of the average rtt
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/observatory/command/command.proto
package command
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go-grpc. DO NOT EDIT.
// versions:
// - protoc-gen-go-grpc v1.6.0
// - protoc v6.33.5
// - protoc-gen-go-grpc v1.6.2
// - protoc (unknown)
// source: app/observatory/command/command.proto
package command
+14 -14
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/observatory/config.proto
package observatory
@@ -152,23 +152,23 @@ func (x *HealthPingMeasurementResult) GetMin() int64 {
type OutboundStatus struct {
state protoimpl.MessageState `protogen:"open.v1"`
// @Document Whether this outbound is usable
// @Restriction ReadOnlyForUser
//@Restriction ReadOnlyForUser
Alive bool `protobuf:"varint,1,opt,name=alive,proto3" json:"alive,omitempty"`
// @Document The time for probe request to finish.
// @Type time.ms
// @Restriction ReadOnlyForUser
//@Type time.ms
//@Restriction ReadOnlyForUser
Delay int64 `protobuf:"varint,2,opt,name=delay,proto3" json:"delay,omitempty"`
// @Document The last error caused this outbound failed to relay probe request
// @Restriction NotMachineReadable
//@Restriction NotMachineReadable
LastErrorReason string `protobuf:"bytes,3,opt,name=last_error_reason,json=lastErrorReason,proto3" json:"last_error_reason,omitempty"`
// @Document The outbound tag for this Server
// @Type id.outboundTag
//@Type id.outboundTag
OutboundTag string `protobuf:"bytes,4,opt,name=outbound_tag,json=outboundTag,proto3" json:"outbound_tag,omitempty"`
// @Document The time this outbound is known to be alive
// @Type id.outboundTag
//@Type id.outboundTag
LastSeenTime int64 `protobuf:"varint,5,opt,name=last_seen_time,json=lastSeenTime,proto3" json:"last_seen_time,omitempty"`
// @Document The time this outbound is tried
// @Type id.outboundTag
//@Type id.outboundTag
LastTryTime int64 `protobuf:"varint,6,opt,name=last_try_time,json=lastTryTime,proto3" json:"last_try_time,omitempty"`
HealthPing *HealthPingMeasurementResult `protobuf:"bytes,7,opt,name=health_ping,json=healthPing,proto3" json:"health_ping,omitempty"`
unknownFields protoimpl.UnknownFields
@@ -257,14 +257,14 @@ func (x *OutboundStatus) GetHealthPing() *HealthPingMeasurementResult {
type ProbeResult struct {
state protoimpl.MessageState `protogen:"open.v1"`
// @Document Whether this outbound is usable
// @Restriction ReadOnlyForUser
//@Restriction ReadOnlyForUser
Alive bool `protobuf:"varint,1,opt,name=alive,proto3" json:"alive,omitempty"`
// @Document The time for probe request to finish.
// @Type time.ms
// @Restriction ReadOnlyForUser
//@Type time.ms
//@Restriction ReadOnlyForUser
Delay int64 `protobuf:"varint,2,opt,name=delay,proto3" json:"delay,omitempty"`
// @Document The error caused this outbound failed to relay probe request
// @Restriction NotMachineReadable
//@Restriction NotMachineReadable
LastErrorReason string `protobuf:"bytes,3,opt,name=last_error_reason,json=lastErrorReason,proto3" json:"last_error_reason,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
@@ -324,7 +324,7 @@ func (x *ProbeResult) GetLastErrorReason() string {
type Intensity struct {
state protoimpl.MessageState `protogen:"open.v1"`
// @Document The time interval for a probe request in ms.
// @Type time.ms
//@Type time.ms
ProbeInterval uint32 `protobuf:"varint,1,opt,name=probe_interval,json=probeInterval,proto3" json:"probe_interval,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/policy/config.proto
package policy
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/proxyman/command/command.proto
package command
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go-grpc. DO NOT EDIT.
// versions:
// - protoc-gen-go-grpc v1.6.0
// - protoc v6.33.5
// - protoc-gen-go-grpc v1.6.2
// - protoc (unknown)
// source: app/proxyman/command/command.proto
package command
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/proxyman/config.proto
package proxyman
+1 -1
View File
@@ -66,7 +66,7 @@ func NewAlwaysOnInboundHandler(ctx context.Context, tag string, receiverConfig *
}
mss, err := internet.ToMemoryStreamConfig(receiverConfig.StreamSettings)
if err != nil {
return nil, errors.New("failed to parse stream config").Base(err).AtWarning()
return nil, errors.New("failed to parse stream config").Base(err)
}
newCtx := session.ContextWithInbound(ctx, &session.Inbound{Tag: tag, Source: src})
+2 -2
View File
@@ -165,7 +165,7 @@ func NewHandler(ctx context.Context, config *core.InboundHandlerConfig) (inbound
receiverSettings, ok := rawReceiverSettings.(*proxyman.ReceiverConfig)
if !ok {
return nil, errors.New("not a ReceiverConfig").AtError()
return nil, errors.New("not a ReceiverConfig")
}
streamSettings := receiverSettings.StreamSettings
@@ -174,7 +174,7 @@ func NewHandler(ctx context.Context, config *core.InboundHandlerConfig) (inbound
Mark: streamSettings.SocketSettings.Mark,
})
}
if streamSettings != nil && streamSettings.ProtocolName == "splithttp" {
if streamSettings != nil && (streamSettings.ProtocolName == "splithttp" || streamSettings.ProtocolName == "masque") {
ctx = session.ContextWithAllowedNetwork(ctx, net.Network_UDP)
}
+2 -2
View File
@@ -142,7 +142,7 @@ func (w *tcpWorker) Start() error {
go w.callback(conn)
})
if err != nil {
return errors.New("failed to listen TCP on ", w.port).AtWarning().Base(err)
return errors.New("failed to listen TCP on ", w.port).Base(err)
}
w.hub = hub
return nil
@@ -528,7 +528,7 @@ func (w *dsWorker) Start() error {
go w.callback(conn)
})
if err != nil {
return errors.New("failed to listen Unix Domain Socket on ", w.address).AtWarning().Base(err)
return errors.New("failed to listen Unix Domain Socket on ", w.address).Base(err)
}
w.hub = hub
return nil
+2 -2
View File
@@ -87,7 +87,7 @@ func NewHandler(ctx context.Context, config *core.OutboundHandlerConfig) (outbou
h.senderSettings = s
mss, err := internet.ToMemoryStreamConfig(s.StreamSettings)
if err != nil {
return nil, errors.New("failed to parse stream settings").Base(err).AtWarning()
return nil, errors.New("failed to parse stream settings").Base(err)
}
h.streamSettings = mss
default:
@@ -217,7 +217,7 @@ func (h *Handler) Dispatch(ctx context.Context, link *transport.Link) {
if ob.Target.Network == net.Network_UDP && ob.Target.Port == 443 {
switch h.udp443 {
case "reject":
test(errors.New("XUDP rejected UDP/443 traffic").AtInfo())
test(errors.New("XUDP rejected UDP/443 traffic"))
return
case "skip":
goto out
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/reverse/config.proto
package reverse
+2 -2
View File
@@ -68,13 +68,13 @@ func (p *Portal) HandleConnection(ctx context.Context, link *transport.Link) err
outbounds := session.OutboundsFromContext(ctx)
ob := outbounds[len(outbounds)-1]
if ob == nil {
return errors.New("outbound metadata not found").AtError()
return errors.New("outbound metadata not found")
}
if isDomain(ob.Target, p.domain) {
muxClient, err := mux.NewClientWorker(*link, mux.ClientStrategy{})
if err != nil {
return errors.New("failed to create mux client worker").Base(err).AtWarning()
return errors.New("failed to create mux client worker").Base(err)
}
worker, err := NewPortalWorker(muxClient)
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/router/command/command.proto
package command
+13 -12
View File
@@ -34,18 +34,19 @@ message RoutingContext {
// opened by xray-core.
// * FieldSelectors selects a subset of fields in routing statistics to return.
// Valid selectors:
// - inbound: Selects connection's inbound tag.
// - network: Selects connection's network.
// - ip: Equivalent as "ip_source" and "ip_target", selects both source and
// target IP.
// - port: Equivalent as "port_source" and "port_target", selects both source
// and target port.
// - domain: Selects target domain.
// - protocol: Select connection's protocol.
// - user: Select connection's inbound user email.
// - attributes: Select connection's additional attributes.
// - outbound: Equivalent as "outbound" and "outbound_group", select both
// outbound tag and outbound group tags.
// - inbound: Selects connection's inbound tag.
// - network: Selects connection's network.
// - ip: Equivalent as "ip_source" and "ip_target", selects both source and
// target IP.
// - port: Equivalent as "port_source" and "port_target", selects both source
// and target port.
// - domain: Selects target domain.
// - protocol: Select connection's protocol.
// - user: Select connection's inbound user email.
// - attributes: Select connection's additional attributes.
// - outbound: Equivalent as "outbound" and "outbound_group", select both
// outbound tag and outbound group tags.
//
// * If FieldSelectors is left empty, all fields will be returned.
message SubscribeRoutingStatsRequest {
repeated string FieldSelectors = 1;
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go-grpc. DO NOT EDIT.
// versions:
// - protoc-gen-go-grpc v1.6.0
// - protoc v6.33.5
// - protoc-gen-go-grpc v1.6.2
// - protoc (unknown)
// source: app/router/command/command.proto
package command
+1 -1
View File
@@ -415,7 +415,7 @@ func BenchmarkMphDomainMatcher(b *testing.B) {
func BenchmarkMultiGeoIPMatcher(b *testing.B) {
b.Setenv("xray.location.asset", filepath.Join("..", "..", "resources"))
rules, err := geodata.ParseIPRules([]string{"geoip:cn", "geoip:jp", "geoip:ca", "geoip:us"})
rules, err := geodata.ParseIPRules([]string{"geoip:cn", "geoip:ir", "geoip:ru"})
common.Must(err)
matcher, err := NewIPMatcher(rules, MatcherAsType_Target)
+2 -2
View File
@@ -115,7 +115,7 @@ func (rr *RoutingRule) BuildCondition() (Condition, error) {
}
if conds.Len() == 0 {
return nil, errors.New("this rule has no effective fields").AtWarning()
return nil, errors.New("this rule has no effective fields")
}
return conds, nil
@@ -145,7 +145,7 @@ func (br *BalancingRule) Build(ohm outbound.Manager, dispatcher routing.Dispatch
}
s, ok := i.(*StrategyLeastLoadConfig)
if !ok {
return nil, errors.New("not a StrategyLeastLoadConfig").AtError()
return nil, errors.New("not a StrategyLeastLoadConfig")
}
leastLoadStrategy := NewLeastLoadStrategy(s)
return &Balancer{
+13 -4
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/router/config.proto
package router
@@ -587,6 +587,7 @@ type Config struct {
DomainStrategy Config_DomainStrategy `protobuf:"varint,1,opt,name=domain_strategy,json=domainStrategy,proto3,enum=xray.app.router.Config_DomainStrategy" json:"domain_strategy,omitempty"`
Rule []*RoutingRule `protobuf:"bytes,2,rep,name=rule,proto3" json:"rule,omitempty"`
BalancingRule []*BalancingRule `protobuf:"bytes,3,rep,name=balancing_rule,json=balancingRule,proto3" json:"balancing_rule,omitempty"`
Script string `protobuf:"bytes,4,opt,name=script,proto3" json:"script,omitempty"`
unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache
}
@@ -642,6 +643,13 @@ func (x *Config) GetBalancingRule() []*BalancingRule {
return nil
}
func (x *Config) GetScript() string {
if x != nil {
return x.Script
}
return ""
}
var File_app_router_config_proto protoreflect.FileDescriptor
const file_app_router_config_proto_rawDesc = "" +
@@ -699,11 +707,12 @@ const file_app_router_config_proto_rawDesc = "" +
"\tbaselines\x18\x03 \x03(\x03R\tbaselines\x12\x1a\n" +
"\bexpected\x18\x04 \x01(\x05R\bexpected\x12\x16\n" +
"\x06maxRTT\x18\x05 \x01(\x03R\x06maxRTT\x12\x1c\n" +
"\ttolerance\x18\x06 \x01(\x02R\ttolerance\"\x96\x02\n" +
"\ttolerance\x18\x06 \x01(\x02R\ttolerance\"\xae\x02\n" +
"\x06Config\x12O\n" +
"\x0fdomain_strategy\x18\x01 \x01(\x0e2&.xray.app.router.Config.DomainStrategyR\x0edomainStrategy\x120\n" +
"\x04rule\x18\x02 \x03(\v2\x1c.xray.app.router.RoutingRuleR\x04rule\x12E\n" +
"\x0ebalancing_rule\x18\x03 \x03(\v2\x1e.xray.app.router.BalancingRuleR\rbalancingRule\"B\n" +
"\x0ebalancing_rule\x18\x03 \x03(\v2\x1e.xray.app.router.BalancingRuleR\rbalancingRule\x12\x16\n" +
"\x06script\x18\x04 \x01(\tR\x06script\"B\n" +
"\x0eDomainStrategy\x12\b\n" +
"\x04AsIs\x10\x00\x12\x10\n" +
"\fIpIfNonMatch\x10\x02\x12\x0e\n" +
+1
View File
@@ -110,4 +110,5 @@ message Config {
DomainStrategy domain_strategy = 1;
repeated RoutingRule rule = 2;
repeated BalancingRule balancing_rule = 3;
string script = 4;
}
+175
View File
@@ -0,0 +1,175 @@
package router
import (
"runtime"
"strings"
"github.com/xtls/xray-core/common/errors"
xlua "github.com/xtls/xray-core/common/lua"
"github.com/xtls/xray-core/common/net"
"github.com/xtls/xray-core/features/routing"
lua "github.com/yuin/gopher-lua"
)
const (
luaContextType = "xray.router.Context"
luaAttributesType = "xray.router.Attributes"
)
// RegisterLua makes xray.router available to routing scripts.
func (r *Router) RegisterLua(L *lua.LState) {
registerLuaContext(L)
L.PreloadModule("xray.router", func(L *lua.LState) int {
module := L.CreateTable(0, 7)
module.RawSetString("NetworkUnknown", lua.LNumber(net.Network_Unknown))
module.RawSetString("NetworkTCP", lua.LNumber(net.Network_TCP))
module.RawSetString("NetworkUDP", lua.LNumber(net.Network_UDP))
module.RawSetString("NetworkUNIX", lua.LNumber(net.Network_UNIX))
module.RawSetString("LocalOS", lua.LString(runtime.GOOS))
module.RawSetString("PickOutbound", L.NewFunction(func(L *lua.LState) int {
tag, ok := L.Get(1).(lua.LString)
if !ok {
L.ArgError(1, "balancer tag must be a string")
return 0
}
balancer, found := (*r.balancers.Load())[string(tag)]
if !found {
xlua.PushNil(L)
xlua.PushError(L, errors.New("balancer ", tag, " not found"))
return 2
}
outboundTag, err := balancer.PickOutbound()
xlua.PushString(L, outboundTag)
xlua.PushError(L, err)
return 2
}))
module.RawSetString("FindProcess", L.NewFunction(func(L *lua.LState) int {
pid, name, path, err := findProcess(checkLuaContext(L), net.FindProcess)
xlua.PushNumber(L, pid)
xlua.PushString(L, name)
xlua.PushString(L, path)
xlua.PushError(L, err)
return 4
}))
L.Push(module)
return 1
})
}
func registerLuaContext(L *lua.LState) {
pushIPs := xlua.NewReadOnlySlicePusher[net.IP](L)
attributes := L.NewTypeMetatable(luaAttributesType)
L.SetField(attributes, "__index", L.NewFunction(func(L *lua.LState) int {
values := L.CheckUserData(1).Value.(map[string]string)
key := L.CheckString(2)
if value, found := values[key]; found {
xlua.PushString(L, value)
} else {
xlua.PushNil(L)
}
return 1
}))
methods := L.CreateTable(0, 4)
L.SetFuncs(methods, map[string]lua.LGFunction{
"GetSourceIPs": func(L *lua.LState) int {
pushIPs(L, checkLuaContext(L).GetSourceIPs())
return 1
},
"GetTargetIPs": func(L *lua.LState) int {
pushIPs(L, checkLuaContext(L).GetTargetIPs())
return 1
},
"GetLocalIPs": func(L *lua.LState) int {
pushIPs(L, checkLuaContext(L).GetLocalIPs())
return 1
},
"GetAttributes": func(L *lua.LState) int {
values := L.NewUserData()
values.Value = checkLuaContext(L).GetAttributes()
L.SetMetatable(values, attributes)
L.Push(values)
return 1
},
})
L.SetField(L.NewTypeMetatable(luaContextType), "__index", methods)
}
func checkLuaContext(L *lua.LState) routing.Context {
ctx, ok := L.CheckUserData(1).Value.(routing.Context)
if !ok {
L.ArgError(1, "routing context expected")
}
return ctx
}
// callLuaRoute runs HandleRoute and leaves (outboundTag, ruleTag, err) on the stack.
func callLuaRoute(L *lua.LState, ctx routing.Context) error {
fn := L.GetGlobal("HandleRoute")
if fn.Type() != lua.LTFunction {
return errors.New("routing script must define HandleRoute(...)")
}
value := L.NewUserData()
value.Value = ctx
L.SetMetatable(value, L.GetTypeMetatable(luaContextType))
return L.CallByParam(lua.P{Fn: fn, NRet: 3, Protect: true},
value,
lua.LString(ctx.GetInboundTag()),
lua.LNumber(ctx.GetSourcePort()),
lua.LNumber(ctx.GetTargetPort()),
lua.LNumber(ctx.GetLocalPort()),
lua.LString(strings.ToLower(ctx.GetTargetDomain())),
lua.LNumber(ctx.GetNetwork()),
lua.LString(ctx.GetProtocol()),
lua.LString(ctx.GetUser()),
lua.LNumber(ctx.GetVlessRoute()),
lua.LBool(ctx.GetSkipDNSResolve()))
}
// readLuaRouteResult reads (outboundTag, ruleTag, err) from the stack.
func readLuaRouteResult(L *lua.LState) (string, string, error) {
if err := xlua.ReadError(L.Get(-1), "routing script error must be an error or string"); err != nil {
return "", "", err
}
outboundTag, err := xlua.ReadOptionalString(L.Get(-3), "routing script outboundTag must be a string or nil")
if err != nil || outboundTag == "" {
return "", "", err
}
ruleTag, err := xlua.ReadOptionalString(L.Get(-2), "routing script ruleTag must be a string")
if err != nil {
return "", "", err
}
return outboundTag, ruleTag, nil
}
type processFinder func(string, string, uint16, string, uint16) (int, string, string, error)
func findProcess(ctx routing.Context, finder processFinder) (int, string, string, error) {
sources := ctx.GetSourceIPs()
if len(sources) == 0 {
return 0, "", "", errors.New("process lookup requires a source IP")
}
var network string
switch ctx.GetNetwork() {
case net.Network_TCP:
network = "tcp"
case net.Network_UDP:
network = "udp"
default:
return 0, "", "", errors.New("process lookup requires TCP or UDP")
}
targetIP, targetPort := "", uint16(0)
if targets := ctx.GetTargetIPs(); len(targets) > 0 {
targetIP, targetPort = targets[0].String(), uint16(ctx.GetTargetPort())
}
return finder(network, sources[0].String(), uint16(ctx.GetSourcePort()), targetIP, targetPort)
}
+223
View File
@@ -0,0 +1,223 @@
package router
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"github.com/xtls/xray-core/common/geodata"
"github.com/xtls/xray-core/common/net"
"github.com/xtls/xray-core/common/session"
"github.com/xtls/xray-core/features/routing"
routing_session "github.com/xtls/xray-core/features/routing/session"
lua "github.com/yuin/gopher-lua"
)
func benchmarkRouteContext(target net.Destination) *routing_session.Context {
// Use the production context: its IP getters construct a slice per call.
// The cached IP slices in luaRouteTestContext would undercount this cost.
return &routing_session.Context{
Inbound: &session.Inbound{
Tag: "in",
Source: net.TCPDestination(net.LocalHostIP, 1234),
Local: net.TCPDestination(net.LocalHostIP, 5678),
},
Outbound: &session.Outbound{Target: target},
Content: &session.Content{Protocol: "tls"},
}
}
func benchmarkRouteState(b *testing.B, r *Router, script string) *lua.LState {
b.Helper()
L := lua.NewState()
b.Cleanup(L.Close)
r.RegisterLua(L)
geodata.RegisterLua(L)
if err := L.DoString(script); err != nil {
b.Fatal(err)
}
L.SetContext(context.Background())
return L
}
// BenchmarkLuaRouteHook isolates argument bridging and a fixed-return hook.
// It excludes rules, result decoding, the state pool, and Route construction.
func BenchmarkLuaRouteHook(b *testing.B) {
L := benchmarkRouteState(b, new(Router), `
function HandleRoute(ctx, inboundTag, sourcePort, targetPort, localPort,
targetDomain, network, protocol, user, vlessRoute, skipDNSResolve)
return "out", "rule"
end
`)
ctx := benchmarkRouteContext(net.TCPDestination(net.LocalHostIP, 443))
if err := callLuaRoute(L, ctx); err != nil {
b.Fatal(err)
}
outboundTag, ruleTag, err := readLuaRouteResult(L)
L.Pop(3)
if err != nil || outboundTag != "out" || ruleTag != "rule" {
b.Fatalf("hook() = %q, %q, %v", outboundTag, ruleTag, err)
}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
if err := callLuaRoute(L, ctx); err != nil {
b.Fatal(err)
}
L.Pop(3)
}
}
// BenchmarkLuaRoute compares equivalent ordered rules on the same session.
// rules returns tags only; pick_route uses Router.PickRoute on both sides.
// All compilation, matcher construction, and pool startup are outside timing.
func BenchmarkLuaRoute(b *testing.B) {
for _, name := range []string{"scalar", "ip", "domain", "domain_32_last"} {
b.Run(name, func(b *testing.B) {
config, script, ctx, wantTag, wantRule := benchmarkRouteFixture(b, name)
native := new(Router)
if err := native.Init(context.Background(), config, nil, nil, nil); err != nil {
b.Fatal(err)
}
L := benchmarkRouteState(b, native, script)
path := filepath.Join(b.TempDir(), "route.lua")
if err := os.WriteFile(path, []byte(script), 0o600); err != nil {
b.Fatal(err)
}
scripted := new(Router)
if err := scripted.Init(context.Background(), &Config{Script: path}, nil, nil, nil); err != nil {
b.Fatal(err)
}
if err := scripted.Start(); err != nil {
b.Fatal(err)
}
b.Cleanup(func() {
if err := scripted.Close(); err != nil {
b.Error(err)
}
})
for _, bench := range []struct {
name string
route func() (string, string, error)
}{
{"rules/native", func() (string, string, error) {
rule, _, err := native.pickRouteInternal(ctx)
if err != nil {
return "", "", err
}
tag, err := rule.GetTag()
return tag, rule.RuleTag, err
}},
{"rules/lua", func() (string, string, error) {
if err := callLuaRoute(L, ctx); err != nil {
return "", "", err
}
tag, ruleTag, err := readLuaRouteResult(L)
L.Pop(3)
return tag, ruleTag, err
}},
{"pick_route/native", func() (string, string, error) {
return benchmarkPickRoute(native, ctx)
}},
{"pick_route/lua", func() (string, string, error) {
return benchmarkPickRoute(scripted, ctx)
}},
} {
b.Run(bench.name, func(b *testing.B) {
// Validate and warm both paths before measuring steady state.
tag, ruleTag, err := bench.route()
if err != nil || tag != wantTag || ruleTag != wantRule {
b.Fatalf("route() = %q, %q, %v; want %q, %q", tag, ruleTag, err, wantTag, wantRule)
}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
tag, ruleTag, err = bench.route()
if err != nil {
b.Fatal(err)
}
}
b.StopTimer()
if tag != wantTag || ruleTag != wantRule {
b.Fatalf("route() = %q, %q; want %q, %q", tag, ruleTag, wantTag, wantRule)
}
})
}
})
}
}
func benchmarkPickRoute(r *Router, ctx routing.Context) (string, string, error) {
route, err := r.PickRoute(ctx)
if err != nil {
return "", "", err
}
return route.GetOutboundTag(), route.GetRuleTag(), nil
}
func benchmarkRouteFixture(b *testing.B, name string) (*Config, string, routing.Context, string, string) {
b.Helper()
config := new(Config)
ctx := benchmarkRouteContext(net.TCPDestination(net.LocalHostIP, 443))
prelude := `local router = require("xray.router")
local geodata = require("xray.geodata")
`
body := `if inboundTag == "in" and network == router.NetworkTCP then return "out", "rule" end`
wantTag, wantRule := "out", "rule"
if name == "scalar" || name == "ip" {
rule := &RoutingRule{
TargetTag: &RoutingRule_Tag{Tag: wantTag},
RuleTag: wantRule,
InboundTag: []string{"in"},
Networks: []net.Network{net.Network_TCP},
}
if name == "ip" {
var err error
rule.Ip, err = geodata.ParseIPRules([]string{"127.0.0.0/8"})
if err != nil {
b.Fatal(err)
}
prelude += `local matcher = geodata.BuildIPMatcher("127.0.0.0/8")` + "\n"
body = `if inboundTag == "in" and network == router.NetworkTCP and matcher:AnyMatch(ctx:GetTargetIPs()) then return "out", "rule" end`
}
config.Rule = []*RoutingRule{rule}
} else {
count := 1
if name == "domain_32_last" {
count = 32
}
var rules strings.Builder
rules.WriteString("local rules = {\n")
for i := 0; i < count; i++ {
domain := fmt.Sprintf("route-%d.example.com", i)
tag, ruleTag := fmt.Sprintf("out-%d", i), fmt.Sprintf("rule-%d", i)
domains, err := geodata.ParseDomainRules([]string{"full:" + domain}, geodata.Domain_Domain)
if err != nil {
b.Fatal(err)
}
config.Rule = append(config.Rule, &RoutingRule{
TargetTag: &RoutingRule_Tag{Tag: tag}, RuleTag: ruleTag, Domain: domains,
})
fmt.Fprintf(&rules, "{geodata.BuildDomainMatcher(%q), %q, %q},\n", "full:"+domain, tag, ruleTag)
if i == count-1 {
ctx.Outbound.Target = net.TCPDestination(net.DomainAddress(domain), 443)
wantTag, wantRule = tag, ruleTag
}
}
rules.WriteString("}\n")
prelude += rules.String()
body = `for i = 1, #rules do
local rule = rules[i]
if rule[1]:MatchAny(targetDomain) then return rule[2], rule[3] end
end`
}
script := prelude + `function HandleRoute(ctx, inboundTag, sourcePort, targetPort, localPort,
targetDomain, network, protocol, user, vlessRoute, skipDNSResolve)
` + body + "\nend\n"
return config, script, ctx, wantTag, wantRule
}
+274
View File
@@ -0,0 +1,274 @@
package router
import (
"context"
go_errors "errors"
"runtime"
"strings"
"testing"
"github.com/xtls/xray-core/common/geodata"
"github.com/xtls/xray-core/common/net"
"github.com/xtls/xray-core/common/protocol"
"github.com/xtls/xray-core/common/session"
"github.com/xtls/xray-core/features/routing"
routing_session "github.com/xtls/xray-core/features/routing/session"
lua "github.com/yuin/gopher-lua"
)
type luaRouteTestContext struct {
*routing_session.Context
sourceIPs, targetIPs, localIPs []net.IP
}
func (c *luaRouteTestContext) GetSourceIPs() []net.IP { return c.sourceIPs }
func (c *luaRouteTestContext) GetTargetIPs() []net.IP { return c.targetIPs }
func (c *luaRouteTestContext) GetLocalIPs() []net.IP { return c.localIPs }
func newLuaRouteTestContext() *luaRouteTestContext {
return &luaRouteTestContext{
Context: &routing_session.Context{
Inbound: &session.Inbound{
Tag: "in", VlessRoute: 4321,
Source: net.TCPDestination(net.LocalHostIP, 1234),
Local: net.TCPDestination(net.LocalHostIP, 5678),
User: &protocol.MemoryUser{Email: "user@example.com"},
},
Outbound: &session.Outbound{
Target: net.TCPDestination(net.LocalHostIP, 443),
RouteTarget: net.TCPDestination(net.DomainAddress("MiXeD.Example."), 443),
},
Content: &session.Content{
Protocol: "tls", Attributes: map[string]string{"key": "value"}, SkipDNSResolve: true,
},
},
sourceIPs: []net.IP{{127, 0, 0, 2}},
targetIPs: []net.IP{{127, 0, 0, 3}},
localIPs: []net.IP{{127, 0, 0, 1}},
}
}
func newLuaRouterState(t *testing.T, script string) *lua.LState {
t.Helper()
r := new(Router)
if err := r.Init(context.Background(), &Config{}, nil, nil, nil); err != nil {
t.Fatal(err)
}
L := lua.NewState()
t.Cleanup(L.Close)
r.RegisterLua(L)
geodata.RegisterLua(L)
if err := L.DoString(script); err != nil {
t.Fatal(err)
}
return L
}
func TestLuaRouteBinding(t *testing.T) {
L := newLuaRouterState(t, `
local router = require("xray.router")
local matcher = require("xray.geodata").BuildIPMatcher("127.0.0.0/8")
assert(router.NetworkUnknown == 0 and router.NetworkTCP == 2)
assert(router.NetworkUDP == 3 and router.NetworkUNIX == 4)
assert(router.BuildIPMatcher == nil and router.BuildDomainMatcher == nil)
function HandleRoute(ctx, inboundTag, sourcePort, targetPort, localPort,
targetDomain, network, protocol, user, vlessRoute, skipDNSResolve, ...)
assert(select("#", ...) == 0)
assert(inboundTag == "in" and sourcePort == 1234 and targetPort == 443 and localPort == 5678)
assert(targetDomain == "mixed.example." and network == router.NetworkTCP)
assert(protocol == "tls" and user == "user@example.com" and vlessRoute == 4321 and skipDNSResolve)
assert(ctx.GetNetwork == nil and ctx.Context == nil)
savedContext = ctx
sourceIPs, targetIPs, localIPs = ctx:GetSourceIPs(), ctx:GetTargetIPs(), ctx:GetLocalIPs()
attributes = ctx:GetAttributes()
assert(#sourceIPs == 1 and #targetIPs == 1 and #localIPs == 1)
assert(sourceIPs[1]:String() == "127.0.0.2" and targetIPs[1]:String() == "127.0.0.3")
assert(localIPs[1]:String() == "127.0.0.1")
assert(matcher:Match(sourceIPs[1]) and matcher:Match(targetIPs[1]) and matcher:Match(localIPs[1]))
assert(matcher:AnyMatch(sourceIPs) and matcher:AnyMatch(targetIPs) and matcher:AnyMatch(localIPs))
local matched = matcher:FilterIPs(targetIPs)
assert(#matched == 1 and matched[1]:Equal(targetIPs[1]))
assert(attributes.key == "value" and attributes.missing == nil)
assert(not pcall(function() attributes.key = "changed" end))
return "out", "rule"
end`)
ctx := newLuaRouteTestContext()
if err := callLuaRoute(L, ctx); err != nil {
t.Fatal(err)
}
if L.GetTop() != 3 || L.Get(1) != lua.LString("out") || L.Get(2) != lua.LString("rule") || L.Get(3) != lua.LNil {
t.Fatal("callLuaRoute did not leave the three route results on the stack")
}
if L.GetGlobal("savedContext").(*lua.LUserData).Value != ctx {
t.Fatal("routing context was copied")
}
for _, tc := range []struct {
name string
want []net.IP
}{
{"sourceIPs", ctx.sourceIPs},
{"targetIPs", ctx.targetIPs},
{"localIPs", ctx.localIPs},
} {
got := L.GetGlobal(tc.name).(*lua.LUserData).Value.([]net.IP)
if &got[0] != &tc.want[0] {
t.Fatalf("%s storage was copied", tc.name)
}
}
ctx.Content.Attributes["key"] = "updated"
L.SetGlobal("expectedOS", lua.LString(runtime.GOOS))
if err := L.DoString(`
assert(attributes.key == "updated")
assert(require("xray.router").LocalOS == expectedOS)`); err != nil {
t.Fatal(err)
}
}
func TestLuaRouteEmptyIPs(t *testing.T) {
for _, tc := range []struct {
name string
ips []net.IP
}{
{"nil", nil},
{"empty", []net.IP{}},
} {
t.Run(tc.name, func(t *testing.T) {
L := newLuaRouterState(t, `
function HandleRoute(ctx)
for _, name in ipairs({"GetSourceIPs", "GetTargetIPs", "GetLocalIPs"}) do
local ips = ctx[name](ctx)
if expectNil then
assert(ips == nil)
else
assert(type(ips) == "userdata" and #ips == 0)
assert(not pcall(function() return ips[1] end))
end
end
return "out"
end
`)
L.SetGlobal("expectNil", lua.LBool(tc.ips == nil))
ctx := newLuaRouteTestContext()
ctx.sourceIPs, ctx.targetIPs, ctx.localIPs = tc.ips, tc.ips, tc.ips
if err := callLuaRoute(L, ctx); err != nil {
t.Fatal(err)
}
})
}
}
func TestReadLuaRouteResult(t *testing.T) {
nativeErr := go_errors.New("native failure")
for _, tc := range []struct {
name, values string
wantTag, wantRule string
wantErr error
wantMessage string
}{
{name: "route", values: `"out", "rule"`, wantTag: "out", wantRule: "rule"},
{name: "no match", values: `nil`},
{name: "empty tag", values: `""`},
{name: "no match ignores rule", values: `nil, false`},
{name: "empty tag ignores rule", values: `"", false`},
{name: "missing rule", values: `"out"`, wantTag: "out"},
{name: "invalid tag", values: `1`, wantMessage: "outboundTag"},
{name: "invalid rule", values: `"out", false`, wantMessage: "ruleTag"},
{name: "string error", values: `nil, nil, "script failure"`, wantMessage: "script failure"},
{name: "native error", values: `nil, nil, nativeError`, wantErr: nativeErr},
{name: "error overrides invalid tags", values: `false, false, nativeError`, wantErr: nativeErr},
{name: "invalid error", values: `"out", "rule", false`, wantMessage: "error or string"},
{name: "wrong error userdata", values: `"out", "rule", wrongError`, wantMessage: "error or string"},
} {
t.Run(tc.name, func(t *testing.T) {
L := lua.NewState()
defer L.Close()
for name, value := range map[string]any{"nativeError": nativeErr, "wrongError": "not a native error"} {
ud := L.NewUserData()
ud.Value = value
L.SetGlobal(name, ud)
}
fn, err := L.LoadString("return " + tc.values)
if err != nil {
t.Fatal(err)
}
if err := L.CallByParam(lua.P{Fn: fn, NRet: 3, Protect: true}); err != nil {
t.Fatal(err)
}
outboundTag, ruleTag, err := readLuaRouteResult(L)
if outboundTag != tc.wantTag || ruleTag != tc.wantRule {
t.Fatalf("result = %q, %q, %v; want %q, %q", outboundTag, ruleTag, err, tc.wantTag, tc.wantRule)
}
switch {
case tc.wantErr != nil:
if err != tc.wantErr {
t.Fatalf("error = %v, want original error", err)
}
case tc.wantMessage != "":
if err == nil || !strings.Contains(err.Error(), tc.wantMessage) {
t.Fatalf("error = %v, want %q", err, tc.wantMessage)
}
case err != nil:
t.Fatal(err)
}
})
}
}
func TestCallLuaRouteCancellation(t *testing.T) {
L := newLuaRouterState(t, `function HandleRoute() while true do end end`)
ctx, cancel := context.WithCancel(context.Background())
cancel()
L.SetContext(ctx)
if err := callLuaRoute(L, &routing_session.Context{}); err == nil {
t.Fatal("callLuaRoute did not stop after context cancellation")
}
if L.Context() != ctx {
t.Fatal("callLuaRoute changed the Lua state's context")
}
}
func TestFindProcess(t *testing.T) {
for _, tc := range []struct {
name, network, target string
targetPort uint16
modify func(*luaRouteTestContext)
wantErr bool
}{
{name: "TCP", network: "tcp", target: "127.0.0.3", targetPort: 443},
{name: "UDP", network: "udp", target: "127.0.0.3", targetPort: 443, modify: func(c *luaRouteTestContext) {
c.Outbound.Target.Network = net.Network_UDP
}},
{name: "domain target", network: "tcp", modify: func(c *luaRouteTestContext) { c.targetIPs = nil }},
{name: "missing source", modify: func(c *luaRouteTestContext) { c.sourceIPs = nil }, wantErr: true},
{name: "unsupported network", modify: func(c *luaRouteTestContext) {
c.Outbound.Target.Network = net.Network_UNIX
}, wantErr: true},
} {
t.Run(tc.name, func(t *testing.T) {
ctx := newLuaRouteTestContext()
if tc.modify != nil {
tc.modify(ctx)
}
called := false
pid, name, path, err := findProcess(ctx, func(network, source string, sourcePort uint16, target string, targetPort uint16) (int, string, string, error) {
called = true
if network != tc.network || source != "127.0.0.2" || sourcePort != 1234 || target != tc.target || targetPort != tc.targetPort {
t.Fatalf("endpoints = %s %s:%d -> %s:%d", network, source, sourcePort, target, targetPort)
}
return 42, "process", "/path/process", nil
})
if tc.wantErr {
if err == nil || called {
t.Fatalf("findProcess = %d, %q, %q, %v", pid, name, path, err)
}
return
}
if err != nil || !called || pid != 42 || name != "process" || path != "/path/process" {
t.Fatalf("findProcess = %d, %q, %q, %v", pid, name, path, err)
}
})
}
}
var _ routing.Context = (*luaRouteTestContext)(nil)
+21
View File
@@ -20,6 +20,7 @@ import (
type Router struct {
domainStrategy Config_DomainStrategy
rules atomic.Pointer[[]*Rule]
script *scriptEngine
balancers atomic.Pointer[map[string]*Balancer]
dns dns.Client
@@ -45,6 +46,14 @@ func (r *Router) Init(ctx context.Context, config *Config, d dns.Client, ohm out
r.ohm = ohm
r.dispatcher = dispatcher
if config.Script != "" {
script, err := newScriptEngine(config.Script)
if err != nil {
return errors.New("failed to prepare routing script").Base(err)
}
r.script = script
}
r.rules.Store(new([]*Rule))
r.balancers.Store(&map[string]*Balancer{})
return r.ReloadRules(config, false)
@@ -52,6 +61,10 @@ func (r *Router) Init(ctx context.Context, config *Config, d dns.Client, ohm out
// PickRoute implements routing.Router.
func (r *Router) PickRoute(ctx routing.Context) (routing.Route, error) {
if r.script != nil {
return r.script.pickRoute(ctx)
}
originalCtx := ctx
rule, ctx, err := r.pickRouteInternal(ctx)
if err != nil {
@@ -221,6 +234,11 @@ func (r *Router) pickRouteInternal(ctx routing.Context) (*Rule, routing.Context,
// Start implements common.Runnable.
func (r *Router) Start() error {
if r.script != nil {
if err := r.script.start(r); err != nil {
return errors.New("failed to initialize routing script").Base(err)
}
}
return nil
}
@@ -235,6 +253,9 @@ func closeWebhooks(rules []*Rule) {
// Close implements common.Closable.
func (r *Router) Close() error {
if r.script != nil {
r.script.close()
}
r.mu.Lock()
defer r.mu.Unlock()
closeWebhooks(*r.rules.Load())
+84
View File
@@ -0,0 +1,84 @@
package router
import (
"time"
"github.com/xtls/xray-core/app/dns"
"github.com/xtls/xray-core/common"
"github.com/xtls/xray-core/common/errors"
"github.com/xtls/xray-core/common/geodata"
"github.com/xtls/xray-core/common/log"
xlua "github.com/xtls/xray-core/common/lua"
"github.com/xtls/xray-core/features/routing"
lua "github.com/yuin/gopher-lua"
)
const scriptExecutionTimeout = 6 * time.Second
type scriptEngine struct {
program *xlua.Program
pool *xlua.Pool
}
func newScriptEngine(script string) (*scriptEngine, error) {
program, err := xlua.Compile(script)
if err != nil {
return nil, err
}
return &scriptEngine{program: program}, nil
}
func (e *scriptEngine) start(router *Router) error {
pool, err := xlua.NewPool(router.ctx, scriptExecutionTimeout, e.program.NewStateFactory(
scriptExecutionTimeout*20,
256,
func(L *lua.LState) {
geodata.RegisterLua(L)
log.RegisterLua(L)
router.RegisterLua(L)
dns.RegisterLua(L, router.dns)
},
func(L *lua.LState) error {
if L.GetGlobal("HandleRoute").Type() != lua.LTFunction {
return errors.New("routing script must define HandleRoute(...)")
}
return nil
}))
if err != nil {
return err
}
e.pool = pool
errors.LogInfo(router.ctx, "routing script initialized")
return nil
}
func (e *scriptEngine) close() {
if e.pool != nil {
e.pool.Close()
}
}
func (e *scriptEngine) pickRoute(ctx routing.Context) (routing.Route, error) {
var outboundTag, ruleTag string
var routeErr error
if err := e.pool.WithState(nil, 0, func(L *lua.LState) error {
if err := callLuaRoute(L, ctx); err != nil {
return err
}
outboundTag, ruleTag, routeErr = readLuaRouteResult(L)
return nil
}); err != nil {
return nil, err
}
if routeErr != nil {
return nil, routeErr
}
if outboundTag == "" {
return nil, common.ErrNoClue
}
return &Route{Context: ctx, outboundTag: outboundTag, ruleTag: ruleTag}, nil
}
+421
View File
@@ -0,0 +1,421 @@
package router
import (
"context"
stdnet "net"
"os"
"path/filepath"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
wireDNS "github.com/miekg/dns"
"github.com/xtls/xray-core/app/dispatcher"
appdns "github.com/xtls/xray-core/app/dns"
"github.com/xtls/xray-core/app/proxyman"
_ "github.com/xtls/xray-core/app/proxyman/outbound"
"github.com/xtls/xray-core/common"
"github.com/xtls/xray-core/common/net"
"github.com/xtls/xray-core/common/serial"
"github.com/xtls/xray-core/core"
featureDNS "github.com/xtls/xray-core/features/dns"
"github.com/xtls/xray-core/features/outbound"
"github.com/xtls/xray-core/features/routing"
routing_session "github.com/xtls/xray-core/features/routing/session"
"github.com/xtls/xray-core/proxy/blackhole"
"github.com/xtls/xray-core/proxy/freedom"
)
type luaRouteDNSClient struct {
featureDNS.Client
lookup func(string, featureDNS.IPOption) ([]net.IP, uint32, error)
}
func (d *luaRouteDNSClient) LookupIP(domain string, option featureDNS.IPOption) ([]net.IP, uint32, error) {
return d.lookup(domain, option)
}
type luaRouteOutboundManager struct{ outbound.Manager }
func (*luaRouteOutboundManager) Select(selectors []string) []string { return selectors }
func writeRouteScript(t *testing.T, script string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "route.lua")
if err := os.WriteFile(path, []byte(script), 0o600); err != nil {
t.Fatal(err)
}
return path
}
func startLuaRouter(t *testing.T, script string, d featureDNS.Client, config *Config) *Router {
t.Helper()
if config == nil {
config = &Config{}
}
config.Script = writeRouteScript(t, script)
r := new(Router)
if err := r.Init(context.Background(), config, d, &luaRouteOutboundManager{}, nil); err != nil {
t.Fatal(err)
}
if err := r.Start(); err != nil {
t.Fatal(err)
}
t.Cleanup(func() {
if err := r.Close(); err != nil {
t.Error(err)
}
})
return r
}
func TestRouterScriptStartup(t *testing.T) {
for _, tc := range []struct {
name, script string
compileError bool
}{
{"syntax error", "function HandleRoute(", true},
{"missing hook", "value = 1", false},
{"invalid hook", "HandleRoute = true", false},
{"initialization error", `error("setup failed")`, false},
} {
t.Run(tc.name, func(t *testing.T) {
r := new(Router)
err := r.Init(context.Background(), &Config{Script: writeRouteScript(t, tc.script)}, nil, nil, nil)
if tc.compileError {
if r.script != nil || err == nil || !strings.Contains(err.Error(), "failed to prepare routing script") {
t.Fatalf("Init = %v; want no engine and script preparation error", err)
}
return
}
if err != nil {
t.Fatal(err)
}
defer r.Close()
if r.script.program == nil || r.script.pool != nil {
t.Fatal("Init did not compile the script without starting its pool")
}
if err := r.Close(); err != nil {
t.Fatal(err)
}
if err := r.Start(); err == nil {
t.Fatal("Start accepted an invalid routing script")
}
if r.script.pool != nil {
t.Fatal("Start retained a script pool after failure")
}
})
}
}
func TestRouterScriptProgramSnapshot(t *testing.T) {
path := writeRouteScript(t, `function HandleRoute() return "compiled", "compiled-rule" end`)
engine, err := newScriptEngine(path)
if err != nil {
t.Fatal(err)
}
defer engine.close()
if err := os.Remove(path); err != nil {
t.Fatal(err)
}
if err := engine.start(&Router{ctx: context.Background()}); err != nil {
t.Fatal(err)
}
route, err := engine.pickRoute(newLuaRouteTestContext())
if err != nil || route == nil || route.GetOutboundTag() != "compiled" || route.GetRuleTag() != "compiled-rule" {
t.Fatalf("route = %v, %v; want compiled, compiled-rule", route, err)
}
}
func TestRouterScriptRouting(t *testing.T) {
for _, tc := range []struct {
name, body string
wantTag, wantRule string
wantErr error
wantMessage string
wantCalls string
}{
{name: "route", body: `return "lua-out", "lua-rule"`, wantTag: "lua-out", wantRule: "lua-rule", wantCalls: "2"},
{name: "no match", body: `return nil`, wantErr: common.ErrNoClue, wantCalls: "2"},
{name: "empty tag", body: `return ""`, wantErr: common.ErrNoClue, wantCalls: "2"},
{name: "balancer error", body: `local tag, err = router.PickOutbound("missing"); return tag, nil, err`, wantMessage: "not found", wantCalls: "2"},
{name: "string error", body: `return nil, nil, "blocked"`, wantMessage: "blocked", wantCalls: "2"},
{name: "invalid tag", body: `return false`, wantMessage: "outboundTag", wantCalls: "2"},
{name: "invalid rule", body: `return "lua-out", false`, wantMessage: "ruleTag", wantCalls: "2"},
{name: "execution error", body: `error("execution failed")`, wantMessage: "execution failed", wantCalls: "1"},
} {
t.Run(tc.name, func(t *testing.T) {
var dnsCalls atomic.Int32
d := &luaRouteDNSClient{lookup: func(string, featureDNS.IPOption) ([]net.IP, uint32, error) {
dnsCalls.Add(1)
return []net.IP{{1, 2, 3, 4}}, 60, nil
}}
script := `
local router = require("xray.router")
local calls = 0
function HandleRoute(ctx, inbound)
calls = calls + 1
if inbound == "count" then return "lua-out", tostring(calls) end
` + tc.body + `
end
`
r := startLuaRouter(t, script, d, &Config{
DomainStrategy: Config_IpOnDemand,
Rule: []*RoutingRule{{
TargetTag: &RoutingRule_Tag{Tag: "json-out"},
Networks: []net.Network{net.Network_TCP},
}},
})
ctx := newLuaRouteTestContext()
ctx.Content.SkipDNSResolve = false
route, err := r.PickRoute(ctx)
switch {
case tc.wantErr != nil:
if err != tc.wantErr {
t.Fatalf("route error = %v, want %v", err, tc.wantErr)
}
case tc.wantMessage != "":
if err == nil || !strings.Contains(err.Error(), tc.wantMessage) {
t.Fatalf("route error = %v, want %q", err, tc.wantMessage)
}
case err != nil:
t.Fatal(err)
}
if tc.wantTag == "" {
if route != nil {
t.Fatalf("route = %v, want nil", route)
}
} else if route == nil || route.GetOutboundTag() != tc.wantTag || route.GetRuleTag() != tc.wantRule || route.(*Route).Context != ctx {
t.Fatalf("route = %v; want %q, %q and original context", route, tc.wantTag, tc.wantRule)
}
ctx.Inbound.Tag = "count"
route, err = r.PickRoute(ctx)
if err != nil || route == nil || route.GetOutboundTag() != "lua-out" || route.GetRuleTag() != tc.wantCalls {
t.Fatalf("next route = %v, %v; want lua-out, calls %s", route, err, tc.wantCalls)
}
if dnsCalls.Load() != 0 {
t.Fatal("script routing implicitly resolved DNS")
}
})
}
}
func TestRouterScriptModules(t *testing.T) {
ips := []net.IP{{127, 0, 0, 7}}
calls := 0
d := &luaRouteDNSClient{lookup: func(domain string, option featureDNS.IPOption) ([]net.IP, uint32, error) {
calls++
if domain != "mixed.example." || !option.IPv4Enable || option.IPv6Enable || !option.FakeEnable {
t.Fatalf("dns.Query arguments = %q, %+v", domain, option)
}
return ips, 17, nil
}}
r := startLuaRouter(t, `
local dns = require("xray.dns")
local matcher = require("xray.geodata").BuildIPMatcher("127.0.0.0/8")
assert(dns.Servers == nil and type(dns.Query) == "function")
assert(type(require("xray.log").Info) == "function")
function HandleRoute(ctx, inbound, sourcePort, targetPort, localPort, domain)
local ips, ttl, err = dns.Query(domain, true, false, true)
assert(not err and ttl == 17)
assert(matcher:AnyMatch(ips) and matcher:AnyMatch(ctx:GetTargetIPs()))
return "out"
end`, d, nil)
if _, err := r.PickRoute(newLuaRouteTestContext()); err != nil {
t.Fatal(err)
}
if calls != 1 {
t.Fatalf("DNS calls = %d, want 1", calls)
}
}
func TestRouterScriptBalancerReload(t *testing.T) {
config := func(tag string) *Config {
return &Config{BalancingRule: []*BalancingRule{{
Tag: "balance", Strategy: "roundrobin", OutboundSelector: []string{tag},
}}}
}
r := startLuaRouter(t, `
local router = require("xray.router")
function HandleRoute()
local tag, err = router.PickOutbound("balance")
return tag, "balanced", err
end`, nil, config("old"))
pick := func(want string) {
t.Helper()
route, err := r.PickRoute(&routing_session.Context{})
if err != nil || route.GetOutboundTag() != want || route.GetRuleTag() != "balanced" {
t.Fatalf("route = %v, %v, want %q", route, err, want)
}
}
pick("old")
if err := r.SetOverrideTarget("balance", "override"); err != nil {
t.Fatal(err)
}
pick("override")
if err := r.SetOverrideTarget("balance", ""); err != nil {
t.Fatal(err)
}
if err := r.ReloadRules(config("new"), false); err != nil {
t.Fatal(err)
}
pick("new")
}
func TestRouterScriptConcurrentBalancerReload(t *testing.T) {
config := func(tag string) *Config {
return &Config{BalancingRule: []*BalancingRule{{
Tag: "balance", Strategy: "roundrobin", OutboundSelector: []string{tag},
}}}
}
r := startLuaRouter(t, `
local router = require("xray.router")
function HandleRoute()
local tag, err = router.PickOutbound("balance")
return tag, nil, err
end`, nil, config("a"))
var wg sync.WaitGroup
for range 4 {
wg.Go(func() {
for range 20 {
route, err := r.PickRoute(&routing_session.Context{})
if err != nil {
t.Errorf("PickRoute: %v", err)
return
}
if tag := route.GetOutboundTag(); tag != "a" && tag != "b" {
t.Errorf("unexpected tag %q", tag)
}
}
})
}
wg.Go(func() {
for range 20 {
for _, tag := range []string{"a", "b"} {
if err := r.ReloadRules(config(tag), false); err != nil {
t.Error(err)
return
}
}
}
})
wg.Wait()
}
func TestRouterScriptDNSDispatcherReentry(t *testing.T) {
conn, err := stdnet.ListenPacket("udp4", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
port := conn.LocalAddr().(*stdnet.UDPAddr).Port
ready, stopped := make(chan struct{}), make(chan error, 1)
var queries atomic.Int32
server := &wireDNS.Server{
PacketConn: conn,
NotifyStartedFunc: func() {
close(ready)
},
Handler: wireDNS.HandlerFunc(func(w wireDNS.ResponseWriter, query *wireDNS.Msg) {
queries.Add(1)
response := new(wireDNS.Msg).SetReply(query)
for _, question := range query.Question {
if question.Name == "nested.example." && question.Qtype == wireDNS.TypeA {
response.Answer = append(response.Answer, &wireDNS.A{
Hdr: wireDNS.RR_Header{Name: question.Name, Rrtype: wireDNS.TypeA, Class: wireDNS.ClassINET, Ttl: 60},
A: stdnet.IP{127, 0, 0, 7},
})
}
}
if err := w.WriteMsg(response); err != nil {
t.Error(err)
}
}),
}
go func() { stopped <- server.ActivateAndServe() }()
defer func() {
server.Shutdown()
select {
case err := <-stopped:
if err != nil {
t.Error(err)
}
case <-time.After(3 * time.Second):
t.Error("DNS server did not stop")
}
}()
select {
case <-ready:
case err := <-stopped:
t.Fatalf("DNS server startup: %v", err)
case <-time.After(3 * time.Second):
t.Fatal("DNS server did not start")
}
dnsScript := writeRouteScript(t, `
local server = require("xray.dns").Servers[1]
function HandleDNSQuery(domain, ipv4, ipv6, fake)
return server:Query(domain, ipv4, ipv6, fake)
end`)
routerScript := writeRouteScript(t, `
local router = require("xray.router")
local dns = require("xray.dns")
local matcher = require("xray.geodata").BuildIPMatcher("127.0.0.7")
local active = false
function HandleRoute(ctx, inbound, sourcePort, targetPort, localPort, domain, network,
protocol, user, vlessRoute, skipDNSResolve)
assert(not active, "borrowed Router VM reentered")
if inbound == "dns" then
assert(network == router.NetworkUDP and skipDNSResolve == false)
return "direct", "dns-route"
end
active = true
local ips, ttl, err = dns.Query("nested.example", true, false, false)
assert(not err and matcher:AnyMatch(ips) and active)
active = false
return "direct", "outer-route"
end`)
instance, err := core.New(&core.Config{
App: []*serial.TypedMessage{
serial.ToTypedMessage(&appdns.Config{
Tag: "dns", Script: dnsScript, DisableCache: true,
NameServer: []*appdns.NameServer{{
Id: "upstream", TimeoutMs: 1000,
Address: &net.Endpoint{
Network: net.Network_UDP,
Address: &net.IPOrDomain{Address: &net.IPOrDomain_Ip{Ip: []byte{127, 0, 0, 1}}},
Port: uint32(port),
},
}},
}),
serial.ToTypedMessage(&Config{Script: routerScript}),
serial.ToTypedMessage(&dispatcher.Config{}),
serial.ToTypedMessage(&proxyman.OutboundConfig{}),
},
Outbound: []*core.OutboundHandlerConfig{
{Tag: "default", ProxySettings: serial.ToTypedMessage(&blackhole.Config{})},
{Tag: "direct", ProxySettings: serial.ToTypedMessage(&freedom.Config{
FinalRules: []*freedom.FinalRuleConfig{{Action: freedom.RuleAction_Allow}},
})},
},
})
if err != nil {
t.Fatal(err)
}
if err := instance.Start(); err != nil {
t.Fatal(err)
}
defer instance.Close()
r := instance.GetFeature(routing.RouterType()).(*Router)
route, err := r.PickRoute(newLuaRouteTestContext())
if err != nil || route.GetOutboundTag() != "direct" || route.GetRuleTag() != "outer-route" {
t.Fatalf("nested DNS routing = %v, %v", route, err)
}
if queries.Load() == 0 {
t.Fatal("DNS query did not pass through the dispatcher")
}
}
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/stats/command/command.proto
package command
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go-grpc. DO NOT EDIT.
// versions:
// - protoc-gen-go-grpc v1.6.0
// - protoc v6.33.5
// - protoc-gen-go-grpc v1.6.2
// - protoc (unknown)
// source: app/stats/command/command.proto
package command
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/stats/config.proto
package stats
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: app/version/config.proto
package version
+6
View File
@@ -0,0 +1,6 @@
package base64
const (
Prefix = "base64:"
GzipPrefix = "base64+gzip:"
)
+38
View File
@@ -0,0 +1,38 @@
package base64
import (
"bytes"
"compress/gzip"
"encoding/base64"
"io"
"strings"
"github.com/xtls/xray-core/common/errors"
)
// Decode decodes Base64 text with an optional [Prefix] or [GzipPrefix].
func Decode(value string) ([]byte, error) {
value, zip := strings.CutPrefix(value, GzipPrefix)
if !zip {
value = strings.TrimPrefix(value, Prefix)
}
data, err := base64.StdEncoding.DecodeString(value)
if err != nil {
return nil, errors.New("failed to decode base64").Base(err)
}
if !zip {
return data, nil
}
reader, err := gzip.NewReader(bytes.NewReader(data))
if err != nil {
return nil, errors.New("failed to decompress gzip").Base(err)
}
defer reader.Close()
data, err = io.ReadAll(reader)
if err != nil {
return nil, errors.New("failed to decompress gzip").Base(err)
}
return data, nil
}
+28
View File
@@ -0,0 +1,28 @@
package base64
import (
"bytes"
"testing"
)
func TestDecodeFormats(t *testing.T) {
compressed, err := Encode([]byte("x"), true)
if err != nil {
t.Fatal(err)
}
for _, tc := range []struct {
name, value, want string
}{
{"unprefixed", "eA==", "x"},
{"base64", Prefix + "eA==", "x"},
{"gzip", compressed, "x"},
{"empty", Prefix, ""},
} {
t.Run(tc.name, func(t *testing.T) {
data, err := Decode(tc.value)
if err != nil || !bytes.Equal(data, []byte(tc.want)) {
t.Fatalf("Decode = %q, %v; want %q", data, err, tc.want)
}
})
}
}
+29
View File
@@ -0,0 +1,29 @@
package base64
import (
"bytes"
"compress/gzip"
"encoding/base64"
"github.com/xtls/xray-core/common/errors"
)
// Encode returns prefixed Base64 text, optionally compressed with gzip at the highest level.
func Encode(data []byte, zip bool) (string, error) {
if !zip {
return Prefix + base64.StdEncoding.EncodeToString(data), nil
}
var buf bytes.Buffer
writer, err := gzip.NewWriterLevel(&buf, gzip.BestCompression)
if err != nil {
return "", errors.New("failed to compress gzip").Base(err)
}
if _, err := writer.Write(data); err != nil {
return "", errors.New("failed to compress gzip").Base(err)
}
if err := writer.Close(); err != nil {
return "", errors.New("failed to compress gzip").Base(err)
}
return GzipPrefix + base64.StdEncoding.EncodeToString(buf.Bytes()), nil
}
+62
View File
@@ -0,0 +1,62 @@
package base64
import (
"bytes"
"compress/gzip"
"encoding/base64"
"io"
"strings"
"testing"
)
func TestEncodeFormats(t *testing.T) {
for _, tc := range []struct {
name string
data []byte
}{
{"empty", nil},
{"bytes", []byte("你好\r\n\x00\xff")},
} {
t.Run(tc.name, func(t *testing.T) {
for _, zip := range []bool{false, true} {
prefix := Prefix
if zip {
prefix = GzipPrefix
}
t.Run(prefix, func(t *testing.T) {
value, err := Encode(tc.data, zip)
if err != nil {
t.Fatal(err)
}
payload, ok := strings.CutPrefix(value, prefix)
if !ok {
t.Fatalf("encoded value does not start with %q", prefix)
}
// Read the output independently of Decode so matching bugs cannot hide.
data, err := base64.StdEncoding.DecodeString(payload)
if err != nil {
t.Fatal(err)
}
if zip {
// gzip's XFL byte identifies the compression level we selected.
if len(data) < 10 || data[8] != 2 {
t.Fatal("gzip output does not use the highest compression level")
}
reader, err := gzip.NewReader(bytes.NewReader(data))
if err != nil {
t.Fatal(err)
}
defer reader.Close()
data, err = io.ReadAll(reader)
if err != nil {
t.Fatal(err)
}
}
if !bytes.Equal(data, tc.data) {
t.Fatal("encoded output does not preserve the input bytes")
}
})
}
})
}
}
+11 -1
View File
@@ -5,7 +5,8 @@ import (
)
type windowsReader struct {
bufs []syscall.WSABuf
bufs []syscall.WSABuf
ready bool
}
func (r *windowsReader) Init(bs []*Buffer) {
@@ -15,6 +16,7 @@ func (r *windowsReader) Init(bs []*Buffer) {
for _, b := range bs {
r.bufs = append(r.bufs, syscall.WSABuf{Len: uint32(Size), Buf: &b.v[0]})
}
r.ready = false
}
func (r *windowsReader) Clear() {
@@ -25,6 +27,14 @@ func (r *windowsReader) Clear() {
}
func (r *windowsReader) Read(fd uintptr) int32 {
// On the first invocation, we return -1 to indicate "not ready"
// to make rawConn.Read wait for readability using the runtime's own mechanism
// because syscall.WSARecv() is a blocking call when used with nil OVERLAPPED
if !r.ready {
r.ready = true
return -1
}
var nBytes uint32
var flags uint32
err := syscall.WSARecv(syscall.Handle(fd), &r.bufs[0], uint32(len(r.bufs)), &nBytes, &flags, nil, nil)
+3 -3
View File
@@ -10,12 +10,12 @@ import (
// [,)
func RandBetween(from int64, to int64) int64 {
if from == to {
return from
}
if from > to {
from, to = to, from
}
if d := to - from; d == 0 || d == 1 {
return from
}
bigInt, _ := rand.Int(rand.Reader, big.NewInt(to-from))
return from + bigInt.Int64()
}
+13 -65
View File
@@ -18,17 +18,12 @@ type hasInnerError interface {
Unwrap() error
}
type hasSeverity interface {
Severity() log.Severity
}
// Error is an error object with underlying error.
type Error struct {
prefix []interface{}
message []interface{}
caller string
inner error
severity log.Severity
prefix []interface{}
message []interface{}
caller string
inner error
}
// Error implements error.Error().
@@ -69,46 +64,6 @@ func (err *Error) Base(e error) *Error {
return err
}
func (err *Error) atSeverity(s log.Severity) *Error {
err.severity = s
return err
}
func (err *Error) Severity() log.Severity {
if err.inner == nil {
return err.severity
}
if s, ok := err.inner.(hasSeverity); ok {
as := s.Severity()
if as < err.severity {
return as
}
}
return err.severity
}
// AtDebug sets the severity to debug.
func (err *Error) AtDebug() *Error {
return err.atSeverity(log.Severity_Debug)
}
// AtInfo sets the severity to info.
func (err *Error) AtInfo() *Error {
return err.atSeverity(log.Severity_Info)
}
// AtWarning sets the severity to warning.
func (err *Error) AtWarning() *Error {
return err.atSeverity(log.Severity_Warning)
}
// AtError sets the severity to error.
func (err *Error) AtError() *Error {
return err.atSeverity(log.Severity_Error)
}
// String returns the string representation of this error.
func (err *Error) String() string {
return err.Error()
@@ -132,9 +87,8 @@ func New(msg ...interface{}) *Error {
details = details[:i]
}
return &Error{
message: msg,
severity: log.Severity_Info,
caller: details,
message: msg,
caller: details,
}
}
@@ -171,6 +125,9 @@ func LogErrorInner(ctx context.Context, inner error, msg ...interface{}) {
}
func doLog(ctx context.Context, inner error, severity log.Severity, msg ...interface{}) {
if log.GetSeverity() < severity {
return
}
pc, _, _, _ := runtime.Caller(2)
details := runtime.FuncForPC(pc).Name()
if len(details) >= trim {
@@ -181,10 +138,9 @@ func doLog(ctx context.Context, inner error, severity log.Severity, msg ...inter
details = details[:i]
}
err := &Error{
message: msg,
severity: severity,
caller: details,
inner: inner,
message: msg,
caller: details,
inner: inner,
}
if ctx != nil && ctx != context.Background() {
id := uint32(c.IDFromContext(ctx))
@@ -193,7 +149,7 @@ func doLog(ctx context.Context, inner error, severity log.Severity, msg ...inter
}
}
log.Record(&log.GeneralMessage{
Severity: GetSeverity(err),
Severity: severity,
Content: err,
})
}
@@ -217,11 +173,3 @@ L:
}
return err
}
// GetSeverity returns the actual severity of the error, including inner errors.
func GetSeverity(err error) log.Severity {
if s, ok := err.(hasSeverity); ok {
return s.Severity()
}
return log.Severity_Info
}
+6 -15
View File
@@ -7,30 +7,21 @@ import (
"github.com/google/go-cmp/cmp"
. "github.com/xtls/xray-core/common/errors"
"github.com/xtls/xray-core/common/log"
)
func TestError(t *testing.T) {
err := New("TestError")
if v := GetSeverity(err); v != log.Severity_Info {
t.Error("severity: ", v)
if v := err.Error(); !strings.Contains(v, "TestError") {
t.Error("error: ", v)
}
err = New("TestError2").Base(io.EOF)
if v := GetSeverity(err); v != log.Severity_Info {
t.Error("severity: ", v)
if v := err.Error(); !strings.Contains(v, "EOF") {
t.Error("error: ", v)
}
err = New("TestError3").Base(io.EOF).AtWarning()
if v := GetSeverity(err); v != log.Severity_Warning {
t.Error("severity: ", v)
}
err = New("TestError4").Base(io.EOF).AtWarning()
err = New("TestError5").Base(err)
if v := GetSeverity(err); v != log.Severity_Warning {
t.Error("severity: ", v)
}
err = New("TestError3").Base(io.EOF)
err = New("TestError4").Base(err)
if v := err.Error(); !strings.Contains(v, "EOF") {
t.Error("error: ", v)
}
+59 -17
View File
@@ -8,23 +8,65 @@ import (
var privateIPMatcher = sync.OnceValue(func() IPMatcher {
return common.Must2(IPReg.BuildIPMatcher(common.Must2(ParseIPRules([]string{
"0.0.0.0/8",
"10.0.0.0/8",
"100.64.0.0/10",
"127.0.0.0/8",
"169.254.0.0/16",
"172.16.0.0/12",
"192.0.0.0/24",
"192.0.2.0/24",
"192.88.99.0/24",
"192.168.0.0/16",
"198.18.0.0/15",
"198.51.100.0/24",
"203.0.113.0/24",
"224.0.0.0/3",
"::/127",
"fc00::/7",
"fe80::/10",
// https://www.iana.org/assignments/iana-ipv4-special-registry
"0.0.0.0/8", // "This network"
"0.0.0.0/32", // "This host on this network"
"10.0.0.0/8", // Private-Use
"100.64.0.0/10", // Shared Address Space
"127.0.0.0/8", // Loopback
"169.254.0.0/16", // Link Local
"172.16.0.0/12", // Private-Use
// "192.0.0.0/24", // IETF Protocol Assignments
"192.0.0.0/29", // IPv4 Service Continuity Prefix
"192.0.0.8/32", // IPv4 dummy address
"192.0.0.9/32", // Port Control Protocol Anycast
"192.0.0.10/32", // Traversal Using Relays around NAT Anycast
"192.0.0.170/32", // NAT64/DNS64 Discovery
"192.0.0.171/32", // NAT64/DNS64 Discovery
"192.0.2.0/24", // Documentation (TEST-NET-1)
// "192.31.196.0/24", // AS112-v4
"192.52.193.0/24", // AMT
// "192.88.99.0/24", // Deprecated (6to4 Relay Anycast)
"192.88.99.2/32", // 6a44-relay anycast address
"192.168.0.0/16", // Private-Use
// "192.175.48.0/24", // Direct Delegation AS112 Service
"198.18.0.0/15", // Benchmarking
"198.51.100.0/24", // Documentation (TEST-NET-2)
"203.0.113.0/24", // Documentation (TEST-NET-3)
"240.0.0.0/4", // Reserved
"255.255.255.255/32", // Limited Broadcast
// https://www.iana.org/assignments/multicast-addresses
"224.0.0.0/4",
// https://www.iana.org/assignments/iana-ipv6-special-registry
"::1/128", // Loopback Address
"::/128", // Unspecified Address
// "::ffff:0:0/96", // IPv4-mapped Address
"64:ff9b::/96", // IPv4-IPv6 Translat.
"64:ff9b:1::/48", // IPv4-IPv6 Translat.
"100::/64", // Discard-Only Address Block
"100:0:0:1::/64", // Dummy IPv6 Prefix
// "2001::/23", // IETF Protocol Assignments
"2001::/32", // TEREDO
"2001:1::1/128", // Port Control Protocol Anycast
"2001:1::2/128", // Traversal Using Relays around NAT Anycast
"2001:1::3/128", // DNS-SD Service Registration Protocol Anycast
"2001:2::/48", // Benchmarking
"2001:3::/32", // AMT
// "2001:4:112::/48", // AS112-v6
// "2001:10::/28", // Deprecated (previously ORCHID)
"2001:20::/28", // ORCHIDv2
"2001:30::/28", // Drone Remote ID Protocol Entity Tags (DETs) Prefix
"2001:db8::/32", // Documentation
"2002::/16", // 6to4
// "2620:4f:8000::/48", // Direct Delegation AS112 Service
"3fff::/20", // Documentation
"5f00::/16", // Segment Routing (SRv6) SIDs
"fc00::/7", // Unique-Local
"fe80::/10", // Link-Local Unicast
// https://www.iana.org/assignments/ipv6-multicast-addresses
"ff00::/8",
}))))
})
+33 -50
View File
@@ -82,19 +82,10 @@ func (f *MphDomainMatcherFactory) BuildMatcher(rules []*DomainRule) (DomainMatch
}
g.Add(m, uint32(i))
case *DomainRule_Geosite:
domains, err := loadSiteWithAttrs(v.Geosite.File, v.Geosite.Code, v.Geosite.Attrs)
err := loadSiteMatchers(v.Geosite, func(m strmatcher.Matcher) { g.Add(m, uint32(i)) })
if err != nil {
return nil, err
}
for j, d := range domains {
domains[j] = nil // peak mem
m, err := parseDomain(d)
if err != nil {
errors.LogError(context.Background(), "ignore invalid geosite entry in ", v.Geosite.File, ":", v.Geosite.Code, " at index ", j, ", ", err)
continue
}
g.Add(m, uint32(i))
}
default:
panic("unknown domain rule type")
}
@@ -108,12 +99,12 @@ func (f *MphDomainMatcherFactory) BuildMatcher(rules []*DomainRule) (DomainMatch
return g, nil
}
type CompactDomainMatcherFactory struct {
type CompactMphDomainMatcherFactory struct {
sync.Mutex
shared *utils.WeakCacheMap[string, strmatcher.LinearAnyMatcher]
shared *utils.WeakCacheMap[string, strmatcher.MphValueMatcher]
}
func (f *CompactDomainMatcherFactory) getOrCreateFrom(rule *GeoSiteRule) (strmatcher.MatcherSet, error) {
func (f *CompactMphDomainMatcherFactory) getOrCreateFrom(rule *GeoSiteRule) (*strmatcher.MphValueMatcher, error) {
key := rule.File + ":" + rule.Code + "@" + rule.Attrs
f.Lock()
@@ -125,33 +116,23 @@ func (f *CompactDomainMatcherFactory) getOrCreateFrom(rule *GeoSiteRule) (strmat
}
errors.LogDebug(context.Background(), "geodata geosite matcher cache MISS ", key)
s := strmatcher.NewLinearAnyMatcher()
domains, err := loadSiteWithAttrs(rule.File, rule.Code, rule.Attrs)
if err != nil {
s := strmatcher.NewMphValueMatcher()
if err := loadSiteMatchers(rule, func(m strmatcher.Matcher) { s.Add(m, 0) }); err != nil {
return nil, err
}
for i, d := range domains {
domains[i] = nil // peak mem
m, err := parseDomain(d)
if err != nil {
errors.LogError(context.Background(), "ignore invalid geosite entry in ", rule.File, ":", rule.Code, " at index ", i, ", ", err)
continue
}
s.Add(m)
if err := s.Build(); err != nil {
return nil, err
}
f.shared.Store(key, s)
return s, err
return s, nil
}
// BuildMatcher implements DomainMatcherFactory.
func (f *CompactDomainMatcherFactory) BuildMatcher(rules []*DomainRule) (DomainMatcher, error) {
func (f *CompactMphDomainMatcherFactory) BuildMatcher(rules []*DomainRule) (DomainMatcher, error) {
if len(rules) == 0 {
return nil, errors.New("empty domain rule list")
}
compact := &CompactDomainMatcher{
matchers: make([]strmatcher.MatcherSet, 0, len(rules)),
values: make([]uint32, 0, len(rules)),
}
compact := new(CompactMphDomainMatcher)
for i, r := range rules {
switch v := r.Value.(type) {
case *DomainRule_Custom:
@@ -168,8 +149,7 @@ func (f *CompactDomainMatcherFactory) BuildMatcher(rules []*DomainRule) (DomainM
if err != nil {
return nil, err
}
compact.matchers = append(compact.matchers, m)
compact.values = append(compact.values, uint32(i))
compact.combiner.Add(m, uint32(i))
default:
panic("unknown domain rule type")
}
@@ -177,37 +157,40 @@ func (f *CompactDomainMatcherFactory) BuildMatcher(rules []*DomainRule) (DomainM
return compact, nil
}
type CompactDomainMatcher struct {
type CompactMphDomainMatcher struct {
custom strmatcher.ValueMatcher
matchers []strmatcher.MatcherSet
values []uint32
combiner strmatcher.MphValueMatcherCombiner
}
// Match implements DomainMatcher.
func (c *CompactDomainMatcher) Match(input string) []uint32 {
var result []uint32
func (c *CompactMphDomainMatcher) Match(input string) []uint32 {
result := c.combiner.Match(input)
if c.custom != nil {
result = append(result, c.custom.Match(input)...)
}
for i, m := range c.matchers {
if m.MatchAny(input) {
result = append(result, c.values[i])
}
result = append(c.custom.Match(input), result...)
}
return result
}
// MatchAny implements DomainMatcher.
func (c *CompactDomainMatcher) MatchAny(input string) bool {
func (c *CompactMphDomainMatcher) MatchAny(input string) bool {
if c.custom != nil && c.custom.MatchAny(input) {
return true
}
for _, m := range c.matchers {
if m.MatchAny(input) {
return true
return c.combiner.MatchAny(input)
}
// loadSiteMatchers calls add with a matcher for every domain of the geosite rule and logs the invalid ones.
func loadSiteMatchers(rule *GeoSiteRule, add func(strmatcher.Matcher)) error {
i := 0
return loadSite(rule.File, rule.Code, rule.Attrs, func(t Domain_Type, value []byte) {
m, err := parseDomain(&Domain{Type: t, Value: string(value)})
if err != nil {
errors.LogError(context.Background(), "ignore invalid geosite entry in ", rule.File, ":", rule.Code, " at index ", i, ", ", err)
} else {
add(m)
}
}
return false
i++
})
}
func parseDomain(d *Domain) (strmatcher.Matcher, error) {
@@ -231,7 +214,7 @@ func parseDomain(d *Domain) (strmatcher.Matcher, error) {
func newDomainMatcherFactory() DomainMatcherFactory {
switch runtime.GOOS {
case "ios", "android":
return &CompactDomainMatcherFactory{shared: utils.NewWeakCacheMap[string, strmatcher.LinearAnyMatcher]()}
return &CompactMphDomainMatcherFactory{shared: utils.NewWeakCacheMap[string, strmatcher.MphValueMatcher]()}
default:
return &MphDomainMatcherFactory{shared: utils.NewWeakCacheMap[string, strmatcher.MphValueMatcher]()}
}
+76 -2
View File
@@ -4,6 +4,7 @@ import (
"path/filepath"
"reflect"
"slices"
"sync"
"testing"
"github.com/xtls/xray-core/common/geodata/strmatcher"
@@ -11,7 +12,7 @@ import (
)
func TestCompactDomainMatcher_PreservesCustomRuleIndices(t *testing.T) {
factory := &CompactDomainMatcherFactory{shared: utils.NewWeakCacheMap[string, strmatcher.LinearAnyMatcher]()}
factory := &CompactMphDomainMatcherFactory{shared: utils.NewWeakCacheMap[string, strmatcher.MphValueMatcher]()}
matcher, err := factory.BuildMatcher([]*DomainRule{
{Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Full, Value: "example.com"}}},
{Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Domain, Value: "example.com"}}},
@@ -32,7 +33,7 @@ func TestCompactDomainMatcher_PreservesCustomRuleIndices(t *testing.T) {
func TestCompactDomainMatcher_PreservesMixedRuleIndices(t *testing.T) {
t.Setenv("xray.location.asset", filepath.Join("..", "..", "resources"))
factory := &CompactDomainMatcherFactory{shared: utils.NewWeakCacheMap[string, strmatcher.LinearAnyMatcher]()}
factory := &CompactMphDomainMatcherFactory{shared: utils.NewWeakCacheMap[string, strmatcher.MphValueMatcher]()}
matcher, err := factory.BuildMatcher([]*DomainRule{
{Value: &DomainRule_Geosite{Geosite: &GeoSiteRule{File: DefaultGeoSiteDat, Code: "CN"}}},
{Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Full, Value: "163.com"}}},
@@ -72,3 +73,76 @@ func TestMphDomainMatcher_MatchReturnsDetachedSlice(t *testing.T) {
t.Fatalf("Match() after caller mutation = %v, want %v", gotAgain, []uint32{0, 1})
}
}
// DNS sorts every Match result in place, so a matcher must never hand out a
// slice it keeps, also when only its keyword or regex part matches.
func TestDomainMatcher_MatchResultsCanBeSortedConcurrently(t *testing.T) {
t.Setenv("xray.location.asset", filepath.Join("..", "..", "resources"))
rules := []*DomainRule{
{Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Full, Value: "example.com"}}},
{Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Domain, Value: "example.com"}}},
{Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Substr, Value: "exam"}}},
{Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Regex, Value: `^ex.*\.org$`}}},
{Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Substr, Value: "exam"}}},
{Value: &DomainRule_Geosite{Geosite: &GeoSiteRule{File: DefaultGeoSiteDat, Code: "CN"}}},
{Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Full, Value: "only.full.test"}}},
}
cases := []struct {
input string
want []uint32
}{
{"example.com", []uint32{0, 1, 2, 4}},
{"www.example.com", []uint32{1, 2, 4}},
{"exam.net", []uint32{2, 4}}, // keyword part only
{"example.org", []uint32{2, 3, 4}},
{"163.com", []uint32{5}},
{"www.163.com", []uint32{5}},
{"only.full.test", []uint32{6}}, // full part only
{"nomatch.test", nil},
}
factories := map[string]DomainMatcherFactory{
"mph": &MphDomainMatcherFactory{shared: utils.NewWeakCacheMap[string, strmatcher.MphValueMatcher]()},
"compact": &CompactMphDomainMatcherFactory{shared: utils.NewWeakCacheMap[string, strmatcher.MphValueMatcher]()},
}
for name, factory := range factories {
t.Run(name, func(t *testing.T) {
matcher, err := factory.BuildMatcher(rules)
if err != nil {
t.Fatalf("BuildMatcher() failed: %v", err)
}
for _, c := range cases {
got := matcher.Match(c.input)
if sorted := slices.Sorted(slices.Values(got)); !slices.Equal(sorted, c.want) {
t.Fatalf("Match(%q) = %v, want %v", c.input, sorted, c.want)
}
got = got[:cap(got)]
for j := range got {
got[j] = ^uint32(0)
}
if again := slices.Sorted(slices.Values(matcher.Match(c.input))); !slices.Equal(again, c.want) {
t.Fatalf("Match(%q) after caller mutation = %v, want %v", c.input, again, c.want)
}
}
var wg sync.WaitGroup
for range 8 {
wg.Add(1)
go func() {
defer wg.Done()
for range 500 {
for _, c := range cases {
got := matcher.Match(c.input)
slices.Sort(got)
if !slices.Equal(got, c.want) {
t.Errorf("Match(%q) = %v, want %v", c.input, got, c.want)
return
}
}
}
}()
}
wg.Wait()
})
}
}
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: common/geodata/geodat.proto
package geodata
+226 -70
View File
@@ -5,21 +5,24 @@ import (
"bytes"
"io"
"runtime"
"slices"
"strings"
"unicode/utf8"
"github.com/xtls/xray-core/common/errors"
"github.com/xtls/xray-core/common/platform/filesystem"
"google.golang.org/protobuf/encoding/protowire"
"google.golang.org/protobuf/proto"
)
func checkFile(file, code string) error {
r, err := filesystem.OpenAsset(file)
r, size, err := filesystem.OpenAsset(file)
if err != nil {
return errors.New("failed to open ", file).Base(err)
}
defer r.Close()
if _, err := find(r, []byte(code), false); err != nil {
if _, err := find(r, []byte(code), false, size); err != nil {
return errors.New("failed to check code ", code, " from ", file).Base(err)
}
return nil
@@ -27,12 +30,12 @@ func checkFile(file, code string) error {
func loadFile(file, code string) ([]byte, error) {
runtime.GC() // peak mem
r, err := filesystem.OpenAsset(file)
r, size, err := filesystem.OpenAsset(file)
if err != nil {
return nil, errors.New("failed to open ", file).Base(err)
}
defer r.Close()
bs, err := find(r, []byte(code), true)
bs, err := find(r, []byte(code), true, size)
if err != nil {
return nil, errors.New("failed to load code ", code, " from ", file).Base(err)
}
@@ -52,17 +55,56 @@ func loadIP(file, code string) ([]*CIDR, error) {
return geoip.Cidr, nil
}
func loadSite(file, code string) ([]*Domain, error) {
bs, err := loadFile(file, code)
// loadSite calls fn, in file order, with the type and value of every domain of the geosite code
// that has all the "@"-separated attrs. It decodes the entry while reading the file instead of
// unmarshalling it into a []*Domain, so value is only valid during fn.
func loadSite(file, code, attrs string, fn func(Domain_Type, []byte)) error {
runtime.GC() // peak mem
r, size, err := filesystem.OpenAsset(file)
if err != nil {
return nil, err
return errors.New("failed to open ", file).Base(err)
}
defer runtime.GC() // peak mem
var geosite GeoSite
if err := proto.Unmarshal(bs, &geosite); err != nil {
return nil, errors.New("error unmarshal Site in ", file, ":", code).Base(err)
defer r.Close()
br := bufio.NewReaderSize(r, 64*1024)
n, err := seek(br, []byte(code), size)
if err != nil {
return errors.New("failed to load code ", code, " from ", file).Base(err)
}
return geosite.Domain, nil
loadErr := func(err error) error {
if err == io.EOF {
err = io.ErrUnexpectedEOF
}
return errors.New("failed to load code ", code, " from ", file).Base(err)
}
unmarshalErr := func(err error) error {
return errors.New("error unmarshal Site in ", file, ":", code).Base(err)
}
d := newSiteDecoder(attrs, fn)
for n > 0 {
w, err := br.Peek(min(n, br.Size()))
if err != nil {
return loadErr(err)
}
used, err := d.decode(w, len(w) < n)
if err != nil {
return unmarshalErr(err)
}
if used == 0 {
break // a field longer than the buffer
}
br.Discard(used)
n -= used
}
if n > 0 {
w := make([]byte, n)
if _, err := io.ReadFull(br, w); err != nil {
return loadErr(err)
}
if _, err := d.decode(w, false); err != nil {
return unmarshalErr(err)
}
}
return nil
}
func decodeVarint(br *bufio.Reader) (uint64, error) {
@@ -72,6 +114,9 @@ func decodeVarint(br *bufio.Reader) (uint64, error) {
if err != nil {
return 0, err
}
if shift == 63 && b > 1 {
break // Only one value bit fits in the tenth byte.
}
x |= (uint64(b) & 0x7F) << shift
if (b & 0x80) == 0 {
return x, nil
@@ -81,69 +126,66 @@ func decodeVarint(br *bufio.Reader) (uint64, error) {
return 0, errors.New("varint overflow")
}
func find(r io.Reader, code []byte, readBody bool) ([]byte, error) {
func find(r io.Reader, code []byte, readBody bool, size int64) ([]byte, error) {
br := bufio.NewReaderSize(r, 64*1024)
bodyL, err := seek(br, code, size)
if err != nil || !readBody {
return nil, err
}
out := make([]byte, bodyL)
if _, err := io.ReadFull(br, out); err != nil {
return nil, err
}
return out, nil
}
// seek advances br to the body of the entry for code and returns the body length.
func seek(br *bufio.Reader, code []byte, size int64) (int, error) {
codeL := len(code)
if codeL == 0 {
return nil, errors.New("empty code")
return 0, errors.New("empty code")
}
br := bufio.NewReaderSize(r, 64*1024)
need := 2 + codeL // TODO: if code too long
prefixBuf := make([]byte, need)
for {
if _, err := br.ReadByte(); err != nil {
return nil, err
tag, err := br.ReadByte()
if err != nil {
return 0, err
}
x, err := decodeVarint(br)
if err != nil {
return nil, err
return 0, err
}
if x == 0 || x > uint64(size) || x > uint64(^uint(0)>>1) {
return 0, errors.New("invalid body length: ", x)
}
bodyL := int(x)
if bodyL <= 0 {
return nil, errors.New("invalid body length: ", bodyL)
}
prefixL := bodyL
if prefixL > need {
prefixL = need
}
prefix := prefixBuf[:prefixL]
if _, err := io.ReadFull(br, prefix); err != nil {
return nil, err
}
match := false
if bodyL >= need {
if int(prefix[1]) == codeL && bytes.Equal(prefix[2:need], code) {
if !readBody {
return nil, nil
}
match = true
// Peek no more than the buffer holds: a code longer than the buffer cannot match a single
// length byte anyway, so a short peek only skips it, as base find (io.ReadFull) does.
prefix, err := br.Peek(min(bodyL, need, br.Size()))
if err != nil {
if err == io.EOF && len(prefix) > 0 {
err = io.ErrUnexpectedEOF // as io.ReadFull
}
return 0, err
}
remain := bodyL - prefixL
if match {
out := make([]byte, bodyL)
copy(out, prefix)
if remain > 0 {
if _, err := io.ReadFull(br, out[prefixL:]); err != nil {
return nil, err
}
}
return out, nil
if tag == 0x0a && bodyL >= need && len(prefix) >= need &&
prefix[0] == 0x0a && int(prefix[1]) == codeL && bytes.Equal(prefix[2:], code) {
return bodyL, nil
}
if remain > 0 {
if _, err := br.Discard(remain); err != nil {
return nil, err
}
if _, err := br.Discard(bodyL); err != nil {
return 0, err
}
}
}
// AttributeMatcher, HasAttrMatcher, AllAttrsMatcher and NewAllAttrsMatcher are the exported
// attribute helpers that have been part of this package's API since #5814. The streaming loader
// above filters attributes itself without building a *Domain, so it does not use them, but they
// are kept for external callers. Their behaviour is unchanged.
type AttributeMatcher interface {
Match(*Domain) bool
}
@@ -185,23 +227,137 @@ func NewAllAttrsMatcher(attrs string) AttributeMatcher {
return m
}
func loadSiteWithAttrs(file, code, attrs string) ([]*Domain, error) {
domains, err := loadSite(file, code)
if err != nil {
return nil, err
}
var errInvalidUTF8 = errors.New("string field contains invalid UTF-8")
matcher := NewAllAttrsMatcher(attrs)
if matcher == nil {
return domains, nil
}
type siteDecoder struct {
want []string
has []bool
fn func(Domain_Type, []byte)
}
filtered := make([]*Domain, 0, len(domains))
for _, d := range domains {
if matcher.Match(d) {
filtered = append(filtered, d)
func newSiteDecoder(attrs string, fn func(Domain_Type, []byte)) *siteDecoder {
d := &siteDecoder{fn: fn}
if attrs != "" {
d.want = strings.Split(attrs, "@")
d.has = make([]bool, len(d.want))
}
return d
}
// decode walks the whole fields at the start of b, a part of an encoded GeoSite (see geodat.proto),
// calls fn for every domain that has all attrs and returns how many bytes it used. A field cut off
// by the end of b is an error unless more is set. It accepts and rejects what proto.Unmarshal does.
func (d *siteDecoder) decode(b []byte, more bool) (int, error) {
used := 0
for used < len(b) {
f, n, err := consumeField(b[used:])
if err == io.ErrUnexpectedEOF && more {
break
}
if err != nil {
return used, err
}
used += n
if f.typ != protowire.BytesType {
continue
}
switch f.num {
case 1: // code
if !utf8.Valid(f.v) {
return used, errInvalidUTF8
}
case 2: // domain
t, value, err := decodeDomain(f.v, d.want, d.has)
if err != nil {
return used, err
}
if !slices.Contains(d.has, false) {
d.fn(t, value)
}
}
}
return filtered, nil
return used, nil
}
// decodeDomain decodes an encoded Domain and sets has[i] if one of its attributes has the key want[i].
func decodeDomain(b []byte, want []string, has []bool) (t Domain_Type, value []byte, err error) {
clear(has)
for len(b) > 0 {
f, n, err := consumeField(b)
if err != nil {
return 0, nil, err
}
b = b[n:]
switch {
case f.num == 1 && f.typ == protowire.VarintType: // type
t = Domain_Type(f.x)
case f.num == 2 && f.typ == protowire.BytesType: // value
if !utf8.Valid(f.v) {
return 0, nil, errInvalidUTF8
}
value = f.v
case f.num == 3 && f.typ == protowire.BytesType: // attribute
key, err := decodeAttributeKey(f.v)
if err != nil {
return 0, nil, err
}
for i, w := range want {
if string(key) == w {
has[i] = true
}
}
}
}
return t, value, nil
}
// decodeAttributeKey returns the key of an encoded Domain.Attribute.
func decodeAttributeKey(b []byte) ([]byte, error) {
var key []byte
for len(b) > 0 {
f, n, err := consumeField(b)
if err != nil {
return nil, err
}
b = b[n:]
if f.num == 1 && f.typ == protowire.BytesType {
if !utf8.Valid(f.v) {
return nil, errInvalidUTF8
}
key = f.v
}
}
return key, nil
}
type protoField struct {
num protowire.Number
typ protowire.Type
v []byte // payload of a length-delimited field
x uint64 // value of a varint field
}
// consumeField parses the first field of an encoded message and returns it with its length.
func consumeField(b []byte) (protoField, int, error) {
num, typ, n := protowire.ConsumeTag(b)
if n < 0 {
return protoField{}, 0, protowire.ParseError(n)
}
if num > protowire.MaxValidNumber {
return protoField{}, 0, errors.New("invalid field number ", num)
}
f := protoField{num: num, typ: typ}
var m int
switch typ {
case protowire.BytesType:
f.v, m = protowire.ConsumeBytes(b[n:])
case protowire.VarintType:
f.x, m = protowire.ConsumeVarint(b[n:])
default:
m = protowire.ConsumeFieldValue(num, typ, b[n:])
}
if m < 0 {
return protoField{}, 0, protowire.ParseError(m)
}
return f, n + m, nil
}
+336
View File
@@ -0,0 +1,336 @@
package geodata
import (
"bytes"
"fmt"
"os"
"path/filepath"
"slices"
"strings"
"testing"
"google.golang.org/protobuf/encoding/protowire"
"google.golang.org/protobuf/proto"
)
func TestFindChecksEntryAndCodeTags(t *testing.T) {
for _, payload := range [][]byte{
{0x12, 6, 0x0a, 4, 'T', 'E', 'S', 'T'}, // Unknown field posing as entry.
{0x08, 6, 0x0a, 4, 'T', 'E', 'S', 'T'}, // Wrong entry wire type.
{0x0a, 6, 0x1a, 4, 'T', 'E', 'S', 'T'}, // Unknown field posing as code.
{0x0a, 6, 0x08, 4, 'T', 'E', 'S', 'T'}, // Wrong code wire type.
} {
if _, err := find(bytes.NewReader(payload), []byte("TEST"), true, int64(len(payload))); err == nil {
t.Fatalf("matched incorrect entry/code tags in %x", payload)
}
}
}
func TestLoadFileRejectsVarintOverflow(t *testing.T) {
payload := []byte{
0x0a,
0x86, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x02, // 2^64 + 6
0x0a, 0x04, 'T', 'E', 'S', 'T',
}
dir := t.TempDir()
t.Setenv("xray.location.asset", dir)
if err := os.WriteFile(filepath.Join(dir, "overflow.dat"), payload, 0o600); err != nil {
t.Fatal(err)
}
if _, err := loadFile("overflow.dat", "TEST"); err == nil || !strings.Contains(err.Error(), "varint overflow") {
t.Fatalf("want varint overflow, got %v", err)
}
}
func TestLoadFileRejectsOversizedEntry(t *testing.T) {
payload := []byte{
0x0a, 0x03, 0x0a, 0x01, 'A', // A valid entry before the oversized one.
0x0a, 0x80, 0x80, 0x80, 0x80, 0x04, // Entry length: 1 GiB.
0x0a, 0x04, 'T', 'E', 'S', 'T',
}
dir := t.TempDir()
t.Setenv("xray.location.asset", dir)
if err := os.WriteFile(filepath.Join(dir, "oversized.dat"), payload, 0o600); err != nil {
t.Fatal(err)
}
// Check without allocating the body first, so a regression fails safely.
if err := checkFile("oversized.dat", "TEST"); err == nil {
t.Fatal("accepted an entry length larger than the file")
}
if _, err := loadFile("oversized.dat", "TEST"); err == nil || !strings.Contains(err.Error(), "invalid body length") {
t.Fatalf("want invalid body length, got %v", err)
}
if err := loadSite("oversized.dat", "TEST", "", func(Domain_Type, []byte) {}); err == nil || !strings.Contains(err.Error(), "invalid body length") {
t.Fatalf("want invalid body length, got %v", err)
}
}
type siteEntry struct {
Type Domain_Type
Value string
}
// unmarshalSite is what loadSite used to do: proto.Unmarshal, then keep the domains that have all attrs.
func unmarshalSite(b []byte, attrs string) ([]siteEntry, error) {
var site GeoSite
if err := proto.Unmarshal(b, &site); err != nil {
return nil, err
}
var entries []siteEntry
for _, d := range site.Domain {
ok := true
for _, key := range strings.Split(attrs, "@") {
ok = ok && (attrs == "" || slices.ContainsFunc(d.Attribute, func(a *Domain_Attribute) bool { return a.Key == key }))
}
if ok {
entries = append(entries, siteEntry{d.Type, d.Value})
}
}
return entries, nil
}
func checkDecodeSite(t *testing.T, name string, b []byte, attrs string) {
t.Helper()
want, wantErr := unmarshalSite(b, attrs)
var got []siteEntry
_, err := newSiteDecoder(attrs, func(typ Domain_Type, value []byte) {
got = append(got, siteEntry{typ, string(value)})
}).decode(b, false)
if (err == nil) != (wantErr == nil) {
t.Fatalf("%s@%s: error %v, proto.Unmarshal: %v", name, attrs, err, wantErr)
}
if err == nil && !slices.Equal(got, want) {
t.Fatalf("%s@%s: got %v, want %v", name, attrs, got, want)
}
}
func TestDecodeSiteMatchesUnmarshal(t *testing.T) {
bs, err := os.ReadFile(filepath.Join("..", "..", "resources", DefaultGeoSiteDat))
if err != nil {
t.Fatal(err)
}
for len(bs) > 0 {
num, typ, n := protowire.ConsumeTag(bs)
if n < 0 || num != 1 || typ != protowire.BytesType {
t.Fatal("unexpected GeoSiteList field")
}
entry, m := protowire.ConsumeBytes(bs[n:])
if m < 0 {
t.Fatal(protowire.ParseError(m))
}
bs = bs[n+m:]
var site GeoSite
if err := proto.Unmarshal(entry, &site); err != nil {
t.Fatal(err)
}
queries := []string{"", "none"}
for _, d := range site.Domain {
for _, a := range d.Attribute {
if !slices.Contains(queries, a.Key) {
queries = append(queries, a.Key, a.Key+"@none")
}
}
}
for _, attrs := range queries {
checkDecodeSite(t, site.Code, entry, attrs)
}
}
}
func TestDecodeSiteUnusualEncodings(t *testing.T) {
field := func(num protowire.Number, v []byte) []byte {
return protowire.AppendBytes(protowire.AppendTag(nil, num, protowire.BytesType), v)
}
typ := func(v Domain_Type) []byte {
return protowire.AppendVarint(protowire.AppendTag(nil, 1, protowire.VarintType), uint64(v))
}
value := func(s string) []byte { return field(2, []byte(s)) }
attr := func(keys ...string) []byte {
var b []byte
for _, k := range keys {
b = append(b, field(1, []byte(k))...)
}
return field(3, b)
}
domain := func(fields ...[]byte) []byte { return field(2, slices.Concat(fields...)) }
unknown := protowire.AppendFixed32(protowire.AppendTag(nil, 9, protowire.Fixed32Type), 1)
for name, b := range map[string][]byte{
"unknown field": domain(typ(Domain_Full), unknown, value("example.com")),
"repeated value": domain(value("a.com"), typ(Domain_Full), value("b.com")),
"repeated type": domain(typ(Domain_Full), value("a.com"), typ(Domain_Regex)),
"repeated key": domain(value("a.com"), attr("cn", "ads")),
"type as bytes": domain(field(1, []byte("x")), value("a.com")),
"no value": domain(typ(Domain_Domain), attr("cn")),
"truncated": domain(typ(Domain_Full), value("example.com"))[:10],
"invalid utf8": domain(value("example.\xff")),
"invalid key": domain(value("a.com"), attr("\xff")),
"bad field": protowire.AppendVarint(protowire.AppendTag(nil, protowire.MaxValidNumber+1, protowire.VarintType), 1),
"stray end group": protowire.AppendTag(nil, 5, protowire.EndGroupType),
} {
for _, attrs := range []string{"", "cn", "ads", "cn@ads"} {
checkDecodeSite(t, name, b, attrs)
}
}
}
// TestLoadSiteReadsInPieces covers what real lists never do: an entry far longer than the read
// buffer, with a field longer than the buffer in the middle, and a file cut short.
func TestLoadSiteReadsInPieces(t *testing.T) {
site := &GeoSite{Code: "BIG"}
for i := range 5000 {
d := &Domain{Type: Domain_Domain, Value: strings.Repeat("x", i%40) + ".example.com"}
if i%3 == 0 {
d.Attribute = []*Domain_Attribute{{Key: "cn"}}
}
if i == 2500 {
d = &Domain{Type: Domain_Regex, Value: strings.Repeat("a", 100_000)}
}
site.Domain = append(site.Domain, d)
}
list := &GeoSiteList{Entry: []*GeoSite{{Code: "SMALL", Domain: []*Domain{{Type: Domain_Full, Value: "a.com"}}}, site}}
bs, err := proto.Marshal(list)
if err != nil {
t.Fatal(err)
}
entry, err := proto.Marshal(site)
if err != nil {
t.Fatal(err)
}
dir := t.TempDir()
t.Setenv("xray.location.asset", dir)
write := func(b []byte) {
if err := os.WriteFile(filepath.Join(dir, "big.dat"), b, 0o644); err != nil {
t.Fatal(err)
}
}
for _, attrs := range []string{"", "cn"} {
want, _ := unmarshalSite(entry, attrs)
var got []siteEntry
write(bs)
err := loadSite("big.dat", "BIG", attrs, func(typ Domain_Type, value []byte) {
got = append(got, siteEntry{typ, string(value)})
})
if err != nil || !slices.Equal(got, want) {
t.Fatalf("attrs %q: %d entries, want %d, error %v", attrs, len(got), len(want), err)
}
for _, cut := range []int{30_000, len(bs) - 150_000, len(bs) - 1} {
write(bs[:cut])
if err := loadSite("big.dat", "BIG", attrs, func(Domain_Type, []byte) {}); err == nil {
t.Fatalf("file cut at %d of %d: no error", cut, len(bs))
}
}
}
}
// oneEntryGeoSiteFile wraps an encoded GeoSite as a one-entry GeoSiteList, the file loadSite reads.
func oneEntryGeoSiteFile(entry []byte) []byte {
return protowire.AppendBytes(protowire.AppendTag(nil, 1, protowire.BytesType), entry)
}
// TestLoadSiteWindowedMatchesSingleShot checks that the windowed reader in loadSite (its Peek/Discard
// loop, the more-break when a field is cut by a window edge, the used==0 fallback for a field longer
// than the buffer, and the tail path) reaches exactly the same result as decoding the whole entry at
// once, for a category several 64 KiB windows long, valid and then mutated near a window edge and
// early in the file: same error-or-not, and the same emitted (type, value) sequence when both accept.
func TestLoadSiteWindowedMatchesSingleShot(t *testing.T) {
const window = 64 * 1024
site := &GeoSite{Code: "BIG"}
for i := range 12000 { // ~250 KiB, four windows
d := &Domain{Type: Domain_Domain, Value: fmt.Sprintf("host%d.%s.example.com", i, strings.Repeat("y", i%30))}
if i%3 == 0 {
d.Attribute = []*Domain_Attribute{{Key: "cn"}}
}
site.Domain = append(site.Domain, d)
}
// a field longer than the buffer, straddling the third window, to force the used==0 fallback
site.Domain = slices.Insert(site.Domain, 8000, &Domain{Type: Domain_Regex, Value: strings.Repeat("a", 90_000)})
entry, err := proto.Marshal(site)
if err != nil {
t.Fatal(err)
}
dir := t.TempDir()
t.Setenv("xray.location.asset", dir)
// mutations of the encoded entry: unchanged, a byte flipped at several offsets (early windows and
// either side of a window edge), and truncations at the same places.
type mut struct {
name string
make func([]byte) []byte
}
muts := []mut{{"valid", func(b []byte) []byte { return b }}}
for _, off := range []int{3, 40, 4000, window - 1, window, window + 1, 2*window - 2, 2 * window} {
if off < len(entry) {
off := off
muts = append(muts, mut{fmt.Sprintf("flip@%d", off), func(b []byte) []byte {
c := slices.Clone(b)
c[off] ^= 0xff
return c
}})
muts = append(muts, mut{fmt.Sprintf("cut@%d", off), func(b []byte) []byte { return slices.Clone(b[:off]) }})
}
}
for _, attrs := range []string{"", "cn"} {
for _, m := range muts {
e := m.make(entry)
// single-shot reference: decode the whole entry in one call
var want []siteEntry
_, wantErr := newSiteDecoder(attrs, func(typ Domain_Type, value []byte) {
want = append(want, siteEntry{typ, string(value)})
}).decode(e, false)
// windowed: loadSite reads the file 64 KiB at a time
if err := os.WriteFile(filepath.Join(dir, "w.dat"), oneEntryGeoSiteFile(e), 0o644); err != nil {
t.Fatal(err)
}
var got []siteEntry
gotErr := loadSite("w.dat", "BIG", attrs, func(typ Domain_Type, value []byte) {
got = append(got, siteEntry{typ, string(value)})
})
if (gotErr == nil) != (wantErr == nil) {
t.Fatalf("%s attrs=%q: windowed err %v, single-shot err %v", m.name, attrs, gotErr, wantErr)
}
if gotErr == nil && !slices.Equal(got, want) {
t.Fatalf("%s attrs=%q: windowed got %d entries, single-shot %d", m.name, attrs, len(got), len(want))
}
}
}
}
// TestLoadSiteLongCode covers a geosite entry whose code is longer than the 64 KiB read buffer. seek
// must skip it (find compares a single length byte, so it never matches such a code) and still find a
// later entry, and looking the long code up must fail cleanly, like a missing code, not panic.
func TestLoadSiteLongCode(t *testing.T) {
longCode := strings.Repeat("Z", 70000)
list := &GeoSiteList{Entry: []*GeoSite{
{Code: "FIRST", Domain: []*Domain{{Type: Domain_Full, Value: "first.com"}}},
{Code: longCode, Domain: []*Domain{{Type: Domain_Full, Value: "huge.com"}}},
{Code: "AFTER", Domain: []*Domain{{Type: Domain_Domain, Value: "after.com"}}},
}}
bs, err := proto.Marshal(list)
if err != nil {
t.Fatal(err)
}
dir := t.TempDir()
t.Setenv("xray.location.asset", dir)
if err := os.WriteFile(filepath.Join(dir, "lc.dat"), bs, 0o644); err != nil {
t.Fatal(err)
}
collect := func(code string) ([]siteEntry, error) {
var got []siteEntry
err := loadSite("lc.dat", code, "", func(typ Domain_Type, value []byte) {
got = append(got, siteEntry{typ, string(value)})
})
return got, err
}
if got, err := collect("FIRST"); err != nil || !slices.Equal(got, []siteEntry{{Domain_Full, "first.com"}}) {
t.Fatalf("FIRST: %v %v", got, err)
}
if got, err := collect("AFTER"); err != nil || !slices.Equal(got, []siteEntry{{Domain_Domain, "after.com"}}) {
t.Fatalf("AFTER (past the oversized entry): %v %v", got, err)
}
if _, err := collect(longCode); err == nil {
t.Fatal("oversized code: expected a not-found error, got nil")
}
}
+7 -7
View File
@@ -400,13 +400,13 @@ func TestIPMatcher4CN(t *testing.T) {
}
}
func TestIPMatcher6US(t *testing.T) {
func TestIPMatcher6RU(t *testing.T) {
t.Setenv("xray.location.asset", filepath.Join("..", "..", "resources"))
matcher := buildIPMatcher("geoip:us")
matcher := buildIPMatcher("geoip:ru")
if !matcher.Match(xnet.ParseAddress("2001:4860:4860::8888").IP()) {
t.Error("expect US geoip contain 2001:4860:4860::8888, but actually not")
if !matcher.Match(xnet.ParseAddress("2a02:6b8::feed:0ff").IP()) {
t.Error("expect RU geoip contain 2a02:6b8::feed:0ff, but actually not")
}
}
@@ -423,11 +423,11 @@ func BenchmarkIPMatcher4CN(b *testing.B) {
}
}
func BenchmarkIPMatcher6US(b *testing.B) {
func BenchmarkIPMatcher6RU(b *testing.B) {
b.Setenv("xray.location.asset", filepath.Join("..", "..", "resources"))
matcher := buildIPMatcher("geoip:us")
ip := xnet.ParseAddress("2001:4860:4860::8888").IP()
matcher := buildIPMatcher("geoip:ru")
ip := xnet.ParseAddress("2a02:6b8::feed:0ff").IP()
b.ResetTimer()
+163
View File
@@ -0,0 +1,163 @@
package geodata
import (
xlua "github.com/xtls/xray-core/common/lua"
"github.com/xtls/xray-core/common/net"
lua "github.com/yuin/gopher-lua"
)
// RegisterLua makes xray.geodata available to require in an LState.
func RegisterLua(L *lua.LState) {
L.PreloadModule("xray.geodata", func(L *lua.LState) int {
module := L.CreateTable(0, 2)
module.RawSetString("BuildDomainMatcher", L.NewFunction(func(L *lua.LState) int {
parsed, err := ParseDomainRules(luaRules(L), Domain_Domain)
if err != nil {
L.RaiseError("%v", err)
return 0
}
matcher, err := DomainReg.BuildDomainMatcher(parsed)
if err != nil {
L.RaiseError("%v", err)
return 0
}
xlua.PushWithDirectMethods(L, matcher, map[string]xlua.DirectMethod{
"Match": newLuaDomainMatch(xlua.NewSlicePusher[uint32](L)),
"MatchAny": luaDomainMatchAny,
})
return 1
}))
module.RawSetString("BuildIPMatcher", L.NewFunction(func(L *lua.LState) int {
parsed, err := ParseIPRules(luaRules(L))
if err != nil {
L.RaiseError("%v", err)
return 0
}
matcher, err := IPReg.BuildIPMatcher(parsed)
if err != nil {
L.RaiseError("%v", err)
return 0
}
xlua.PushWithDirectMethods(L, matcher, map[string]xlua.DirectMethod{
"Match": luaIPMatch,
"AnyMatch": luaIPAnyMatch,
"Matches": luaIPMatches,
"FilterIPs": newLuaIPFilterIPs(xlua.NewReadOnlySlicePusher[net.IP](L)),
}, "SetReverse", "ToggleReverse")
return 1
}))
L.Push(module)
return 1
})
}
// Read native Go values by type assertion; slices keep their original storage.
func readLuaIPMatcherArgs[T any](L *lua.LState) (IPMatcher, T, bool) {
var input T
if L.GetTop() != 2 {
return nil, input, false
}
value, ok := L.Get(1).(*lua.LUserData)
if !ok {
return nil, input, false
}
matcher, ok := value.Value.(IPMatcher)
if !ok {
return nil, input, false
}
if L.Get(2) == lua.LNil {
return matcher, input, true
}
value, ok = L.Get(2).(*lua.LUserData)
if !ok {
return nil, input, false
}
input, ok = value.Value.(T)
return matcher, input, ok
}
func luaIPMatch(L *lua.LState) (int, bool) {
matcher, ip, ok := readLuaIPMatcherArgs[net.IP](L)
if !ok {
return 0, false
}
L.Push(lua.LBool(matcher.Match(ip)))
return 1, true
}
func luaIPAnyMatch(L *lua.LState) (int, bool) {
matcher, ips, ok := readLuaIPMatcherArgs[[]net.IP](L)
if !ok {
return 0, false
}
L.Push(lua.LBool(matcher.AnyMatch(ips)))
return 1, true
}
func luaIPMatches(L *lua.LState) (int, bool) {
matcher, ips, ok := readLuaIPMatcherArgs[[]net.IP](L)
if !ok {
return 0, false
}
L.Push(lua.LBool(matcher.Matches(ips)))
return 1, true
}
func newLuaIPFilterIPs(pushIPs func(*lua.LState, []net.IP)) xlua.DirectMethod {
return func(L *lua.LState) (int, bool) {
matcher, ips, ok := readLuaIPMatcherArgs[[]net.IP](L)
if !ok {
return 0, false
}
matched, unmatched := matcher.FilterIPs(ips)
pushIPs(L, matched)
pushIPs(L, unmatched)
return 2, true
}
}
func newLuaDomainMatch(pushMatches func(*lua.LState, []uint32)) xlua.DirectMethod {
return func(L *lua.LState) (int, bool) {
if L.GetTop() == 2 {
if value, ok := L.Get(1).(*lua.LUserData); ok {
matcher, validMatcher := value.Value.(DomainMatcher)
domain, validDomain := L.Get(2).(lua.LString)
if validMatcher && validDomain {
pushMatches(L, matcher.Match(string(domain)))
return 1, true
}
}
}
return 0, false
}
}
func luaDomainMatchAny(L *lua.LState) (int, bool) {
if L.GetTop() == 2 {
if value, ok := L.Get(1).(*lua.LUserData); ok {
matcher, validMatcher := value.Value.(DomainMatcher)
domain, validDomain := L.Get(2).(lua.LString)
if validMatcher && validDomain {
L.Push(lua.LBool(matcher.MatchAny(string(domain))))
return 1, true
}
}
}
return 0, false
}
func luaRules(L *lua.LState) []string {
rules := make([]string, L.GetTop())
for i := range rules {
value, ok := L.Get(i + 1).(lua.LString)
if !ok {
L.RaiseError("geodata rules must be strings")
return nil
}
rules[i] = string(value)
}
return rules
}
+178
View File
@@ -0,0 +1,178 @@
package geodata
import (
"fmt"
"testing"
"github.com/xtls/xray-core/common/net"
lua "github.com/yuin/gopher-lua"
)
func TestLuaIPMatcher(t *testing.T) {
L := lua.NewState()
defer L.Close()
RegisterLua(L)
ip := L.NewUserData()
ip.Value = net.ParseIP("127.0.0.1")
L.SetGlobal("ip", ip)
ips := L.NewUserData()
ips.Value = []net.IP{ip.Value.(net.IP), net.ParseIP("8.8.8.8")}
L.SetGlobal("ips", ips)
if err := L.DoString(`
local matcher = require("xray.geodata").BuildIPMatcher("127.0.0.0/8", "::1")
assert(matcher:Match(ip))
assert(matcher:AnyMatch(ips))
assert(not matcher:Matches(ips))
local matched, unmatched = matcher:FilterIPs(ips)
assert(type(matched) == "userdata" and type(unmatched) == "userdata")
assert(#matched == 1 and #unmatched == 1)
`); err != nil {
t.Fatal(err)
}
}
func TestLuaDomainMatcher(t *testing.T) {
L := lua.NewState()
defer L.Close()
RegisterLua(L)
if err := L.DoString(`
local matcher = require("xray.geodata").BuildDomainMatcher("example.com", "full:other.com")
assert(matcher:MatchAny("example.com"))
assert(matcher:MatchAny("www.example.com"))
assert(matcher:MatchAny("other.com"))
assert(not matcher:MatchAny("www.other.com"))
assert(#(matcher:Match("www.example.com")) == 1)
`); err != nil {
t.Fatal(err)
}
}
func TestLuaMatchersRejectInvalidRules(t *testing.T) {
for _, tc := range []struct {
name string
script string
}{
{"IP rule", `require("xray.geodata").BuildIPMatcher("not-an-ip")`},
{"non-string domain rule", `require("xray.geodata").BuildDomainMatcher("example.com", true)`},
} {
t.Run(tc.name, func(t *testing.T) {
L := lua.NewState()
defer L.Close()
RegisterLua(L)
if err := L.DoString(tc.script); err == nil {
t.Fatal("invalid geodata rule was accepted")
}
})
}
}
func TestLuaMatcherArgumentsAndAliases(t *testing.T) {
L := lua.NewState()
defer L.Close()
RegisterLua(L)
ip := L.NewUserData()
ip.Value = net.ParseIP("127.0.0.1")
L.SetGlobal("ip", ip)
if err := L.DoString(`
local geodata = require("xray.geodata")
local matcher = geodata.BuildIPMatcher("127.0.0.0/8")
assert(matcher.Match == matcher.match and matcher.AnyMatch == matcher.anyMatch)
assert(matcher.Matches == matcher.matches and matcher.FilterIPs == matcher.filterIPs)
assert(matcher.Reload == nil and matcher.reload == nil)
assert(matcher:match(ip))
assert(matcher:anyMatch({ip}) and matcher:matches({ip}))
assert(not matcher:AnyMatch(nil))
assert(matcher:Matches(nil) == matcher:Matches({}))
local matched, unmatched = matcher:FilterIPs({ip})
assert(#matched == 1 and matched[1]:Equal(ip))
assert(matcher:AnyMatch(matched) and matcher:Matches(matched))
local filtered, excluded = matcher:filterIPs(matched)
assert(#filtered == 1 and #excluded == 0 and filtered[1]:Equal(ip))
local emptyMatched, emptyUnmatched = matcher:FilterIPs(nil)
assert(#emptyMatched == 0 and #emptyUnmatched == 0)
matcher:SetReverse(true)
assert(not matcher:Match(ip) and not matcher:AnyMatch(matched))
matcher:ToggleReverse()
assert(matcher:Match(ip) and matcher:AnyMatch(matched))
assert(matcher.missing == nil)
local domain = geodata.BuildDomainMatcher("full:example.com")
assert(domain.Match == domain.match and domain.MatchAny == domain.matchAny)
assert(domain.Reload == nil and domain.reload == nil)
for _, value in ipairs({matcher, domain}) do
assert(not pcall(function() return -value end))
assert(not pcall(function() return value ^ {} end))
end
assert(domain:matchAny("example.com"))
assert(#domain:Match("example.com") == 1)
assert(domain:match("example.com")[1] == 0)
assert(not pcall(function() matcher:AnyMatch() end))
assert(not pcall(function() matcher:AnyMatch(matched, true) end))
assert(not pcall(function() matcher.AnyMatch(ip, matched) end))
assert(not pcall(function() matcher:Match(true) end))
assert(not pcall(function() domain:MatchAny(123) end))
assert(not pcall(function() domain:MatchAny("example.com", true) end))
assert(not pcall(function() matcher:FilterIPs(true) end))
assert(not pcall(function() matcher:FilterIPs(matched, true) end))
assert(not pcall(function() domain:Match(123) end))
assert(not pcall(function() domain:Match("example.com", true) end))
`); err != nil {
t.Fatal(err)
}
}
// BenchmarkLuaMatcherCall measures repeated calls with prebuilt matchers and inputs.
func BenchmarkLuaMatcherCall(b *testing.B) {
L := lua.NewState()
defer L.Close()
RegisterLua(L)
ip := net.ParseIP("127.0.0.1")
for name, value := range map[string]any{"ip": ip, "ips": []net.IP{ip}} {
ud := L.NewUserData()
ud.Value = value
L.SetGlobal(name, ud)
}
if err := L.DoString(`
local geodata = require("xray.geodata")
ipMatcher = geodata.BuildIPMatcher("127.0.0.0/8")
domainMatcher = geodata.BuildDomainMatcher("full:example.com")
`); err != nil {
b.Fatal(err)
}
for _, benchmark := range []struct {
name, expression string
}{
{"ip_match", "ipMatcher:Match(ip)"},
{"ip_match_lower", "ipMatcher:match(ip)"},
{"ip_any_match", "ipMatcher:AnyMatch(ips)"},
{"ip_any_match_lower", "ipMatcher:anyMatch(ips)"},
{"ip_matches", "ipMatcher:Matches(ips)"},
{"ip_matches_lower", "ipMatcher:matches(ips)"},
{"domain_match_any", `domainMatcher:MatchAny("example.com")`},
{"domain_match_any_lower", `domainMatcher:matchAny("example.com")`},
{"ip_filter", "select(1, ipMatcher:FilterIPs(ips)) ~= nil"},
{"ip_filter_lower", "select(1, ipMatcher:filterIPs(ips)) ~= nil"},
{"domain_match", `#domainMatcher:Match("example.com") == 1`},
{"domain_match_lower", `#domainMatcher:match("example.com") == 1`},
{"ip_lua_table", "ipMatcher:AnyMatch({ip})"},
{"ip_lua_table_lower", "ipMatcher:anyMatch({ip})"},
} {
b.Run(benchmark.name, func(b *testing.B) {
if err := L.DoString(fmt.Sprintf("function benchmarkMatch() return %s end", benchmark.expression)); err != nil {
b.Fatal(err)
}
fn := L.GetGlobal("benchmarkMatch")
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
if err := L.CallByParam(lua.P{Fn: fn, NRet: 1, Protect: true}); err != nil {
b.Fatal(err)
}
if L.Get(-1) != lua.LTrue {
b.Fatal("matcher returned false")
}
L.Pop(1)
}
})
}
}
+3 -3
View File
@@ -11,7 +11,7 @@ func TestParseIPRules(t *testing.T) {
t.Setenv("xray.location.asset", filepath.Join("..", "..", "resources"))
rules := []string{
"geoip:us",
"geoip:ru",
"geoip:cn",
"!geoip:cn",
"!!geoip:cn",
@@ -20,9 +20,9 @@ func TestParseIPRules(t *testing.T) {
"!geoip:!cn",
"ext:geoip.dat:!cn",
"ext:geoip.dat:!!cn",
"ext:geoip.dat:ca",
"ext:geoip.dat:ir",
"ext-ip:geoip.dat:!cn",
"ext-ip:geoip.dat:!ca",
"ext-ip:geoip.dat:!ir",
"192.168.0.0/24",
"!192.168.0.0/24",
"!!192.168.0.0/24",
@@ -1,6 +1,7 @@
package strmatcher_test
import (
"regexp"
"strconv"
"testing"
@@ -72,6 +73,64 @@ func BenchmarkSubstrMatcher(b *testing.B) {
})
}
func BenchmarkRegexMatcher(b *testing.B) {
patterns := []string{ // taken from geosite
`(^|\.)91porn\.(best|com|cool|fun|group|party|plus|site|tw|work)$`,
`(^|\.)91porn[0-9]{3}\.me$`,
`(^|\.)apiproxy-device-prod-nlb-.+\.amazonaws\.com$`,
`(^|\.)dualstack\.apiproxy-.+\.amazonaws\.com$`,
`(^|\.)aqdk[0-9]{3}\.com$`,
`(^|\.)bilibili3(0[1-9]|1[0-2])\.xyz$`,
`(^|\.)byyum([3589]|2[235689]|3[34]|4[1-9]|5[1-79]|6[0134679])?\.com$`,
`(^|\.)fiftymvapi\..+$`,
`(^|\.)gossipfuli[0-9]{3,4}\.xyz$`,
`(^|\.)kpkuang\.(bond|fun|info|one|us)$`,
`(^|\.)rule34\.(asia|us|world|xxx|xyz)$`,
`(^|\.)[a-z][1-9][0-9][a-z]\.com$`,
`.+\.awsdns-[0-9][0-9]\.(co\.uk|com|net|org)$`,
`.+\.dkr\.ecr\.[^\.]+\.amazonaws\.com$`,
`^(.+\.)*zh\.okaapps\.com$`,
`^cdn\d-epicgames-\d+\.file\.myqcloud\.com$`,
`^chatgpt-async-webps-prod-\S+-\d+\.webpubsub\.azure\.com$`,
`^r+[0-9]+(---|\.)sn-(2x3|ni5|j5o)\w{5}\.googlevideo\.com$`,
`^speed\.(coe|open)\.ad\.[a-z]{2,6}\.prod\.hosts\.ooklaserver\.net$`,
`javdb\d+\.com$`,
}
domains := []string{
"www.google.com", "rr3---sn-4g5edndy.googlevideo.com", "r1---sn-2x3abcde.googlevideo.com", "i.ytimg.com",
"graph.facebook.com", "api.twitter.com", "www.baidu.com", "github.com", "objects.githubusercontent.com",
"login.microsoftonline.com", "e1234.dscb.akamaiedge.net", "d1a2b3c4d5e6f7.cloudfront.net",
"s3.us-east-1.amazonaws.com", "123456789012.dkr.ecr.us-east-1.amazonaws.com", "www.wikipedia.org",
"discord.com", "telegram.org", "store.steampowered.com", "www.91porn.com", "ns-1234.awsdns-12.org",
}
bench := func(b *testing.B, ctor func(pattern string) func(string) bool) {
var matchers []func(string) bool
for _, p := range patterns {
matchers = append(matchers, ctor(p))
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
for _, d := range domains {
for _, match := range matchers {
_ = match(d)
}
}
}
}
b.Run("regexp", func(b *testing.B) {
bench(b, func(pattern string) func(string) bool {
return regexp.MustCompile(pattern).MatchString
})
})
b.Run("prefilter", func(b *testing.B) {
bench(b, func(pattern string) func(string) bool {
m, err := Regex.New(pattern)
common.Must(err)
return m.Match
})
})
}
// Utility functions for benchmark
func benchmarkMatcherType(b *testing.B, t Type, ctor func() MatcherGroup) {
+8 -12
View File
@@ -52,7 +52,9 @@ func (g *MphIndexMatcher) Add(matcher Matcher) uint32 {
func (g *MphIndexMatcher) Build() error {
if g.mph != nil {
runtime.GC() // peak mem
g.mph.Build()
if err := g.mph.Build(); err != nil {
return err
}
}
runtime.GC() // peak mem
if g.ac != nil {
@@ -64,23 +66,17 @@ func (g *MphIndexMatcher) Build() error {
// Match implements IndexMatcher.Match.
func (g *MphIndexMatcher) Match(input string) []uint32 {
result := make([][]uint32, 0, 5)
var result []uint32
if g.mph != nil {
if matches := g.mph.Match(input); len(matches) > 0 {
result = append(result, matches)
}
result = g.mph.Match(input) // a new slice, returned without another copy
}
if g.ac != nil {
if matches := g.ac.Match(input); len(matches) > 0 {
result = append(result, matches)
}
result = append(result, g.ac.Match(input)...)
}
if g.regex != nil {
if matches := g.regex.Match(input); len(matches) > 0 {
result = append(result, matches)
}
result = append(result, g.regex.Match(input)...)
}
return CompositeMatches(result)
return result
}
// MatchAny implements IndexMatcher.MatchAny.
@@ -78,6 +78,10 @@ func TestMphIndexMatcher(t *testing.T) {
Input: "example.com",
Output: []uint32{10, 4},
},
{
Input: "apis.org",
Output: []uint32{2, 6},
},
}
matcherGroup := NewMphIndexMatcher()
for _, rule := range rules {
@@ -87,8 +91,13 @@ func TestMphIndexMatcher(t *testing.T) {
}
matcherGroup.Build()
for _, test := range cases {
if m := matcherGroup.Match(test.Input); !reflect.DeepEqual(m, test.Output) {
m := matcherGroup.Match(test.Input)
if !reflect.DeepEqual(m, test.Output) {
t.Error("unexpected output: ", m, " for test case ", test)
}
clear(m) // the caller owns the result, so this must not change the next one
if m := matcherGroup.Match(test.Input); !reflect.DeepEqual(m, test.Output) {
t.Error("unexpected output after clearing the previous one: ", m, " for test case ", test)
}
}
}
+384 -142
View File
@@ -1,198 +1,440 @@
package strmatcher
import (
"math/bits"
"runtime"
"sort"
"bytes"
"cmp"
"encoding/binary"
"errors"
"math"
"slices"
"strings"
"unsafe"
)
// PrimeRK is the prime base used in Rabin-Karp algorithm.
const PrimeRK = 16777619
// RollingHash calculates the rolling murmurHash of given string based on a provided suffix hash.
func RollingHash(hash uint32, input string) uint32 {
for i := len(input) - 1; i >= 0; i-- {
hash = hash*PrimeRK + uint32(input[i])
}
return hash
}
// MemHash is the hash function used by go map, it utilizes available hardware instructions(behaves
// as aeshash if aes instruction is available).
// With different seed, each MemHash<seed> performs as distinct hash functions.
func MemHash(seed uint32, input string) uint32 {
return uint32(strhash(unsafe.Pointer(&input), uintptr(seed))) // nosemgrep
}
// Flags of a level1 slot, stored above the record offset.
const (
mphMatchTypeCount = 2 // Full and Domain
mphDomain = 1 << 31 // matches the pattern and its subdomains
mphFull = 1 << 30 // matches the pattern only
mphParent = 1 << 29 // matches subdomains only, from a pattern with a leading dot
mphOffMask = mphParent - 1
)
type mphRuleInfo struct {
rollingHash uint32
matchers [mphMatchTypeCount][]uint32
// Kinds of an added pattern, indexes of mphKinds.
const (
mphKindFull = iota
mphKindParent
mphKindDomain
)
// mphKinds are the slot flags in the order Match reports their values.
var mphKinds = [...]uint32{mphFull, mphParent, mphDomain}
// mphMultipliers are odd multipliers for the suffix hash. Build moves to the next one if two patterns collide.
var mphMultipliers = [...]uint64{0x9e3779b97f4a7c15, 0xc2b2ae3d27d4eb4f, 0x165667b19e3779f9, 0x27d4eb2f165667c5}
var (
errMphCollision = errors.New("strmatcher: suffix hash collision in MphMatcherGroup")
errMphBuilt = errors.New("strmatcher: MphMatcherGroup is already built")
)
type mphEntry struct {
off uint32 // pattern start in buf
value uint32
n uint32 // pattern length
kind uint8
}
// MphMatcherGroup is an implementation of MatcherGroup.
// It implements Rabin-Karp algorithm and minimal perfect hash table for Full and Domain matcher.
// MphMatcherGroup is an implementation of MatcherGroup for Full and Domain matchers.
// Each distinct pattern is stored once as a record in arena: its length (255 means a uvarint length follows),
// its bytes and, if the group holds more than one distinct value, its values. A minimal perfect hash table
// built with hash, displace and compress (http://cmph.sourceforge.net/papers/esa09.pdf) maps a pattern to its
// record. Patterns are hashed from the right, so one pass over the input hashes all its parent domains.
type MphMatcherGroup struct {
rules []string // RuleIdx -> pattern string, index 0 reserved for failed lookup
values [][]uint32 // RuleIdx -> registered matcher values for the pattern (Full Matcher takes precedence)
level0 []uint32 // RollingHash & Mask -> seed for Memhash
level0Mask uint32 // Mask restricting RollingHash to 0 ~ len(level0)
level1 []uint32 // Memhash<seed> & Mask -> stored index for rules
level1Mask uint32 // Mask for restricting Memhash<seed> to 0 ~ len(level1)
ruleInfos *map[string]mphRuleInfo
arena string
level0 []uint16 // bucket -> seed
level1 []uint32 // slot -> flags | record offset
fp []uint8 // slot -> low byte of its pattern's hash, rejects most misses without reading arena
n0, n1 uint32
mul uint64 // multiplier of the suffix hash
single uint32 // the only value if !multi
multi bool
buf []byte // build only, patterns in Add order
entries []mphEntry
}
func NewMphMatcherGroup() *MphMatcherGroup {
return &MphMatcherGroup{
rules: []string{""},
values: [][]uint32{nil},
level0: nil,
level0Mask: 0,
level1: nil,
level1Mask: 0,
ruleInfos: &map[string]mphRuleInfo{}, // Only used for building, destroyed after build complete
}
return new(MphMatcherGroup)
}
// AddFullMatcher implements MatcherGroupForFull.
func (g *MphMatcherGroup) AddFullMatcher(matcher FullMatcher, value uint32) {
pattern := strings.ToLower(matcher.Pattern())
g.addPattern(0, "", pattern, matcher.Type(), value)
g.add(matcher.Pattern(), mphKindFull, value)
}
// AddDomainMatcher implements MatcherGroupForDomain.
func (g *MphMatcherGroup) AddDomainMatcher(matcher DomainMatcher, value uint32) {
pattern := strings.ToLower(matcher.Pattern())
hash := g.addPattern(0, "", pattern, matcher.Type(), value) // For full domain match
g.addPattern(hash, pattern, ".", matcher.Type(), value) // For partial domain match
g.add(matcher.Pattern(), mphKindDomain, value)
}
func (g *MphMatcherGroup) addPattern(suffixHash uint32, suffixPattern string, pattern string, matcherType Type, value uint32) uint32 {
fullPattern := pattern + suffixPattern
info, found := (*g.ruleInfos)[fullPattern]
if !found {
info = mphRuleInfo{rollingHash: RollingHash(suffixHash, pattern)}
g.rules = append(g.rules, fullPattern)
g.values = append(g.values, nil)
func (g *MphMatcherGroup) add(pattern string, kind uint8, value uint32) {
if g.arena != "" {
panic(errMphBuilt)
}
pattern = strings.ToLower(pattern)
off := uint32(len(g.buf))
g.buf = append(g.buf, pattern...)
g.entries = append(g.entries, mphEntry{off: off, value: value, n: uint32(len(pattern)), kind: kind})
if len(pattern) > 0 && pattern[0] == '.' {
// ".x" has always matched "*.x" as well, so it also gets a parent-only record for "x"
g.entries = append(g.entries, mphEntry{off: off + 1, value: value, n: uint32(len(pattern) - 1), kind: mphKindParent})
}
info.matchers[matcherType] = append(info.matchers[matcherType], value)
(*g.ruleInfos)[fullPattern] = info
return info.rollingHash
}
// Build builds a minimal perfect hash table for insert rules.
// Algorithm used: Hash, displace, and compress. See http://cmph.sourceforge.net/papers/esa09.pdf
func (g *MphMatcherGroup) key(i uint32) []byte {
e := &g.entries[i]
return g.buf[e.off : e.off+e.n]
}
// Build builds the hash table. It must be called once, after the last Add.
func (g *MphMatcherGroup) Build() error {
ruleCount := len(*g.ruleInfos)
g.level0 = make([]uint32, nextPow2(ruleCount/4))
g.level0Mask = uint32(len(g.level0) - 1)
g.level1 = make([]uint32, nextPow2(ruleCount))
g.level1Mask = uint32(len(g.level1) - 1)
// Create buckets based on all rule's rolling hash
buckets := make([][]uint32, len(g.level0))
for ruleIdx := 1; ruleIdx < len(g.rules); ruleIdx++ { // Traverse rules starting from index 1 (0 reserved for failed lookup)
ruleInfo := (*g.ruleInfos)[g.rules[ruleIdx]]
bucketIdx := ruleInfo.rollingHash & g.level0Mask
buckets[bucketIdx] = append(buckets[bucketIdx], uint32(ruleIdx))
g.values[ruleIdx] = append(ruleInfo.matchers[Full], ruleInfo.matchers[Domain]...) // nolint:gocritic
if g.arena != "" {
return errMphBuilt
}
g.ruleInfos = nil // Set ruleInfos nil to release memory
runtime.GC() // peak mem
// Sort buckets in descending order with respect to each bucket's size
bucketIdxs := make([]int, len(buckets))
for bucketIdx := range buckets {
bucketIdxs[bucketIdx] = bucketIdx
if uint64(len(g.buf)) > math.MaxUint32 {
return errors.New("too many rules for MphMatcherGroup")
}
sort.Slice(bucketIdxs, func(i, j int) bool { return len(buckets[bucketIdxs[i]]) > len(buckets[bucketIdxs[j]]) })
recs := g.writeRecords()
if len(g.arena) > mphOffMask {
return errors.New("too many rules for MphMatcherGroup")
}
hashes := make([]uint64, len(recs))
for _, mul := range mphMultipliers {
for i, rec := range recs {
hashes[i] = mphMix(mphHash(mul, g.recKey(rec)))
}
g.mul = mul
if err := g.place(recs, hashes); err != errMphCollision {
return err
}
}
return errMphCollision
}
// Exercise Hash, Displace, and Compress algorithm to construct minimal perfect hash table
occupied := make([]bool, len(g.level1)) // Whether a second-level hash has been already used
hashedBucket := make([]uint32, 0, 4) // Second-level hashes for each rule in a specific bucket
for _, bucketIdx := range bucketIdxs {
bucket := buckets[bucketIdx]
hashedBucket = hashedBucket[:0]
seed := uint32(0)
for len(hashedBucket) != len(bucket) {
for _, ruleIdx := range bucket {
memHash := MemHash(seed, g.rules[ruleIdx]) & g.level1Mask
if occupied[memHash] { // Collision occurred with this seed
for _, hash := range hashedBucket { // Revert all values in this hashed bucket
occupied[hash] = false
g.level1[hash] = 0
}
hashedBucket = hashedBucket[:0]
seed++ // Try next seed
break
}
occupied[memHash] = true
g.level1[memHash] = ruleIdx // The final value in the hash table
hashedBucket = append(hashedBucket, memHash)
// writeRecords writes one record per distinct pattern to arena and returns flags | offset of each.
func (g *MphMatcherGroup) writeRecords() []uint32 {
g.multi = false
if len(g.entries) > 0 {
g.single = g.entries[0].value
for _, e := range g.entries {
if e.value != g.single {
g.multi = true
break
}
}
g.level0[bucketIdx] = seed // Displacement value for this bucket
}
// Equal patterns become neighbours in Add order, so their values keep their priority
order := make([]uint32, len(g.entries))
for i := range order {
order[i] = uint32(i)
}
slices.SortFunc(order, func(a, b uint32) int {
return cmp.Or(bytes.Compare(g.key(a), g.key(b)), cmp.Compare(a, b))
})
size := len(g.buf) + len(g.entries) + 2
if g.multi {
size += 3 * len(g.entries)
}
arena := make([]byte, 0, size)
recs := make([]uint32, 0, len(order))
var vals [len(mphKinds)][]uint32
for i := 0; i < len(order); {
k := g.key(order[i])
for t := range vals {
vals[t] = vals[t][:0]
}
for ; i < len(order) && bytes.Equal(g.key(order[i]), k); i++ {
e := &g.entries[order[i]]
if !slices.Contains(vals[e.kind], e.value) {
vals[e.kind] = append(vals[e.kind], e.value)
}
}
rec := uint32(len(arena))
if len(k) < 255 {
arena = append(arena, byte(len(k)))
} else {
arena = binary.AppendUvarint(append(arena, 255), uint64(len(k)))
}
arena = append(arena, k...)
for t, v := range vals {
if len(v) == 0 {
continue
}
rec |= mphKinds[t]
if g.multi {
arena = binary.AppendUvarint(arena, uint64(len(v)))
for _, x := range v {
arena = binary.AppendUvarint(arena, uint64(x))
}
}
}
recs = append(recs, rec)
}
// Lookups may point one byte past a pattern, and an empty group needs a record at offset 0 for empty slots
arena = append(arena, 0)
if len(recs) == 0 {
arena = append(arena, 0)
}
g.buf, g.entries = nil, nil
if cap(arena)-len(arena) > len(arena)/32 {
arena = slices.Clone(arena)
}
g.arena = unsafe.String(unsafe.SliceData(arena), len(arena)) // arena is not written after this
return recs
}
// place fills level0, level1 and fp: records are bucketed by hash, and each bucket, largest first, gets
// the first seed that puts all its records in free slots.
func (g *MphMatcherGroup) place(recs []uint32, hashes []uint64) error {
r := len(recs)
n0, n1 := max(1, r/3), max(1, r+r/99)
g.n0, g.n1 = uint32(n0), uint32(n1)
g.level0 = make([]uint16, n0)
g.level1 = make([]uint32, n1)
g.fp = make([]uint8, n1)
start := make([]uint32, n0+1)
for _, h := range hashes {
start[g.bucket(h)+1]++
}
for b := range n0 {
start[b+1] += start[b]
}
members := make([]uint32, r)
fill := slices.Clone(start[:n0])
for i, h := range hashes {
b := g.bucket(h)
members[fill[b]] = uint32(i)
fill[b]++
}
fill = nil
buckets := make([]uint32, n0)
for b := range buckets {
buckets[b] = uint32(b)
}
slices.SortStableFunc(buckets, func(a, b uint32) int {
return cmp.Compare(start[b+1]-start[b], start[a+1]-start[a])
})
occupied := make([]uint64, (n1+63)/64)
var slots []uint32
next:
for _, b := range buckets {
m := members[start[b]:start[b+1]]
if len(m) == 0 {
break
}
for i := range m {
for j := range i {
if hashes[m[i]] == hashes[m[j]] {
return errMphCollision // no seed can separate them
}
}
}
search:
for seed := range math.MaxUint16 + 1 {
slots = slots[:0]
for _, ri := range m {
s := g.slot(hashes[ri], uint16(seed))
if occupied[s/64]&(1<<(s%64)) != 0 || slices.Contains(slots, s) {
continue search
}
slots = append(slots, s)
}
for k, ri := range m {
s := slots[k]
occupied[s/64] |= 1 << (s % 64)
g.level1[s] = recs[ri]
g.fp[s] = uint8(hashes[ri])
}
g.level0[b] = uint16(seed)
continue next
}
return errors.New("strmatcher: no seed found for a bucket in MphMatcherGroup")
}
return nil
}
// Lookup searches for input in minimal perfect hash table and returns its index. 0 indicates not found.
func (g *MphMatcherGroup) Lookup(rollingHash uint32, input string) uint32 {
i0 := rollingHash & g.level0Mask
seed := g.level0[i0]
i1 := MemHash(seed, input) & g.level1Mask
if n := g.level1[i1]; g.rules[n] == input {
return n
// mphHash is the suffix hash of s, taken from the right: the hash of s[i:] is the state after reading s[i].
func mphHash(mul uint64, s string) uint64 {
h := uint64(0)
for i := len(s) - 1; i >= 0; i-- {
h = h*mul + uint64(s[i])
}
return h
}
// mphMix spreads the weak low bits of a suffix hash.
func mphMix(h uint64) uint64 {
h ^= h >> 32
h *= 0xd6e8feb86659fd93
return h ^ h>>32
}
func (g *MphMatcherGroup) bucket(f uint64) uint32 {
return uint32(((f >> 32) * uint64(g.n0)) >> 32)
}
func (g *MphMatcherGroup) slot(f uint64, seed uint16) uint32 {
x := ((f ^ uint64(seed)*0x9e3779b97f4a7c15) * 0xc4ceb9fe1a85ec53) >> 32
return uint32((x * uint64(g.n1)) >> 32)
}
func (g *MphMatcherGroup) uvarint(p uint32) (x, next uint32) {
for shift := 0; ; shift += 7 {
c := g.arena[p]
p++
x |= uint32(c&0x7f) << shift
if c < 0x80 {
return x, p
}
}
}
// recSpan returns where the pattern of the record at off starts and how long it is.
func (g *MphMatcherGroup) recSpan(off uint32) (p, n uint32) {
n, p = uint32(g.arena[off]), off+1
if n == 255 {
n, p = g.uvarint(p)
}
return p, n
}
func (g *MphMatcherGroup) recKey(rec uint32) string {
p, n := g.recSpan(rec & mphOffMask)
return g.arena[p : p+n]
}
// lookup returns the level1 entry of s, or 0 if s is not a pattern. h is the suffix hash of s.
func (g *MphMatcherGroup) lookup(h uint64, s string) uint32 {
f := mphMix(h)
// bucket < n0 == len(level0) and slot < n1 == len(level1) == len(fp), skip the bounds checks
seed := *(*uint16)(unsafe.Add(unsafe.Pointer(unsafe.SliceData(g.level0)), uintptr(g.bucket(f))*2))
slot := uintptr(g.slot(f, seed))
if *(*uint8)(unsafe.Add(unsafe.Pointer(unsafe.SliceData(g.fp)), slot)) != uint8(f) {
return 0
}
e := *(*uint32)(unsafe.Add(unsafe.Pointer(unsafe.SliceData(g.level1)), slot*4))
if len(s) < 255 {
// A record whose length byte is len(s) has len(s) pattern bytes after it
p := unsafe.Add(unsafe.Pointer(unsafe.StringData(g.arena)), e&mphOffMask)
if int(*(*byte)(p)) == len(s) && unsafe.String((*byte)(unsafe.Add(p, 1)), len(s)) == s {
return e
}
return 0
}
if g.recKey(e) == s {
return e
}
return 0
}
// Match implements MatcherGroup.Match.
func (g *MphMatcherGroup) Match(input string) []uint32 {
matches := make([][]uint32, 0, 5)
hash := uint32(0)
for i := len(input) - 1; i >= 0; i-- {
hash = hash*PrimeRK + uint32(input[i])
if input[i] == '.' {
if mphIdx := g.Lookup(hash, input[i:]); mphIdx != 0 {
matches = append(matches, g.values[mphIdx])
// appendValues appends the values of record e for the flags in want, in mphKinds order.
func (g *MphMatcherGroup) appendValues(dst []uint32, e, want uint32) []uint32 {
if !g.multi {
for _, flag := range mphKinds {
if e&want&flag != 0 {
dst = append(dst, g.single)
}
}
return dst
}
if e&want == 0 {
return dst
}
p, n := g.recSpan(e & mphOffMask)
p += n
for _, flag := range mphKinds {
if e&flag == 0 {
continue
}
var count, v uint32
for count, p = g.uvarint(p); count > 0; count-- {
v, p = g.uvarint(p)
if want&flag != 0 {
dst = append(dst, v)
}
}
}
if mphIdx := g.Lookup(hash, input); mphIdx != 0 {
matches = append(matches, g.values[mphIdx])
return dst
}
// Match implements MatcherGroup.Match. Values of an exact match come first (Full, then Domain), then those of
// the parent domains, nearest first.
func (g *MphMatcherGroup) Match(input string) []uint32 {
var stack [8]uint32
parents := stack[:0] // TLD side first
h, mul := uint64(0), g.mul
for i := len(input) - 1; i >= 0; i-- {
if input[i] == '.' {
if e := g.lookup(h, input[i+1:]); e&(mphDomain|mphParent) != 0 {
parents = append(parents, e)
}
}
h = h*mul + uint64(input[i])
}
return CompositeMatchesReverse(matches)
exact := g.lookup(h, input)
if exact&(mphFull|mphDomain) == 0 && len(parents) == 0 {
return nil
}
result := g.appendValues(make([]uint32, 0, len(parents)+1), exact, mphFull|mphDomain)
for k := len(parents) - 1; k >= 0; k-- {
result = g.appendValues(result, parents[k], mphParent|mphDomain)
}
return result
}
// MatchAny implements MatcherGroup.MatchAny.
func (g *MphMatcherGroup) MatchAny(input string) bool {
hash := uint32(0)
h, mul := uint64(0), g.mul
for i := len(input) - 1; i >= 0; i-- {
if input[i] == '.' && g.lookup(h, input[i+1:])&(mphDomain|mphParent) != 0 {
return true
}
h = h*mul + uint64(input[i])
}
return g.lookup(h, input)&(mphFull|mphDomain) != 0
}
// mphSuffix is the suffix hash of input[off:], a parent domain of the input.
type mphSuffix struct {
h uint64
off int
}
// mphSuffixes appends the suffix hashes of the parent domains of input to dst, TLD side first, and returns them
// with the hash of input itself: what MatchAny computes, computed once for several groups.
func mphSuffixes(dst []mphSuffix, mul uint64, input string) ([]mphSuffix, uint64) {
h := uint64(0)
for i := len(input) - 1; i >= 0; i-- {
hash = hash*PrimeRK + uint32(input[i])
if input[i] == '.' {
if g.Lookup(hash, input[i:]) != 0 {
return true
}
dst = append(dst, mphSuffix{h, i + 1})
}
h = h*mul + uint64(input[i])
}
return dst, h
}
// matchAnyHashed is MatchAny with parents and h from mphSuffixes(_, mul, input).
func (g *MphMatcherGroup) matchAnyHashed(input string, parents []mphSuffix, h, mul uint64) bool {
if g.mul != mul {
return g.MatchAny(input) // built with a later multiplier after a collision
}
for _, p := range parents {
if g.lookup(p.h, input[p.off:])&(mphDomain|mphParent) != 0 {
return true
}
}
return g.Lookup(hash, input) != 0
return g.lookup(h, input)&(mphFull|mphDomain) != 0
}
func nextPow2(v int) int {
if v <= 1 {
return 1
}
const MaxUInt = ^uint(0)
n := (MaxUInt >> bits.LeadingZeros(uint(v))) + 1
return int(n)
}
//go:noescape
//go:linkname strhash runtime.strhash
func strhash(p unsafe.Pointer, h uintptr) uintptr
@@ -0,0 +1,108 @@
package strmatcher
import (
"slices"
"testing"
)
func TestMphMatcherGroupHashCollision(t *testing.T) {
saved := mphMultipliers
defer func() { mphMultipliers = saved }()
mphMultipliers[0] = 1 // anagrams collide
g := NewMphMatcherGroup()
g.AddFullMatcher(FullMatcher("ab.com"), 1)
g.AddDomainMatcher(DomainMatcher("ba.com"), 2)
g.AddDomainMatcher(DomainMatcher("com"), 3)
if err := g.Build(); err != nil {
t.Fatal(err)
}
if g.mul != saved[1] {
t.Errorf("multiplier %#x, want the second one %#x", g.mul, saved[1])
}
for input, want := range map[string][]uint32{"ab.com": {1, 3}, "x.ba.com": {2, 3}, "x.ab.com": {3}, "ba.com": {2, 3}} {
if m := g.Match(input); !slices.Equal(m, want) {
t.Errorf("Match(%q) = %v, want %v", input, m, want)
}
}
// Thue-Morse strings of 2048 bytes and their complements collide for every odd multiplier
mphMultipliers = saved
a, b := make([]byte, 2048), make([]byte, 2048)
for i := range a {
a[i], b[i] = "ab"[bitsOnes(i)%2], "ba"[bitsOnes(i)%2]
}
g = NewMphMatcherGroup()
g.AddFullMatcher(FullMatcher(a), 1)
g.AddFullMatcher(FullMatcher(b), 1)
if err := g.Build(); err != errMphCollision {
t.Errorf("Build() = %v, want %v", err, errMphCollision)
}
}
func bitsOnes(i int) int {
n := 0
for ; i > 0; i &= i - 1 {
n++
}
return n
}
func TestMphValueMatcherCombiner(t *testing.T) {
build := func(matchers ...Matcher) *MphValueMatcher {
m := NewMphValueMatcher()
for _, x := range matchers {
m.Add(x, 0)
}
if err := m.Build(); err != nil {
t.Fatal(err)
}
return m
}
regex, err := Regex.New(`^a\d+\.net$`)
if err != nil {
t.Fatal(err)
}
saved := mphMultipliers
t.Cleanup(func() { mphMultipliers = saved })
mphMultipliers[0] = 1 // anagrams collide, so this one falls back to its own hash pass
collided := build(FullMatcher("ab.com"), DomainMatcher("ba.com"))
mphMultipliers = saved
if collided.mph.mul == mphMultipliers[0] {
t.Fatal("collided matcher uses the first multiplier")
}
matchers := []*MphValueMatcher{
build(DomainMatcher("example.com"), FullMatcher("full.org"), DomainMatcher(".dot.io")),
collided,
build(regex, SubstrMatcher("keyword")),
build(),
build(DomainMatcher("com"), DomainMatcher("a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s")),
}
var s MphValueMatcherCombiner
for i, m := range matchers {
s.Add(m, uint32(10+i))
}
inputs := []string{
"", ".", "..", "com", "example.com", "www.example.com", "xexample.com", "example.com.", "full.org", "x.full.org",
"dot.io", "x.dot.io", ".dot.io", "ab.com", "x.ab.com", "ba.com", "x.ba.com", "a12.net", "a12.net.x", "my-keyword.org",
"a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s", "0.a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s", "b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s",
"x.y.z.1.2.3.4.5.6.7.8.9.10.11.12.13.14.15.16.17.ab.com", "x.y.z.1.2.3.4.5.6.7.8.9.10.11.12.13.14.15.16.17.org",
}
for _, input := range inputs {
var want []uint32
for i, m := range matchers {
if m.MatchAny(input) {
want = append(want, uint32(10+i))
}
}
if got := s.Match(input); !slices.Equal(got, want) {
t.Errorf("Match(%q) = %v, want %v", input, got, want)
}
if got := s.MatchAny(input); got != (len(want) > 0) {
t.Errorf("MatchAny(%q) = %v", input, got)
}
}
if n := testing.AllocsPerRun(100, func() { s.MatchAny("www.a.b.c.example.org") }); n != 0 {
t.Errorf("MatchAny allocates %v times", n)
}
}
@@ -1,7 +1,10 @@
package strmatcher_test
import (
"math/rand"
"reflect"
"slices"
"strings"
"testing"
"github.com/xtls/xray-core/common"
@@ -276,3 +279,142 @@ func TestEmptyMphMatcherGroup(t *testing.T) {
t.Error("Expect [], but ", r)
}
}
func TestMphMatcherGroupRandom(t *testing.T) {
inputs := []string{""} // All strings over "ab." up to 7 bytes
for i := 0; len(inputs[i]) < 7; i++ {
for _, c := range []string{"a", "b", "."} {
inputs = append(inputs, inputs[i]+c)
}
}
for seed := int64(0); seed < 300; seed++ {
r := rand.New(rand.NewSource(seed))
g := NewMphMatcherGroup()
full, domain := map[string][]uint32{}, map[string][]uint32{} // Stored pattern -> values
for value := uint32(r.Intn(200)); value > 0; value-- {
pattern := make([]byte, r.Intn(8))
for i := range pattern {
pattern[i] = "ab."[r.Intn(3)]
}
if p := string(pattern); r.Intn(2) == 0 {
g.AddFullMatcher(FullMatcher(p), value)
full[p] = append(full[p], value)
} else {
g.AddDomainMatcher(DomainMatcher(p), value)
domain[p] = append(domain[p], value)
domain["."+p] = append(domain["."+p], value)
}
}
common.Must(g.Build())
for _, input := range inputs {
keys := []string{input} // Whole input first, then "." suffixes from longest to shortest
for i := range len(input) {
if input[i] == '.' {
keys = append(keys, input[i:])
}
}
var want []uint32
for _, k := range keys {
want = append(append(want, full[k]...), domain[k]...)
}
// Compared as sets: Match reports a value once per matching pattern, and orders them differently
// from want for patterns and inputs with a leading dot
m := g.Match(input)
if !slices.Equal(sortedSet(m), sortedSet(want)) {
t.Fatalf("seed %d: Match(%q) = %v, want %v", seed, input, m, want)
}
if m := g.MatchAny(input); m != (len(want) > 0) {
t.Fatalf("seed %d: MatchAny(%q) = %v", seed, input, m)
}
}
}
}
func TestMphMatcherGroupAppend(t *testing.T) {
g := NewMphMatcherGroup()
g.AddFullMatcher(FullMatcher("a.com"), 1)
g.AddFullMatcher(FullMatcher("b.com"), 2)
g.Build()
if m := append(g.Match("a.com"), 3); !slices.Equal(m, []uint32{1, 3}) {
t.Error("expect [1 3], but ", m)
}
if m := g.Match("b.com"); !slices.Equal(m, []uint32{2}) {
t.Error("expect [2], but ", m)
}
}
func sortedSet(v []uint32) []uint32 {
v = slices.Clone(v)
slices.Sort(v)
return slices.Compact(v)
}
func TestMphMatcherGroupLongPattern(t *testing.T) {
long := strings.Repeat("a", 300) + ".com"
for _, values := range [][4]uint32{{1, 2, 3, 4}, {7, 7, 7, 7}} {
g := NewMphMatcherGroup()
g.AddDomainMatcher(DomainMatcher(long), values[0])
g.AddFullMatcher(FullMatcher("x."+long), values[1])
g.AddFullMatcher(FullMatcher(long[:255]), values[2]) // the shortest pattern stored with a long length
g.AddFullMatcher(FullMatcher(long[:254]), values[3])
common.Must(g.Build())
cases := []struct {
input string
want []uint32
}{
{long, []uint32{values[0]}},
{"www." + long, []uint32{values[0]}},
{"x." + long, []uint32{values[1], values[0]}},
{long[1:], nil},
{"a" + long, nil},
{long[:255], []uint32{values[2]}},
{long[:254], []uint32{values[3]}},
{long[:256], nil},
{long[:253], nil},
}
for _, c := range cases {
if m := g.Match(c.input); !slices.Equal(m, c.want) {
t.Errorf("Match(%d bytes) = %v, want %v", len(c.input), m, c.want)
}
if m := g.MatchAny(c.input); m != (c.want != nil) {
t.Errorf("MatchAny(%d bytes) = %v", len(c.input), m)
}
}
}
// A pattern longer than 65535 bytes builds and matches: a record's length is a uvarint,
// so the only cap was the build-time length field, now widened to uint32.
huge := strings.Repeat("a", 70000)
g := NewMphMatcherGroup()
g.AddFullMatcher(FullMatcher(strings.Repeat("a", 65535)), 1)
g.AddDomainMatcher(DomainMatcher(huge+".com"), 2)
g.AddFullMatcher(FullMatcher("a.com"), 3)
common.Must(g.Build())
if !g.MatchAny(strings.Repeat("a", 65535)) || g.MatchAny(strings.Repeat("a", 65534)) {
t.Error("wrong answer for a 65535-byte pattern")
}
if m := g.Match(huge + ".com"); !slices.Equal(m, []uint32{2}) {
t.Errorf("Match(%d-byte input) = %v, want [2]", len(huge)+4, m)
}
if m := g.Match("x." + huge + ".com"); !slices.Equal(m, []uint32{2}) {
t.Errorf("Match(subdomain of a %d-byte pattern) = %v, want [2]", len(huge)+4, m)
}
if g.MatchAny(huge) { // the 70000-byte label on its own is not a rule
t.Error("unexpected match for the bare 70000-byte label")
}
}
func TestMphMatcherGroupBuildOnce(t *testing.T) {
g := NewMphMatcherGroup()
g.AddFullMatcher(FullMatcher("a.com"), 1)
common.Must(g.Build())
if err := g.Build(); err == nil || !g.MatchAny("a.com") {
t.Errorf("second Build() = %v, MatchAny(a.com) = %v", err, g.MatchAny("a.com"))
}
defer func() {
if recover() == nil {
t.Error("Add after Build did not panic")
}
}()
g.AddDomainMatcher(DomainMatcher("b.com"), 2)
}
+281 -11
View File
@@ -2,9 +2,12 @@ package strmatcher
import (
"errors"
"math/bits"
"regexp"
"regexp/syntax"
"slices"
"strings"
"unicode"
"unicode/utf8"
"golang.org/x/net/idna"
@@ -73,7 +76,274 @@ func (m SubstrMatcher) Match(s string) bool {
// RegexMatcher is an implementation of Matcher.
type RegexMatcher struct {
pattern *regexp.Regexp
pattern *regexp.Regexp
literals []string // every match contains all of them, longest first
tail []byteSet // tail[i] holds the bytes a matching input can have i bytes before its end
rest *byteSet // the bytes it can have further before, nil if any
}
func newRegexMatcher(pattern string) (Matcher, error) {
regex, err := regexp.Compile(pattern)
if err != nil {
return nil, err
}
m := &RegexMatcher{pattern: regex}
if re, err := syntax.Parse(pattern, syntax.Perl); err == nil { // same flags as regexp.Compile
m.literals = requiredLiterals(re, nil)
slices.SortStableFunc(m.literals, func(a, b string) int { return len(b) - len(a) })
m.tail, m.rest = tailGuard(re)
}
return m, nil
}
// byteSet is a set of bytes. The bytes >= 0x80 share one bit with 0x7f.
type byteSet [4]uint32
func (s *byteSet) add(c byte) { c = min(c, 0x7f); s[c>>5] |= 1 << (c & 31) }
func (s *byteSet) has(c byte) bool { c = min(c, 0x7f); return s[c>>5]&(1<<(c&31)) != 0 }
func (s *byteSet) or(t *byteSet) {
for i := range s {
s[i] |= t[i]
}
}
var allBytes = byteSet{^uint32(0), ^uint32(0), ^uint32(0), ^uint32(0)}
// tailLen is how many positions before the end of the input tailGuard tells apart.
const tailLen = 8
// tailBudget caps how many repetition steps tailGuard walks. Only nested repeats can make the
// walk explode, so only they are charged: a flat pattern, however long, is walked once and keeps
// its guard.
const tailBudget = 100000
// tailWalk is a set of positions in the input, counted in bytes before its end.
type tailWalk struct {
at uint32 // bit i: exactly i bytes before the end, for i < tailLen
far bool // tailLen or more bytes before the end
free bool // not tied to the end of the input yet
}
func (w tailWalk) union(v tailWalk) tailWalk {
return tailWalk{w.at | v.at, w.far || v.far, w.free || v.free}
}
type tailBuilder struct {
tail [tailLen]byteSet
rest byteSet
void bool
work int
}
// tailGuard walks re backwards from the end of the input and collects the bytes an input
// matching re can have at each position before its end. It returns nil, nil when a branch
// of re does not end with $ or when nested repeats push the walk past tailBudget.
func tailGuard(re *syntax.Regexp) ([]byteSet, *byteSet) {
var b tailBuilder
w := b.walk(re, tailWalk{free: true})
b.stop(w)
if b.void {
return nil, nil
}
if w.at != 0 { // a match can start here, so any bytes can come before
for i := bits.TrailingZeros32(w.at); i < tailLen; i++ {
b.tail[i] = allBytes
}
}
if w.at != 0 || w.far {
b.rest = allBytes
}
n := tailLen
for n > 0 && b.tail[n-1] == b.rest {
n--
}
var tail []byteSet
if n > 0 {
tail = slices.Clone(b.tail[:n])
}
if b.rest != allBytes {
rest := b.rest
return tail, &rest
}
return tail, nil
}
// stop ends the paths of w. One that never met $ lets its match be followed by anything.
func (b *tailBuilder) stop(w tailWalk) {
if w.free {
b.void = true
}
}
func (b *tailBuilder) walk(re *syntax.Regexp, w tailWalk) tailWalk {
if w == (tailWalk{}) || b.void {
return w
}
switch re.Op {
case syntax.OpNoMatch:
return tailWalk{}
case syntax.OpLiteral:
for i := len(re.Rune) - 1; i >= 0; i-- {
var set byteSet
set.add(byte(min(re.Rune[i], utf8.RuneSelf)))
if re.Flags&syntax.FoldCase != 0 {
for f := unicode.SimpleFold(re.Rune[i]); f != re.Rune[i]; f = unicode.SimpleFold(f) {
set.add(byte(min(f, utf8.RuneSelf)))
}
}
w = b.step(w, &set)
}
return w
case syntax.OpCharClass:
var set byteSet
for i := 0; i+1 < len(re.Rune); i += 2 {
for r := min(re.Rune[i], utf8.RuneSelf); r <= min(re.Rune[i+1], utf8.RuneSelf); r++ {
set.add(byte(r))
}
}
return b.step(w, &set)
case syntax.OpAnyChar, syntax.OpAnyCharNotNL: // a domain has no \n to reject
return b.step(w, &allBytes)
case syntax.OpBeginText: // nothing comes before
b.stop(w)
return tailWalk{}
case syntax.OpEndText:
out := tailWalk{at: w.at & 1}
if w.free {
out.at = 1
}
return out
case syntax.OpCapture:
return b.walk(re.Sub[0], w)
case syntax.OpConcat:
for i := len(re.Sub) - 1; i >= 0; i-- {
w = b.walk(re.Sub[i], w)
}
return w
case syntax.OpAlternate:
var out tailWalk
for _, sub := range re.Sub {
out = out.union(b.walk(sub, w))
}
return out
case syntax.OpQuest:
return b.repeat(re.Sub[0], w, 1)
case syntax.OpStar:
return b.repeat(re.Sub[0], w, -1)
case syntax.OpPlus:
return b.repeat(re.Sub[0], b.walk(re.Sub[0], w), -1)
case syntax.OpRepeat:
for i := 0; i < re.Min; i++ {
if b.charge() {
return w
}
w = b.walk(re.Sub[0], w)
}
if re.Max < 0 {
return b.repeat(re.Sub[0], w, -1)
}
return b.repeat(re.Sub[0], w, re.Max-re.Min)
}
return w // empty match, line and word boundaries: no constraint
}
// charge counts one repetition step and reports whether the walk has run out of budget. Only
// repeats re-walk their body, so charging them alone bounds the blow-up of nested repeats while
// leaving a single linear pass, of any length, free.
func (b *tailBuilder) charge() bool {
b.work++
if b.work > tailBudget {
b.void = true
}
return b.void
}
// repeat walks back over up to n more repetitions of re, any number if n < 0.
func (b *tailBuilder) repeat(re *syntax.Regexp, w tailWalk, n int) tailWalk {
for ; n != 0; n-- {
if b.charge() {
return w
}
next := w.union(b.walk(re, w))
if next == w {
break
}
w = next
}
return w
}
// step walks back over one character whose last byte is in set. A character that can be
// non-ASCII can take up to 4 bytes, all >= 0x80; regexp matches an invalid byte as U+FFFD.
func (b *tailBuilder) step(w tailWalk, set *byteSet) tailWalk {
out := tailWalk{far: w.far, free: w.free}
if w.far {
b.rest.or(set)
}
width := 1
if set.has(0x80) {
width = utf8.UTFMax
}
for i := 0; i < tailLen; i++ {
if w.at&(1<<i) == 0 {
continue
}
b.tail[i].or(set)
for n := 1; n <= width; n++ {
if j := i + n; j < tailLen {
out.at |= 1 << j
if n < width {
b.tail[j].add(0x80)
}
} else {
out.far = true
if n < width {
b.rest.add(0x80)
}
}
}
}
return out
}
// mayMatch reports whether s passes the tail guard.
func (m *RegexMatcher) mayMatch(s string) bool {
n := len(s)
if m.rest == nil {
n = min(n, len(m.tail))
}
for i := 0; i < n; i++ {
set := m.rest
if i < len(m.tail) {
set = &m.tail[i]
}
if !set.has(s[len(s)-1-i]) {
return false
}
}
return true
}
// requiredLiterals appends to dst the case-sensitive strings that every match of re contains.
func requiredLiterals(re *syntax.Regexp, dst []string) []string {
switch re.Op {
case syntax.OpLiteral:
// regexp matches U+FFFD against invalid UTF-8 bytes, strings.Contains does not
if re.Flags&syntax.FoldCase == 0 && !slices.Contains(re.Rune, utf8.RuneError) {
dst = append(dst, string(re.Rune))
}
case syntax.OpCapture, syntax.OpPlus:
dst = requiredLiterals(re.Sub[0], dst)
case syntax.OpRepeat:
if re.Min > 0 {
dst = requiredLiterals(re.Sub[0], dst)
}
case syntax.OpConcat:
for _, sub := range re.Sub {
dst = requiredLiterals(sub, dst)
}
}
return dst
}
func (*RegexMatcher) Type() Type {
@@ -89,6 +359,14 @@ func (m *RegexMatcher) String() string {
}
func (m *RegexMatcher) Match(s string) bool {
if !m.mayMatch(s) {
return false
}
for _, l := range m.literals {
if !strings.Contains(s, l) {
return false
}
}
return m.pattern.MatchString(s)
}
@@ -102,11 +380,7 @@ func (t Type) New(pattern string) (Matcher, error) {
case Domain:
return DomainMatcher(pattern), nil
case Regex: // 1. regex matching is case-sensitive
regex, err := regexp.Compile(pattern)
if err != nil {
return nil, err
}
return &RegexMatcher{pattern: regex}, nil
return newRegexMatcher(pattern)
default:
return nil, errors.New("unknown matcher type")
}
@@ -135,11 +409,7 @@ func (t Type) NewDomainPattern(pattern string) (Matcher, error) {
}
return DomainMatcher(pattern), nil
case Regex: // Regex's charset not in LDH subset
regex, err := regexp.Compile(pattern)
if err != nil {
return nil, err
}
return &RegexMatcher{pattern: regex}, nil
return newRegexMatcher(pattern)
default:
return nil, errors.New("unknown matcher type")
}
@@ -0,0 +1,233 @@
package strmatcher
import (
"hash/fnv"
"math/rand/v2"
"regexp"
"regexp/syntax"
"slices"
"strconv"
"strings"
"testing"
"unicode"
"unicode/utf8"
)
var regexLiteralCases = []struct {
pattern string
literals []string
}{
{`(^|\.)91porn\.(best|com)$`, []string{"91porn."}},
{`.+\.awsdns-cn-[0-9][0-9]\.(biz|com|net|top)$`, []string{".awsdns-cn-", "."}},
{`^r+[0-9]+(---|\.)sn-(2x3|ni5|j5o)\w{5}\.googlevideo\.com$`, []string{".googlevideo.com", "sn-", "r"}},
{`(?i)abc`, nil},
{`ab(?i:CD)ef`, []string{"ab", "ef"}},
{`(abc)?x`, []string{"x"}},
{`(abc)*x`, []string{"x"}},
{`x{0,3}yy`, []string{"yy"}},
{`(ab)+c{2}`, []string{"ab", "c"}},
{`abc|abd`, []string{"ab"}},
{`\Qa.b\E`, []string{"a.b"}},
{`a\x{FFFD}b`, nil},
{`^[^.]+$`, nil},
}
func TestRegexRequiredLiterals(t *testing.T) {
for _, test := range regexLiteralCases {
m, err := newRegexMatcher(test.pattern)
if err != nil {
t.Fatal(err)
}
if got := m.(*RegexMatcher).literals; !slices.Equal(got, test.literals) {
t.Errorf("%s: got %q, want %q", test.pattern, got, test.literals)
}
}
}
var regexTailCases = []struct {
pattern string
guard bool
match []string // inputs the pattern matches
reject []string // inputs the tail guard alone rejects
}{
{`^[a-z]([a-z0-9-]{0,61}[a-z0-9])?$`, true, []string{"a", "localhost", "x-1"}, []string{"www.example.com", "localhost.", "LOCALHOST", "a b"}},
{`(^|\.)[a-z][1-9][0-9][a-z]\.com$`, true, []string{"a12b.com", "x.q10z.com"}, []string{"google.com", "a12b.co", "a12b.com.", "ab12.com"}},
{`^hses[1-7]?\.akamaized\.net$`, true, []string{"hses.akamaized.net", "hses3.akamaized.net"}, []string{"xhses.akamaized.net", "www.hses.akamaized.net"}},
{`(?i)k\.net$`, true, []string{"k.net", "K.NET", "\u212a.net"}, []string{"x.net", "k.nex"}},
{`[^.]+\.cn$`, true, []string{"a.cn", "\xff.cn", "\u4e2d.cn"}, []string{"a.cnn", "a.c"}},
{`\x{FFFD}$`, true, []string{"\xff", "a\xc3", "\uFFFD"}, []string{"a", "\xff."}},
{`^.\.cn$`, true, []string{"a.cn", "\u4E2D.cn", "\xff.cn"}, []string{"ab.cn"}},
{`^$`, true, []string{""}, []string{"a"}},
{`(^|\.)youyuapi\..+$`, false, []string{"youyuapi.com"}, nil},
{`abc`, false, []string{"abc", "xabcx"}, nil},
{`^ab`, false, []string{"ab", "abc"}, nil},
{`a$|b`, false, []string{"a", "bx"}, nil},
{`(?m)a$`, false, []string{"a", "a\nb"}, nil},
{strings.Repeat(`(?:abcdefgh(?:a`, 20) + strings.Repeat(`)*)*`, 20) + `\.com$`, false, []string{".com", "abcdefgha.com"}, nil}, // over tailBudget
}
func TestRegexTailGuard(t *testing.T) {
for _, test := range regexTailCases {
m, err := newRegexMatcher(test.pattern)
if err != nil {
t.Fatal(err)
}
rm := m.(*RegexMatcher)
if guard := rm.tail != nil || rm.rest != nil; guard != test.guard {
t.Errorf("%s: guard %v, want %v", test.pattern, guard, test.guard)
}
for _, s := range test.match {
if !rm.pattern.MatchString(s) || !rm.Match(s) {
t.Errorf("%s: %q does not match", test.pattern, s)
}
}
for _, s := range test.reject {
if rm.pattern.MatchString(s) || rm.mayMatch(s) {
t.Errorf("%s: %q passes the guard", test.pattern, s)
}
}
}
}
// TestRegexTailGuardFlatAlternation checks that a long but non-recursive pattern keeps its
// guard. Only nested repeats are charged against tailBudget, so a flat alternation of many
// names, however large, is walked once and guarded; its guard is checked against regexp.
func TestRegexTailGuardFlatAlternation(t *testing.T) {
var sb strings.Builder
sb.WriteString("(?:")
for i := 0; i < 20000; i++ {
if i > 0 {
sb.WriteByte('|')
}
sb.WriteString("name")
sb.WriteString(strconv.Itoa(i))
}
sb.WriteString(`)\.example\.com$`)
m, err := newRegexMatcher(sb.String())
if err != nil {
t.Fatal(err)
}
rm := m.(*RegexMatcher)
if rm.tail == nil && rm.rest == nil {
t.Fatal("flat alternation of 20000 names lost its guard")
}
for _, s := range []string{"name0.example.com", "name19999.example.com", "x.name12345.example.com"} {
if !rm.pattern.MatchString(s) || !rm.Match(s) {
t.Errorf("%q should match", s)
}
}
for _, s := range []string{"name0.example.org", "name0.example.com.", "name0.example.con", "google.com"} {
if rm.pattern.MatchString(s) {
t.Fatalf("test bug: %q matches the pattern", s)
}
if rm.mayMatch(s) {
t.Errorf("%q should be rejected by the guard", s)
}
}
}
// sampleMatch appends a string that re matches, assertions aside, unless it runs out of
// budget, which it spends one per call so that nested repeats stay cheap.
func sampleMatch(sb *strings.Builder, re *syntax.Regexp, rnd *rand.Rand, budget *int) {
if *budget <= 0 {
return
}
*budget--
switch re.Op {
case syntax.OpLiteral:
for _, r := range re.Rune {
if re.Flags&syntax.FoldCase != 0 {
for n := rnd.IntN(4); n > 0; n-- {
r = unicode.SimpleFold(r)
}
}
sampleRune(sb, r, rnd)
}
case syntax.OpCharClass:
if len(re.Rune) > 0 {
i := rnd.IntN(len(re.Rune)/2) * 2
sampleRune(sb, re.Rune[i]+rnd.Int32N(min(re.Rune[i+1]-re.Rune[i]+1, 300)), rnd)
}
case syntax.OpAnyChar, syntax.OpAnyCharNotNL:
sampleRune(sb, []rune{'a', '.', '\n', 0xe9, 0x212a, utf8.RuneError}[rnd.IntN(6)], rnd)
case syntax.OpCapture:
sampleMatch(sb, re.Sub[0], rnd, budget)
case syntax.OpConcat:
for _, sub := range re.Sub {
sampleMatch(sb, sub, rnd, budget)
}
case syntax.OpAlternate:
sampleMatch(sb, re.Sub[rnd.IntN(len(re.Sub))], rnd, budget)
case syntax.OpQuest, syntax.OpStar, syntax.OpPlus, syntax.OpRepeat:
lo, hi := 0, 3
switch re.Op {
case syntax.OpQuest:
hi = 1
case syntax.OpPlus:
lo = 1
case syntax.OpRepeat:
lo, hi = re.Min, re.Min+3
if re.Max >= 0 {
hi = min(hi, re.Max)
}
}
for n := lo + rnd.IntN(hi-lo+1); n > 0; n-- {
sampleMatch(sb, re.Sub[0], rnd, budget)
}
}
}
func sampleRune(sb *strings.Builder, r rune, rnd *rand.Rand) {
if r == utf8.RuneError && rnd.IntN(2) == 0 {
sb.WriteByte(0x80 | byte(rnd.IntN(0x80))) // regexp matches an invalid byte as U+FFFD
return
}
sb.WriteRune(r)
}
func FuzzRegexMatcher(f *testing.F) {
inputs := []string{
"", "x", "yy", "abd", "ccc", "ABC", "abCDef", "abcdef", "abababcc", "a.b", "a\xffb", "a\uFFFDb",
"www.91porn.com", "ns1.awsdns-cn-01.top", "r1---sn-2x3abcde.googlevideo.com",
}
for _, test := range regexLiteralCases {
for _, s := range inputs {
f.Add(test.pattern, s)
}
}
for _, test := range regexTailCases {
for _, s := range append(test.match, test.reject...) {
f.Add(test.pattern, s)
}
}
f.Fuzz(func(t *testing.T, pattern, s string) {
re, err := regexp.Compile(pattern)
if err != nil {
return
}
m, _ := newRegexMatcher(pattern)
check := func(s string) {
if got, want := m.Match(s), re.MatchString(s); got != want {
t.Errorf("pattern %q, input %q: got %v, want %v", pattern, s, got, want)
}
}
check(s)
// random inputs seldom match, so also try strings built from the pattern
parsed, _ := syntax.Parse(pattern, syntax.Perl)
h := fnv.New64a()
h.Write([]byte(s))
rnd := rand.New(rand.NewPCG(h.Sum64(), 1))
for range 8 {
var sb strings.Builder
budget := 256
sampleMatch(&sb, parsed, rnd, &budget)
sample := sb.String()
check(sample)
check(s + sample)
if len(sample) > 0 && len(s) > 0 {
i := rnd.IntN(len(sample))
check(sample[:i] + s[:1] + sample[i+1:])
}
}
})
}
+67 -12
View File
@@ -46,7 +46,9 @@ func (g *MphValueMatcher) Add(matcher Matcher, value uint32) {
func (g *MphValueMatcher) Build() error {
if g.mph != nil {
runtime.GC() // peak mem
g.mph.Build()
if err := g.mph.Build(); err != nil {
return err
}
}
runtime.GC() // peak mem
if g.ac != nil {
@@ -58,23 +60,17 @@ func (g *MphValueMatcher) Build() error {
// Match implements ValueMatcher.Match.
func (g *MphValueMatcher) Match(input string) []uint32 {
result := make([][]uint32, 0, 5)
var result []uint32
if g.mph != nil {
if matches := g.mph.Match(input); len(matches) > 0 {
result = append(result, matches)
}
result = g.mph.Match(input) // a new slice, returned without another copy
}
if g.ac != nil {
if matches := g.ac.Match(input); len(matches) > 0 {
result = append(result, matches)
}
result = append(result, g.ac.Match(input)...)
}
if g.regex != nil {
if matches := g.regex.Match(input); len(matches) > 0 {
result = append(result, matches)
}
result = append(result, g.regex.Match(input)...)
}
return CompositeMatches(result)
return result
}
// MatchAny implements ValueMatcher.MatchAny.
@@ -87,3 +83,62 @@ func (g *MphValueMatcher) MatchAny(input string) bool {
}
return g.regex != nil && g.regex.MatchAny(input)
}
func (g *MphValueMatcher) matchAnyHashed(input string, parents []mphSuffix, h, mul uint64) bool {
if g.mph != nil && g.mph.matchAnyHashed(input, parents, h, mul) {
return true
}
if g.ac != nil && g.ac.MatchAny(input) {
return true
}
return g.regex != nil && g.regex.MatchAny(input)
}
// MphValueMatcherCombiner combines several built MphValueMatchers, each bound to one value, and matches an input
// against them as their MatchAny would, hashing the input once for all of them.
type MphValueMatcherCombiner struct {
matchers []*MphValueMatcher
values []uint32
}
// Add adds a built matcher that stands for value.
func (s *MphValueMatcherCombiner) Add(m *MphValueMatcher, value uint32) {
s.matchers = append(s.matchers, m)
s.values = append(s.values, value)
}
// Match returns the values of the matchers that match input, in Add order.
func (s *MphValueMatcherCombiner) Match(input string) []uint32 {
if len(s.matchers) == 0 {
return nil
}
var stack [16]mphSuffix
mul := mphMultipliers[0]
parents, h := mphSuffixes(stack[:0], mul, input)
var result []uint32
for i, m := range s.matchers {
if m.matchAnyHashed(input, parents, h, mul) {
result = append(result, s.values[i])
}
}
return result
}
// MatchAny returns true as soon as one matcher matches input.
func (s *MphValueMatcherCombiner) MatchAny(input string) bool {
switch len(s.matchers) {
case 0:
return false
case 1:
return s.matchers[0].MatchAny(input) // nothing to share, and it stops at the first matching suffix
}
var stack [16]mphSuffix
mul := mphMultipliers[0]
parents, h := mphSuffixes(stack[:0], mul, input)
for _, m := range s.matchers {
if m.matchAnyHashed(input, parents, h, mul) {
return true
}
}
return false
}
+21 -25
View File
@@ -1,7 +1,7 @@
package log // import "github.com/xtls/xray-core/common/log"
import (
"sync"
"sync/atomic"
"github.com/xtls/xray-core/common/serial"
)
@@ -29,36 +29,32 @@ func (m *GeneralMessage) String() string {
// Record writes a message into log stream.
func Record(msg Message) {
logHandler.Handle(msg)
if h := logHandler.Load(); h != nil {
(*h).Handle(msg)
}
}
var logHandler syncHandler
type SeverityLogger interface {
Handler
Severity() Severity
}
func GetSeverity() Severity {
if h := logHandler.Load(); h != nil {
if sh, ok := (*h).(SeverityLogger); ok {
return sh.Severity()
}
}
// log everything by default
return Severity_Debug
}
var logHandler atomic.Pointer[Handler]
// RegisterHandler registers a new handler as current log handler. Previous registered handler will be discarded.
func RegisterHandler(handler Handler) {
if handler == nil {
panic("Log handler is nil")
}
logHandler.Set(handler)
}
type syncHandler struct {
sync.RWMutex
Handler
}
func (h *syncHandler) Handle(msg Message) {
h.RLock()
defer h.RUnlock()
if h.Handler != nil {
h.Handler.Handle(msg)
}
}
func (h *syncHandler) Set(handler Handler) {
h.Lock()
defer h.Unlock()
h.Handler = handler
logHandler.Store(&handler)
}
+2 -2
View File
@@ -1,7 +1,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.36.11
// protoc v6.33.5
// protoc-gen-go v1.36.12
// protoc (unknown)
// source: common/log/log.proto
package log
+4
View File
@@ -68,6 +68,10 @@ func (l *serverityLogger) Handle(msg Message) {
}
}
func (l *serverityLogger) Severity() Severity {
return l.logLevel
}
func (l *generalLogger) run() {
defer l.access.Signal()
+61
View File
@@ -0,0 +1,61 @@
package log
import (
"path/filepath"
"strings"
lua "github.com/yuin/gopher-lua"
)
// RegisterLua makes xray.log available to require in an LState.
func RegisterLua(L *lua.LState) {
L.PreloadModule("xray.log", func(L *lua.LState) int {
module := L.CreateTable(0, 4)
var source, prefix string // cache
for name, severity := range map[string]Severity{
"Debug": Severity_Debug,
"Info": Severity_Info,
"Warning": Severity_Warning,
"Error": Severity_Error,
} {
module.RawSetString(name, L.NewFunction(func(L *lua.LState) int {
if GetSeverity() < severity {
return 0
}
var content strings.Builder
// Prefix with the calling script's filename.
if caller, ok := L.GetStack(1); ok {
if _, err := L.GetInfo("S", caller, lua.LNil); err == nil && caller.Source != "" {
if caller.Source != source {
source = caller.Source
prefix = filepath.Base(strings.TrimPrefix(source, "@")) + ": "
}
content.WriteString(prefix)
}
}
for i := 1; i <= L.GetTop(); i++ {
content.WriteString(luaLogString(L, L.Get(i)))
}
Record(&GeneralMessage{
Severity: severity,
Content: content.String(),
})
return 0
}))
}
L.Push(module)
return 1
})
}
func luaLogString(L *lua.LState, value lua.LValue) string {
if ud, ok := value.(*lua.LUserData); ok {
if err, ok := ud.Value.(error); ok {
return err.Error()
}
}
if _, ok := L.GetMetaField(value, "__tostring").(*lua.LFunction); ok {
return L.ToStringMeta(value).String()
}
return value.String()
}
+213
View File
@@ -0,0 +1,213 @@
package log
import (
"errors"
"fmt"
"os"
"path/filepath"
"testing"
lua "github.com/yuin/gopher-lua"
)
type luaLogHandler struct {
messages []Message
}
func (h *luaLogHandler) Handle(msg Message) {
h.messages = append(h.messages, msg)
}
func TestLuaLog(t *testing.T) {
previous := logHandler.Load()
t.Cleanup(func() { logHandler.Store(previous) })
handler := &luaLogHandler{}
RegisterHandler(handler)
L := lua.NewState()
defer L.Close()
RegisterLua(L)
nativeError := L.NewUserData()
nativeError.Value = fmt.Errorf("lookup failed: %w", errors.New("upstream timeout"))
L.SetGlobal("nativeError", nativeError)
path := filepath.Join(t.TempDir(), "logging.lua")
if err := os.WriteFile(path, []byte(`
local log = require("xray.log")
assert(log == require("xray.log"))
log.Debug("query: ", "example.com")
log.Info("count=", 42, ", enabled=", true, ", value=", nil)
log.Warning(setmetatable({}, {
__tostring = function() return "fallback" end
}))
assert(select("#", log.Error("failed")) == 0)
log.Error("DNS failed: ", nativeError)
log.Warning(nativeError)
local ok, err = pcall(function() error("Lua failure", 0) end)
assert(not ok)
log.Error(err)
local calls = 0
local custom = setmetatable({}, {
__tostring = function() calls = calls + 1; return "custom" end
})
log.Info(custom, custom)
assert(calls == 2)
log.Info("a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l")
log.Info()
function logHook()
log.Info("hook")
end
`), 0o600); err != nil {
t.Fatal(err)
}
if err := L.DoFile(path); err != nil {
t.Fatal(err)
}
if err := L.DoString(`
logHook()
require("xray.log").Info("anonymous")
`); err != nil {
t.Fatal(err)
}
other := filepath.Join(t.TempDir(), "other.lua")
if err := os.WriteFile(other, []byte(`
local log = require("xray.log")
log.Info("other")
logHook()
log.Info("other again")
`), 0o600); err != nil {
t.Fatal(err)
}
if err := L.DoFile(other); err != nil {
t.Fatal(err)
}
want := []struct {
severity Severity
message string
}{
{Severity_Debug, "[Debug] logging.lua: query: example.com"},
{Severity_Info, "[Info] logging.lua: count=42, enabled=true, value=nil"},
{Severity_Warning, "[Warning] logging.lua: fallback"},
{Severity_Error, "[Error] logging.lua: failed"},
{Severity_Error, "[Error] logging.lua: DNS failed: lookup failed: upstream timeout"},
{Severity_Warning, "[Warning] logging.lua: lookup failed: upstream timeout"},
{Severity_Error, "[Error] logging.lua: Lua failure"},
{Severity_Info, "[Info] logging.lua: customcustom"},
{Severity_Info, "[Info] logging.lua: abcdefghijkl"},
{Severity_Info, "[Info] logging.lua: "},
{Severity_Info, "[Info] logging.lua: hook"},
{Severity_Info, "[Info] <string>: anonymous"},
{Severity_Info, "[Info] other.lua: other"},
{Severity_Info, "[Info] logging.lua: hook"},
{Severity_Info, "[Info] other.lua: other again"},
}
if len(handler.messages) != len(want) {
t.Fatalf("logged %d messages, want %d", len(handler.messages), len(want))
}
for i, expected := range want {
msg, ok := handler.messages[i].(*GeneralMessage)
if !ok {
t.Fatalf("message %d has type %T, want *GeneralMessage", i, handler.messages[i])
}
if msg.Severity != expected.severity || msg.String() != expected.message {
t.Errorf("message %d = %q with severity %v, want %q with severity %v", i, msg.String(), msg.Severity, expected.message, expected.severity)
}
}
}
type luaSeverityLogHandler struct {
luaLogHandler
level Severity
}
func (h *luaSeverityLogHandler) Severity() Severity { return h.level }
func TestLuaLogSeverity(t *testing.T) {
previous := logHandler.Load()
t.Cleanup(func() { logHandler.Store(previous) })
L := lua.NewState()
defer L.Close()
RegisterLua(L)
for _, level := range []Severity{Severity_Unknown, Severity_Error, Severity_Warning, Severity_Info, Severity_Debug, Severity_Warning} {
t.Run(level.String(), func(t *testing.T) {
handler := &luaSeverityLogHandler{level: level}
RegisterHandler(handler)
want := []Severity{}
for _, severity := range []Severity{Severity_Error, Severity_Warning, Severity_Info, Severity_Debug} {
if severity <= level {
want = append(want, severity)
}
}
if err := L.DoString(fmt.Sprintf(`
local log = require("xray.log")
local calls = 0
local value = setmetatable({}, {
__tostring = function() calls = calls + 1; return "message" end
})
for _, write in ipairs({log.Error, log.Warning, log.Info, log.Debug}) do
assert(select("#", write(value)) == 0)
end
assert(calls == %d)
`, len(want))); err != nil {
t.Fatal(err)
}
if len(handler.messages) != len(want) {
t.Fatalf("logged %d messages, want %d", len(handler.messages), len(want))
}
for i, severity := range want {
msg := handler.messages[i].(*GeneralMessage)
if msg.Severity != severity || msg.Content != "<string>: message" {
t.Errorf("message %d = %v, want severity %v and content %q", i, msg, severity, "<string>: message")
}
}
})
}
}
type luaDiscardLogHandler struct{ level Severity }
func (luaDiscardLogHandler) Handle(Message) {}
func (h luaDiscardLogHandler) Severity() Severity { return h.level }
func BenchmarkLuaLog(b *testing.B) {
benchmarkLuaLog(b, Severity_Debug)
}
func BenchmarkLuaLogFiltered(b *testing.B) {
benchmarkLuaLog(b, Severity_Warning)
}
func benchmarkLuaLog(b *testing.B, level Severity) {
previous := logHandler.Load()
b.Cleanup(func() { logHandler.Store(previous) })
RegisterHandler(luaDiscardLogHandler{level: level})
L := lua.NewState()
defer L.Close()
RegisterLua(L)
if err := L.DoString(`custom = setmetatable({}, {__tostring = function() return "custom" end})`); err != nil {
b.Fatal(err)
}
for _, benchmark := range []struct {
name, arguments string
}{
{"strings", `"query: ", "example.com"`},
{"mixed", `"count=", 42, ", enabled=", true, ", value=", nil`},
{"many_arguments", `"a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l"`},
{"tostring", "custom"},
} {
b.Run(benchmark.name, func(b *testing.B) {
if err := L.DoString(fmt.Sprintf(`local log = require("xray.log")
function benchmarkLog() log.Info(%s) end`, benchmark.arguments)); err != nil {
b.Fatal(err)
}
fn := L.GetGlobal("benchmarkLog")
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
if err := L.CallByParam(lua.P{Fn: fn, NRet: 0, Protect: true}); err != nil {
b.Fatal(err)
}
}
})
}
}
@@ -0,0 +1,19 @@
// DO NOT EDIT. Regression test for app/geodata's luaGuardHeader.
package lua
import (
"errors"
"strings"
"testing"
"github.com/yuin/gopher-lua/parse"
)
func TestGeodataLuaGuard(t *testing.T) {
_, err := parse.Parse(strings.NewReader("\x12\x01\x00function HandleRoute() end"), "geodata")
var syntaxErr *parse.Error
if !errors.As(err, &syntaxErr) || syntaxErr.Pos.Line != 1 || syntaxErr.Pos.Column != 1 {
t.Fatalf("want Lua rejection at the first byte, got %v", err)
}
}

Some files were not shown because too many files have changed in this diff Show More