Compare commits

..
Author SHA1 Message Date
Meo597 9726bbc9d7 lua: prevent memory address leaks through sandbox string.format and refine comments 2026-10-15 03:07:54 +08:00
Meo597 6cd0c8a414 lua: preserve native Stringer and error output in sandbox tostring() 2026-10-15 02:28:36 +08:00
Meo597 796bbd763a lua: prevent configuration files from being executed as scripts 2026-10-15 01:34:40 +08:00
Meo597 e04191e65b lua: validate script extension after resolving symlinks 2026-10-14 23:12:30 +08:00
Meo597 bd28a87f0b geodata: prevent downloaded assets from compiling as Lua 2026-10-14 22:29:56 +08:00
Meo597 a04770587e geodata: validate entry and code tags before matching 2026-10-14 22:04:54 +08:00
Meo597 eb9c6030ab geodata: reject entry lengths exceeding asset size 2026-10-14 22:03:08 +08:00
Meo597 e4124d3dbb geodata: stricter varint overflow checks 2026-10-14 21:36:24 +08:00
Meo597 4c0f3a94c5 disable inline lua 2026-10-14 10:18:40 +08:00
Meo597 19842ed0c2 luar: whitelist 2026-10-14 09:49:58 +08:00
Meo597 34efc32ff0 sandbox: disable more func 2026-10-14 09:16:32 +08:00
Meo597 4828ed8270 cmd: add Ed25519 key generation and Lua script signing 2026-10-14 08:14:05 +08:00
Meo597 e84f3065dc fix format 2026-10-14 07:07:03 +08:00
Meo597 fa9dc21a55 Replace privileged PSK with Ed25519 signature verification 2026-10-14 06:57:32 +08:00
Meo597 590a5236bb Reduce memory usage 2026-10-13 12:31:04 +08:00
Meo597 1aae252fc3 Lua: Restrict PSK privileges to local Lua files 2026-10-13 11:38:06 +08:00
Meo597 20c4c8f956 Lua: Hide package from sandboxed scripts 2026-10-13 11:24:47 +08:00
Meo597 4c571f5e06 Lua: Add sandbox mechanism 2026-10-13 09:52:31 +08:00
c77a4d0e49 MASQUE client: Support XMUX ("maxConnections": 1 by default); Use Chrome's User-Agent by default (#7121)
https://github.com/XTLS/Xray-core/pull/6810#issuecomment-5842441136
https://github.com/XTLS/Xray-core/pull/6878#issuecomment-6095112915
https://github.com/XTLS/Xray-core/pull/7121#issuecomment-6117636043

---------

Co-authored-by: SVLAVR <skugorov.s@gmail.com>
2026-10-12 19:40:04 +00:00
MeowandGitHub 5a64db009b Lua: Support inline scripts; Add ./xray base64 encode/decode [-gzip] command (#7132)
https://github.com/XTLS/Xray-core/pull/7124#issuecomment-6110557806
https://github.com/XTLS/Xray-core/pull/7132#issuecomment-6118169645
https://github.com/XTLS/Xray-core/pull/7132#issuecomment-6126428359
2026-10-12 16:58:20 +00:00
SVLAVRandGitHub 2bea2df6a1 Hysteria client: Release client map lock before cleanup (#7122)
https://github.com/XTLS/Xray-core/pull/7122#issuecomment-6119280343
2026-10-12 06:56:24 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
fb2dcedbb7 Bump golang.org/x/net from 0.59.0 to 0.60.0 (#7133)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.59.0 to 0.60.0.
- [Commits](https://github.com/golang/net/compare/v0.59.0...v0.60.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-version: 0.60.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-12 05:16:45 +00:00
7c2e05dc47 Hysteria client: Honor cancellation during dialing (#7119)
Fixes https://github.com/XTLS/Xray-core/issues/7120

---------

Co-authored-by: LjhAUMEM <llnu14702@gmail.com>
2026-10-12 05:01:23 +00:00
brookwkoandRPRX 1dd2a0bd3c TUN inbound: Support autoSystemDnsToGateway on FreeBSD as well (#7123)
https://github.com/XTLS/Xray-core/issues/6454#issuecomment-5859521287
https://github.com/XTLS/Xray-core/pull/6691#issuecomment-5445818753
https://github.com/XTLS/Xray-core/pull/6773 & https://github.com/XTLS/Xray-core/pull/6853
2026-10-12 02:48:59 +00:00
Artem LytkinandRPRX 70ef97d564 TUN inbound: Use Xray-core's built-in lightweight TCP/IP stack instead of gVisor (#7130)
https://github.com/XTLS/Xray-core/pull/6771#issuecomment-5752154561
https://github.com/XTLS/Xray-core/pull/6795#issuecomment-6095740152
https://github.com/XTLS/Xray-core/pull/7130#issuecomment-6116258188
2026-10-12 02:18:56 +00:00
RPRXandGitHub 701af60772 Xray-core v26.10.10
Sponsor & Donation & NFTs: https://github.com/XTLS/Xray-core/issues/3668
Project X Channel: https://t.me/projectXtls

Announcement of NFTs by Project X: https://github.com/XTLS/Xray-core/discussions/3633
Project X NFT: https://opensea.io/assets/ethereum/0x5ee362866001613093361eb8569d59c4141b76d1/1

VLESS Post-Quantum Encryption: https://github.com/XTLS/Xray-core/pull/5067
VLESS NFT: https://opensea.io/collection/vless

XHTTP: Beyond REALITY: https://github.com/XTLS/Xray-core/discussions/4113
REALITY NFT: https://opensea.io/assets/ethereum/0x5ee362866001613093361eb8569d59c4141b76d1/2
2026-10-10 09:21:07 +00:00
MeowandGitHub 8855145e58 Xray-core: Add Lua script for dns and routing (#6823)
https://github.com/XTLS/Xray-core/pull/6823#issuecomment-5843754759
https://github.com/XTLS/Xray-core/pull/6823#issuecomment-5861456450
https://github.com/XTLS/Xray-core/pull/6823#issuecomment-6093596069
2026-10-10 09:01:16 +00:00
c7dbfd5e1e Hysteria client: Remove closed instance conn (#7107)
https://github.com/XTLS/Xray-core/pull/7102#issuecomment-6054105250

Fixes https://github.com/XTLS/Xray-core/issues/7100

---------

Co-authored-by: LjhAUMEM <llnu14702@gmail.com>
2026-10-10 08:08:06 +00:00
RPRX aa7aa9ad0a MASQUE client: Ignore publicKey in warp when pcs/vcn exists (domain fronting for WARP)
Closes https://github.com/XTLS/Xray-core/issues/7115#issuecomment-6088708005
2026-10-10 06:56:57 +00:00
2cc08769a5 TLS ECH DoH h2c: Support verifyPeerCertByName's "fromMitm" (after dialerProxy, in RAW outbound's tlsSettings) (#6246)
https://github.com/XTLS/Xray-core/pull/6246#issuecomment-5850758711

Like https://github.com/XTLS/Xray-core/commit/613c63b165829a0b3e5bfc3a72eff21d96ab4874

---------

Co-authored-by: patterniha <71074308+patterniha@users.noreply.github.com>
2026-10-10 06:55:46 +00:00
风扇滑翔翼andRPRX 5af2e9c228 TLS client: Use Xray-core's built-in RootCAs by default (avoids maliciously injected CA in the OS) (#7105)
https://github.com/XTLS/Xray-core/pull/5532#issuecomment-3745598515
https://github.com/XTLS/Xray-core/pull/7098#issuecomment-6053012480
https://github.com/XTLS/Xray-core/pull/7105#issuecomment-6093006914
2026-10-10 06:53:39 +00:00
Artem LytkinandGitHub 836a6fed38 Mux client: Fix b.UDP being overwritten by the next frame (#6822)
https://github.com/XTLS/Xray-core/pull/6822#issuecomment-5993165787
2026-10-08 09:24:11 +00:00
2a14c50775 Socks inbound: Fix stats for UDP Associate traffic (#7101)
https://github.com/XTLS/Xray-core/pull/7101#pullrequestreview-5452526703

---------

Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-10-08 09:21:21 +00:00
8989adfd3f TUN inbound: Handle weak host send and forwarding on the outbound interface on Windows (#7092)
https://github.com/XTLS/Xray-core/issues/6776#issuecomment-5994046421
https://github.com/XTLS/Xray-core/issues/6776#issuecomment-5994796106

Closes https://github.com/XTLS/Xray-core/issues/6776
Closes https://github.com/XTLS/Xray-core/issues/6872

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 05:49:28 +00:00
Hossin AsaadiandGitHub 1a60fc78ff TUN inbound: Duplicate the iOS tunnel file descriptor (#6883)
https://github.com/XTLS/Xray-core/pull/6883#issuecomment-6008368575
2026-10-08 05:14:14 +00:00
Artem LytkinandGitHub 2eedef51a2 DNS: Fix pubsub cleanup sizing every topic's slice by the topic count (#7091)
https://github.com/XTLS/Xray-core/pull/7091#pullrequestreview-5414371870
2026-10-08 05:08:43 +00:00
LjhAUMEMandGitHub 37184948d4 XDNS finalmask: Refine upload (#7095)
Fixes https://github.com/XTLS/Xray-core/pull/7090#issuecomment-6009651970
2026-10-08 04:58:49 +00:00
CluvexandGitHub 9e55a6ed18 MASQUE client: Support WARP (#6878)
https://github.com/XTLS/Xray-core/pull/6844#issuecomment-5859092964
https://github.com/XTLS/Xray-core/pull/6862#issuecomment-5897508127
https://github.com/XTLS/Xray-core/pull/6878#issuecomment-6052448898
2026-10-08 04:53:07 +00:00
7da5dae650 Mux client: Fix a race condition between Dispatch and SessionManager.Close (#6871)
Fixes https://github.com/XTLS/Xray-core/issues/6870

---------

Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-10-05 09:52:18 +00:00
LjhAUMEMandGitHub a3c6cc6aa8 XDNS finalmask: Default values for types; name->names; type+addr->addrs (#7090)
https://github.com/XTLS/Xray-core/pull/7090#issuecomment-5989883544
https://github.com/XTLS/Xray-core/pull/7090#issuecomment-5988755551
https://github.com/XTLS/Xray-core/pull/7089#issuecomment-5987266008
2026-10-05 08:07:19 +00:00
nSQUICKandGitHub 1e08d48d88 TUN inbound: Start on Windows when IPv6 is disabled system-wide (#6874)
https://github.com/XTLS/Xray-core/pull/6874#issuecomment-5989986544
2026-10-05 07:26:00 +00:00
5121c28b85 QUIC sniffer: Handle zero-filled datagram tails (#6882)
https://github.com/XTLS/Xray-core/pull/6882#pullrequestreview-5395639793

---------

Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-10-05 03:53:22 +00:00
92f7e72490 WireGuard kernelTun: Fix IPv6 table index allocation when the first table is occupied (#6881)
https://github.com/XTLS/Xray-core/pull/6881#issuecomment-5927334896

---------

Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-10-05 03:50:22 +00:00
Жора ЗмейкинandGitHub eddad37c69 XDNS finalmask: Fix resolver close deadlock and socket leaks (#7089)
https://github.com/XTLS/Xray-core/pull/7089#issuecomment-5987521972
2026-10-05 03:31:46 +00:00
SVLAVRandGitHub 00cde72a9b Test: Fix process leaks when retrying scenario tests (#6863)
https://github.com/XTLS/Xray-core/pull/6863#pullrequestreview-5362117967
2026-10-05 02:13:03 +00:00
DenisandGitHub 08775afd65 WireGuard inbound: Reduce memory held by idle UDP flows (#6856)
https://github.com/XTLS/Xray-core/pull/6856#issuecomment-5870639238
2026-10-05 01:56:36 +00:00
DenisandGitHub e0bae21201 WireGuard inbound: Reply fake pong to ICMP ping (#6855)
https://github.com/XTLS/Xray-core/pull/6015#issuecomment-4321525342
2026-10-05 01:36:54 +00:00
9d9a7a1c00 gRPC client: Fix redial with the first request's canceled context (#6877)
https://github.com/XTLS/Xray-core/pull/6877#issuecomment-5986476726

Fixes https://github.com/XTLS/Xray-core/issues/6876

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 01:26:12 +00:00
RPRXandGitHub b26a91de4f Xray-core v26.9.30
Sponsor & Donation & NFTs: https://github.com/XTLS/Xray-core/issues/3668
Project X Channel: https://t.me/projectXtls

Announcement of NFTs by Project X: https://github.com/XTLS/Xray-core/discussions/3633
Project X NFT: https://opensea.io/assets/ethereum/0x5ee362866001613093361eb8569d59c4141b76d1/1

VLESS Post-Quantum Encryption: https://github.com/XTLS/Xray-core/pull/5067
VLESS NFT: https://opensea.io/collection/vless

XHTTP: Beyond REALITY: https://github.com/XTLS/Xray-core/discussions/4113
REALITY NFT: https://opensea.io/assets/ethereum/0x5ee362866001613093361eb8569d59c4141b76d1/2
2026-09-30 07:40:04 +00:00
1f304916bd TUN inbound: Add autoSystemWfpBlockLeak on Windows (blocks "dns" and "misconfigtun" IPv4/IPv6 traffic leaks outside the TUN); Rename autoSystemDNS to autoSystemDnsToGateway on Linux (and change some behaviors) (#6853)
https://github.com/XTLS/Xray-core/pull/6853#issuecomment-5899791359
https://github.com/XTLS/Xray-core/pull/6853#issuecomment-5901287980
https://github.com/XTLS/Xray-core/pull/6853#issuecomment-5903680113
https://github.com/XTLS/Xray-core/pull/6853#issuecomment-5904123488
https://github.com/XTLS/Xray-core/pull/6853#issuecomment-5904647772
https://github.com/XTLS/Xray-core/pull/6853#issuecomment-5905047424

Fixes https://github.com/XTLS/Xray-core/issues/6454#issuecomment-5863800676

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 06:26:16 +00:00
风扇滑翔翼 0086362663 SS2022 proxy: Refactor to fix some issues (#6866)
https://github.com/XTLS/Xray-core/pull/6866#issuecomment-5904374529

Fixes https://github.com/XTLS/Xray-core/pull/6831#issuecomment-5884408501 and https://github.com/XTLS/Xray-core/pull/6866#issuecomment-5889568266
2026-09-30 13:16:27 +08:00
CluvexandGitHub e51b3c3621 Noise finalmask: type supports "exp" (#6862)
https://github.com/XTLS/Xray-core/pull/6844#issuecomment-5859092964
https://github.com/XTLS/Xray-core/pull/6844#issuecomment-5859855674
https://github.com/XTLS/Xray-core/pull/6862#issuecomment-5897471145
2026-09-30 03:11:12 +00:00
Artem LytkinandGitHub 6243d2a26e Geodata: Reduce matcher memory on mobile and desktop, skip regexes that cannot match (#6867)
https://github.com/XTLS/Xray-core/pull/6867#issuecomment-5895171934
2026-09-29 19:25:53 +00:00
风扇滑翔翼 35e616d3b9 Xray-core: Move PacketConnWrapper to common/net (#6854)
https://github.com/XTLS/Xray-core/pull/6854#issuecomment-5895541650

Fixes https://github.com/XTLS/Xray-core/issues/6849
2026-09-30 02:33:53 +08:00
v2rayandRPRX 08cb6e6bca WireGuard proxy: Fix potential startup races (#6852)
https://github.com/XTLS/Xray-core/pull/6852#issuecomment-5871816550

Fixes https://github.com/XTLS/Xray-core/issues/6850
2026-09-29 18:13:43 +00:00
wewon-backdownandRPRX 48ad0300ea API: xray api adu supports Hysteria (#6847)
https://github.com/XTLS/Xray-core/pull/6847#issuecomment-5895325181

---------

Co-authored-by: RPRX <63339210+RPRX@users.noreply.github.com>
2026-09-29 18:11:26 +00:00
0fc379203f HTTPUpgrade transport: Send some Sec-WebSocket-* headers (#6835)
https://github.com/XTLS/Xray-core/pull/6835#issuecomment-5853986428

---------

Co-authored-by: 风扇滑翔翼 <Fangliding.fshxy@outlook.com>
2026-09-29 17:25:26 +00:00
LjhAUMEMandGitHub fc8f8a451d XDNS finalmask: Refactor and new parameters (#6718)
https://github.com/XTLS/Xray-core/pull/6718#issuecomment-5894987590

Fixes https://github.com/XTLS/Xray-core/issues/6692
2026-09-29 17:16:59 +00:00
e5e85ca9da XTLS Vision: Suppress outer CloseNotify after switching to direct copy (#6834)
https://github.com/XTLS/Xray-core/pull/6816#issuecomment-5828082031
https://github.com/XTLS/Xray-core/issues/6794#issuecomment-5847200576
https://github.com/XTLS/Xray-core/pull/6834#issuecomment-5860728689

Fixes https://github.com/XTLS/Xray-core/issues/6794#issuecomment-5754894283
Fixes https://github.com/XTLS/Xray-core/issues/6124#issuecomment-4439918073
Fixes https://github.com/XTLS/Xray-core/issues/4878#issuecomment-5754638401

---------

Co-authored-by: Artem Lytkin <146867384+4RH1T3CT0R7@users.noreply.github.com>
2026-09-27 23:39:21 +00:00
202 changed files with 18369 additions and 5163 deletions
+3
View File
@@ -470,6 +470,9 @@ func (d *DefaultDispatcher) routedDispatch(ctx context.Context, link *transport.
return // DO NOT CHANGE: the traffic shouldn't be processed by default outbound if the specified outbound tag doesn't exist (yet), e.g., VLESS Reverse Proxy return // DO NOT CHANGE: the traffic shouldn't be processed by default outbound if the specified outbound tag doesn't exist (yet), e.g., VLESS Reverse Proxy
} }
} else { } else {
if err != common.ErrNoClue {
errors.LogErrorInner(ctx, err, "failed to pick route for ", destination)
}
errors.LogInfo(ctx, "default route for ", destination) errors.LogInfo(ctx, "default route for ", destination)
} }
} }
+22 -4
View File
@@ -93,6 +93,7 @@ type NameServer struct {
UnexpectedIp []*geodata.IPRule `protobuf:"bytes,13,rep,name=unexpected_ip,json=unexpectedIp,proto3" json:"unexpected_ip,omitempty"` UnexpectedIp []*geodata.IPRule `protobuf:"bytes,13,rep,name=unexpected_ip,json=unexpectedIp,proto3" json:"unexpected_ip,omitempty"`
ActUnprior bool `protobuf:"varint,14,opt,name=actUnprior,proto3" json:"actUnprior,omitempty"` ActUnprior bool `protobuf:"varint,14,opt,name=actUnprior,proto3" json:"actUnprior,omitempty"`
PolicyID uint32 `protobuf:"varint,17,opt,name=policyID,proto3" json:"policyID,omitempty"` PolicyID uint32 `protobuf:"varint,17,opt,name=policyID,proto3" json:"policyID,omitempty"`
Id string `protobuf:"bytes,18,opt,name=id,proto3" json:"id,omitempty"`
unknownFields protoimpl.UnknownFields unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache sizeCache protoimpl.SizeCache
} }
@@ -239,6 +240,13 @@ func (x *NameServer) GetPolicyID() uint32 {
return 0 return 0
} }
func (x *NameServer) GetId() string {
if x != nil {
return x.Id
}
return ""
}
type Config struct { type Config struct {
state protoimpl.MessageState `protogen:"open.v1"` state protoimpl.MessageState `protogen:"open.v1"`
// NameServer list used by this DNS client. // NameServer list used by this DNS client.
@@ -258,6 +266,7 @@ type Config struct {
DisableFallback bool `protobuf:"varint,10,opt,name=disableFallback,proto3" json:"disableFallback,omitempty"` DisableFallback bool `protobuf:"varint,10,opt,name=disableFallback,proto3" json:"disableFallback,omitempty"`
DisableFallbackIfMatch bool `protobuf:"varint,11,opt,name=disableFallbackIfMatch,proto3" json:"disableFallbackIfMatch,omitempty"` DisableFallbackIfMatch bool `protobuf:"varint,11,opt,name=disableFallbackIfMatch,proto3" json:"disableFallbackIfMatch,omitempty"`
EnableParallelQuery bool `protobuf:"varint,14,opt,name=enableParallelQuery,proto3" json:"enableParallelQuery,omitempty"` EnableParallelQuery bool `protobuf:"varint,14,opt,name=enableParallelQuery,proto3" json:"enableParallelQuery,omitempty"`
Script string `protobuf:"bytes,15,opt,name=script,proto3" json:"script,omitempty"`
unknownFields protoimpl.UnknownFields unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache sizeCache protoimpl.SizeCache
} }
@@ -369,6 +378,13 @@ func (x *Config) GetEnableParallelQuery() bool {
return false return false
} }
func (x *Config) GetScript() string {
if x != nil {
return x.Script
}
return ""
}
type Config_HostMapping struct { type Config_HostMapping struct {
state protoimpl.MessageState `protogen:"open.v1"` state protoimpl.MessageState `protogen:"open.v1"`
Domain *geodata.DomainRule `protobuf:"bytes,2,opt,name=domain,proto3" json:"domain,omitempty"` Domain *geodata.DomainRule `protobuf:"bytes,2,opt,name=domain,proto3" json:"domain,omitempty"`
@@ -435,7 +451,7 @@ var File_app_dns_config_proto protoreflect.FileDescriptor
const file_app_dns_config_proto_rawDesc = "" + const file_app_dns_config_proto_rawDesc = "" +
"\n" + "\n" +
"\x14app/dns/config.proto\x12\fxray.app.dns\x1a\x1ccommon/net/destination.proto\x1a\x1bcommon/geodata/geodat.proto\"\xde\x05\n" + "\x14app/dns/config.proto\x12\fxray.app.dns\x1a\x1ccommon/net/destination.proto\x1a\x1bcommon/geodata/geodat.proto\"\xee\x05\n" +
"\n" + "\n" +
"NameServer\x123\n" + "NameServer\x123\n" +
"\aaddress\x18\x01 \x01(\v2\x19.xray.common.net.EndpointR\aaddress\x12\x1b\n" + "\aaddress\x18\x01 \x01(\v2\x19.xray.common.net.EndpointR\aaddress\x12\x1b\n" +
@@ -461,10 +477,11 @@ const file_app_dns_config_proto_rawDesc = "" +
"\n" + "\n" +
"actUnprior\x18\x0e \x01(\bR\n" + "actUnprior\x18\x0e \x01(\bR\n" +
"actUnprior\x12\x1a\n" + "actUnprior\x12\x1a\n" +
"\bpolicyID\x18\x11 \x01(\rR\bpolicyIDB\x0f\n" + "\bpolicyID\x18\x11 \x01(\rR\bpolicyID\x12\x0e\n" +
"\x02id\x18\x12 \x01(\tR\x02idB\x0f\n" +
"\r_disableCacheB\r\n" + "\r_disableCacheB\r\n" +
"\v_serveStaleB\x12\n" + "\v_serveStaleB\x12\n" +
"\x10_serveExpiredTTLJ\x04\b\x04\x10\x05\"\x82\x05\n" + "\x10_serveExpiredTTLJ\x04\b\x04\x10\x05\"\x9a\x05\n" +
"\x06Config\x129\n" + "\x06Config\x129\n" +
"\vname_server\x18\x05 \x03(\v2\x18.xray.app.dns.NameServerR\n" + "\vname_server\x18\x05 \x03(\v2\x18.xray.app.dns.NameServerR\n" +
"nameServer\x12\x1b\n" + "nameServer\x12\x1b\n" +
@@ -480,7 +497,8 @@ const file_app_dns_config_proto_rawDesc = "" +
"\x0fdisableFallback\x18\n" + "\x0fdisableFallback\x18\n" +
" \x01(\bR\x0fdisableFallback\x126\n" + " \x01(\bR\x0fdisableFallback\x126\n" +
"\x16disableFallbackIfMatch\x18\v \x01(\bR\x16disableFallbackIfMatch\x120\n" + "\x16disableFallbackIfMatch\x18\v \x01(\bR\x16disableFallbackIfMatch\x120\n" +
"\x13enableParallelQuery\x18\x0e \x01(\bR\x13enableParallelQuery\x1a}\n" + "\x13enableParallelQuery\x18\x0e \x01(\bR\x13enableParallelQuery\x12\x16\n" +
"\x06script\x18\x0f \x01(\tR\x06script\x1a}\n" +
"\vHostMapping\x127\n" + "\vHostMapping\x127\n" +
"\x06domain\x18\x02 \x01(\v2\x1f.xray.common.geodata.DomainRuleR\x06domain\x12\x0e\n" + "\x06domain\x18\x02 \x01(\v2\x1f.xray.common.geodata.DomainRuleR\x06domain\x12\x0e\n" +
"\x02ip\x18\x03 \x03(\fR\x02ip\x12%\n" + "\x02ip\x18\x03 \x03(\fR\x02ip\x12%\n" +
+3
View File
@@ -27,6 +27,7 @@ message NameServer {
repeated xray.common.geodata.IPRule unexpected_ip = 13; repeated xray.common.geodata.IPRule unexpected_ip = 13;
bool actUnprior = 14; bool actUnprior = 14;
uint32 policyID = 17; uint32 policyID = 17;
string id = 18;
} }
enum QueryStrategy { enum QueryStrategy {
@@ -73,4 +74,6 @@ message Config {
bool disableFallbackIfMatch = 11; bool disableFallbackIfMatch = 11;
bool enableParallelQuery = 14; bool enableParallelQuery = 14;
string script = 15;
} }
+20
View File
@@ -31,6 +31,7 @@ type DNS struct {
domainMatcher geodata.DomainMatcher domainMatcher geodata.DomainMatcher
matcherInfos []*DomainMatcherInfo matcherInfos []*DomainMatcherInfo
checkSystem bool checkSystem bool
script *scriptEngine
} }
// DomainMatcherInfo contains information attached to index returned by Server.domainMatcher. // DomainMatcherInfo contains information attached to index returned by Server.domainMatcher.
@@ -169,6 +170,13 @@ func New(ctx context.Context, config *Config) (*DNS, error) {
clients = append(clients, NewLocalDNSClient(ipOption)) clients = append(clients, NewLocalDNSClient(ipOption))
} }
var script *scriptEngine
if config.Script != "" {
if script, err = newScriptEngine(config.Script); err != nil {
return nil, errors.New("failed to prepare DNS script").Base(err)
}
}
return &DNS{ return &DNS{
hosts: hosts, hosts: hosts,
ipOption: &ipOption, ipOption: &ipOption,
@@ -180,6 +188,7 @@ func New(ctx context.Context, config *Config) (*DNS, error) {
disableFallbackIfMatch: config.DisableFallbackIfMatch, disableFallbackIfMatch: config.DisableFallbackIfMatch,
enableParallelQuery: config.EnableParallelQuery, enableParallelQuery: config.EnableParallelQuery,
checkSystem: checkSystem, checkSystem: checkSystem,
script: script,
}, nil }, nil
} }
@@ -190,11 +199,19 @@ func (*DNS) Type() interface{} {
// Start implements common.Runnable. // Start implements common.Runnable.
func (s *DNS) Start() error { func (s *DNS) Start() error {
if s.script != nil {
if err := s.script.start(s); err != nil {
return errors.New("failed to initialize DNS script").Base(err)
}
}
return nil return nil
} }
// Close implements common.Closable. // Close implements common.Closable.
func (s *DNS) Close() error { func (s *DNS) Close() error {
if s.script != nil {
s.script.close()
}
return nil return nil
} }
@@ -279,6 +296,9 @@ func (s *DNS) LookupIP(domain string, option dns.IPOption) ([]net.IP, uint32, er
} }
// Name servers lookup // Name servers lookup
if s.script != nil {
return s.script.query(domain, option)
}
if s.enableParallelQuery { if s.enableParallelQuery {
return s.parallelQuery(domain, option) return s.parallelQuery(domain, option)
} else { } else {
+169
View File
@@ -0,0 +1,169 @@
package dns
import (
"context"
"strings"
"github.com/xtls/xray-core/common/errors"
xlua "github.com/xtls/xray-core/common/lua"
"github.com/xtls/xray-core/common/net"
featureDNS "github.com/xtls/xray-core/features/dns"
"github.com/xtls/xray-core/features/dns/localdns"
lua "github.com/yuin/gopher-lua"
)
// luaDNSServer adapts configured and local DNS to the same Lua API.
type luaDNSServer struct {
id string
name string
query func(context.Context, string, featureDNS.IPOption) ([]net.IP, uint32, error)
}
// RegisterLua makes xray.dns available to scripts backed by client.
func RegisterLua(L *lua.LState, client featureDNS.Client) {
var servers []luaDNSServer
switch client := client.(type) {
case *DNS:
servers = luaServers(client)
case *localdns.Client:
servers = []luaDNSServer{{
id: "localhost",
name: "localhost",
query: func(_ context.Context, domain string, option featureDNS.IPOption) ([]net.IP, uint32, error) {
return client.LookupIP(domain, option)
},
}}
}
registerLua(L, servers, client)
}
// registerLua makes xray.dns available to DNS scripts.
func (s *DNS) registerLua(L *lua.LState) {
registerLua(L, luaServers(s), nil)
}
func luaServers(s *DNS) []luaDNSServer {
servers := make([]luaDNSServer, len(s.clients))
for i, client := range s.clients {
servers[i] = luaDNSServer{id: client.id, name: client.Name(), query: client.QueryIP}
}
return servers
}
func registerLua(L *lua.LState, servers []luaDNSServer, client featureDNS.Client) {
L.PreloadModule("xray.dns", func(L *lua.LState) int {
pushIPs := xlua.NewSlicePusher[net.IP](L)
serverList := L.CreateTable(len(servers), 0)
for i, client := range servers {
server := L.CreateTable(0, 2)
server.RawSetString("ID", lua.LString(client.id))
server.RawSetString("Query", L.NewFunction(func(L *lua.LState) int {
domain, ok := L.Get(2).(lua.LString)
if !ok {
L.RaiseError("server:Query requires a domain")
return 0
}
option := featureDNS.IPOption{
IPv4Enable: L.CheckBool(3),
IPv6Enable: L.CheckBool(4),
FakeEnable: L.CheckBool(5),
}
ctx := L.Context()
if ctx == nil {
L.RaiseError("server:Query requires an active DNS query")
return 0
}
var ips []net.IP
var ttl uint32
var err error
if !option.FakeEnable && strings.EqualFold(client.name, "FakeDNS") {
err = featureDNS.ErrEmptyResponse
} else {
ips, ttl, err = client.query(ctx, string(domain), option)
}
pushIPs(L, ips)
xlua.PushNumber(L, ttl)
xlua.PushError(L, err)
return 3
}))
serverList.RawSetInt(i+1, server)
}
module := L.CreateTable(0, 2)
if servers != nil {
module.RawSetString("Servers", serverList)
}
if client != nil {
module.RawSetString("Query", newLuaClientQuery(L, client, pushIPs))
}
L.Push(module)
return 1
})
}
func newLuaClientQuery(L *lua.LState, client featureDNS.Client, pushIPs func(*lua.LState, []net.IP)) *lua.LFunction {
return L.NewFunction(func(L *lua.LState) int {
domain, ok := L.Get(1).(lua.LString)
if !ok {
L.RaiseError("dns.Query requires a domain")
return 0
}
option := featureDNS.IPOption{
IPv4Enable: L.CheckBool(2),
IPv6Enable: L.CheckBool(3),
FakeEnable: L.CheckBool(4),
}
if L.Context() == nil {
L.RaiseError("dns.Query requires an active DNS query")
return 0
}
ips, ttl, err := client.LookupIP(string(domain), option)
pushIPs(L, ips)
xlua.PushNumber(L, ttl)
xlua.PushError(L, err)
return 3
})
}
// callLuaQuery runs HandleDNSQuery and leaves (ips, ttl, err) on the stack.
func callLuaQuery(L *lua.LState, domain string, option featureDNS.IPOption) error {
fn := L.GetGlobal("HandleDNSQuery")
if fn.Type() != lua.LTFunction {
return errors.New("DNS script must define HandleDNSQuery(...)")
}
return L.CallByParam(lua.P{Fn: fn, NRet: 3, Protect: true},
lua.LString(strings.ToLower(domain)),
lua.LBool(option.IPv4Enable),
lua.LBool(option.IPv6Enable),
lua.LBool(option.FakeEnable))
}
// readLuaQueryResult reads (ips, ttl, err) from the stack without copying the IPs.
func readLuaQueryResult(L *lua.LState) ([]net.IP, uint32, error) {
if err := xlua.ReadError(L.Get(-1), "DNS script error must be an error or string"); err != nil {
return nil, 0, err
}
ttl, err := xlua.ReadUint32(L.Get(-2), "DNS script returned invalid TTL")
if err != nil {
return nil, 0, err
}
addresses := L.Get(-3)
if addresses == lua.LNil {
return nil, 0, featureDNS.ErrEmptyResponse
}
ips, err := xlua.ReadUserData[[]net.IP](addresses, "DNS script IPs must be native IP slice userdata")
if err != nil {
return nil, 0, err
}
if len(ips) == 0 {
return nil, 0, featureDNS.ErrEmptyResponse
}
return ips, ttl, nil
}
+121
View File
@@ -0,0 +1,121 @@
package dns
import (
"context"
"os"
"path/filepath"
"testing"
"time"
"github.com/xtls/xray-core/common/net"
featureDNS "github.com/xtls/xray-core/features/dns"
lua "github.com/yuin/gopher-lua"
)
// BenchmarkLuaDNSHook isolates scalar argument bridging and a fixed return.
// It excludes upstream queries, result decoding, and state pool management.
func BenchmarkLuaDNSHook(b *testing.B) {
L := lua.NewState()
b.Cleanup(L.Close)
if err := L.DoString(`
function HandleDNSQuery(domain, ipv4, ipv6, fake)
return true
end
`); err != nil {
b.Fatal(err)
}
L.SetContext(context.Background())
option := featureDNS.IPOption{IPv4Enable: true}
if err := callLuaQuery(L, "example.com", option); err != nil {
b.Fatal(err)
}
if L.Get(-3) != lua.LTrue {
b.Fatal("hook did not return true")
}
L.Pop(3)
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
if err := callLuaQuery(L, "example.com", option); err != nil {
b.Fatal(err)
}
L.Pop(3)
}
}
// BenchmarkLuaDNSQuery queries the same preselected, in-memory upstream.
// client_query compares Client.QueryIP to a preloaded server:Query hook.
// script_query additionally measures production pool and timeout management.
// These cases do not measure DNS.LookupIP server selection or network latency.
func BenchmarkLuaDNSQuery(b *testing.B) {
ctx := context.Background()
option := featureDNS.IPOption{IPv4Enable: true}
ip := net.ParseIP("127.0.0.1")
upstream := &benchmarkLuaNameServer{ips: []net.IP{ip}}
client := &Client{server: upstream, ipOption: &option, timeoutMs: time.Second}
server := &DNS{ctx: ctx, clients: []*Client{client}}
const script = `
local server = require("xray.dns").Servers[1]
function HandleDNSQuery(domain, ipv4, ipv6, fake)
return server:Query(domain, ipv4, ipv6, fake)
end
`
L := lua.NewState()
b.Cleanup(L.Close)
server.registerLua(L)
if err := L.DoString(script); err != nil {
b.Fatal(err)
}
L.SetContext(ctx)
path := filepath.Join(b.TempDir(), "query.lua")
if err := os.WriteFile(path, []byte(script), 0o600); err != nil {
b.Fatal(err)
}
engine, err := newScriptEngine(path)
if err != nil {
b.Fatal(err)
}
if err := engine.start(server); err != nil {
b.Fatal(err)
}
b.Cleanup(engine.close)
for _, bench := range []struct {
name string
query func() ([]net.IP, uint32, error)
}{
{"client_query/native", func() ([]net.IP, uint32, error) {
return client.QueryIP(ctx, "example.com", option)
}},
{"client_query/lua", func() ([]net.IP, uint32, error) {
if err := callLuaQuery(L, "example.com", option); err != nil {
return nil, 0, err
}
ips, ttl, err := readLuaQueryResult(L)
L.Pop(3)
return ips, ttl, err
}},
{"script_query/lua", func() ([]net.IP, uint32, error) {
return engine.query("example.com", option)
}},
} {
b.Run(bench.name, func(b *testing.B) {
ips, ttl, err := bench.query()
if err != nil || ttl != 60 || len(ips) != 1 || !ips[0].Equal(ip) {
b.Fatalf("query() = %v, TTL %d, %v; want %v, TTL 60", ips, ttl, err, ip)
}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
ips, ttl, err = bench.query()
if err != nil {
b.Fatal(err)
}
}
b.StopTimer()
if ttl != 60 || len(ips) != 1 || !ips[0].Equal(ip) {
b.Fatalf("query() = %v, TTL %d; want %v, TTL 60", ips, ttl, ip)
}
})
}
}
+272
View File
@@ -0,0 +1,272 @@
package dns
import (
"context"
go_errors "errors"
"strings"
"testing"
"time"
"github.com/xtls/xray-core/common/geodata"
"github.com/xtls/xray-core/common/net"
featureDNS "github.com/xtls/xray-core/features/dns"
"github.com/xtls/xray-core/features/dns/localdns"
lua "github.com/yuin/gopher-lua"
)
func TestReadLuaQueryResult(t *testing.T) {
wantIPs := []net.IP{net.ParseIP("8.8.8.8"), {127, 0, 0, 1}, net.ParseIP("::1")}
nativeErr := go_errors.New("upstream failed")
for _, tc := range []struct {
name, values string
wantIPs []net.IP
wantTTL uint32
wantErr error
wantMessage string
}{
{name: "IPs", values: `ips, 45`, wantIPs: wantIPs, wantTTL: 45},
{name: "nil IPs", values: `nil, 0`, wantErr: featureDNS.ErrEmptyResponse},
{name: "empty IPs", values: `emptyIPs, 0`, wantErr: featureDNS.ErrEmptyResponse},
{name: "native error", values: `nil, nil, nativeError`, wantErr: nativeErr},
{name: "string error", values: `nil, nil, "blocked"`, wantMessage: "blocked"},
{name: "fractional TTL", values: `ips, 1.5`, wantMessage: "invalid TTL"},
{name: "oversized TTL", values: `ips, 4294967296`, wantMessage: "invalid TTL"},
{name: "negative TTL", values: `ips, -1`, wantMessage: "invalid TTL"},
{name: "NaN TTL", values: `ips, 0/0`, wantMessage: "invalid TTL"},
{name: "missing TTL", values: `ips`, wantMessage: "invalid TTL"},
{name: "string IPs", values: `"127.0.0.1", 60`, wantMessage: "native IP slice"},
{name: "wrong userdata", values: `ip, 60`, wantMessage: "native IP slice"},
{name: "invalid error", values: `ips, 60, false`, wantMessage: "error or string"},
} {
t.Run(tc.name, func(t *testing.T) {
L := lua.NewState()
defer L.Close()
for name, value := range map[string]any{"ips": wantIPs, "ip": wantIPs[0], "emptyIPs": []net.IP(nil), "nativeError": nativeErr} {
ud := L.NewUserData()
ud.Value = value
L.SetGlobal(name, ud)
}
fn, err := L.LoadString("return " + tc.values)
if err != nil {
t.Fatal(err)
}
if err := L.CallByParam(lua.P{Fn: fn, NRet: 3, Protect: true}); err != nil {
t.Fatal(err)
}
ips, ttl, err := readLuaQueryResult(L)
switch {
case tc.wantErr != nil:
if err != tc.wantErr {
t.Fatalf("error = %v, want original error %v", err, tc.wantErr)
}
case tc.wantMessage != "":
if err == nil || !strings.Contains(err.Error(), tc.wantMessage) {
t.Fatalf("error = %v, want %q", err, tc.wantMessage)
}
case err != nil:
t.Fatal(err)
}
if ttl != tc.wantTTL || len(ips) != len(tc.wantIPs) {
t.Fatalf("result = %v, TTL %d; want %v, TTL %d", ips, ttl, tc.wantIPs, tc.wantTTL)
}
for i := range ips {
if !ips[i].Equal(tc.wantIPs[i]) {
t.Fatalf("IP %d = %v, want %v", i, ips[i], tc.wantIPs[i])
}
}
if len(ips) != 0 && &ips[0] != &tc.wantIPs[0] {
t.Fatal("result copied the IP slice")
}
})
}
}
func TestCallLuaQueryCancellation(t *testing.T) {
L := lua.NewState()
defer L.Close()
if err := L.DoString(`function HandleDNSQuery(domain, ipv4, ipv6, fake) while true do end end`); err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
L.SetContext(ctx)
err := callLuaQuery(L, "example.com", featureDNS.IPOption{IPv4Enable: true})
if err == nil {
t.Fatal("callLuaQuery did not stop after context cancellation")
}
if L.Context() != ctx {
t.Fatal("callLuaQuery changed the Lua state's context")
}
}
func TestCallLuaQuery(t *testing.T) {
L := lua.NewState()
defer L.Close()
addresses := L.NewUserData()
addresses.Value = []net.IP{net.ParseIP("127.0.0.1")}
L.SetGlobal("ips", addresses)
if err := L.DoString(`
function HandleDNSQuery(domain, ipv4, ipv6, fake)
assert(domain == "example.com")
assert(ipv4 and not ipv6 and not fake)
return ips, 60, nil
end
`); err != nil {
t.Fatal(err)
}
if err := callLuaQuery(L, "ExAmPlE.CoM", featureDNS.IPOption{IPv4Enable: true}); err != nil {
t.Fatal(err)
}
if L.GetTop() != 3 || L.Get(1) != addresses || L.Get(2) != lua.LNumber(60) || L.Get(3) != lua.LNil {
t.Fatal("callLuaQuery did not leave the three query results on the stack")
}
}
func TestLuaDNSServerQuery(t *testing.T) {
L := lua.NewState()
defer L.Close()
geodata.RegisterLua(L)
option := featureDNS.IPOption{IPv4Enable: true}
ips := []net.IP{net.ParseIP("127.0.0.1"), net.ParseIP("8.8.8.8")}
server := &DNS{clients: []*Client{{server: &benchmarkLuaNameServer{ips: ips}, ipOption: &option, timeoutMs: time.Second}}}
server.registerLua(L)
if err := L.DoString(`
local server = require("xray.dns").Servers[1]
local matcher = require("xray.geodata").BuildIPMatcher("127.0.0.0/8")
function HandleDNSQuery(domain, ipv4, ipv6, fake)
local ips, ttl, err = server:Query(domain, ipv4, ipv6, fake)
assert(type(ips) == "userdata" and not err)
assert(#ips == 2 and ips[1]:String() == "127.0.0.1" and ips[2]:String() == "8.8.8.8")
assert(matcher:Match(ips[1]) and not matcher:Match(ips[2]))
assert(matcher:AnyMatch(ips))
local matched, unmatched = matcher:FilterIPs(ips)
assert(#matched == 1 and #unmatched == 1)
assert(matched[1]:Equal(ips[1]) and unmatched[1]:Equal(ips[2]))
return matched, ttl, err
end
`); err != nil {
t.Fatal(err)
}
L.SetContext(context.Background())
if err := callLuaQuery(L, "example.com", option); err != nil {
t.Fatal(err)
}
got, ttl, err := readLuaQueryResult(L)
if err != nil || ttl != 60 || len(got) != 1 || !got[0].Equal(ips[0]) {
t.Fatalf("server query = %v, TTL %d, %v", got, ttl, err)
}
}
type luaDNSClient struct {
featureDNS.Client
lookup func(string, featureDNS.IPOption) ([]net.IP, uint32, error)
}
func (c *luaDNSClient) LookupIP(domain string, option featureDNS.IPOption) ([]net.IP, uint32, error) {
return c.lookup(domain, option)
}
func TestLuaDNSClientQuery(t *testing.T) {
L := lua.NewState()
defer L.Close()
L.SetContext(context.Background())
geodata.RegisterLua(L)
want := []net.IP{{127, 0, 0, 1}, net.ParseIP("::1")}
client := &luaDNSClient{lookup: func(domain string, option featureDNS.IPOption) ([]net.IP, uint32, error) {
if domain != "MiXeD.Example." || !option.IPv4Enable || option.IPv6Enable || !option.FakeEnable {
t.Fatalf("dns.Query arguments = %q, %+v", domain, option)
}
return want, 42, nil
}}
RegisterLua(L, client)
if err := L.DoString(`
local dns = require("xray.dns")
local matcher = require("xray.geodata").BuildIPMatcher("127.0.0.1")
assert(dns.Servers == nil)
ips, ttl, err = dns.Query("MiXeD.Example.", true, false, true)
assert(not err and ttl == 42 and matcher:AnyMatch(ips))
assert(#ips == 2 and ips[1]:String() == "127.0.0.1" and ips[2]:String() == "::1")
assert(matcher:Match(ips[1]) and not matcher:Match(ips[2]))
`); err != nil {
t.Fatal(err)
}
got := L.GetGlobal("ips").(*lua.LUserData).Value.([]net.IP)
if &got[0] != &want[0] {
t.Fatal("dns.Query copied the IP slice")
}
}
func TestLuaDNSLocalClient(t *testing.T) {
L := lua.NewState()
defer L.Close()
L.SetContext(context.Background())
RegisterLua(L, localdns.New())
if err := L.DoString(`
local dns = require("xray.dns")
assert(dns.Servers[1].ID == "localhost")
serverIPs, _, serverErr = dns.Servers[1]:Query("127.0.0.1", true, false, false)
clientIPs, _, clientErr = dns.Query("127.0.0.1", true, false, false)
assert(not serverErr and not clientErr)
assert(#serverIPs == 1 and #clientIPs == 1)
assert(serverIPs[1]:String() == "127.0.0.1" and serverIPs[1]:Equal(clientIPs[1]))
`); err != nil {
t.Fatal(err)
}
for _, name := range []string{"serverIPs", "clientIPs"} {
ips := L.GetGlobal(name).(*lua.LUserData).Value.([]net.IP)
if len(ips) != 1 || !ips[0].Equal(net.ParseIP("127.0.0.1")) {
t.Fatalf("%s = %v", name, ips)
}
}
}
func TestLuaDNSQueryEmptyIPs(t *testing.T) {
for _, tc := range []struct {
name string
ips []net.IP
}{
{"nil", nil},
{"empty", []net.IP{}},
} {
t.Run(tc.name, func(t *testing.T) {
L := lua.NewState()
defer L.Close()
L.SetContext(context.Background())
L.SetGlobal("expectNil", lua.LBool(tc.ips == nil))
client := &luaDNSClient{lookup: func(string, featureDNS.IPOption) ([]net.IP, uint32, error) {
return tc.ips, 0, featureDNS.ErrEmptyResponse
}}
registerLua(L, []luaDNSServer{{query: func(_ context.Context, domain string, option featureDNS.IPOption) ([]net.IP, uint32, error) {
return client.LookupIP(domain, option)
}}}, client)
if err := L.DoString(`
local dns = require("xray.dns")
for _, query in ipairs({
function() return dns.Servers[1]:Query("empty.example", true, false, false) end,
function() return dns.Query("empty.example", true, false, false) end,
}) do
local ips, ttl, err = query()
assert(ttl == 0 and err)
if expectNil then
assert(ips == nil)
else
assert(type(ips) == "userdata" and #ips == 0)
assert(not pcall(function() return ips[1] end))
end
end
`); err != nil {
t.Fatal(err)
}
})
}
}
type benchmarkLuaNameServer struct {
ips []net.IP
}
func (*benchmarkLuaNameServer) Name() string { return "benchmark" }
func (*benchmarkLuaNameServer) IsDisableCache() bool { return true }
func (s *benchmarkLuaNameServer) QueryIP(context.Context, string, featureDNS.IPOption) ([]net.IP, uint32, error) {
return s.ips, 60, nil
}
+2 -1
View File
@@ -29,6 +29,7 @@ type Server interface {
// Client is the interface for DNS client. // Client is the interface for DNS client.
type Client struct { type Client struct {
id string
server Server server Server
skipFallback bool skipFallback bool
expectedIPs geodata.IPMatcher expectedIPs geodata.IPMatcher
@@ -97,7 +98,7 @@ func NewClient(
ipOption dns.IPOption, ipOption dns.IPOption,
updateRules func(bool), updateRules func(bool),
) (*Client, error) { ) (*Client, error) {
client := &Client{} client := &Client{id: ns.Id}
err := core.RequireFeatures(ctx, func(dispatcher routing.Dispatcher) error { err := core.RequireFeatures(ctx, func(dispatcher routing.Dispatcher) error {
// Create a new server for each client for now // Create a new server for each client for now
server, err := NewServer(ctx, ns.Address.AsDestination(), dispatcher, disableCache, serveStale, serveExpiredTTL, clientIP) server, err := NewServer(ctx, ns.Address.AsDestination(), dispatcher, disableCache, serveStale, serveExpiredTTL, clientIP)
+1 -1
View File
@@ -49,5 +49,5 @@ func NewLocalNameServer() *LocalNameServer {
// NewLocalDNSClient creates localdns client object for directly lookup in system DNS. // NewLocalDNSClient creates localdns client object for directly lookup in system DNS.
func NewLocalDNSClient(ipOption dns.IPOption) *Client { func NewLocalDNSClient(ipOption dns.IPOption) *Client {
return &Client{server: NewLocalNameServer(), ipOption: &ipOption} return &Client{id: "localhost", server: NewLocalNameServer(), ipOption: &ipOption}
} }
+70
View File
@@ -0,0 +1,70 @@
package dns
import (
"time"
"github.com/xtls/xray-core/common/errors"
"github.com/xtls/xray-core/common/geodata"
"github.com/xtls/xray-core/common/log"
xlua "github.com/xtls/xray-core/common/lua"
"github.com/xtls/xray-core/common/net"
"github.com/xtls/xray-core/features/dns"
lua "github.com/yuin/gopher-lua"
)
const scriptExecutionTimeout = 6 * time.Second
type scriptEngine struct {
program *xlua.Program
pool *xlua.Pool
}
func newScriptEngine(script string) (*scriptEngine, error) {
program, err := xlua.Compile(script)
if err != nil {
return nil, err
}
return &scriptEngine{program: program}, nil
}
func (e *scriptEngine) start(server *DNS) error {
pool, err := xlua.NewPool(server.ctx, scriptExecutionTimeout, e.program.NewStateFactory(
scriptExecutionTimeout*20,
func(L *lua.LState) {
geodata.RegisterLua(L)
log.RegisterLua(L)
server.registerLua(L)
},
func(L *lua.LState) error {
if L.GetGlobal("HandleDNSQuery").Type() != lua.LTFunction {
return errors.New("DNS script must define HandleDNSQuery(...)")
}
return nil
}))
if err != nil {
return err
}
e.pool = pool
errors.LogInfo(server.ctx, "DNS script initialized")
return nil
}
func (e *scriptEngine) close() {
if e.pool != nil {
e.pool.Close()
}
}
func (e *scriptEngine) query(domain string, option dns.IPOption) (ips []net.IP, ttl uint32, queryErr error) {
if err := e.pool.WithState(nil, 0, func(L *lua.LState) error {
if err := callLuaQuery(L, domain, option); err != nil {
return err
}
ips, ttl, queryErr = readLuaQueryResult(L)
return nil
}); err != nil {
return nil, 0, err
}
return ips, ttl, queryErr
}
+329
View File
@@ -0,0 +1,329 @@
package dns
import (
"context"
go_errors "errors"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/xtls/xray-core/common/net"
featureDNS "github.com/xtls/xray-core/features/dns"
)
type scriptNameServer struct {
name string
answers map[string]net.IP
errors map[string]error
ttl uint32
calls int
}
func (s *scriptNameServer) Name() string { return s.name }
func (s *scriptNameServer) IsDisableCache() bool { return true }
func (s *scriptNameServer) QueryIP(ctx context.Context, domain string, _ featureDNS.IPOption) ([]net.IP, uint32, error) {
if err := ctx.Err(); err != nil {
return nil, 0, err
}
s.calls++
if err := s.errors[domain]; err != nil {
return nil, 0, err
}
ip, ok := s.answers[domain]
if !ok {
return nil, 0, featureDNS.ErrEmptyResponse
}
return []net.IP{ip}, s.ttl, nil
}
func TestDNSScriptQuery(t *testing.T) {
wantIP := net.ParseIP("127.0.0.1")
upstreamErr := go_errors.New("upstream failed")
for _, tc := range []struct {
name, body string
wantIPs []net.IP
wantTTL uint32
wantErr error
wantMessage string
wantCalls uint32
}{
{name: "IPs", body: `return server:Query(domain, ipv4, ipv6, fake)`, wantIPs: []net.IP{wantIP}, wantTTL: 60, wantCalls: 2},
{name: "empty result", body: `return nil, 0`, wantErr: featureDNS.ErrEmptyResponse, wantCalls: 2},
{name: "upstream error", body: `return server:Query("failed.example", ipv4, ipv6, fake)`, wantErr: upstreamErr, wantCalls: 2},
{name: "string error", body: `return nil, nil, "blocked"`, wantMessage: "blocked", wantCalls: 2},
{name: "invalid result", body: `return false, 0`, wantMessage: "native IP slice", wantCalls: 2},
{name: "execution error", body: `error("execution failed")`, wantMessage: "execution failed", wantCalls: 1},
} {
t.Run(tc.name, func(t *testing.T) {
script := `
local server = require("xray.dns").Servers[1]
local calls = 0
function HandleDNSQuery(domain, ipv4, ipv6, fake)
calls = calls + 1
if domain == "count.example" then
local ips, _, err = server:Query("good.example", ipv4, ipv6, fake)
return ips, calls, err
end
` + tc.body + `
end
`
path := filepath.Join(t.TempDir(), "query.lua")
if err := os.WriteFile(path, []byte(script), 0o600); err != nil {
t.Fatal(err)
}
option := featureDNS.IPOption{IPv4Enable: true}
upstream := &scriptNameServer{
name: "test",
answers: map[string]net.IP{"good.example": wantIP},
errors: map[string]error{"failed.example": upstreamErr},
ttl: 60,
}
server := &DNS{
ctx: context.Background(),
clients: []*Client{{server: upstream, ipOption: &option, timeoutMs: time.Second}},
}
engine, err := newScriptEngine(path)
if err != nil {
t.Fatal(err)
}
if err := engine.start(server); err != nil {
t.Fatal(err)
}
defer engine.close()
ips, ttl, err := engine.query("good.example", option)
switch {
case tc.wantErr != nil:
if err != tc.wantErr {
t.Fatalf("query error = %v, want original error %v", err, tc.wantErr)
}
case tc.wantMessage != "":
if err == nil || !strings.Contains(err.Error(), tc.wantMessage) {
t.Fatalf("query error = %v, want %q", err, tc.wantMessage)
}
case err != nil:
t.Fatal(err)
}
if ttl != tc.wantTTL || len(ips) != len(tc.wantIPs) {
t.Fatalf("query = %v, TTL %d; want %v, TTL %d", ips, ttl, tc.wantIPs, tc.wantTTL)
}
for i := range ips {
if !ips[i].Equal(tc.wantIPs[i]) {
t.Fatalf("IP %d = %v, want %v", i, ips[i], tc.wantIPs[i])
}
}
ips, calls, err := engine.query("count.example", option)
if err != nil || calls != tc.wantCalls || len(ips) != 1 || !ips[0].Equal(wantIP) {
t.Fatalf("next query = %v, calls %d, %v; want %v, calls %d", ips, calls, err, wantIP, tc.wantCalls)
}
})
}
}
func TestDNSScriptGeoIPFallback(t *testing.T) {
t.Setenv("xray.location.asset", filepath.Join("..", "..", "resources"))
script := `
local servers = require("xray.dns").Servers
local us_ips = require("xray.geodata").BuildIPMatcher("geoip:us")
local by_id = {}
for _, server in ipairs(servers) do
by_id[server.ID] = server
end
assert(by_id.primary and by_id.fallback, "primary and fallback DNS servers are required")
function HandleDNSQuery(domain, ipv4, ipv6, fake)
local ips, ttl, err = by_id.primary:Query(domain, ipv4, ipv6, fake)
if not err and us_ips:AnyMatch(ips) then
return ips, ttl, nil
end
return by_id.fallback:Query(domain, ipv4, ipv6, fake)
end
`
scriptPath := filepath.Join(t.TempDir(), "geoip_fallback.lua")
if err := os.WriteFile(scriptPath, []byte(script), 0o600); err != nil {
t.Fatal(err)
}
primary := &scriptNameServer{
name: "primary",
answers: map[string]net.IP{
"us.example": net.ParseIP("2001:4860:4860::8888"),
"other.example": net.ParseIP("127.0.0.1"),
},
ttl: 30,
}
fallback := &scriptNameServer{
name: "fallback",
answers: map[string]net.IP{"other.example": net.ParseIP("9.9.9.9")},
ttl: 60,
}
option := featureDNS.IPOption{IPv4Enable: true, IPv6Enable: true}
hosts, err := NewStaticHosts(nil)
if err != nil {
t.Fatal(err)
}
engine, err := newScriptEngine(scriptPath)
if err != nil {
t.Fatal(err)
}
server := &DNS{
ctx: context.Background(),
hosts: hosts,
ipOption: &option,
script: engine,
clients: []*Client{
{id: "primary", server: primary, ipOption: &option, timeoutMs: 2 * time.Second},
{id: "fallback", server: fallback, ipOption: &option, timeoutMs: 2 * time.Second},
},
}
if err := server.Start(); err != nil {
t.Fatal(err)
}
defer server.Close()
for _, tc := range []struct {
domain string
ip net.IP
ttl uint32
}{
{"Us.Example.", net.ParseIP("2001:4860:4860::8888"), 30},
{"other.example", net.ParseIP("9.9.9.9"), 60},
} {
ips, ttl, err := server.LookupIP(tc.domain, option)
if err != nil {
t.Fatalf("LookupIP(%q): %v", tc.domain, err)
}
if ttl != tc.ttl || len(ips) != 1 || !ips[0].Equal(tc.ip) {
t.Fatalf("LookupIP(%q) = %v, TTL %d; want %v, TTL %d", tc.domain, ips, ttl, tc.ip, tc.ttl)
}
}
if primary.calls != 2 || fallback.calls != 1 {
t.Fatalf("upstream calls: primary %d, fallback %d; want 2 and 1", primary.calls, fallback.calls)
}
}
func TestDNSScriptRejectsInvalidStartup(t *testing.T) {
for _, tc := range []struct {
name, script string
compileError bool
}{
{"syntax", "function HandleDNSQuery(", true},
{"missing hook", "value = 1", false},
{"invalid hook", "HandleDNSQuery = true", false},
{"top-level error", `error("setup failed")`, false},
} {
t.Run(tc.name, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "script.lua")
if err := os.WriteFile(path, []byte(tc.script), 0o600); err != nil {
t.Fatal(err)
}
server, err := New(context.Background(), &Config{Script: path})
if tc.compileError {
if server != nil || err == nil || !strings.Contains(err.Error(), "failed to prepare DNS script") {
t.Fatalf("New = %v, %v; want nil server and script preparation error", server, err)
}
return
}
if err != nil {
t.Fatal(err)
}
defer server.Close()
if server.script.program == nil || server.script.pool != nil {
t.Fatal("New did not compile the script without starting its pool")
}
if err := server.Close(); err != nil {
t.Fatal(err)
}
if err := server.Start(); err == nil {
t.Fatal("Start accepted an invalid DNS script")
}
if server.script.pool != nil {
t.Fatal("Start retained a script pool after failure")
}
})
}
}
func TestDNSScriptProgramSnapshot(t *testing.T) {
path := filepath.Join(t.TempDir(), "script.lua")
if err := os.WriteFile(path, []byte(`function HandleDNSQuery() return nil, nil, "compiled" end`), 0o600); err != nil {
t.Fatal(err)
}
engine, err := newScriptEngine(path)
if err != nil {
t.Fatal(err)
}
defer engine.close()
if err := os.Remove(path); err != nil {
t.Fatal(err)
}
if err := engine.start(&DNS{ctx: context.Background()}); err != nil {
t.Fatal(err)
}
if _, _, err := engine.query("example.com", featureDNS.IPOption{}); err == nil || !strings.Contains(err.Error(), "compiled") {
t.Fatalf("query error = %v, want compiled", err)
}
}
func TestDNSScriptFakeDNSOption(t *testing.T) {
path := filepath.Join(t.TempDir(), "script.lua")
script := `
local server = require("xray.dns").Servers[1]
local log = require("xray.log")
log.Info("DNS script loaded")
function HandleDNSQuery(domain, ipv4, ipv6, fake)
log.Debug("DNS query: ", domain)
local ips, ttl, err = server:Query(domain, ipv4, ipv6, fake)
if err then log.Error("DNS failed: ", err) end
return ips, ttl, err
end
`
if err := os.WriteFile(path, []byte(script), 0o600); err != nil {
t.Fatal(err)
}
option := featureDNS.IPOption{IPv4Enable: true}
hosts, err := NewStaticHosts(nil)
if err != nil {
t.Fatal(err)
}
upstream := &scriptNameServer{
name: "FakeDNS",
answers: map[string]net.IP{"good.example": net.ParseIP("198.18.0.1")},
ttl: 30,
}
engine, err := newScriptEngine(path)
if err != nil {
t.Fatal(err)
}
server := &DNS{
ctx: context.Background(),
hosts: hosts,
ipOption: &option,
script: engine,
clients: []*Client{{id: "fake", server: upstream, ipOption: &option, timeoutMs: time.Second}},
}
if err := server.Start(); err != nil {
t.Fatal(err)
}
defer server.Close()
if _, _, err := server.LookupIP("good.example", option); err != featureDNS.ErrEmptyResponse {
t.Fatalf("FakeDNS without FakeEnable = %v, want ErrEmptyResponse", err)
}
if upstream.calls != 0 {
t.Fatalf("FakeDNS was queried without FakeEnable: %d calls", upstream.calls)
}
withFake := featureDNS.IPOption{IPv4Enable: true, FakeEnable: true}
ips, ttl, err := server.LookupIP("good.example", withFake)
if err != nil || ttl != 30 || len(ips) != 1 || !ips[0].Equal(net.ParseIP("198.18.0.1")) {
t.Fatalf("FakeDNS with FakeEnable = %v, TTL %d, %v", ips, ttl, err)
}
if upstream.calls != 1 {
t.Fatalf("FakeDNS query count = %d, want 1", upstream.calls)
}
}
+7
View File
@@ -23,6 +23,9 @@ import (
const idleTimeout = 30 * time.Second const idleTimeout = 30 * time.Second
// DO NOT EDIT. Ignored by geodata, rejected by Lua.
const luaGuardHeader = "\x12\x01\x00"
type stage struct { type stage struct {
target string target string
temp string temp string
@@ -170,6 +173,10 @@ func (d *downloader) downloadOne(asset *Asset) (stage, error) {
} }
}() }()
if _, err := temp.WriteString(luaGuardHeader); err != nil {
temp.Close()
return stage{}, err
}
if err := d.fetch(asset.Url, temp); err != nil { if err := d.fetch(asset.Url, temp); err != nil {
temp.Close() temp.Close()
return stage{}, err return stage{}, err
+73
View File
@@ -0,0 +1,73 @@
package geodata
import (
"context"
"errors"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
commongeodata "github.com/xtls/xray-core/common/geodata"
xlua "github.com/xtls/xray-core/common/lua"
"github.com/yuin/gopher-lua/parse"
"google.golang.org/protobuf/proto"
)
func TestDownloadOneLuaGuard(t *testing.T) {
for _, tc := range []struct {
name string
body string
code string
}{
{"geodata", "\x0a\x03\x0a\x01A", "A"},
{"prefixed geodata", luaGuardHeader + "\x0a\x03\x0a\x01A", "A"},
{"Lua", "function HandleRoute() end", ""},
} {
t.Run(tc.name, func(t *testing.T) {
dir := t.TempDir()
t.Setenv("xray.location.asset", dir)
if err := os.WriteFile(filepath.Join(dir, "asset.lua"), nil, 0o600); err != nil {
t.Fatal(err)
}
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(tc.body))
}))
defer server.Close()
d := &downloader{ctx: context.Background(), httpClient: server.Client()}
staged, err := d.downloadOne(&Asset{Url: server.URL, File: "asset.lua"})
if err != nil {
t.Fatal(err)
}
tx, err := swapAll([]stage{staged})
if err != nil {
t.Fatal(err)
}
if err := tx.commit(); err != nil {
t.Fatal(err)
}
data, err := os.ReadFile(staged.target)
if err != nil || string(data) != luaGuardHeader+tc.body {
t.Fatalf("downloaded data = % x, %v", data, err)
}
program, err := xlua.Compile(staged.target)
var syntaxErr *parse.Error
if program != nil || !errors.As(err, &syntaxErr) || syntaxErr.Pos.Line != 1 || syntaxErr.Pos.Column != 1 {
t.Fatalf("want Lua rejection at the first byte, got %v, %v", program, err)
}
if tc.code != "" {
var list commongeodata.GeoSiteList
if err := proto.Unmarshal(data, &list); err != nil {
t.Fatal(err)
}
if len(list.Entry) != 1 || list.Entry[0].Code != tc.code {
t.Fatalf("header changed geodata: %v", &list)
}
if _, err := commongeodata.ParseDomainRules([]string{"ext:" + filepath.Base(staged.target) + ":" + tc.code}, commongeodata.Domain_Full); err != nil {
t.Fatalf("geodata loader rejected the header: %v", err)
}
}
})
}
}
+1 -1
View File
@@ -174,7 +174,7 @@ func NewHandler(ctx context.Context, config *core.InboundHandlerConfig) (inbound
Mark: streamSettings.SocketSettings.Mark, Mark: streamSettings.SocketSettings.Mark,
}) })
} }
if streamSettings != nil && streamSettings.ProtocolName == "splithttp" { if streamSettings != nil && (streamSettings.ProtocolName == "splithttp" || streamSettings.ProtocolName == "masque") {
ctx = session.ContextWithAllowedNetwork(ctx, net.Network_UDP) ctx = session.ContextWithAllowedNetwork(ctx, net.Network_UDP)
} }
+11 -2
View File
@@ -587,6 +587,7 @@ type Config struct {
DomainStrategy Config_DomainStrategy `protobuf:"varint,1,opt,name=domain_strategy,json=domainStrategy,proto3,enum=xray.app.router.Config_DomainStrategy" json:"domain_strategy,omitempty"` DomainStrategy Config_DomainStrategy `protobuf:"varint,1,opt,name=domain_strategy,json=domainStrategy,proto3,enum=xray.app.router.Config_DomainStrategy" json:"domain_strategy,omitempty"`
Rule []*RoutingRule `protobuf:"bytes,2,rep,name=rule,proto3" json:"rule,omitempty"` Rule []*RoutingRule `protobuf:"bytes,2,rep,name=rule,proto3" json:"rule,omitempty"`
BalancingRule []*BalancingRule `protobuf:"bytes,3,rep,name=balancing_rule,json=balancingRule,proto3" json:"balancing_rule,omitempty"` BalancingRule []*BalancingRule `protobuf:"bytes,3,rep,name=balancing_rule,json=balancingRule,proto3" json:"balancing_rule,omitempty"`
Script string `protobuf:"bytes,4,opt,name=script,proto3" json:"script,omitempty"`
unknownFields protoimpl.UnknownFields unknownFields protoimpl.UnknownFields
sizeCache protoimpl.SizeCache sizeCache protoimpl.SizeCache
} }
@@ -642,6 +643,13 @@ func (x *Config) GetBalancingRule() []*BalancingRule {
return nil return nil
} }
func (x *Config) GetScript() string {
if x != nil {
return x.Script
}
return ""
}
var File_app_router_config_proto protoreflect.FileDescriptor var File_app_router_config_proto protoreflect.FileDescriptor
const file_app_router_config_proto_rawDesc = "" + const file_app_router_config_proto_rawDesc = "" +
@@ -699,11 +707,12 @@ const file_app_router_config_proto_rawDesc = "" +
"\tbaselines\x18\x03 \x03(\x03R\tbaselines\x12\x1a\n" + "\tbaselines\x18\x03 \x03(\x03R\tbaselines\x12\x1a\n" +
"\bexpected\x18\x04 \x01(\x05R\bexpected\x12\x16\n" + "\bexpected\x18\x04 \x01(\x05R\bexpected\x12\x16\n" +
"\x06maxRTT\x18\x05 \x01(\x03R\x06maxRTT\x12\x1c\n" + "\x06maxRTT\x18\x05 \x01(\x03R\x06maxRTT\x12\x1c\n" +
"\ttolerance\x18\x06 \x01(\x02R\ttolerance\"\x96\x02\n" + "\ttolerance\x18\x06 \x01(\x02R\ttolerance\"\xae\x02\n" +
"\x06Config\x12O\n" + "\x06Config\x12O\n" +
"\x0fdomain_strategy\x18\x01 \x01(\x0e2&.xray.app.router.Config.DomainStrategyR\x0edomainStrategy\x120\n" + "\x0fdomain_strategy\x18\x01 \x01(\x0e2&.xray.app.router.Config.DomainStrategyR\x0edomainStrategy\x120\n" +
"\x04rule\x18\x02 \x03(\v2\x1c.xray.app.router.RoutingRuleR\x04rule\x12E\n" + "\x04rule\x18\x02 \x03(\v2\x1c.xray.app.router.RoutingRuleR\x04rule\x12E\n" +
"\x0ebalancing_rule\x18\x03 \x03(\v2\x1e.xray.app.router.BalancingRuleR\rbalancingRule\"B\n" + "\x0ebalancing_rule\x18\x03 \x03(\v2\x1e.xray.app.router.BalancingRuleR\rbalancingRule\x12\x16\n" +
"\x06script\x18\x04 \x01(\tR\x06script\"B\n" +
"\x0eDomainStrategy\x12\b\n" + "\x0eDomainStrategy\x12\b\n" +
"\x04AsIs\x10\x00\x12\x10\n" + "\x04AsIs\x10\x00\x12\x10\n" +
"\fIpIfNonMatch\x10\x02\x12\x0e\n" + "\fIpIfNonMatch\x10\x02\x12\x0e\n" +
+1
View File
@@ -110,4 +110,5 @@ message Config {
DomainStrategy domain_strategy = 1; DomainStrategy domain_strategy = 1;
repeated RoutingRule rule = 2; repeated RoutingRule rule = 2;
repeated BalancingRule balancing_rule = 3; repeated BalancingRule balancing_rule = 3;
string script = 4;
} }
+175
View File
@@ -0,0 +1,175 @@
package router
import (
"runtime"
"strings"
"github.com/xtls/xray-core/common/errors"
xlua "github.com/xtls/xray-core/common/lua"
"github.com/xtls/xray-core/common/net"
"github.com/xtls/xray-core/features/routing"
lua "github.com/yuin/gopher-lua"
)
const (
luaContextType = "xray.router.Context"
luaAttributesType = "xray.router.Attributes"
)
// RegisterLua makes xray.router available to routing scripts.
func (r *Router) RegisterLua(L *lua.LState) {
registerLuaContext(L)
L.PreloadModule("xray.router", func(L *lua.LState) int {
module := L.CreateTable(0, 7)
module.RawSetString("NetworkUnknown", lua.LNumber(net.Network_Unknown))
module.RawSetString("NetworkTCP", lua.LNumber(net.Network_TCP))
module.RawSetString("NetworkUDP", lua.LNumber(net.Network_UDP))
module.RawSetString("NetworkUNIX", lua.LNumber(net.Network_UNIX))
module.RawSetString("LocalOS", lua.LString(runtime.GOOS))
module.RawSetString("PickOutbound", L.NewFunction(func(L *lua.LState) int {
tag, ok := L.Get(1).(lua.LString)
if !ok {
L.ArgError(1, "balancer tag must be a string")
return 0
}
balancer, found := (*r.balancers.Load())[string(tag)]
if !found {
xlua.PushNil(L)
xlua.PushError(L, errors.New("balancer ", tag, " not found"))
return 2
}
outboundTag, err := balancer.PickOutbound()
xlua.PushString(L, outboundTag)
xlua.PushError(L, err)
return 2
}))
module.RawSetString("FindProcess", L.NewFunction(func(L *lua.LState) int {
pid, name, path, err := findProcess(checkLuaContext(L), net.FindProcess)
xlua.PushNumber(L, pid)
xlua.PushString(L, name)
xlua.PushString(L, path)
xlua.PushError(L, err)
return 4
}))
L.Push(module)
return 1
})
}
func registerLuaContext(L *lua.LState) {
pushIPs := xlua.NewSlicePusher[net.IP](L)
attributes := L.NewTypeMetatable(luaAttributesType)
L.SetField(attributes, "__index", L.NewFunction(func(L *lua.LState) int {
values := L.CheckUserData(1).Value.(map[string]string)
key := L.CheckString(2)
if value, found := values[key]; found {
xlua.PushString(L, value)
} else {
xlua.PushNil(L)
}
return 1
}))
methods := L.CreateTable(0, 4)
L.SetFuncs(methods, map[string]lua.LGFunction{
"GetSourceIPs": func(L *lua.LState) int {
pushIPs(L, checkLuaContext(L).GetSourceIPs())
return 1
},
"GetTargetIPs": func(L *lua.LState) int {
pushIPs(L, checkLuaContext(L).GetTargetIPs())
return 1
},
"GetLocalIPs": func(L *lua.LState) int {
pushIPs(L, checkLuaContext(L).GetLocalIPs())
return 1
},
"GetAttributes": func(L *lua.LState) int {
values := L.NewUserData()
values.Value = checkLuaContext(L).GetAttributes()
L.SetMetatable(values, attributes)
L.Push(values)
return 1
},
})
L.SetField(L.NewTypeMetatable(luaContextType), "__index", methods)
}
func checkLuaContext(L *lua.LState) routing.Context {
ctx, ok := L.CheckUserData(1).Value.(routing.Context)
if !ok {
L.ArgError(1, "routing context expected")
}
return ctx
}
// callLuaRoute runs HandleRoute and leaves (outboundTag, ruleTag, err) on the stack.
func callLuaRoute(L *lua.LState, ctx routing.Context) error {
fn := L.GetGlobal("HandleRoute")
if fn.Type() != lua.LTFunction {
return errors.New("routing script must define HandleRoute(...)")
}
value := L.NewUserData()
value.Value = ctx
L.SetMetatable(value, L.GetTypeMetatable(luaContextType))
return L.CallByParam(lua.P{Fn: fn, NRet: 3, Protect: true},
value,
lua.LString(ctx.GetInboundTag()),
lua.LNumber(ctx.GetSourcePort()),
lua.LNumber(ctx.GetTargetPort()),
lua.LNumber(ctx.GetLocalPort()),
lua.LString(strings.ToLower(ctx.GetTargetDomain())),
lua.LNumber(ctx.GetNetwork()),
lua.LString(ctx.GetProtocol()),
lua.LString(ctx.GetUser()),
lua.LNumber(ctx.GetVlessRoute()),
lua.LBool(ctx.GetSkipDNSResolve()))
}
// readLuaRouteResult reads (outboundTag, ruleTag, err) from the stack.
func readLuaRouteResult(L *lua.LState) (string, string, error) {
if err := xlua.ReadError(L.Get(-1), "routing script error must be an error or string"); err != nil {
return "", "", err
}
outboundTag, err := xlua.ReadOptionalString(L.Get(-3), "routing script outboundTag must be a string or nil")
if err != nil || outboundTag == "" {
return "", "", err
}
ruleTag, err := xlua.ReadOptionalString(L.Get(-2), "routing script ruleTag must be a string")
if err != nil {
return "", "", err
}
return outboundTag, ruleTag, nil
}
type processFinder func(string, string, uint16, string, uint16) (int, string, string, error)
func findProcess(ctx routing.Context, finder processFinder) (int, string, string, error) {
sources := ctx.GetSourceIPs()
if len(sources) == 0 {
return 0, "", "", errors.New("process lookup requires a source IP")
}
var network string
switch ctx.GetNetwork() {
case net.Network_TCP:
network = "tcp"
case net.Network_UDP:
network = "udp"
default:
return 0, "", "", errors.New("process lookup requires TCP or UDP")
}
targetIP, targetPort := "", uint16(0)
if targets := ctx.GetTargetIPs(); len(targets) > 0 {
targetIP, targetPort = targets[0].String(), uint16(ctx.GetTargetPort())
}
return finder(network, sources[0].String(), uint16(ctx.GetSourcePort()), targetIP, targetPort)
}
+223
View File
@@ -0,0 +1,223 @@
package router
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"github.com/xtls/xray-core/common/geodata"
"github.com/xtls/xray-core/common/net"
"github.com/xtls/xray-core/common/session"
"github.com/xtls/xray-core/features/routing"
routing_session "github.com/xtls/xray-core/features/routing/session"
lua "github.com/yuin/gopher-lua"
)
func benchmarkRouteContext(target net.Destination) *routing_session.Context {
// Use the production context: its IP getters construct a slice per call.
// The cached IP slices in luaRouteTestContext would undercount this cost.
return &routing_session.Context{
Inbound: &session.Inbound{
Tag: "in",
Source: net.TCPDestination(net.LocalHostIP, 1234),
Local: net.TCPDestination(net.LocalHostIP, 5678),
},
Outbound: &session.Outbound{Target: target},
Content: &session.Content{Protocol: "tls"},
}
}
func benchmarkRouteState(b *testing.B, r *Router, script string) *lua.LState {
b.Helper()
L := lua.NewState()
b.Cleanup(L.Close)
r.RegisterLua(L)
geodata.RegisterLua(L)
if err := L.DoString(script); err != nil {
b.Fatal(err)
}
L.SetContext(context.Background())
return L
}
// BenchmarkLuaRouteHook isolates argument bridging and a fixed-return hook.
// It excludes rules, result decoding, the state pool, and Route construction.
func BenchmarkLuaRouteHook(b *testing.B) {
L := benchmarkRouteState(b, new(Router), `
function HandleRoute(ctx, inboundTag, sourcePort, targetPort, localPort,
targetDomain, network, protocol, user, vlessRoute, skipDNSResolve)
return "out", "rule"
end
`)
ctx := benchmarkRouteContext(net.TCPDestination(net.LocalHostIP, 443))
if err := callLuaRoute(L, ctx); err != nil {
b.Fatal(err)
}
outboundTag, ruleTag, err := readLuaRouteResult(L)
L.Pop(3)
if err != nil || outboundTag != "out" || ruleTag != "rule" {
b.Fatalf("hook() = %q, %q, %v", outboundTag, ruleTag, err)
}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
if err := callLuaRoute(L, ctx); err != nil {
b.Fatal(err)
}
L.Pop(3)
}
}
// BenchmarkLuaRoute compares equivalent ordered rules on the same session.
// rules returns tags only; pick_route uses Router.PickRoute on both sides.
// All compilation, matcher construction, and pool startup are outside timing.
func BenchmarkLuaRoute(b *testing.B) {
for _, name := range []string{"scalar", "ip", "domain", "domain_32_last"} {
b.Run(name, func(b *testing.B) {
config, script, ctx, wantTag, wantRule := benchmarkRouteFixture(b, name)
native := new(Router)
if err := native.Init(context.Background(), config, nil, nil, nil); err != nil {
b.Fatal(err)
}
L := benchmarkRouteState(b, native, script)
path := filepath.Join(b.TempDir(), "route.lua")
if err := os.WriteFile(path, []byte(script), 0o600); err != nil {
b.Fatal(err)
}
scripted := new(Router)
if err := scripted.Init(context.Background(), &Config{Script: path}, nil, nil, nil); err != nil {
b.Fatal(err)
}
if err := scripted.Start(); err != nil {
b.Fatal(err)
}
b.Cleanup(func() {
if err := scripted.Close(); err != nil {
b.Error(err)
}
})
for _, bench := range []struct {
name string
route func() (string, string, error)
}{
{"rules/native", func() (string, string, error) {
rule, _, err := native.pickRouteInternal(ctx)
if err != nil {
return "", "", err
}
tag, err := rule.GetTag()
return tag, rule.RuleTag, err
}},
{"rules/lua", func() (string, string, error) {
if err := callLuaRoute(L, ctx); err != nil {
return "", "", err
}
tag, ruleTag, err := readLuaRouteResult(L)
L.Pop(3)
return tag, ruleTag, err
}},
{"pick_route/native", func() (string, string, error) {
return benchmarkPickRoute(native, ctx)
}},
{"pick_route/lua", func() (string, string, error) {
return benchmarkPickRoute(scripted, ctx)
}},
} {
b.Run(bench.name, func(b *testing.B) {
// Validate and warm both paths before measuring steady state.
tag, ruleTag, err := bench.route()
if err != nil || tag != wantTag || ruleTag != wantRule {
b.Fatalf("route() = %q, %q, %v; want %q, %q", tag, ruleTag, err, wantTag, wantRule)
}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
tag, ruleTag, err = bench.route()
if err != nil {
b.Fatal(err)
}
}
b.StopTimer()
if tag != wantTag || ruleTag != wantRule {
b.Fatalf("route() = %q, %q; want %q, %q", tag, ruleTag, wantTag, wantRule)
}
})
}
})
}
}
func benchmarkPickRoute(r *Router, ctx routing.Context) (string, string, error) {
route, err := r.PickRoute(ctx)
if err != nil {
return "", "", err
}
return route.GetOutboundTag(), route.GetRuleTag(), nil
}
func benchmarkRouteFixture(b *testing.B, name string) (*Config, string, routing.Context, string, string) {
b.Helper()
config := new(Config)
ctx := benchmarkRouteContext(net.TCPDestination(net.LocalHostIP, 443))
prelude := `local router = require("xray.router")
local geodata = require("xray.geodata")
`
body := `if inboundTag == "in" and network == router.NetworkTCP then return "out", "rule" end`
wantTag, wantRule := "out", "rule"
if name == "scalar" || name == "ip" {
rule := &RoutingRule{
TargetTag: &RoutingRule_Tag{Tag: wantTag},
RuleTag: wantRule,
InboundTag: []string{"in"},
Networks: []net.Network{net.Network_TCP},
}
if name == "ip" {
var err error
rule.Ip, err = geodata.ParseIPRules([]string{"127.0.0.0/8"})
if err != nil {
b.Fatal(err)
}
prelude += `local matcher = geodata.BuildIPMatcher("127.0.0.0/8")` + "\n"
body = `if inboundTag == "in" and network == router.NetworkTCP and matcher:AnyMatch(ctx:GetTargetIPs()) then return "out", "rule" end`
}
config.Rule = []*RoutingRule{rule}
} else {
count := 1
if name == "domain_32_last" {
count = 32
}
var rules strings.Builder
rules.WriteString("local rules = {\n")
for i := 0; i < count; i++ {
domain := fmt.Sprintf("route-%d.example.com", i)
tag, ruleTag := fmt.Sprintf("out-%d", i), fmt.Sprintf("rule-%d", i)
domains, err := geodata.ParseDomainRules([]string{"full:" + domain}, geodata.Domain_Domain)
if err != nil {
b.Fatal(err)
}
config.Rule = append(config.Rule, &RoutingRule{
TargetTag: &RoutingRule_Tag{Tag: tag}, RuleTag: ruleTag, Domain: domains,
})
fmt.Fprintf(&rules, "{geodata.BuildDomainMatcher(%q), %q, %q},\n", "full:"+domain, tag, ruleTag)
if i == count-1 {
ctx.Outbound.Target = net.TCPDestination(net.DomainAddress(domain), 443)
wantTag, wantRule = tag, ruleTag
}
}
rules.WriteString("}\n")
prelude += rules.String()
body = `for i = 1, #rules do
local rule = rules[i]
if rule[1]:MatchAny(targetDomain) then return rule[2], rule[3] end
end`
}
script := prelude + `function HandleRoute(ctx, inboundTag, sourcePort, targetPort, localPort,
targetDomain, network, protocol, user, vlessRoute, skipDNSResolve)
` + body + "\nend\n"
return config, script, ctx, wantTag, wantRule
}
+274
View File
@@ -0,0 +1,274 @@
package router
import (
"context"
go_errors "errors"
"runtime"
"strings"
"testing"
"github.com/xtls/xray-core/common/geodata"
"github.com/xtls/xray-core/common/net"
"github.com/xtls/xray-core/common/protocol"
"github.com/xtls/xray-core/common/session"
"github.com/xtls/xray-core/features/routing"
routing_session "github.com/xtls/xray-core/features/routing/session"
lua "github.com/yuin/gopher-lua"
)
type luaRouteTestContext struct {
*routing_session.Context
sourceIPs, targetIPs, localIPs []net.IP
}
func (c *luaRouteTestContext) GetSourceIPs() []net.IP { return c.sourceIPs }
func (c *luaRouteTestContext) GetTargetIPs() []net.IP { return c.targetIPs }
func (c *luaRouteTestContext) GetLocalIPs() []net.IP { return c.localIPs }
func newLuaRouteTestContext() *luaRouteTestContext {
return &luaRouteTestContext{
Context: &routing_session.Context{
Inbound: &session.Inbound{
Tag: "in", VlessRoute: 4321,
Source: net.TCPDestination(net.LocalHostIP, 1234),
Local: net.TCPDestination(net.LocalHostIP, 5678),
User: &protocol.MemoryUser{Email: "user@example.com"},
},
Outbound: &session.Outbound{
Target: net.TCPDestination(net.LocalHostIP, 443),
RouteTarget: net.TCPDestination(net.DomainAddress("MiXeD.Example."), 443),
},
Content: &session.Content{
Protocol: "tls", Attributes: map[string]string{"key": "value"}, SkipDNSResolve: true,
},
},
sourceIPs: []net.IP{{127, 0, 0, 2}},
targetIPs: []net.IP{{127, 0, 0, 3}},
localIPs: []net.IP{{127, 0, 0, 1}},
}
}
func newLuaRouterState(t *testing.T, script string) *lua.LState {
t.Helper()
r := new(Router)
if err := r.Init(context.Background(), &Config{}, nil, nil, nil); err != nil {
t.Fatal(err)
}
L := lua.NewState()
t.Cleanup(L.Close)
r.RegisterLua(L)
geodata.RegisterLua(L)
if err := L.DoString(script); err != nil {
t.Fatal(err)
}
return L
}
func TestLuaRouteBinding(t *testing.T) {
L := newLuaRouterState(t, `
local router = require("xray.router")
local matcher = require("xray.geodata").BuildIPMatcher("127.0.0.0/8")
assert(router.NetworkUnknown == 0 and router.NetworkTCP == 2)
assert(router.NetworkUDP == 3 and router.NetworkUNIX == 4)
assert(router.BuildIPMatcher == nil and router.BuildDomainMatcher == nil)
function HandleRoute(ctx, inboundTag, sourcePort, targetPort, localPort,
targetDomain, network, protocol, user, vlessRoute, skipDNSResolve, ...)
assert(select("#", ...) == 0)
assert(inboundTag == "in" and sourcePort == 1234 and targetPort == 443 and localPort == 5678)
assert(targetDomain == "mixed.example." and network == router.NetworkTCP)
assert(protocol == "tls" and user == "user@example.com" and vlessRoute == 4321 and skipDNSResolve)
assert(ctx.GetNetwork == nil and ctx.Context == nil)
savedContext = ctx
sourceIPs, targetIPs, localIPs = ctx:GetSourceIPs(), ctx:GetTargetIPs(), ctx:GetLocalIPs()
attributes = ctx:GetAttributes()
assert(#sourceIPs == 1 and #targetIPs == 1 and #localIPs == 1)
assert(sourceIPs[1]:String() == "127.0.0.2" and targetIPs[1]:String() == "127.0.0.3")
assert(localIPs[1]:String() == "127.0.0.1")
assert(matcher:Match(sourceIPs[1]) and matcher:Match(targetIPs[1]) and matcher:Match(localIPs[1]))
assert(matcher:AnyMatch(sourceIPs) and matcher:AnyMatch(targetIPs) and matcher:AnyMatch(localIPs))
local matched = matcher:FilterIPs(targetIPs)
assert(#matched == 1 and matched[1]:Equal(targetIPs[1]))
assert(attributes.key == "value" and attributes.missing == nil)
assert(not pcall(function() attributes.key = "changed" end))
return "out", "rule"
end`)
ctx := newLuaRouteTestContext()
if err := callLuaRoute(L, ctx); err != nil {
t.Fatal(err)
}
if L.GetTop() != 3 || L.Get(1) != lua.LString("out") || L.Get(2) != lua.LString("rule") || L.Get(3) != lua.LNil {
t.Fatal("callLuaRoute did not leave the three route results on the stack")
}
if L.GetGlobal("savedContext").(*lua.LUserData).Value != ctx {
t.Fatal("routing context was copied")
}
for _, tc := range []struct {
name string
want []net.IP
}{
{"sourceIPs", ctx.sourceIPs},
{"targetIPs", ctx.targetIPs},
{"localIPs", ctx.localIPs},
} {
got := L.GetGlobal(tc.name).(*lua.LUserData).Value.([]net.IP)
if &got[0] != &tc.want[0] {
t.Fatalf("%s storage was copied", tc.name)
}
}
ctx.Content.Attributes["key"] = "updated"
L.SetGlobal("expectedOS", lua.LString(runtime.GOOS))
if err := L.DoString(`
assert(attributes.key == "updated")
assert(require("xray.router").LocalOS == expectedOS)`); err != nil {
t.Fatal(err)
}
}
func TestLuaRouteEmptyIPs(t *testing.T) {
for _, tc := range []struct {
name string
ips []net.IP
}{
{"nil", nil},
{"empty", []net.IP{}},
} {
t.Run(tc.name, func(t *testing.T) {
L := newLuaRouterState(t, `
function HandleRoute(ctx)
for _, name in ipairs({"GetSourceIPs", "GetTargetIPs", "GetLocalIPs"}) do
local ips = ctx[name](ctx)
if expectNil then
assert(ips == nil)
else
assert(type(ips) == "userdata" and #ips == 0)
assert(not pcall(function() return ips[1] end))
end
end
return "out"
end
`)
L.SetGlobal("expectNil", lua.LBool(tc.ips == nil))
ctx := newLuaRouteTestContext()
ctx.sourceIPs, ctx.targetIPs, ctx.localIPs = tc.ips, tc.ips, tc.ips
if err := callLuaRoute(L, ctx); err != nil {
t.Fatal(err)
}
})
}
}
func TestReadLuaRouteResult(t *testing.T) {
nativeErr := go_errors.New("native failure")
for _, tc := range []struct {
name, values string
wantTag, wantRule string
wantErr error
wantMessage string
}{
{name: "route", values: `"out", "rule"`, wantTag: "out", wantRule: "rule"},
{name: "no match", values: `nil`},
{name: "empty tag", values: `""`},
{name: "no match ignores rule", values: `nil, false`},
{name: "empty tag ignores rule", values: `"", false`},
{name: "missing rule", values: `"out"`, wantTag: "out"},
{name: "invalid tag", values: `1`, wantMessage: "outboundTag"},
{name: "invalid rule", values: `"out", false`, wantMessage: "ruleTag"},
{name: "string error", values: `nil, nil, "script failure"`, wantMessage: "script failure"},
{name: "native error", values: `nil, nil, nativeError`, wantErr: nativeErr},
{name: "error overrides invalid tags", values: `false, false, nativeError`, wantErr: nativeErr},
{name: "invalid error", values: `"out", "rule", false`, wantMessage: "error or string"},
{name: "wrong error userdata", values: `"out", "rule", wrongError`, wantMessage: "error or string"},
} {
t.Run(tc.name, func(t *testing.T) {
L := lua.NewState()
defer L.Close()
for name, value := range map[string]any{"nativeError": nativeErr, "wrongError": "not a native error"} {
ud := L.NewUserData()
ud.Value = value
L.SetGlobal(name, ud)
}
fn, err := L.LoadString("return " + tc.values)
if err != nil {
t.Fatal(err)
}
if err := L.CallByParam(lua.P{Fn: fn, NRet: 3, Protect: true}); err != nil {
t.Fatal(err)
}
outboundTag, ruleTag, err := readLuaRouteResult(L)
if outboundTag != tc.wantTag || ruleTag != tc.wantRule {
t.Fatalf("result = %q, %q, %v; want %q, %q", outboundTag, ruleTag, err, tc.wantTag, tc.wantRule)
}
switch {
case tc.wantErr != nil:
if err != tc.wantErr {
t.Fatalf("error = %v, want original error", err)
}
case tc.wantMessage != "":
if err == nil || !strings.Contains(err.Error(), tc.wantMessage) {
t.Fatalf("error = %v, want %q", err, tc.wantMessage)
}
case err != nil:
t.Fatal(err)
}
})
}
}
func TestCallLuaRouteCancellation(t *testing.T) {
L := newLuaRouterState(t, `function HandleRoute() while true do end end`)
ctx, cancel := context.WithCancel(context.Background())
cancel()
L.SetContext(ctx)
if err := callLuaRoute(L, &routing_session.Context{}); err == nil {
t.Fatal("callLuaRoute did not stop after context cancellation")
}
if L.Context() != ctx {
t.Fatal("callLuaRoute changed the Lua state's context")
}
}
func TestFindProcess(t *testing.T) {
for _, tc := range []struct {
name, network, target string
targetPort uint16
modify func(*luaRouteTestContext)
wantErr bool
}{
{name: "TCP", network: "tcp", target: "127.0.0.3", targetPort: 443},
{name: "UDP", network: "udp", target: "127.0.0.3", targetPort: 443, modify: func(c *luaRouteTestContext) {
c.Outbound.Target.Network = net.Network_UDP
}},
{name: "domain target", network: "tcp", modify: func(c *luaRouteTestContext) { c.targetIPs = nil }},
{name: "missing source", modify: func(c *luaRouteTestContext) { c.sourceIPs = nil }, wantErr: true},
{name: "unsupported network", modify: func(c *luaRouteTestContext) {
c.Outbound.Target.Network = net.Network_UNIX
}, wantErr: true},
} {
t.Run(tc.name, func(t *testing.T) {
ctx := newLuaRouteTestContext()
if tc.modify != nil {
tc.modify(ctx)
}
called := false
pid, name, path, err := findProcess(ctx, func(network, source string, sourcePort uint16, target string, targetPort uint16) (int, string, string, error) {
called = true
if network != tc.network || source != "127.0.0.2" || sourcePort != 1234 || target != tc.target || targetPort != tc.targetPort {
t.Fatalf("endpoints = %s %s:%d -> %s:%d", network, source, sourcePort, target, targetPort)
}
return 42, "process", "/path/process", nil
})
if tc.wantErr {
if err == nil || called {
t.Fatalf("findProcess = %d, %q, %q, %v", pid, name, path, err)
}
return
}
if err != nil || !called || pid != 42 || name != "process" || path != "/path/process" {
t.Fatalf("findProcess = %d, %q, %q, %v", pid, name, path, err)
}
})
}
}
var _ routing.Context = (*luaRouteTestContext)(nil)
+21
View File
@@ -20,6 +20,7 @@ import (
type Router struct { type Router struct {
domainStrategy Config_DomainStrategy domainStrategy Config_DomainStrategy
rules atomic.Pointer[[]*Rule] rules atomic.Pointer[[]*Rule]
script *scriptEngine
balancers atomic.Pointer[map[string]*Balancer] balancers atomic.Pointer[map[string]*Balancer]
dns dns.Client dns dns.Client
@@ -45,6 +46,14 @@ func (r *Router) Init(ctx context.Context, config *Config, d dns.Client, ohm out
r.ohm = ohm r.ohm = ohm
r.dispatcher = dispatcher r.dispatcher = dispatcher
if config.Script != "" {
script, err := newScriptEngine(config.Script)
if err != nil {
return errors.New("failed to prepare routing script").Base(err)
}
r.script = script
}
r.rules.Store(new([]*Rule)) r.rules.Store(new([]*Rule))
r.balancers.Store(&map[string]*Balancer{}) r.balancers.Store(&map[string]*Balancer{})
return r.ReloadRules(config, false) return r.ReloadRules(config, false)
@@ -52,6 +61,10 @@ func (r *Router) Init(ctx context.Context, config *Config, d dns.Client, ohm out
// PickRoute implements routing.Router. // PickRoute implements routing.Router.
func (r *Router) PickRoute(ctx routing.Context) (routing.Route, error) { func (r *Router) PickRoute(ctx routing.Context) (routing.Route, error) {
if r.script != nil {
return r.script.pickRoute(ctx)
}
originalCtx := ctx originalCtx := ctx
rule, ctx, err := r.pickRouteInternal(ctx) rule, ctx, err := r.pickRouteInternal(ctx)
if err != nil { if err != nil {
@@ -221,6 +234,11 @@ func (r *Router) pickRouteInternal(ctx routing.Context) (*Rule, routing.Context,
// Start implements common.Runnable. // Start implements common.Runnable.
func (r *Router) Start() error { func (r *Router) Start() error {
if r.script != nil {
if err := r.script.start(r); err != nil {
return errors.New("failed to initialize routing script").Base(err)
}
}
return nil return nil
} }
@@ -235,6 +253,9 @@ func closeWebhooks(rules []*Rule) {
// Close implements common.Closable. // Close implements common.Closable.
func (r *Router) Close() error { func (r *Router) Close() error {
if r.script != nil {
r.script.close()
}
r.mu.Lock() r.mu.Lock()
defer r.mu.Unlock() defer r.mu.Unlock()
closeWebhooks(*r.rules.Load()) closeWebhooks(*r.rules.Load())
+83
View File
@@ -0,0 +1,83 @@
package router
import (
"time"
"github.com/xtls/xray-core/app/dns"
"github.com/xtls/xray-core/common"
"github.com/xtls/xray-core/common/errors"
"github.com/xtls/xray-core/common/geodata"
"github.com/xtls/xray-core/common/log"
xlua "github.com/xtls/xray-core/common/lua"
"github.com/xtls/xray-core/features/routing"
lua "github.com/yuin/gopher-lua"
)
const scriptExecutionTimeout = 6 * time.Second
type scriptEngine struct {
program *xlua.Program
pool *xlua.Pool
}
func newScriptEngine(script string) (*scriptEngine, error) {
program, err := xlua.Compile(script)
if err != nil {
return nil, err
}
return &scriptEngine{program: program}, nil
}
func (e *scriptEngine) start(router *Router) error {
pool, err := xlua.NewPool(router.ctx, scriptExecutionTimeout, e.program.NewStateFactory(
scriptExecutionTimeout*20,
func(L *lua.LState) {
geodata.RegisterLua(L)
log.RegisterLua(L)
router.RegisterLua(L)
dns.RegisterLua(L, router.dns)
},
func(L *lua.LState) error {
if L.GetGlobal("HandleRoute").Type() != lua.LTFunction {
return errors.New("routing script must define HandleRoute(...)")
}
return nil
}))
if err != nil {
return err
}
e.pool = pool
errors.LogInfo(router.ctx, "routing script initialized")
return nil
}
func (e *scriptEngine) close() {
if e.pool != nil {
e.pool.Close()
}
}
func (e *scriptEngine) pickRoute(ctx routing.Context) (routing.Route, error) {
var outboundTag, ruleTag string
var routeErr error
if err := e.pool.WithState(nil, 0, func(L *lua.LState) error {
if err := callLuaRoute(L, ctx); err != nil {
return err
}
outboundTag, ruleTag, routeErr = readLuaRouteResult(L)
return nil
}); err != nil {
return nil, err
}
if routeErr != nil {
return nil, routeErr
}
if outboundTag == "" {
return nil, common.ErrNoClue
}
return &Route{Context: ctx, outboundTag: outboundTag, ruleTag: ruleTag}, nil
}
+421
View File
@@ -0,0 +1,421 @@
package router
import (
"context"
stdnet "net"
"os"
"path/filepath"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
wireDNS "github.com/miekg/dns"
"github.com/xtls/xray-core/app/dispatcher"
appdns "github.com/xtls/xray-core/app/dns"
"github.com/xtls/xray-core/app/proxyman"
_ "github.com/xtls/xray-core/app/proxyman/outbound"
"github.com/xtls/xray-core/common"
"github.com/xtls/xray-core/common/net"
"github.com/xtls/xray-core/common/serial"
"github.com/xtls/xray-core/core"
featureDNS "github.com/xtls/xray-core/features/dns"
"github.com/xtls/xray-core/features/outbound"
"github.com/xtls/xray-core/features/routing"
routing_session "github.com/xtls/xray-core/features/routing/session"
"github.com/xtls/xray-core/proxy/blackhole"
"github.com/xtls/xray-core/proxy/freedom"
)
type luaRouteDNSClient struct {
featureDNS.Client
lookup func(string, featureDNS.IPOption) ([]net.IP, uint32, error)
}
func (d *luaRouteDNSClient) LookupIP(domain string, option featureDNS.IPOption) ([]net.IP, uint32, error) {
return d.lookup(domain, option)
}
type luaRouteOutboundManager struct{ outbound.Manager }
func (*luaRouteOutboundManager) Select(selectors []string) []string { return selectors }
func writeRouteScript(t *testing.T, script string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "route.lua")
if err := os.WriteFile(path, []byte(script), 0o600); err != nil {
t.Fatal(err)
}
return path
}
func startLuaRouter(t *testing.T, script string, d featureDNS.Client, config *Config) *Router {
t.Helper()
if config == nil {
config = &Config{}
}
config.Script = writeRouteScript(t, script)
r := new(Router)
if err := r.Init(context.Background(), config, d, &luaRouteOutboundManager{}, nil); err != nil {
t.Fatal(err)
}
if err := r.Start(); err != nil {
t.Fatal(err)
}
t.Cleanup(func() {
if err := r.Close(); err != nil {
t.Error(err)
}
})
return r
}
func TestRouterScriptStartup(t *testing.T) {
for _, tc := range []struct {
name, script string
compileError bool
}{
{"syntax error", "function HandleRoute(", true},
{"missing hook", "value = 1", false},
{"invalid hook", "HandleRoute = true", false},
{"initialization error", `error("setup failed")`, false},
} {
t.Run(tc.name, func(t *testing.T) {
r := new(Router)
err := r.Init(context.Background(), &Config{Script: writeRouteScript(t, tc.script)}, nil, nil, nil)
if tc.compileError {
if r.script != nil || err == nil || !strings.Contains(err.Error(), "failed to prepare routing script") {
t.Fatalf("Init = %v; want no engine and script preparation error", err)
}
return
}
if err != nil {
t.Fatal(err)
}
defer r.Close()
if r.script.program == nil || r.script.pool != nil {
t.Fatal("Init did not compile the script without starting its pool")
}
if err := r.Close(); err != nil {
t.Fatal(err)
}
if err := r.Start(); err == nil {
t.Fatal("Start accepted an invalid routing script")
}
if r.script.pool != nil {
t.Fatal("Start retained a script pool after failure")
}
})
}
}
func TestRouterScriptProgramSnapshot(t *testing.T) {
path := writeRouteScript(t, `function HandleRoute() return "compiled", "compiled-rule" end`)
engine, err := newScriptEngine(path)
if err != nil {
t.Fatal(err)
}
defer engine.close()
if err := os.Remove(path); err != nil {
t.Fatal(err)
}
if err := engine.start(&Router{ctx: context.Background()}); err != nil {
t.Fatal(err)
}
route, err := engine.pickRoute(newLuaRouteTestContext())
if err != nil || route == nil || route.GetOutboundTag() != "compiled" || route.GetRuleTag() != "compiled-rule" {
t.Fatalf("route = %v, %v; want compiled, compiled-rule", route, err)
}
}
func TestRouterScriptRouting(t *testing.T) {
for _, tc := range []struct {
name, body string
wantTag, wantRule string
wantErr error
wantMessage string
wantCalls string
}{
{name: "route", body: `return "lua-out", "lua-rule"`, wantTag: "lua-out", wantRule: "lua-rule", wantCalls: "2"},
{name: "no match", body: `return nil`, wantErr: common.ErrNoClue, wantCalls: "2"},
{name: "empty tag", body: `return ""`, wantErr: common.ErrNoClue, wantCalls: "2"},
{name: "balancer error", body: `local tag, err = router.PickOutbound("missing"); return tag, nil, err`, wantMessage: "not found", wantCalls: "2"},
{name: "string error", body: `return nil, nil, "blocked"`, wantMessage: "blocked", wantCalls: "2"},
{name: "invalid tag", body: `return false`, wantMessage: "outboundTag", wantCalls: "2"},
{name: "invalid rule", body: `return "lua-out", false`, wantMessage: "ruleTag", wantCalls: "2"},
{name: "execution error", body: `error("execution failed")`, wantMessage: "execution failed", wantCalls: "1"},
} {
t.Run(tc.name, func(t *testing.T) {
var dnsCalls atomic.Int32
d := &luaRouteDNSClient{lookup: func(string, featureDNS.IPOption) ([]net.IP, uint32, error) {
dnsCalls.Add(1)
return []net.IP{{1, 2, 3, 4}}, 60, nil
}}
script := `
local router = require("xray.router")
local calls = 0
function HandleRoute(ctx, inbound)
calls = calls + 1
if inbound == "count" then return "lua-out", tostring(calls) end
` + tc.body + `
end
`
r := startLuaRouter(t, script, d, &Config{
DomainStrategy: Config_IpOnDemand,
Rule: []*RoutingRule{{
TargetTag: &RoutingRule_Tag{Tag: "json-out"},
Networks: []net.Network{net.Network_TCP},
}},
})
ctx := newLuaRouteTestContext()
ctx.Content.SkipDNSResolve = false
route, err := r.PickRoute(ctx)
switch {
case tc.wantErr != nil:
if err != tc.wantErr {
t.Fatalf("route error = %v, want %v", err, tc.wantErr)
}
case tc.wantMessage != "":
if err == nil || !strings.Contains(err.Error(), tc.wantMessage) {
t.Fatalf("route error = %v, want %q", err, tc.wantMessage)
}
case err != nil:
t.Fatal(err)
}
if tc.wantTag == "" {
if route != nil {
t.Fatalf("route = %v, want nil", route)
}
} else if route == nil || route.GetOutboundTag() != tc.wantTag || route.GetRuleTag() != tc.wantRule || route.(*Route).Context != ctx {
t.Fatalf("route = %v; want %q, %q and original context", route, tc.wantTag, tc.wantRule)
}
ctx.Inbound.Tag = "count"
route, err = r.PickRoute(ctx)
if err != nil || route == nil || route.GetOutboundTag() != "lua-out" || route.GetRuleTag() != tc.wantCalls {
t.Fatalf("next route = %v, %v; want lua-out, calls %s", route, err, tc.wantCalls)
}
if dnsCalls.Load() != 0 {
t.Fatal("script routing implicitly resolved DNS")
}
})
}
}
func TestRouterScriptModules(t *testing.T) {
ips := []net.IP{{127, 0, 0, 7}}
calls := 0
d := &luaRouteDNSClient{lookup: func(domain string, option featureDNS.IPOption) ([]net.IP, uint32, error) {
calls++
if domain != "mixed.example." || !option.IPv4Enable || option.IPv6Enable || !option.FakeEnable {
t.Fatalf("dns.Query arguments = %q, %+v", domain, option)
}
return ips, 17, nil
}}
r := startLuaRouter(t, `
local dns = require("xray.dns")
local matcher = require("xray.geodata").BuildIPMatcher("127.0.0.0/8")
assert(dns.Servers == nil and type(dns.Query) == "function")
assert(type(require("xray.log").Info) == "function")
function HandleRoute(ctx, inbound, sourcePort, targetPort, localPort, domain)
local ips, ttl, err = dns.Query(domain, true, false, true)
assert(not err and ttl == 17)
assert(matcher:AnyMatch(ips) and matcher:AnyMatch(ctx:GetTargetIPs()))
return "out"
end`, d, nil)
if _, err := r.PickRoute(newLuaRouteTestContext()); err != nil {
t.Fatal(err)
}
if calls != 1 {
t.Fatalf("DNS calls = %d, want 1", calls)
}
}
func TestRouterScriptBalancerReload(t *testing.T) {
config := func(tag string) *Config {
return &Config{BalancingRule: []*BalancingRule{{
Tag: "balance", Strategy: "roundrobin", OutboundSelector: []string{tag},
}}}
}
r := startLuaRouter(t, `
local router = require("xray.router")
function HandleRoute()
local tag, err = router.PickOutbound("balance")
return tag, "balanced", err
end`, nil, config("old"))
pick := func(want string) {
t.Helper()
route, err := r.PickRoute(&routing_session.Context{})
if err != nil || route.GetOutboundTag() != want || route.GetRuleTag() != "balanced" {
t.Fatalf("route = %v, %v, want %q", route, err, want)
}
}
pick("old")
if err := r.SetOverrideTarget("balance", "override"); err != nil {
t.Fatal(err)
}
pick("override")
if err := r.SetOverrideTarget("balance", ""); err != nil {
t.Fatal(err)
}
if err := r.ReloadRules(config("new"), false); err != nil {
t.Fatal(err)
}
pick("new")
}
func TestRouterScriptConcurrentBalancerReload(t *testing.T) {
config := func(tag string) *Config {
return &Config{BalancingRule: []*BalancingRule{{
Tag: "balance", Strategy: "roundrobin", OutboundSelector: []string{tag},
}}}
}
r := startLuaRouter(t, `
local router = require("xray.router")
function HandleRoute()
local tag, err = router.PickOutbound("balance")
return tag, nil, err
end`, nil, config("a"))
var wg sync.WaitGroup
for range 4 {
wg.Go(func() {
for range 20 {
route, err := r.PickRoute(&routing_session.Context{})
if err != nil {
t.Errorf("PickRoute: %v", err)
return
}
if tag := route.GetOutboundTag(); tag != "a" && tag != "b" {
t.Errorf("unexpected tag %q", tag)
}
}
})
}
wg.Go(func() {
for range 20 {
for _, tag := range []string{"a", "b"} {
if err := r.ReloadRules(config(tag), false); err != nil {
t.Error(err)
return
}
}
}
})
wg.Wait()
}
func TestRouterScriptDNSDispatcherReentry(t *testing.T) {
conn, err := stdnet.ListenPacket("udp4", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
port := conn.LocalAddr().(*stdnet.UDPAddr).Port
ready, stopped := make(chan struct{}), make(chan error, 1)
var queries atomic.Int32
server := &wireDNS.Server{
PacketConn: conn,
NotifyStartedFunc: func() {
close(ready)
},
Handler: wireDNS.HandlerFunc(func(w wireDNS.ResponseWriter, query *wireDNS.Msg) {
queries.Add(1)
response := new(wireDNS.Msg).SetReply(query)
for _, question := range query.Question {
if question.Name == "nested.example." && question.Qtype == wireDNS.TypeA {
response.Answer = append(response.Answer, &wireDNS.A{
Hdr: wireDNS.RR_Header{Name: question.Name, Rrtype: wireDNS.TypeA, Class: wireDNS.ClassINET, Ttl: 60},
A: stdnet.IP{127, 0, 0, 7},
})
}
}
if err := w.WriteMsg(response); err != nil {
t.Error(err)
}
}),
}
go func() { stopped <- server.ActivateAndServe() }()
defer func() {
server.Shutdown()
select {
case err := <-stopped:
if err != nil {
t.Error(err)
}
case <-time.After(3 * time.Second):
t.Error("DNS server did not stop")
}
}()
select {
case <-ready:
case err := <-stopped:
t.Fatalf("DNS server startup: %v", err)
case <-time.After(3 * time.Second):
t.Fatal("DNS server did not start")
}
dnsScript := writeRouteScript(t, `
local server = require("xray.dns").Servers[1]
function HandleDNSQuery(domain, ipv4, ipv6, fake)
return server:Query(domain, ipv4, ipv6, fake)
end`)
routerScript := writeRouteScript(t, `
local router = require("xray.router")
local dns = require("xray.dns")
local matcher = require("xray.geodata").BuildIPMatcher("127.0.0.7")
local active = false
function HandleRoute(ctx, inbound, sourcePort, targetPort, localPort, domain, network,
protocol, user, vlessRoute, skipDNSResolve)
assert(not active, "borrowed Router VM reentered")
if inbound == "dns" then
assert(network == router.NetworkUDP and skipDNSResolve == false)
return "direct", "dns-route"
end
active = true
local ips, ttl, err = dns.Query("nested.example", true, false, false)
assert(not err and matcher:AnyMatch(ips) and active)
active = false
return "direct", "outer-route"
end`)
instance, err := core.New(&core.Config{
App: []*serial.TypedMessage{
serial.ToTypedMessage(&appdns.Config{
Tag: "dns", Script: dnsScript, DisableCache: true,
NameServer: []*appdns.NameServer{{
Id: "upstream", TimeoutMs: 1000,
Address: &net.Endpoint{
Network: net.Network_UDP,
Address: &net.IPOrDomain{Address: &net.IPOrDomain_Ip{Ip: []byte{127, 0, 0, 1}}},
Port: uint32(port),
},
}},
}),
serial.ToTypedMessage(&Config{Script: routerScript}),
serial.ToTypedMessage(&dispatcher.Config{}),
serial.ToTypedMessage(&proxyman.OutboundConfig{}),
},
Outbound: []*core.OutboundHandlerConfig{
{Tag: "default", ProxySettings: serial.ToTypedMessage(&blackhole.Config{})},
{Tag: "direct", ProxySettings: serial.ToTypedMessage(&freedom.Config{
FinalRules: []*freedom.FinalRuleConfig{{Action: freedom.RuleAction_Allow}},
})},
},
})
if err != nil {
t.Fatal(err)
}
if err := instance.Start(); err != nil {
t.Fatal(err)
}
defer instance.Close()
r := instance.GetFeature(routing.RouterType()).(*Router)
route, err := r.PickRoute(newLuaRouteTestContext())
if err != nil || route.GetOutboundTag() != "direct" || route.GetRuleTag() != "outer-route" {
t.Fatalf("nested DNS routing = %v, %v", route, err)
}
if queries.Load() == 0 {
t.Fatal("DNS query did not pass through the dispatcher")
}
}
+6
View File
@@ -0,0 +1,6 @@
package base64
const (
Prefix = "base64:"
GzipPrefix = "base64+gzip:"
)
+38
View File
@@ -0,0 +1,38 @@
package base64
import (
"bytes"
"compress/gzip"
"encoding/base64"
"io"
"strings"
"github.com/xtls/xray-core/common/errors"
)
// Decode decodes Base64 text with an optional [Prefix] or [GzipPrefix].
func Decode(value string) ([]byte, error) {
value, zip := strings.CutPrefix(value, GzipPrefix)
if !zip {
value = strings.TrimPrefix(value, Prefix)
}
data, err := base64.StdEncoding.DecodeString(value)
if err != nil {
return nil, errors.New("failed to decode base64").Base(err)
}
if !zip {
return data, nil
}
reader, err := gzip.NewReader(bytes.NewReader(data))
if err != nil {
return nil, errors.New("failed to decompress gzip").Base(err)
}
defer reader.Close()
data, err = io.ReadAll(reader)
if err != nil {
return nil, errors.New("failed to decompress gzip").Base(err)
}
return data, nil
}
+28
View File
@@ -0,0 +1,28 @@
package base64
import (
"bytes"
"testing"
)
func TestDecodeFormats(t *testing.T) {
compressed, err := Encode([]byte("x"), true)
if err != nil {
t.Fatal(err)
}
for _, tc := range []struct {
name, value, want string
}{
{"unprefixed", "eA==", "x"},
{"base64", Prefix + "eA==", "x"},
{"gzip", compressed, "x"},
{"empty", Prefix, ""},
} {
t.Run(tc.name, func(t *testing.T) {
data, err := Decode(tc.value)
if err != nil || !bytes.Equal(data, []byte(tc.want)) {
t.Fatalf("Decode = %q, %v; want %q", data, err, tc.want)
}
})
}
}
+29
View File
@@ -0,0 +1,29 @@
package base64
import (
"bytes"
"compress/gzip"
"encoding/base64"
"github.com/xtls/xray-core/common/errors"
)
// Encode returns prefixed Base64 text, optionally compressed with gzip at the highest level.
func Encode(data []byte, zip bool) (string, error) {
if !zip {
return Prefix + base64.StdEncoding.EncodeToString(data), nil
}
var buf bytes.Buffer
writer, err := gzip.NewWriterLevel(&buf, gzip.BestCompression)
if err != nil {
return "", errors.New("failed to compress gzip").Base(err)
}
if _, err := writer.Write(data); err != nil {
return "", errors.New("failed to compress gzip").Base(err)
}
if err := writer.Close(); err != nil {
return "", errors.New("failed to compress gzip").Base(err)
}
return GzipPrefix + base64.StdEncoding.EncodeToString(buf.Bytes()), nil
}
+62
View File
@@ -0,0 +1,62 @@
package base64
import (
"bytes"
"compress/gzip"
"encoding/base64"
"io"
"strings"
"testing"
)
func TestEncodeFormats(t *testing.T) {
for _, tc := range []struct {
name string
data []byte
}{
{"empty", nil},
{"bytes", []byte("你好\r\n\x00\xff")},
} {
t.Run(tc.name, func(t *testing.T) {
for _, zip := range []bool{false, true} {
prefix := Prefix
if zip {
prefix = GzipPrefix
}
t.Run(prefix, func(t *testing.T) {
value, err := Encode(tc.data, zip)
if err != nil {
t.Fatal(err)
}
payload, ok := strings.CutPrefix(value, prefix)
if !ok {
t.Fatalf("encoded value does not start with %q", prefix)
}
// Read the output independently of Decode so matching bugs cannot hide.
data, err := base64.StdEncoding.DecodeString(payload)
if err != nil {
t.Fatal(err)
}
if zip {
// gzip's XFL byte identifies the compression level we selected.
if len(data) < 10 || data[8] != 2 {
t.Fatal("gzip output does not use the highest compression level")
}
reader, err := gzip.NewReader(bytes.NewReader(data))
if err != nil {
t.Fatal(err)
}
defer reader.Close()
data, err = io.ReadAll(reader)
if err != nil {
t.Fatal(err)
}
}
if !bytes.Equal(data, tc.data) {
t.Fatal("encoded output does not preserve the input bytes")
}
})
}
})
}
}
+33 -50
View File
@@ -82,19 +82,10 @@ func (f *MphDomainMatcherFactory) BuildMatcher(rules []*DomainRule) (DomainMatch
} }
g.Add(m, uint32(i)) g.Add(m, uint32(i))
case *DomainRule_Geosite: case *DomainRule_Geosite:
domains, err := loadSiteWithAttrs(v.Geosite.File, v.Geosite.Code, v.Geosite.Attrs) err := loadSiteMatchers(v.Geosite, func(m strmatcher.Matcher) { g.Add(m, uint32(i)) })
if err != nil { if err != nil {
return nil, err return nil, err
} }
for j, d := range domains {
domains[j] = nil // peak mem
m, err := parseDomain(d)
if err != nil {
errors.LogError(context.Background(), "ignore invalid geosite entry in ", v.Geosite.File, ":", v.Geosite.Code, " at index ", j, ", ", err)
continue
}
g.Add(m, uint32(i))
}
default: default:
panic("unknown domain rule type") panic("unknown domain rule type")
} }
@@ -108,12 +99,12 @@ func (f *MphDomainMatcherFactory) BuildMatcher(rules []*DomainRule) (DomainMatch
return g, nil return g, nil
} }
type CompactDomainMatcherFactory struct { type CompactMphDomainMatcherFactory struct {
sync.Mutex sync.Mutex
shared *utils.WeakCacheMap[string, strmatcher.LinearAnyMatcher] shared *utils.WeakCacheMap[string, strmatcher.MphValueMatcher]
} }
func (f *CompactDomainMatcherFactory) getOrCreateFrom(rule *GeoSiteRule) (strmatcher.MatcherSet, error) { func (f *CompactMphDomainMatcherFactory) getOrCreateFrom(rule *GeoSiteRule) (*strmatcher.MphValueMatcher, error) {
key := rule.File + ":" + rule.Code + "@" + rule.Attrs key := rule.File + ":" + rule.Code + "@" + rule.Attrs
f.Lock() f.Lock()
@@ -125,33 +116,23 @@ func (f *CompactDomainMatcherFactory) getOrCreateFrom(rule *GeoSiteRule) (strmat
} }
errors.LogDebug(context.Background(), "geodata geosite matcher cache MISS ", key) errors.LogDebug(context.Background(), "geodata geosite matcher cache MISS ", key)
s := strmatcher.NewLinearAnyMatcher() s := strmatcher.NewMphValueMatcher()
domains, err := loadSiteWithAttrs(rule.File, rule.Code, rule.Attrs) if err := loadSiteMatchers(rule, func(m strmatcher.Matcher) { s.Add(m, 0) }); err != nil {
if err != nil {
return nil, err return nil, err
} }
for i, d := range domains { if err := s.Build(); err != nil {
domains[i] = nil // peak mem return nil, err
m, err := parseDomain(d)
if err != nil {
errors.LogError(context.Background(), "ignore invalid geosite entry in ", rule.File, ":", rule.Code, " at index ", i, ", ", err)
continue
}
s.Add(m)
} }
f.shared.Store(key, s) f.shared.Store(key, s)
return s, err return s, nil
} }
// BuildMatcher implements DomainMatcherFactory. // BuildMatcher implements DomainMatcherFactory.
func (f *CompactDomainMatcherFactory) BuildMatcher(rules []*DomainRule) (DomainMatcher, error) { func (f *CompactMphDomainMatcherFactory) BuildMatcher(rules []*DomainRule) (DomainMatcher, error) {
if len(rules) == 0 { if len(rules) == 0 {
return nil, errors.New("empty domain rule list") return nil, errors.New("empty domain rule list")
} }
compact := &CompactDomainMatcher{ compact := new(CompactMphDomainMatcher)
matchers: make([]strmatcher.MatcherSet, 0, len(rules)),
values: make([]uint32, 0, len(rules)),
}
for i, r := range rules { for i, r := range rules {
switch v := r.Value.(type) { switch v := r.Value.(type) {
case *DomainRule_Custom: case *DomainRule_Custom:
@@ -168,8 +149,7 @@ func (f *CompactDomainMatcherFactory) BuildMatcher(rules []*DomainRule) (DomainM
if err != nil { if err != nil {
return nil, err return nil, err
} }
compact.matchers = append(compact.matchers, m) compact.combiner.Add(m, uint32(i))
compact.values = append(compact.values, uint32(i))
default: default:
panic("unknown domain rule type") panic("unknown domain rule type")
} }
@@ -177,37 +157,40 @@ func (f *CompactDomainMatcherFactory) BuildMatcher(rules []*DomainRule) (DomainM
return compact, nil return compact, nil
} }
type CompactDomainMatcher struct { type CompactMphDomainMatcher struct {
custom strmatcher.ValueMatcher custom strmatcher.ValueMatcher
matchers []strmatcher.MatcherSet combiner strmatcher.MphValueMatcherCombiner
values []uint32
} }
// Match implements DomainMatcher. // Match implements DomainMatcher.
func (c *CompactDomainMatcher) Match(input string) []uint32 { func (c *CompactMphDomainMatcher) Match(input string) []uint32 {
var result []uint32 result := c.combiner.Match(input)
if c.custom != nil { if c.custom != nil {
result = append(result, c.custom.Match(input)...) result = append(c.custom.Match(input), result...)
}
for i, m := range c.matchers {
if m.MatchAny(input) {
result = append(result, c.values[i])
}
} }
return result return result
} }
// MatchAny implements DomainMatcher. // MatchAny implements DomainMatcher.
func (c *CompactDomainMatcher) MatchAny(input string) bool { func (c *CompactMphDomainMatcher) MatchAny(input string) bool {
if c.custom != nil && c.custom.MatchAny(input) { if c.custom != nil && c.custom.MatchAny(input) {
return true return true
} }
for _, m := range c.matchers { return c.combiner.MatchAny(input)
if m.MatchAny(input) { }
return true
// loadSiteMatchers calls add with a matcher for every domain of the geosite rule and logs the invalid ones.
func loadSiteMatchers(rule *GeoSiteRule, add func(strmatcher.Matcher)) error {
i := 0
return loadSite(rule.File, rule.Code, rule.Attrs, func(t Domain_Type, value []byte) {
m, err := parseDomain(&Domain{Type: t, Value: string(value)})
if err != nil {
errors.LogError(context.Background(), "ignore invalid geosite entry in ", rule.File, ":", rule.Code, " at index ", i, ", ", err)
} else {
add(m)
} }
} i++
return false })
} }
func parseDomain(d *Domain) (strmatcher.Matcher, error) { func parseDomain(d *Domain) (strmatcher.Matcher, error) {
@@ -231,7 +214,7 @@ func parseDomain(d *Domain) (strmatcher.Matcher, error) {
func newDomainMatcherFactory() DomainMatcherFactory { func newDomainMatcherFactory() DomainMatcherFactory {
switch runtime.GOOS { switch runtime.GOOS {
case "ios", "android": case "ios", "android":
return &CompactDomainMatcherFactory{shared: utils.NewWeakCacheMap[string, strmatcher.LinearAnyMatcher]()} return &CompactMphDomainMatcherFactory{shared: utils.NewWeakCacheMap[string, strmatcher.MphValueMatcher]()}
default: default:
return &MphDomainMatcherFactory{shared: utils.NewWeakCacheMap[string, strmatcher.MphValueMatcher]()} return &MphDomainMatcherFactory{shared: utils.NewWeakCacheMap[string, strmatcher.MphValueMatcher]()}
} }
+76 -2
View File
@@ -4,6 +4,7 @@ import (
"path/filepath" "path/filepath"
"reflect" "reflect"
"slices" "slices"
"sync"
"testing" "testing"
"github.com/xtls/xray-core/common/geodata/strmatcher" "github.com/xtls/xray-core/common/geodata/strmatcher"
@@ -11,7 +12,7 @@ import (
) )
func TestCompactDomainMatcher_PreservesCustomRuleIndices(t *testing.T) { func TestCompactDomainMatcher_PreservesCustomRuleIndices(t *testing.T) {
factory := &CompactDomainMatcherFactory{shared: utils.NewWeakCacheMap[string, strmatcher.LinearAnyMatcher]()} factory := &CompactMphDomainMatcherFactory{shared: utils.NewWeakCacheMap[string, strmatcher.MphValueMatcher]()}
matcher, err := factory.BuildMatcher([]*DomainRule{ matcher, err := factory.BuildMatcher([]*DomainRule{
{Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Full, Value: "example.com"}}}, {Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Full, Value: "example.com"}}},
{Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Domain, Value: "example.com"}}}, {Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Domain, Value: "example.com"}}},
@@ -32,7 +33,7 @@ func TestCompactDomainMatcher_PreservesCustomRuleIndices(t *testing.T) {
func TestCompactDomainMatcher_PreservesMixedRuleIndices(t *testing.T) { func TestCompactDomainMatcher_PreservesMixedRuleIndices(t *testing.T) {
t.Setenv("xray.location.asset", filepath.Join("..", "..", "resources")) t.Setenv("xray.location.asset", filepath.Join("..", "..", "resources"))
factory := &CompactDomainMatcherFactory{shared: utils.NewWeakCacheMap[string, strmatcher.LinearAnyMatcher]()} factory := &CompactMphDomainMatcherFactory{shared: utils.NewWeakCacheMap[string, strmatcher.MphValueMatcher]()}
matcher, err := factory.BuildMatcher([]*DomainRule{ matcher, err := factory.BuildMatcher([]*DomainRule{
{Value: &DomainRule_Geosite{Geosite: &GeoSiteRule{File: DefaultGeoSiteDat, Code: "CN"}}}, {Value: &DomainRule_Geosite{Geosite: &GeoSiteRule{File: DefaultGeoSiteDat, Code: "CN"}}},
{Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Full, Value: "163.com"}}}, {Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Full, Value: "163.com"}}},
@@ -72,3 +73,76 @@ func TestMphDomainMatcher_MatchReturnsDetachedSlice(t *testing.T) {
t.Fatalf("Match() after caller mutation = %v, want %v", gotAgain, []uint32{0, 1}) t.Fatalf("Match() after caller mutation = %v, want %v", gotAgain, []uint32{0, 1})
} }
} }
// DNS sorts every Match result in place, so a matcher must never hand out a
// slice it keeps, also when only its keyword or regex part matches.
func TestDomainMatcher_MatchResultsCanBeSortedConcurrently(t *testing.T) {
t.Setenv("xray.location.asset", filepath.Join("..", "..", "resources"))
rules := []*DomainRule{
{Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Full, Value: "example.com"}}},
{Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Domain, Value: "example.com"}}},
{Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Substr, Value: "exam"}}},
{Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Regex, Value: `^ex.*\.org$`}}},
{Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Substr, Value: "exam"}}},
{Value: &DomainRule_Geosite{Geosite: &GeoSiteRule{File: DefaultGeoSiteDat, Code: "CN"}}},
{Value: &DomainRule_Custom{Custom: &Domain{Type: Domain_Full, Value: "only.full.test"}}},
}
cases := []struct {
input string
want []uint32
}{
{"example.com", []uint32{0, 1, 2, 4}},
{"www.example.com", []uint32{1, 2, 4}},
{"exam.net", []uint32{2, 4}}, // keyword part only
{"example.org", []uint32{2, 3, 4}},
{"163.com", []uint32{5}},
{"www.163.com", []uint32{5}},
{"only.full.test", []uint32{6}}, // full part only
{"nomatch.test", nil},
}
factories := map[string]DomainMatcherFactory{
"mph": &MphDomainMatcherFactory{shared: utils.NewWeakCacheMap[string, strmatcher.MphValueMatcher]()},
"compact": &CompactMphDomainMatcherFactory{shared: utils.NewWeakCacheMap[string, strmatcher.MphValueMatcher]()},
}
for name, factory := range factories {
t.Run(name, func(t *testing.T) {
matcher, err := factory.BuildMatcher(rules)
if err != nil {
t.Fatalf("BuildMatcher() failed: %v", err)
}
for _, c := range cases {
got := matcher.Match(c.input)
if sorted := slices.Sorted(slices.Values(got)); !slices.Equal(sorted, c.want) {
t.Fatalf("Match(%q) = %v, want %v", c.input, sorted, c.want)
}
got = got[:cap(got)]
for j := range got {
got[j] = ^uint32(0)
}
if again := slices.Sorted(slices.Values(matcher.Match(c.input))); !slices.Equal(again, c.want) {
t.Fatalf("Match(%q) after caller mutation = %v, want %v", c.input, again, c.want)
}
}
var wg sync.WaitGroup
for range 8 {
wg.Add(1)
go func() {
defer wg.Done()
for range 500 {
for _, c := range cases {
got := matcher.Match(c.input)
slices.Sort(got)
if !slices.Equal(got, c.want) {
t.Errorf("Match(%q) = %v, want %v", c.input, got, c.want)
return
}
}
}
}()
}
wg.Wait()
})
}
}
+226 -70
View File
@@ -5,21 +5,24 @@ import (
"bytes" "bytes"
"io" "io"
"runtime" "runtime"
"slices"
"strings" "strings"
"unicode/utf8"
"github.com/xtls/xray-core/common/errors" "github.com/xtls/xray-core/common/errors"
"github.com/xtls/xray-core/common/platform/filesystem" "github.com/xtls/xray-core/common/platform/filesystem"
"google.golang.org/protobuf/encoding/protowire"
"google.golang.org/protobuf/proto" "google.golang.org/protobuf/proto"
) )
func checkFile(file, code string) error { func checkFile(file, code string) error {
r, err := filesystem.OpenAsset(file) r, size, err := filesystem.OpenAsset(file)
if err != nil { if err != nil {
return errors.New("failed to open ", file).Base(err) return errors.New("failed to open ", file).Base(err)
} }
defer r.Close() defer r.Close()
if _, err := find(r, []byte(code), false); err != nil { if _, err := find(r, []byte(code), false, size); err != nil {
return errors.New("failed to check code ", code, " from ", file).Base(err) return errors.New("failed to check code ", code, " from ", file).Base(err)
} }
return nil return nil
@@ -27,12 +30,12 @@ func checkFile(file, code string) error {
func loadFile(file, code string) ([]byte, error) { func loadFile(file, code string) ([]byte, error) {
runtime.GC() // peak mem runtime.GC() // peak mem
r, err := filesystem.OpenAsset(file) r, size, err := filesystem.OpenAsset(file)
if err != nil { if err != nil {
return nil, errors.New("failed to open ", file).Base(err) return nil, errors.New("failed to open ", file).Base(err)
} }
defer r.Close() defer r.Close()
bs, err := find(r, []byte(code), true) bs, err := find(r, []byte(code), true, size)
if err != nil { if err != nil {
return nil, errors.New("failed to load code ", code, " from ", file).Base(err) return nil, errors.New("failed to load code ", code, " from ", file).Base(err)
} }
@@ -52,17 +55,56 @@ func loadIP(file, code string) ([]*CIDR, error) {
return geoip.Cidr, nil return geoip.Cidr, nil
} }
func loadSite(file, code string) ([]*Domain, error) { // loadSite calls fn, in file order, with the type and value of every domain of the geosite code
bs, err := loadFile(file, code) // that has all the "@"-separated attrs. It decodes the entry while reading the file instead of
// unmarshalling it into a []*Domain, so value is only valid during fn.
func loadSite(file, code, attrs string, fn func(Domain_Type, []byte)) error {
runtime.GC() // peak mem
r, size, err := filesystem.OpenAsset(file)
if err != nil { if err != nil {
return nil, err return errors.New("failed to open ", file).Base(err)
} }
defer runtime.GC() // peak mem defer r.Close()
var geosite GeoSite br := bufio.NewReaderSize(r, 64*1024)
if err := proto.Unmarshal(bs, &geosite); err != nil { n, err := seek(br, []byte(code), size)
return nil, errors.New("error unmarshal Site in ", file, ":", code).Base(err) if err != nil {
return errors.New("failed to load code ", code, " from ", file).Base(err)
} }
return geosite.Domain, nil loadErr := func(err error) error {
if err == io.EOF {
err = io.ErrUnexpectedEOF
}
return errors.New("failed to load code ", code, " from ", file).Base(err)
}
unmarshalErr := func(err error) error {
return errors.New("error unmarshal Site in ", file, ":", code).Base(err)
}
d := newSiteDecoder(attrs, fn)
for n > 0 {
w, err := br.Peek(min(n, br.Size()))
if err != nil {
return loadErr(err)
}
used, err := d.decode(w, len(w) < n)
if err != nil {
return unmarshalErr(err)
}
if used == 0 {
break // a field longer than the buffer
}
br.Discard(used)
n -= used
}
if n > 0 {
w := make([]byte, n)
if _, err := io.ReadFull(br, w); err != nil {
return loadErr(err)
}
if _, err := d.decode(w, false); err != nil {
return unmarshalErr(err)
}
}
return nil
} }
func decodeVarint(br *bufio.Reader) (uint64, error) { func decodeVarint(br *bufio.Reader) (uint64, error) {
@@ -72,6 +114,9 @@ func decodeVarint(br *bufio.Reader) (uint64, error) {
if err != nil { if err != nil {
return 0, err return 0, err
} }
if shift == 63 && b > 1 {
break // Only one value bit fits in the tenth byte.
}
x |= (uint64(b) & 0x7F) << shift x |= (uint64(b) & 0x7F) << shift
if (b & 0x80) == 0 { if (b & 0x80) == 0 {
return x, nil return x, nil
@@ -81,69 +126,66 @@ func decodeVarint(br *bufio.Reader) (uint64, error) {
return 0, errors.New("varint overflow") return 0, errors.New("varint overflow")
} }
func find(r io.Reader, code []byte, readBody bool) ([]byte, error) { func find(r io.Reader, code []byte, readBody bool, size int64) ([]byte, error) {
br := bufio.NewReaderSize(r, 64*1024)
bodyL, err := seek(br, code, size)
if err != nil || !readBody {
return nil, err
}
out := make([]byte, bodyL)
if _, err := io.ReadFull(br, out); err != nil {
return nil, err
}
return out, nil
}
// seek advances br to the body of the entry for code and returns the body length.
func seek(br *bufio.Reader, code []byte, size int64) (int, error) {
codeL := len(code) codeL := len(code)
if codeL == 0 { if codeL == 0 {
return nil, errors.New("empty code") return 0, errors.New("empty code")
} }
br := bufio.NewReaderSize(r, 64*1024)
need := 2 + codeL // TODO: if code too long need := 2 + codeL // TODO: if code too long
prefixBuf := make([]byte, need)
for { for {
if _, err := br.ReadByte(); err != nil { tag, err := br.ReadByte()
return nil, err if err != nil {
return 0, err
} }
x, err := decodeVarint(br) x, err := decodeVarint(br)
if err != nil { if err != nil {
return nil, err return 0, err
}
if x == 0 || x > uint64(size) || x > uint64(^uint(0)>>1) {
return 0, errors.New("invalid body length: ", x)
} }
bodyL := int(x) bodyL := int(x)
if bodyL <= 0 {
return nil, errors.New("invalid body length: ", bodyL)
}
prefixL := bodyL // Peek no more than the buffer holds: a code longer than the buffer cannot match a single
if prefixL > need { // length byte anyway, so a short peek only skips it, as base find (io.ReadFull) does.
prefixL = need prefix, err := br.Peek(min(bodyL, need, br.Size()))
} if err != nil {
prefix := prefixBuf[:prefixL] if err == io.EOF && len(prefix) > 0 {
if _, err := io.ReadFull(br, prefix); err != nil { err = io.ErrUnexpectedEOF // as io.ReadFull
return nil, err
}
match := false
if bodyL >= need {
if int(prefix[1]) == codeL && bytes.Equal(prefix[2:need], code) {
if !readBody {
return nil, nil
}
match = true
} }
return 0, err
} }
if tag == 0x0a && bodyL >= need && len(prefix) >= need &&
remain := bodyL - prefixL prefix[0] == 0x0a && int(prefix[1]) == codeL && bytes.Equal(prefix[2:], code) {
if match { return bodyL, nil
out := make([]byte, bodyL)
copy(out, prefix)
if remain > 0 {
if _, err := io.ReadFull(br, out[prefixL:]); err != nil {
return nil, err
}
}
return out, nil
} }
if _, err := br.Discard(bodyL); err != nil {
if remain > 0 { return 0, err
if _, err := br.Discard(remain); err != nil {
return nil, err
}
} }
} }
} }
// AttributeMatcher, HasAttrMatcher, AllAttrsMatcher and NewAllAttrsMatcher are the exported
// attribute helpers that have been part of this package's API since #5814. The streaming loader
// above filters attributes itself without building a *Domain, so it does not use them, but they
// are kept for external callers. Their behaviour is unchanged.
type AttributeMatcher interface { type AttributeMatcher interface {
Match(*Domain) bool Match(*Domain) bool
} }
@@ -185,23 +227,137 @@ func NewAllAttrsMatcher(attrs string) AttributeMatcher {
return m return m
} }
func loadSiteWithAttrs(file, code, attrs string) ([]*Domain, error) { var errInvalidUTF8 = errors.New("string field contains invalid UTF-8")
domains, err := loadSite(file, code)
if err != nil {
return nil, err
}
matcher := NewAllAttrsMatcher(attrs) type siteDecoder struct {
if matcher == nil { want []string
return domains, nil has []bool
} fn func(Domain_Type, []byte)
}
filtered := make([]*Domain, 0, len(domains)) func newSiteDecoder(attrs string, fn func(Domain_Type, []byte)) *siteDecoder {
for _, d := range domains { d := &siteDecoder{fn: fn}
if matcher.Match(d) { if attrs != "" {
filtered = append(filtered, d) d.want = strings.Split(attrs, "@")
d.has = make([]bool, len(d.want))
}
return d
}
// decode walks the whole fields at the start of b, a part of an encoded GeoSite (see geodat.proto),
// calls fn for every domain that has all attrs and returns how many bytes it used. A field cut off
// by the end of b is an error unless more is set. It accepts and rejects what proto.Unmarshal does.
func (d *siteDecoder) decode(b []byte, more bool) (int, error) {
used := 0
for used < len(b) {
f, n, err := consumeField(b[used:])
if err == io.ErrUnexpectedEOF && more {
break
}
if err != nil {
return used, err
}
used += n
if f.typ != protowire.BytesType {
continue
}
switch f.num {
case 1: // code
if !utf8.Valid(f.v) {
return used, errInvalidUTF8
}
case 2: // domain
t, value, err := decodeDomain(f.v, d.want, d.has)
if err != nil {
return used, err
}
if !slices.Contains(d.has, false) {
d.fn(t, value)
}
} }
} }
return used, nil
return filtered, nil }
// decodeDomain decodes an encoded Domain and sets has[i] if one of its attributes has the key want[i].
func decodeDomain(b []byte, want []string, has []bool) (t Domain_Type, value []byte, err error) {
clear(has)
for len(b) > 0 {
f, n, err := consumeField(b)
if err != nil {
return 0, nil, err
}
b = b[n:]
switch {
case f.num == 1 && f.typ == protowire.VarintType: // type
t = Domain_Type(f.x)
case f.num == 2 && f.typ == protowire.BytesType: // value
if !utf8.Valid(f.v) {
return 0, nil, errInvalidUTF8
}
value = f.v
case f.num == 3 && f.typ == protowire.BytesType: // attribute
key, err := decodeAttributeKey(f.v)
if err != nil {
return 0, nil, err
}
for i, w := range want {
if string(key) == w {
has[i] = true
}
}
}
}
return t, value, nil
}
// decodeAttributeKey returns the key of an encoded Domain.Attribute.
func decodeAttributeKey(b []byte) ([]byte, error) {
var key []byte
for len(b) > 0 {
f, n, err := consumeField(b)
if err != nil {
return nil, err
}
b = b[n:]
if f.num == 1 && f.typ == protowire.BytesType {
if !utf8.Valid(f.v) {
return nil, errInvalidUTF8
}
key = f.v
}
}
return key, nil
}
type protoField struct {
num protowire.Number
typ protowire.Type
v []byte // payload of a length-delimited field
x uint64 // value of a varint field
}
// consumeField parses the first field of an encoded message and returns it with its length.
func consumeField(b []byte) (protoField, int, error) {
num, typ, n := protowire.ConsumeTag(b)
if n < 0 {
return protoField{}, 0, protowire.ParseError(n)
}
if num > protowire.MaxValidNumber {
return protoField{}, 0, errors.New("invalid field number ", num)
}
f := protoField{num: num, typ: typ}
var m int
switch typ {
case protowire.BytesType:
f.v, m = protowire.ConsumeBytes(b[n:])
case protowire.VarintType:
f.x, m = protowire.ConsumeVarint(b[n:])
default:
m = protowire.ConsumeFieldValue(num, typ, b[n:])
}
if m < 0 {
return protoField{}, 0, protowire.ParseError(m)
}
return f, n + m, nil
} }
+336
View File
@@ -0,0 +1,336 @@
package geodata
import (
"bytes"
"fmt"
"os"
"path/filepath"
"slices"
"strings"
"testing"
"google.golang.org/protobuf/encoding/protowire"
"google.golang.org/protobuf/proto"
)
func TestFindChecksEntryAndCodeTags(t *testing.T) {
for _, payload := range [][]byte{
{0x12, 6, 0x0a, 4, 'T', 'E', 'S', 'T'}, // Unknown field posing as entry.
{0x08, 6, 0x0a, 4, 'T', 'E', 'S', 'T'}, // Wrong entry wire type.
{0x0a, 6, 0x1a, 4, 'T', 'E', 'S', 'T'}, // Unknown field posing as code.
{0x0a, 6, 0x08, 4, 'T', 'E', 'S', 'T'}, // Wrong code wire type.
} {
if _, err := find(bytes.NewReader(payload), []byte("TEST"), true, int64(len(payload))); err == nil {
t.Fatalf("matched incorrect entry/code tags in %x", payload)
}
}
}
func TestLoadFileRejectsVarintOverflow(t *testing.T) {
payload := []byte{
0x0a,
0x86, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0x02, // 2^64 + 6
0x0a, 0x04, 'T', 'E', 'S', 'T',
}
dir := t.TempDir()
t.Setenv("xray.location.asset", dir)
if err := os.WriteFile(filepath.Join(dir, "overflow.dat"), payload, 0o600); err != nil {
t.Fatal(err)
}
if _, err := loadFile("overflow.dat", "TEST"); err == nil || !strings.Contains(err.Error(), "varint overflow") {
t.Fatalf("want varint overflow, got %v", err)
}
}
func TestLoadFileRejectsOversizedEntry(t *testing.T) {
payload := []byte{
0x0a, 0x03, 0x0a, 0x01, 'A', // A valid entry before the oversized one.
0x0a, 0x80, 0x80, 0x80, 0x80, 0x04, // Entry length: 1 GiB.
0x0a, 0x04, 'T', 'E', 'S', 'T',
}
dir := t.TempDir()
t.Setenv("xray.location.asset", dir)
if err := os.WriteFile(filepath.Join(dir, "oversized.dat"), payload, 0o600); err != nil {
t.Fatal(err)
}
// Check without allocating the body first, so a regression fails safely.
if err := checkFile("oversized.dat", "TEST"); err == nil {
t.Fatal("accepted an entry length larger than the file")
}
if _, err := loadFile("oversized.dat", "TEST"); err == nil || !strings.Contains(err.Error(), "invalid body length") {
t.Fatalf("want invalid body length, got %v", err)
}
if err := loadSite("oversized.dat", "TEST", "", func(Domain_Type, []byte) {}); err == nil || !strings.Contains(err.Error(), "invalid body length") {
t.Fatalf("want invalid body length, got %v", err)
}
}
type siteEntry struct {
Type Domain_Type
Value string
}
// unmarshalSite is what loadSite used to do: proto.Unmarshal, then keep the domains that have all attrs.
func unmarshalSite(b []byte, attrs string) ([]siteEntry, error) {
var site GeoSite
if err := proto.Unmarshal(b, &site); err != nil {
return nil, err
}
var entries []siteEntry
for _, d := range site.Domain {
ok := true
for _, key := range strings.Split(attrs, "@") {
ok = ok && (attrs == "" || slices.ContainsFunc(d.Attribute, func(a *Domain_Attribute) bool { return a.Key == key }))
}
if ok {
entries = append(entries, siteEntry{d.Type, d.Value})
}
}
return entries, nil
}
func checkDecodeSite(t *testing.T, name string, b []byte, attrs string) {
t.Helper()
want, wantErr := unmarshalSite(b, attrs)
var got []siteEntry
_, err := newSiteDecoder(attrs, func(typ Domain_Type, value []byte) {
got = append(got, siteEntry{typ, string(value)})
}).decode(b, false)
if (err == nil) != (wantErr == nil) {
t.Fatalf("%s@%s: error %v, proto.Unmarshal: %v", name, attrs, err, wantErr)
}
if err == nil && !slices.Equal(got, want) {
t.Fatalf("%s@%s: got %v, want %v", name, attrs, got, want)
}
}
func TestDecodeSiteMatchesUnmarshal(t *testing.T) {
bs, err := os.ReadFile(filepath.Join("..", "..", "resources", DefaultGeoSiteDat))
if err != nil {
t.Fatal(err)
}
for len(bs) > 0 {
num, typ, n := protowire.ConsumeTag(bs)
if n < 0 || num != 1 || typ != protowire.BytesType {
t.Fatal("unexpected GeoSiteList field")
}
entry, m := protowire.ConsumeBytes(bs[n:])
if m < 0 {
t.Fatal(protowire.ParseError(m))
}
bs = bs[n+m:]
var site GeoSite
if err := proto.Unmarshal(entry, &site); err != nil {
t.Fatal(err)
}
queries := []string{"", "none"}
for _, d := range site.Domain {
for _, a := range d.Attribute {
if !slices.Contains(queries, a.Key) {
queries = append(queries, a.Key, a.Key+"@none")
}
}
}
for _, attrs := range queries {
checkDecodeSite(t, site.Code, entry, attrs)
}
}
}
func TestDecodeSiteUnusualEncodings(t *testing.T) {
field := func(num protowire.Number, v []byte) []byte {
return protowire.AppendBytes(protowire.AppendTag(nil, num, protowire.BytesType), v)
}
typ := func(v Domain_Type) []byte {
return protowire.AppendVarint(protowire.AppendTag(nil, 1, protowire.VarintType), uint64(v))
}
value := func(s string) []byte { return field(2, []byte(s)) }
attr := func(keys ...string) []byte {
var b []byte
for _, k := range keys {
b = append(b, field(1, []byte(k))...)
}
return field(3, b)
}
domain := func(fields ...[]byte) []byte { return field(2, slices.Concat(fields...)) }
unknown := protowire.AppendFixed32(protowire.AppendTag(nil, 9, protowire.Fixed32Type), 1)
for name, b := range map[string][]byte{
"unknown field": domain(typ(Domain_Full), unknown, value("example.com")),
"repeated value": domain(value("a.com"), typ(Domain_Full), value("b.com")),
"repeated type": domain(typ(Domain_Full), value("a.com"), typ(Domain_Regex)),
"repeated key": domain(value("a.com"), attr("cn", "ads")),
"type as bytes": domain(field(1, []byte("x")), value("a.com")),
"no value": domain(typ(Domain_Domain), attr("cn")),
"truncated": domain(typ(Domain_Full), value("example.com"))[:10],
"invalid utf8": domain(value("example.\xff")),
"invalid key": domain(value("a.com"), attr("\xff")),
"bad field": protowire.AppendVarint(protowire.AppendTag(nil, protowire.MaxValidNumber+1, protowire.VarintType), 1),
"stray end group": protowire.AppendTag(nil, 5, protowire.EndGroupType),
} {
for _, attrs := range []string{"", "cn", "ads", "cn@ads"} {
checkDecodeSite(t, name, b, attrs)
}
}
}
// TestLoadSiteReadsInPieces covers what real lists never do: an entry far longer than the read
// buffer, with a field longer than the buffer in the middle, and a file cut short.
func TestLoadSiteReadsInPieces(t *testing.T) {
site := &GeoSite{Code: "BIG"}
for i := range 5000 {
d := &Domain{Type: Domain_Domain, Value: strings.Repeat("x", i%40) + ".example.com"}
if i%3 == 0 {
d.Attribute = []*Domain_Attribute{{Key: "cn"}}
}
if i == 2500 {
d = &Domain{Type: Domain_Regex, Value: strings.Repeat("a", 100_000)}
}
site.Domain = append(site.Domain, d)
}
list := &GeoSiteList{Entry: []*GeoSite{{Code: "SMALL", Domain: []*Domain{{Type: Domain_Full, Value: "a.com"}}}, site}}
bs, err := proto.Marshal(list)
if err != nil {
t.Fatal(err)
}
entry, err := proto.Marshal(site)
if err != nil {
t.Fatal(err)
}
dir := t.TempDir()
t.Setenv("xray.location.asset", dir)
write := func(b []byte) {
if err := os.WriteFile(filepath.Join(dir, "big.dat"), b, 0o644); err != nil {
t.Fatal(err)
}
}
for _, attrs := range []string{"", "cn"} {
want, _ := unmarshalSite(entry, attrs)
var got []siteEntry
write(bs)
err := loadSite("big.dat", "BIG", attrs, func(typ Domain_Type, value []byte) {
got = append(got, siteEntry{typ, string(value)})
})
if err != nil || !slices.Equal(got, want) {
t.Fatalf("attrs %q: %d entries, want %d, error %v", attrs, len(got), len(want), err)
}
for _, cut := range []int{30_000, len(bs) - 150_000, len(bs) - 1} {
write(bs[:cut])
if err := loadSite("big.dat", "BIG", attrs, func(Domain_Type, []byte) {}); err == nil {
t.Fatalf("file cut at %d of %d: no error", cut, len(bs))
}
}
}
}
// oneEntryGeoSiteFile wraps an encoded GeoSite as a one-entry GeoSiteList, the file loadSite reads.
func oneEntryGeoSiteFile(entry []byte) []byte {
return protowire.AppendBytes(protowire.AppendTag(nil, 1, protowire.BytesType), entry)
}
// TestLoadSiteWindowedMatchesSingleShot checks that the windowed reader in loadSite (its Peek/Discard
// loop, the more-break when a field is cut by a window edge, the used==0 fallback for a field longer
// than the buffer, and the tail path) reaches exactly the same result as decoding the whole entry at
// once, for a category several 64 KiB windows long, valid and then mutated near a window edge and
// early in the file: same error-or-not, and the same emitted (type, value) sequence when both accept.
func TestLoadSiteWindowedMatchesSingleShot(t *testing.T) {
const window = 64 * 1024
site := &GeoSite{Code: "BIG"}
for i := range 12000 { // ~250 KiB, four windows
d := &Domain{Type: Domain_Domain, Value: fmt.Sprintf("host%d.%s.example.com", i, strings.Repeat("y", i%30))}
if i%3 == 0 {
d.Attribute = []*Domain_Attribute{{Key: "cn"}}
}
site.Domain = append(site.Domain, d)
}
// a field longer than the buffer, straddling the third window, to force the used==0 fallback
site.Domain = slices.Insert(site.Domain, 8000, &Domain{Type: Domain_Regex, Value: strings.Repeat("a", 90_000)})
entry, err := proto.Marshal(site)
if err != nil {
t.Fatal(err)
}
dir := t.TempDir()
t.Setenv("xray.location.asset", dir)
// mutations of the encoded entry: unchanged, a byte flipped at several offsets (early windows and
// either side of a window edge), and truncations at the same places.
type mut struct {
name string
make func([]byte) []byte
}
muts := []mut{{"valid", func(b []byte) []byte { return b }}}
for _, off := range []int{3, 40, 4000, window - 1, window, window + 1, 2*window - 2, 2 * window} {
if off < len(entry) {
off := off
muts = append(muts, mut{fmt.Sprintf("flip@%d", off), func(b []byte) []byte {
c := slices.Clone(b)
c[off] ^= 0xff
return c
}})
muts = append(muts, mut{fmt.Sprintf("cut@%d", off), func(b []byte) []byte { return slices.Clone(b[:off]) }})
}
}
for _, attrs := range []string{"", "cn"} {
for _, m := range muts {
e := m.make(entry)
// single-shot reference: decode the whole entry in one call
var want []siteEntry
_, wantErr := newSiteDecoder(attrs, func(typ Domain_Type, value []byte) {
want = append(want, siteEntry{typ, string(value)})
}).decode(e, false)
// windowed: loadSite reads the file 64 KiB at a time
if err := os.WriteFile(filepath.Join(dir, "w.dat"), oneEntryGeoSiteFile(e), 0o644); err != nil {
t.Fatal(err)
}
var got []siteEntry
gotErr := loadSite("w.dat", "BIG", attrs, func(typ Domain_Type, value []byte) {
got = append(got, siteEntry{typ, string(value)})
})
if (gotErr == nil) != (wantErr == nil) {
t.Fatalf("%s attrs=%q: windowed err %v, single-shot err %v", m.name, attrs, gotErr, wantErr)
}
if gotErr == nil && !slices.Equal(got, want) {
t.Fatalf("%s attrs=%q: windowed got %d entries, single-shot %d", m.name, attrs, len(got), len(want))
}
}
}
}
// TestLoadSiteLongCode covers a geosite entry whose code is longer than the 64 KiB read buffer. seek
// must skip it (find compares a single length byte, so it never matches such a code) and still find a
// later entry, and looking the long code up must fail cleanly, like a missing code, not panic.
func TestLoadSiteLongCode(t *testing.T) {
longCode := strings.Repeat("Z", 70000)
list := &GeoSiteList{Entry: []*GeoSite{
{Code: "FIRST", Domain: []*Domain{{Type: Domain_Full, Value: "first.com"}}},
{Code: longCode, Domain: []*Domain{{Type: Domain_Full, Value: "huge.com"}}},
{Code: "AFTER", Domain: []*Domain{{Type: Domain_Domain, Value: "after.com"}}},
}}
bs, err := proto.Marshal(list)
if err != nil {
t.Fatal(err)
}
dir := t.TempDir()
t.Setenv("xray.location.asset", dir)
if err := os.WriteFile(filepath.Join(dir, "lc.dat"), bs, 0o644); err != nil {
t.Fatal(err)
}
collect := func(code string) ([]siteEntry, error) {
var got []siteEntry
err := loadSite("lc.dat", code, "", func(typ Domain_Type, value []byte) {
got = append(got, siteEntry{typ, string(value)})
})
return got, err
}
if got, err := collect("FIRST"); err != nil || !slices.Equal(got, []siteEntry{{Domain_Full, "first.com"}}) {
t.Fatalf("FIRST: %v %v", got, err)
}
if got, err := collect("AFTER"); err != nil || !slices.Equal(got, []siteEntry{{Domain_Domain, "after.com"}}) {
t.Fatalf("AFTER (past the oversized entry): %v %v", got, err)
}
if _, err := collect(longCode); err == nil {
t.Fatal("oversized code: expected a not-found error, got nil")
}
}
+163
View File
@@ -0,0 +1,163 @@
package geodata
import (
xlua "github.com/xtls/xray-core/common/lua"
"github.com/xtls/xray-core/common/net"
lua "github.com/yuin/gopher-lua"
)
// RegisterLua makes xray.geodata available to require in an LState.
func RegisterLua(L *lua.LState) {
L.PreloadModule("xray.geodata", func(L *lua.LState) int {
module := L.CreateTable(0, 2)
module.RawSetString("BuildDomainMatcher", L.NewFunction(func(L *lua.LState) int {
parsed, err := ParseDomainRules(luaRules(L), Domain_Domain)
if err != nil {
L.RaiseError("%v", err)
return 0
}
matcher, err := DomainReg.BuildDomainMatcher(parsed)
if err != nil {
L.RaiseError("%v", err)
return 0
}
xlua.PushWithDirectMethods(L, matcher, map[string]xlua.DirectMethod{
"Match": newLuaDomainMatch(xlua.NewSlicePusher[uint32](L)),
"MatchAny": luaDomainMatchAny,
})
return 1
}))
module.RawSetString("BuildIPMatcher", L.NewFunction(func(L *lua.LState) int {
parsed, err := ParseIPRules(luaRules(L))
if err != nil {
L.RaiseError("%v", err)
return 0
}
matcher, err := IPReg.BuildIPMatcher(parsed)
if err != nil {
L.RaiseError("%v", err)
return 0
}
xlua.PushWithDirectMethods(L, matcher, map[string]xlua.DirectMethod{
"Match": luaIPMatch,
"AnyMatch": luaIPAnyMatch,
"Matches": luaIPMatches,
"FilterIPs": newLuaIPFilterIPs(xlua.NewSlicePusher[net.IP](L)),
}, "SetReverse", "ToggleReverse")
return 1
}))
L.Push(module)
return 1
})
}
// Read native Go values by type assertion; slices keep their original storage.
func readLuaIPMatcherArgs[T any](L *lua.LState) (IPMatcher, T, bool) {
var input T
if L.GetTop() != 2 {
return nil, input, false
}
value, ok := L.Get(1).(*lua.LUserData)
if !ok {
return nil, input, false
}
matcher, ok := value.Value.(IPMatcher)
if !ok {
return nil, input, false
}
if L.Get(2) == lua.LNil {
return matcher, input, true
}
value, ok = L.Get(2).(*lua.LUserData)
if !ok {
return nil, input, false
}
input, ok = value.Value.(T)
return matcher, input, ok
}
func luaIPMatch(L *lua.LState) (int, bool) {
matcher, ip, ok := readLuaIPMatcherArgs[net.IP](L)
if !ok {
return 0, false
}
L.Push(lua.LBool(matcher.Match(ip)))
return 1, true
}
func luaIPAnyMatch(L *lua.LState) (int, bool) {
matcher, ips, ok := readLuaIPMatcherArgs[[]net.IP](L)
if !ok {
return 0, false
}
L.Push(lua.LBool(matcher.AnyMatch(ips)))
return 1, true
}
func luaIPMatches(L *lua.LState) (int, bool) {
matcher, ips, ok := readLuaIPMatcherArgs[[]net.IP](L)
if !ok {
return 0, false
}
L.Push(lua.LBool(matcher.Matches(ips)))
return 1, true
}
func newLuaIPFilterIPs(pushIPs func(*lua.LState, []net.IP)) xlua.DirectMethod {
return func(L *lua.LState) (int, bool) {
matcher, ips, ok := readLuaIPMatcherArgs[[]net.IP](L)
if !ok {
return 0, false
}
matched, unmatched := matcher.FilterIPs(ips)
pushIPs(L, matched)
pushIPs(L, unmatched)
return 2, true
}
}
func newLuaDomainMatch(pushMatches func(*lua.LState, []uint32)) xlua.DirectMethod {
return func(L *lua.LState) (int, bool) {
if L.GetTop() == 2 {
if value, ok := L.Get(1).(*lua.LUserData); ok {
matcher, validMatcher := value.Value.(DomainMatcher)
domain, validDomain := L.Get(2).(lua.LString)
if validMatcher && validDomain {
pushMatches(L, matcher.Match(string(domain)))
return 1, true
}
}
}
return 0, false
}
}
func luaDomainMatchAny(L *lua.LState) (int, bool) {
if L.GetTop() == 2 {
if value, ok := L.Get(1).(*lua.LUserData); ok {
matcher, validMatcher := value.Value.(DomainMatcher)
domain, validDomain := L.Get(2).(lua.LString)
if validMatcher && validDomain {
L.Push(lua.LBool(matcher.MatchAny(string(domain))))
return 1, true
}
}
}
return 0, false
}
func luaRules(L *lua.LState) []string {
rules := make([]string, L.GetTop())
for i := range rules {
value, ok := L.Get(i + 1).(lua.LString)
if !ok {
L.RaiseError("geodata rules must be strings")
return nil
}
rules[i] = string(value)
}
return rules
}
+178
View File
@@ -0,0 +1,178 @@
package geodata
import (
"fmt"
"testing"
"github.com/xtls/xray-core/common/net"
lua "github.com/yuin/gopher-lua"
)
func TestLuaIPMatcher(t *testing.T) {
L := lua.NewState()
defer L.Close()
RegisterLua(L)
ip := L.NewUserData()
ip.Value = net.ParseIP("127.0.0.1")
L.SetGlobal("ip", ip)
ips := L.NewUserData()
ips.Value = []net.IP{ip.Value.(net.IP), net.ParseIP("8.8.8.8")}
L.SetGlobal("ips", ips)
if err := L.DoString(`
local matcher = require("xray.geodata").BuildIPMatcher("127.0.0.0/8", "::1")
assert(matcher:Match(ip))
assert(matcher:AnyMatch(ips))
assert(not matcher:Matches(ips))
local matched, unmatched = matcher:FilterIPs(ips)
assert(type(matched) == "userdata" and type(unmatched) == "userdata")
assert(#matched == 1 and #unmatched == 1)
`); err != nil {
t.Fatal(err)
}
}
func TestLuaDomainMatcher(t *testing.T) {
L := lua.NewState()
defer L.Close()
RegisterLua(L)
if err := L.DoString(`
local matcher = require("xray.geodata").BuildDomainMatcher("example.com", "full:other.com")
assert(matcher:MatchAny("example.com"))
assert(matcher:MatchAny("www.example.com"))
assert(matcher:MatchAny("other.com"))
assert(not matcher:MatchAny("www.other.com"))
assert(#(matcher:Match("www.example.com")) == 1)
`); err != nil {
t.Fatal(err)
}
}
func TestLuaMatchersRejectInvalidRules(t *testing.T) {
for _, tc := range []struct {
name string
script string
}{
{"IP rule", `require("xray.geodata").BuildIPMatcher("not-an-ip")`},
{"non-string domain rule", `require("xray.geodata").BuildDomainMatcher("example.com", true)`},
} {
t.Run(tc.name, func(t *testing.T) {
L := lua.NewState()
defer L.Close()
RegisterLua(L)
if err := L.DoString(tc.script); err == nil {
t.Fatal("invalid geodata rule was accepted")
}
})
}
}
func TestLuaMatcherArgumentsAndAliases(t *testing.T) {
L := lua.NewState()
defer L.Close()
RegisterLua(L)
ip := L.NewUserData()
ip.Value = net.ParseIP("127.0.0.1")
L.SetGlobal("ip", ip)
if err := L.DoString(`
local geodata = require("xray.geodata")
local matcher = geodata.BuildIPMatcher("127.0.0.0/8")
assert(matcher.Match == matcher.match and matcher.AnyMatch == matcher.anyMatch)
assert(matcher.Matches == matcher.matches and matcher.FilterIPs == matcher.filterIPs)
assert(matcher.Reload == nil and matcher.reload == nil)
assert(matcher:match(ip))
assert(matcher:anyMatch({ip}) and matcher:matches({ip}))
assert(not matcher:AnyMatch(nil))
assert(matcher:Matches(nil) == matcher:Matches({}))
local matched, unmatched = matcher:FilterIPs({ip})
assert(#matched == 1 and matched[1]:Equal(ip))
assert(matcher:AnyMatch(matched) and matcher:Matches(matched))
local filtered, excluded = matcher:filterIPs(matched)
assert(#filtered == 1 and #excluded == 0 and filtered[1]:Equal(ip))
local emptyMatched, emptyUnmatched = matcher:FilterIPs(nil)
assert(#emptyMatched == 0 and #emptyUnmatched == 0)
matcher:SetReverse(true)
assert(not matcher:Match(ip) and not matcher:AnyMatch(matched))
matcher:ToggleReverse()
assert(matcher:Match(ip) and matcher:AnyMatch(matched))
assert(matcher.missing == nil)
local domain = geodata.BuildDomainMatcher("full:example.com")
assert(domain.Match == domain.match and domain.MatchAny == domain.matchAny)
assert(domain.Reload == nil and domain.reload == nil)
for _, value in ipairs({matcher, domain}) do
assert(not pcall(function() return -value end))
assert(not pcall(function() return value ^ {} end))
end
assert(domain:matchAny("example.com"))
assert(#domain:Match("example.com") == 1)
assert(domain:match("example.com")[1] == 0)
assert(not pcall(function() matcher:AnyMatch() end))
assert(not pcall(function() matcher:AnyMatch(matched, true) end))
assert(not pcall(function() matcher.AnyMatch(ip, matched) end))
assert(not pcall(function() matcher:Match(true) end))
assert(not pcall(function() domain:MatchAny(123) end))
assert(not pcall(function() domain:MatchAny("example.com", true) end))
assert(not pcall(function() matcher:FilterIPs(true) end))
assert(not pcall(function() matcher:FilterIPs(matched, true) end))
assert(not pcall(function() domain:Match(123) end))
assert(not pcall(function() domain:Match("example.com", true) end))
`); err != nil {
t.Fatal(err)
}
}
// BenchmarkLuaMatcherCall measures repeated calls with prebuilt matchers and inputs.
func BenchmarkLuaMatcherCall(b *testing.B) {
L := lua.NewState()
defer L.Close()
RegisterLua(L)
ip := net.ParseIP("127.0.0.1")
for name, value := range map[string]any{"ip": ip, "ips": []net.IP{ip}} {
ud := L.NewUserData()
ud.Value = value
L.SetGlobal(name, ud)
}
if err := L.DoString(`
local geodata = require("xray.geodata")
ipMatcher = geodata.BuildIPMatcher("127.0.0.0/8")
domainMatcher = geodata.BuildDomainMatcher("full:example.com")
`); err != nil {
b.Fatal(err)
}
for _, benchmark := range []struct {
name, expression string
}{
{"ip_match", "ipMatcher:Match(ip)"},
{"ip_match_lower", "ipMatcher:match(ip)"},
{"ip_any_match", "ipMatcher:AnyMatch(ips)"},
{"ip_any_match_lower", "ipMatcher:anyMatch(ips)"},
{"ip_matches", "ipMatcher:Matches(ips)"},
{"ip_matches_lower", "ipMatcher:matches(ips)"},
{"domain_match_any", `domainMatcher:MatchAny("example.com")`},
{"domain_match_any_lower", `domainMatcher:matchAny("example.com")`},
{"ip_filter", "select(1, ipMatcher:FilterIPs(ips)) ~= nil"},
{"ip_filter_lower", "select(1, ipMatcher:filterIPs(ips)) ~= nil"},
{"domain_match", `#domainMatcher:Match("example.com") == 1`},
{"domain_match_lower", `#domainMatcher:match("example.com") == 1`},
{"ip_lua_table", "ipMatcher:AnyMatch({ip})"},
{"ip_lua_table_lower", "ipMatcher:anyMatch({ip})"},
} {
b.Run(benchmark.name, func(b *testing.B) {
if err := L.DoString(fmt.Sprintf("function benchmarkMatch() return %s end", benchmark.expression)); err != nil {
b.Fatal(err)
}
fn := L.GetGlobal("benchmarkMatch")
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
if err := L.CallByParam(lua.P{Fn: fn, NRet: 1, Protect: true}); err != nil {
b.Fatal(err)
}
if L.Get(-1) != lua.LTrue {
b.Fatal("matcher returned false")
}
L.Pop(1)
}
})
}
}
+8 -12
View File
@@ -52,7 +52,9 @@ func (g *MphIndexMatcher) Add(matcher Matcher) uint32 {
func (g *MphIndexMatcher) Build() error { func (g *MphIndexMatcher) Build() error {
if g.mph != nil { if g.mph != nil {
runtime.GC() // peak mem runtime.GC() // peak mem
g.mph.Build() if err := g.mph.Build(); err != nil {
return err
}
} }
runtime.GC() // peak mem runtime.GC() // peak mem
if g.ac != nil { if g.ac != nil {
@@ -64,23 +66,17 @@ func (g *MphIndexMatcher) Build() error {
// Match implements IndexMatcher.Match. // Match implements IndexMatcher.Match.
func (g *MphIndexMatcher) Match(input string) []uint32 { func (g *MphIndexMatcher) Match(input string) []uint32 {
result := make([][]uint32, 0, 5) var result []uint32
if g.mph != nil { if g.mph != nil {
if matches := g.mph.Match(input); len(matches) > 0 { result = g.mph.Match(input) // a new slice, returned without another copy
result = append(result, matches)
}
} }
if g.ac != nil { if g.ac != nil {
if matches := g.ac.Match(input); len(matches) > 0 { result = append(result, g.ac.Match(input)...)
result = append(result, matches)
}
} }
if g.regex != nil { if g.regex != nil {
if matches := g.regex.Match(input); len(matches) > 0 { result = append(result, g.regex.Match(input)...)
result = append(result, matches)
}
} }
return CompositeMatches(result) return result
} }
// MatchAny implements IndexMatcher.MatchAny. // MatchAny implements IndexMatcher.MatchAny.
@@ -78,6 +78,10 @@ func TestMphIndexMatcher(t *testing.T) {
Input: "example.com", Input: "example.com",
Output: []uint32{10, 4}, Output: []uint32{10, 4},
}, },
{
Input: "apis.org",
Output: []uint32{2, 6},
},
} }
matcherGroup := NewMphIndexMatcher() matcherGroup := NewMphIndexMatcher()
for _, rule := range rules { for _, rule := range rules {
@@ -87,8 +91,13 @@ func TestMphIndexMatcher(t *testing.T) {
} }
matcherGroup.Build() matcherGroup.Build()
for _, test := range cases { for _, test := range cases {
if m := matcherGroup.Match(test.Input); !reflect.DeepEqual(m, test.Output) { m := matcherGroup.Match(test.Input)
if !reflect.DeepEqual(m, test.Output) {
t.Error("unexpected output: ", m, " for test case ", test) t.Error("unexpected output: ", m, " for test case ", test)
} }
clear(m) // the caller owns the result, so this must not change the next one
if m := matcherGroup.Match(test.Input); !reflect.DeepEqual(m, test.Output) {
t.Error("unexpected output after clearing the previous one: ", m, " for test case ", test)
}
} }
} }
+375 -166
View File
@@ -1,231 +1,440 @@
package strmatcher package strmatcher
import ( import (
"bytes"
"cmp"
"encoding/binary"
"errors" "errors"
"math" "math"
"math/bits" "slices"
"runtime"
"sort"
"strings" "strings"
"unsafe" "unsafe"
) )
// PrimeRK is the prime base used in Rabin-Karp algorithm. // Flags of a level1 slot, stored above the record offset.
const PrimeRK = 16777619
// RollingHash calculates the rolling murmurHash of given string based on a provided suffix hash.
func RollingHash(hash uint32, input string) uint32 {
for i := len(input) - 1; i >= 0; i-- {
hash = hash*PrimeRK + uint32(input[i])
}
return hash
}
// MemHash is the hash function used by go map, it utilizes available hardware instructions(behaves
// as aeshash if aes instruction is available).
// With different seed, each MemHash<seed> performs as distinct hash functions.
func MemHash(seed uint32, input string) uint32 {
return uint32(strhash(unsafe.Pointer(&input), uintptr(seed))) // nosemgrep
}
const ( const (
mphMatchTypeCount = 2 // Full and Domain mphDomain = 1 << 31 // matches the pattern and its subdomains
mphFull = 1 << 30 // matches the pattern only
mphParent = 1 << 29 // matches subdomains only, from a pattern with a leading dot
mphOffMask = mphParent - 1
) )
type mphRuleInfo struct { // Kinds of an added pattern, indexes of mphKinds.
rollingHash uint32 const (
matchers [mphMatchTypeCount][]uint32 mphKindFull = iota
mphKindParent
mphKindDomain
)
// mphKinds are the slot flags in the order Match reports their values.
var mphKinds = [...]uint32{mphFull, mphParent, mphDomain}
// mphMultipliers are odd multipliers for the suffix hash. Build moves to the next one if two patterns collide.
var mphMultipliers = [...]uint64{0x9e3779b97f4a7c15, 0xc2b2ae3d27d4eb4f, 0x165667b19e3779f9, 0x27d4eb2f165667c5}
var (
errMphCollision = errors.New("strmatcher: suffix hash collision in MphMatcherGroup")
errMphBuilt = errors.New("strmatcher: MphMatcherGroup is already built")
)
type mphEntry struct {
off uint32 // pattern start in buf
value uint32
n uint32 // pattern length
kind uint8
} }
// MphMatcherGroup is an implementation of MatcherGroup. // MphMatcherGroup is an implementation of MatcherGroup for Full and Domain matchers.
// It implements Rabin-Karp algorithm and minimal perfect hash table for Full and Domain matcher. // Each distinct pattern is stored once as a record in arena: its length (255 means a uvarint length follows),
// its bytes and, if the group holds more than one distinct value, its values. A minimal perfect hash table
// built with hash, displace and compress (http://cmph.sourceforge.net/papers/esa09.pdf) maps a pattern to its
// record. Patterns are hashed from the right, so one pass over the input hashes all its parent domains.
type MphMatcherGroup struct { type MphMatcherGroup struct {
patterns string // All rule patterns concatenated arena string
patternOffs []uint32 // RuleIdx -> patterns[patternOffs[i]:patternOffs[i+1]], index 0 reserved for failed lookup level0 []uint16 // bucket -> seed
values []uint32 // All registered matcher values concatenated level1 []uint32 // slot -> flags | record offset
valueOffs []uint32 // RuleIdx -> values[valueOffs[i]:valueOffs[i+1]] (Full Matcher takes precedence) fp []uint8 // slot -> low byte of its pattern's hash, rejects most misses without reading arena
level0 []uint32 // RollingHash & Mask -> seed for Memhash n0, n1 uint32
level0Mask uint32 // Mask restricting RollingHash to 0 ~ len(level0) mul uint64 // multiplier of the suffix hash
level1 []uint32 // Memhash<seed> & Mask -> stored index for rules single uint32 // the only value if !multi
level1Mask uint32 // Mask for restricting Memhash<seed> to 0 ~ len(level1) multi bool
rules []string // RuleIdx -> pattern string, only used for building
ruleInfos *map[string]mphRuleInfo buf []byte // build only, patterns in Add order
entries []mphEntry
} }
func NewMphMatcherGroup() *MphMatcherGroup { func NewMphMatcherGroup() *MphMatcherGroup {
return &MphMatcherGroup{ return new(MphMatcherGroup)
rules: []string{""},
level0: nil,
level0Mask: 0,
level1: nil,
level1Mask: 0,
ruleInfos: &map[string]mphRuleInfo{}, // Only used for building, destroyed after build complete
}
} }
// AddFullMatcher implements MatcherGroupForFull. // AddFullMatcher implements MatcherGroupForFull.
func (g *MphMatcherGroup) AddFullMatcher(matcher FullMatcher, value uint32) { func (g *MphMatcherGroup) AddFullMatcher(matcher FullMatcher, value uint32) {
pattern := strings.ToLower(matcher.Pattern()) g.add(matcher.Pattern(), mphKindFull, value)
g.addPattern(0, "", pattern, matcher.Type(), value)
} }
// AddDomainMatcher implements MatcherGroupForDomain. // AddDomainMatcher implements MatcherGroupForDomain.
func (g *MphMatcherGroup) AddDomainMatcher(matcher DomainMatcher, value uint32) { func (g *MphMatcherGroup) AddDomainMatcher(matcher DomainMatcher, value uint32) {
pattern := strings.ToLower(matcher.Pattern()) g.add(matcher.Pattern(), mphKindDomain, value)
hash := g.addPattern(0, "", pattern, matcher.Type(), value) // For full domain match
g.addPattern(hash, pattern, ".", matcher.Type(), value) // For partial domain match
} }
func (g *MphMatcherGroup) addPattern(suffixHash uint32, suffixPattern string, pattern string, matcherType Type, value uint32) uint32 { func (g *MphMatcherGroup) add(pattern string, kind uint8, value uint32) {
fullPattern := pattern + suffixPattern if g.arena != "" {
info, found := (*g.ruleInfos)[fullPattern] panic(errMphBuilt)
if !found { }
info = mphRuleInfo{rollingHash: RollingHash(suffixHash, pattern)} pattern = strings.ToLower(pattern)
g.rules = append(g.rules, fullPattern) off := uint32(len(g.buf))
g.buf = append(g.buf, pattern...)
g.entries = append(g.entries, mphEntry{off: off, value: value, n: uint32(len(pattern)), kind: kind})
if len(pattern) > 0 && pattern[0] == '.' {
// ".x" has always matched "*.x" as well, so it also gets a parent-only record for "x"
g.entries = append(g.entries, mphEntry{off: off + 1, value: value, n: uint32(len(pattern) - 1), kind: mphKindParent})
} }
info.matchers[matcherType] = append(info.matchers[matcherType], value)
(*g.ruleInfos)[fullPattern] = info
return info.rollingHash
} }
// Build builds a minimal perfect hash table for insert rules. func (g *MphMatcherGroup) key(i uint32) []byte {
// Algorithm used: Hash, displace, and compress. See http://cmph.sourceforge.net/papers/esa09.pdf e := &g.entries[i]
return g.buf[e.off : e.off+e.n]
}
// Build builds the hash table. It must be called once, after the last Add.
func (g *MphMatcherGroup) Build() error { func (g *MphMatcherGroup) Build() error {
ruleCount := len(*g.ruleInfos) if g.arena != "" {
g.level0 = make([]uint32, nextPow2(ruleCount/4)) return errMphBuilt
g.level0Mask = uint32(len(g.level0) - 1)
g.level1 = make([]uint32, nextPow2(ruleCount))
g.level1Mask = uint32(len(g.level1) - 1)
// Flatten patterns and values so the built group has no per-rule objects
valueCount := 0
for _, ruleInfo := range *g.ruleInfos {
valueCount += len(ruleInfo.matchers[Full]) + len(ruleInfo.matchers[Domain])
} }
g.patterns = strings.Join(g.rules, "") if uint64(len(g.buf)) > math.MaxUint32 {
if uint64(len(g.patterns)) > math.MaxUint32 || uint64(valueCount) > math.MaxUint32 {
return errors.New("too many rules for MphMatcherGroup") return errors.New("too many rules for MphMatcherGroup")
} }
g.patternOffs = make([]uint32, len(g.rules)+1) recs := g.writeRecords()
g.values = make([]uint32, 0, valueCount) if len(g.arena) > mphOffMask {
g.valueOffs = make([]uint32, len(g.rules)+1) return errors.New("too many rules for MphMatcherGroup")
// Create buckets based on all rule's rolling hash
buckets := make([][]uint32, len(g.level0))
for ruleIdx := 1; ruleIdx < len(g.rules); ruleIdx++ { // Traverse rules starting from index 1 (0 reserved for failed lookup)
ruleInfo := (*g.ruleInfos)[g.rules[ruleIdx]]
bucketIdx := ruleInfo.rollingHash & g.level0Mask
buckets[bucketIdx] = append(buckets[bucketIdx], uint32(ruleIdx))
g.patternOffs[ruleIdx+1] = g.patternOffs[ruleIdx] + uint32(len(g.rules[ruleIdx]))
g.values = append(append(g.values, ruleInfo.matchers[Full]...), ruleInfo.matchers[Domain]...)
g.valueOffs[ruleIdx+1] = uint32(len(g.values))
} }
g.rules = nil hashes := make([]uint64, len(recs))
g.ruleInfos = nil // Set ruleInfos nil to release memory for _, mul := range mphMultipliers {
runtime.GC() // peak mem for i, rec := range recs {
hashes[i] = mphMix(mphHash(mul, g.recKey(rec)))
// Sort buckets in descending order with respect to each bucket's size }
bucketIdxs := make([]int, len(buckets)) g.mul = mul
for bucketIdx := range buckets { if err := g.place(recs, hashes); err != errMphCollision {
bucketIdxs[bucketIdx] = bucketIdx return err
}
} }
sort.Slice(bucketIdxs, func(i, j int) bool { return len(buckets[bucketIdxs[i]]) > len(buckets[bucketIdxs[j]]) }) return errMphCollision
}
// Exercise Hash, Displace, and Compress algorithm to construct minimal perfect hash table // writeRecords writes one record per distinct pattern to arena and returns flags | offset of each.
occupied := make([]bool, len(g.level1)) // Whether a second-level hash has been already used func (g *MphMatcherGroup) writeRecords() []uint32 {
hashedBucket := make([]uint32, 0, 4) // Second-level hashes for each rule in a specific bucket g.multi = false
for _, bucketIdx := range bucketIdxs { if len(g.entries) > 0 {
bucket := buckets[bucketIdx] g.single = g.entries[0].value
hashedBucket = hashedBucket[:0] for _, e := range g.entries {
seed := uint32(0) if e.value != g.single {
for len(hashedBucket) != len(bucket) { g.multi = true
for _, ruleIdx := range bucket { break
memHash := MemHash(seed, g.pattern(ruleIdx)) & g.level1Mask
if occupied[memHash] { // Collision occurred with this seed
for _, hash := range hashedBucket { // Revert all values in this hashed bucket
occupied[hash] = false
g.level1[hash] = 0
}
hashedBucket = hashedBucket[:0]
seed++ // Try next seed
break
}
occupied[memHash] = true
g.level1[memHash] = ruleIdx // The final value in the hash table
hashedBucket = append(hashedBucket, memHash)
} }
} }
g.level0[bucketIdx] = seed // Displacement value for this bucket }
// Equal patterns become neighbours in Add order, so their values keep their priority
order := make([]uint32, len(g.entries))
for i := range order {
order[i] = uint32(i)
}
slices.SortFunc(order, func(a, b uint32) int {
return cmp.Or(bytes.Compare(g.key(a), g.key(b)), cmp.Compare(a, b))
})
size := len(g.buf) + len(g.entries) + 2
if g.multi {
size += 3 * len(g.entries)
}
arena := make([]byte, 0, size)
recs := make([]uint32, 0, len(order))
var vals [len(mphKinds)][]uint32
for i := 0; i < len(order); {
k := g.key(order[i])
for t := range vals {
vals[t] = vals[t][:0]
}
for ; i < len(order) && bytes.Equal(g.key(order[i]), k); i++ {
e := &g.entries[order[i]]
if !slices.Contains(vals[e.kind], e.value) {
vals[e.kind] = append(vals[e.kind], e.value)
}
}
rec := uint32(len(arena))
if len(k) < 255 {
arena = append(arena, byte(len(k)))
} else {
arena = binary.AppendUvarint(append(arena, 255), uint64(len(k)))
}
arena = append(arena, k...)
for t, v := range vals {
if len(v) == 0 {
continue
}
rec |= mphKinds[t]
if g.multi {
arena = binary.AppendUvarint(arena, uint64(len(v)))
for _, x := range v {
arena = binary.AppendUvarint(arena, uint64(x))
}
}
}
recs = append(recs, rec)
}
// Lookups may point one byte past a pattern, and an empty group needs a record at offset 0 for empty slots
arena = append(arena, 0)
if len(recs) == 0 {
arena = append(arena, 0)
}
g.buf, g.entries = nil, nil
if cap(arena)-len(arena) > len(arena)/32 {
arena = slices.Clone(arena)
}
g.arena = unsafe.String(unsafe.SliceData(arena), len(arena)) // arena is not written after this
return recs
}
// place fills level0, level1 and fp: records are bucketed by hash, and each bucket, largest first, gets
// the first seed that puts all its records in free slots.
func (g *MphMatcherGroup) place(recs []uint32, hashes []uint64) error {
r := len(recs)
n0, n1 := max(1, r/3), max(1, r+r/99)
g.n0, g.n1 = uint32(n0), uint32(n1)
g.level0 = make([]uint16, n0)
g.level1 = make([]uint32, n1)
g.fp = make([]uint8, n1)
start := make([]uint32, n0+1)
for _, h := range hashes {
start[g.bucket(h)+1]++
}
for b := range n0 {
start[b+1] += start[b]
}
members := make([]uint32, r)
fill := slices.Clone(start[:n0])
for i, h := range hashes {
b := g.bucket(h)
members[fill[b]] = uint32(i)
fill[b]++
}
fill = nil
buckets := make([]uint32, n0)
for b := range buckets {
buckets[b] = uint32(b)
}
slices.SortStableFunc(buckets, func(a, b uint32) int {
return cmp.Compare(start[b+1]-start[b], start[a+1]-start[a])
})
occupied := make([]uint64, (n1+63)/64)
var slots []uint32
next:
for _, b := range buckets {
m := members[start[b]:start[b+1]]
if len(m) == 0 {
break
}
for i := range m {
for j := range i {
if hashes[m[i]] == hashes[m[j]] {
return errMphCollision // no seed can separate them
}
}
}
search:
for seed := range math.MaxUint16 + 1 {
slots = slots[:0]
for _, ri := range m {
s := g.slot(hashes[ri], uint16(seed))
if occupied[s/64]&(1<<(s%64)) != 0 || slices.Contains(slots, s) {
continue search
}
slots = append(slots, s)
}
for k, ri := range m {
s := slots[k]
occupied[s/64] |= 1 << (s % 64)
g.level1[s] = recs[ri]
g.fp[s] = uint8(hashes[ri])
}
g.level0[b] = uint16(seed)
continue next
}
return errors.New("strmatcher: no seed found for a bucket in MphMatcherGroup")
} }
return nil return nil
} }
func (g *MphMatcherGroup) pattern(ruleIdx uint32) string { // mphHash is the suffix hash of s, taken from the right: the hash of s[i:] is the state after reading s[i].
return g.patterns[g.patternOffs[ruleIdx]:g.patternOffs[ruleIdx+1]] func mphHash(mul uint64, s string) uint64 {
h := uint64(0)
for i := len(s) - 1; i >= 0; i-- {
h = h*mul + uint64(s[i])
}
return h
} }
// valuesOf caps the capacity, so appending to a Match result can't overwrite the next rule's values. // mphMix spreads the weak low bits of a suffix hash.
func (g *MphMatcherGroup) valuesOf(ruleIdx uint32) []uint32 { func mphMix(h uint64) uint64 {
start, end := g.valueOffs[ruleIdx], g.valueOffs[ruleIdx+1] h ^= h >> 32
return g.values[start:end:end] h *= 0xd6e8feb86659fd93
return h ^ h>>32
} }
// Lookup searches for input in minimal perfect hash table and returns its index. 0 indicates not found. func (g *MphMatcherGroup) bucket(f uint64) uint32 {
func (g *MphMatcherGroup) Lookup(rollingHash uint32, input string) uint32 { return uint32(((f >> 32) * uint64(g.n0)) >> 32)
i0 := rollingHash & g.level0Mask }
seed := g.level0[i0]
i1 := MemHash(seed, input) & g.level1Mask func (g *MphMatcherGroup) slot(f uint64, seed uint16) uint32 {
n := g.level1[i1] x := ((f ^ uint64(seed)*0x9e3779b97f4a7c15) * 0xc4ceb9fe1a85ec53) >> 32
// Build only puts valid rule indices in level1, so n+1 < len(patternOffs) and the span is inside patterns. return uint32((x * uint64(g.n1)) >> 32)
// Skip the bounds checks, they made this hot path measurably slower than indexing a []string }
offs := (*[2]uint32)(unsafe.Add(unsafe.Pointer(unsafe.SliceData(g.patternOffs)), uintptr(n)*4))
if start := offs[0]; int(offs[1]-start) == len(input) && unsafe.String((*byte)(unsafe.Add(unsafe.Pointer(unsafe.StringData(g.patterns)), start)), len(input)) == input { func (g *MphMatcherGroup) uvarint(p uint32) (x, next uint32) {
return n for shift := 0; ; shift += 7 {
c := g.arena[p]
p++
x |= uint32(c&0x7f) << shift
if c < 0x80 {
return x, p
}
}
}
// recSpan returns where the pattern of the record at off starts and how long it is.
func (g *MphMatcherGroup) recSpan(off uint32) (p, n uint32) {
n, p = uint32(g.arena[off]), off+1
if n == 255 {
n, p = g.uvarint(p)
}
return p, n
}
func (g *MphMatcherGroup) recKey(rec uint32) string {
p, n := g.recSpan(rec & mphOffMask)
return g.arena[p : p+n]
}
// lookup returns the level1 entry of s, or 0 if s is not a pattern. h is the suffix hash of s.
func (g *MphMatcherGroup) lookup(h uint64, s string) uint32 {
f := mphMix(h)
// bucket < n0 == len(level0) and slot < n1 == len(level1) == len(fp), skip the bounds checks
seed := *(*uint16)(unsafe.Add(unsafe.Pointer(unsafe.SliceData(g.level0)), uintptr(g.bucket(f))*2))
slot := uintptr(g.slot(f, seed))
if *(*uint8)(unsafe.Add(unsafe.Pointer(unsafe.SliceData(g.fp)), slot)) != uint8(f) {
return 0
}
e := *(*uint32)(unsafe.Add(unsafe.Pointer(unsafe.SliceData(g.level1)), slot*4))
if len(s) < 255 {
// A record whose length byte is len(s) has len(s) pattern bytes after it
p := unsafe.Add(unsafe.Pointer(unsafe.StringData(g.arena)), e&mphOffMask)
if int(*(*byte)(p)) == len(s) && unsafe.String((*byte)(unsafe.Add(p, 1)), len(s)) == s {
return e
}
return 0
}
if g.recKey(e) == s {
return e
} }
return 0 return 0
} }
// Match implements MatcherGroup.Match. // appendValues appends the values of record e for the flags in want, in mphKinds order.
func (g *MphMatcherGroup) Match(input string) []uint32 { func (g *MphMatcherGroup) appendValues(dst []uint32, e, want uint32) []uint32 {
matches := make([][]uint32, 0, 5) if !g.multi {
hash := uint32(0) for _, flag := range mphKinds {
for i := len(input) - 1; i >= 0; i-- { if e&want&flag != 0 {
hash = hash*PrimeRK + uint32(input[i]) dst = append(dst, g.single)
if input[i] == '.' { }
if mphIdx := g.Lookup(hash, input[i:]); mphIdx != 0 { }
matches = append(matches, g.valuesOf(mphIdx)) return dst
}
if e&want == 0 {
return dst
}
p, n := g.recSpan(e & mphOffMask)
p += n
for _, flag := range mphKinds {
if e&flag == 0 {
continue
}
var count, v uint32
for count, p = g.uvarint(p); count > 0; count-- {
v, p = g.uvarint(p)
if want&flag != 0 {
dst = append(dst, v)
} }
} }
} }
if mphIdx := g.Lookup(hash, input); mphIdx != 0 { return dst
matches = append(matches, g.valuesOf(mphIdx)) }
// Match implements MatcherGroup.Match. Values of an exact match come first (Full, then Domain), then those of
// the parent domains, nearest first.
func (g *MphMatcherGroup) Match(input string) []uint32 {
var stack [8]uint32
parents := stack[:0] // TLD side first
h, mul := uint64(0), g.mul
for i := len(input) - 1; i >= 0; i-- {
if input[i] == '.' {
if e := g.lookup(h, input[i+1:]); e&(mphDomain|mphParent) != 0 {
parents = append(parents, e)
}
}
h = h*mul + uint64(input[i])
} }
return CompositeMatchesReverse(matches) exact := g.lookup(h, input)
if exact&(mphFull|mphDomain) == 0 && len(parents) == 0 {
return nil
}
result := g.appendValues(make([]uint32, 0, len(parents)+1), exact, mphFull|mphDomain)
for k := len(parents) - 1; k >= 0; k-- {
result = g.appendValues(result, parents[k], mphParent|mphDomain)
}
return result
} }
// MatchAny implements MatcherGroup.MatchAny. // MatchAny implements MatcherGroup.MatchAny.
func (g *MphMatcherGroup) MatchAny(input string) bool { func (g *MphMatcherGroup) MatchAny(input string) bool {
hash := uint32(0) h, mul := uint64(0), g.mul
for i := len(input) - 1; i >= 0; i-- {
if input[i] == '.' && g.lookup(h, input[i+1:])&(mphDomain|mphParent) != 0 {
return true
}
h = h*mul + uint64(input[i])
}
return g.lookup(h, input)&(mphFull|mphDomain) != 0
}
// mphSuffix is the suffix hash of input[off:], a parent domain of the input.
type mphSuffix struct {
h uint64
off int
}
// mphSuffixes appends the suffix hashes of the parent domains of input to dst, TLD side first, and returns them
// with the hash of input itself: what MatchAny computes, computed once for several groups.
func mphSuffixes(dst []mphSuffix, mul uint64, input string) ([]mphSuffix, uint64) {
h := uint64(0)
for i := len(input) - 1; i >= 0; i-- { for i := len(input) - 1; i >= 0; i-- {
hash = hash*PrimeRK + uint32(input[i])
if input[i] == '.' { if input[i] == '.' {
if g.Lookup(hash, input[i:]) != 0 { dst = append(dst, mphSuffix{h, i + 1})
return true }
} h = h*mul + uint64(input[i])
}
return dst, h
}
// matchAnyHashed is MatchAny with parents and h from mphSuffixes(_, mul, input).
func (g *MphMatcherGroup) matchAnyHashed(input string, parents []mphSuffix, h, mul uint64) bool {
if g.mul != mul {
return g.MatchAny(input) // built with a later multiplier after a collision
}
for _, p := range parents {
if g.lookup(p.h, input[p.off:])&(mphDomain|mphParent) != 0 {
return true
} }
} }
return g.Lookup(hash, input) != 0 return g.lookup(h, input)&(mphFull|mphDomain) != 0
} }
func nextPow2(v int) int {
if v <= 1 {
return 1
}
const MaxUInt = ^uint(0)
n := (MaxUInt >> bits.LeadingZeros(uint(v))) + 1
return int(n)
}
//go:noescape
//go:linkname strhash runtime.strhash
func strhash(p unsafe.Pointer, h uintptr) uintptr
@@ -0,0 +1,108 @@
package strmatcher
import (
"slices"
"testing"
)
func TestMphMatcherGroupHashCollision(t *testing.T) {
saved := mphMultipliers
defer func() { mphMultipliers = saved }()
mphMultipliers[0] = 1 // anagrams collide
g := NewMphMatcherGroup()
g.AddFullMatcher(FullMatcher("ab.com"), 1)
g.AddDomainMatcher(DomainMatcher("ba.com"), 2)
g.AddDomainMatcher(DomainMatcher("com"), 3)
if err := g.Build(); err != nil {
t.Fatal(err)
}
if g.mul != saved[1] {
t.Errorf("multiplier %#x, want the second one %#x", g.mul, saved[1])
}
for input, want := range map[string][]uint32{"ab.com": {1, 3}, "x.ba.com": {2, 3}, "x.ab.com": {3}, "ba.com": {2, 3}} {
if m := g.Match(input); !slices.Equal(m, want) {
t.Errorf("Match(%q) = %v, want %v", input, m, want)
}
}
// Thue-Morse strings of 2048 bytes and their complements collide for every odd multiplier
mphMultipliers = saved
a, b := make([]byte, 2048), make([]byte, 2048)
for i := range a {
a[i], b[i] = "ab"[bitsOnes(i)%2], "ba"[bitsOnes(i)%2]
}
g = NewMphMatcherGroup()
g.AddFullMatcher(FullMatcher(a), 1)
g.AddFullMatcher(FullMatcher(b), 1)
if err := g.Build(); err != errMphCollision {
t.Errorf("Build() = %v, want %v", err, errMphCollision)
}
}
func bitsOnes(i int) int {
n := 0
for ; i > 0; i &= i - 1 {
n++
}
return n
}
func TestMphValueMatcherCombiner(t *testing.T) {
build := func(matchers ...Matcher) *MphValueMatcher {
m := NewMphValueMatcher()
for _, x := range matchers {
m.Add(x, 0)
}
if err := m.Build(); err != nil {
t.Fatal(err)
}
return m
}
regex, err := Regex.New(`^a\d+\.net$`)
if err != nil {
t.Fatal(err)
}
saved := mphMultipliers
t.Cleanup(func() { mphMultipliers = saved })
mphMultipliers[0] = 1 // anagrams collide, so this one falls back to its own hash pass
collided := build(FullMatcher("ab.com"), DomainMatcher("ba.com"))
mphMultipliers = saved
if collided.mph.mul == mphMultipliers[0] {
t.Fatal("collided matcher uses the first multiplier")
}
matchers := []*MphValueMatcher{
build(DomainMatcher("example.com"), FullMatcher("full.org"), DomainMatcher(".dot.io")),
collided,
build(regex, SubstrMatcher("keyword")),
build(),
build(DomainMatcher("com"), DomainMatcher("a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s")),
}
var s MphValueMatcherCombiner
for i, m := range matchers {
s.Add(m, uint32(10+i))
}
inputs := []string{
"", ".", "..", "com", "example.com", "www.example.com", "xexample.com", "example.com.", "full.org", "x.full.org",
"dot.io", "x.dot.io", ".dot.io", "ab.com", "x.ab.com", "ba.com", "x.ba.com", "a12.net", "a12.net.x", "my-keyword.org",
"a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s", "0.a.b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s", "b.c.d.e.f.g.h.i.j.k.l.m.n.o.p.q.r.s",
"x.y.z.1.2.3.4.5.6.7.8.9.10.11.12.13.14.15.16.17.ab.com", "x.y.z.1.2.3.4.5.6.7.8.9.10.11.12.13.14.15.16.17.org",
}
for _, input := range inputs {
var want []uint32
for i, m := range matchers {
if m.MatchAny(input) {
want = append(want, uint32(10+i))
}
}
if got := s.Match(input); !slices.Equal(got, want) {
t.Errorf("Match(%q) = %v, want %v", input, got, want)
}
if got := s.MatchAny(input); got != (len(want) > 0) {
t.Errorf("MatchAny(%q) = %v", input, got)
}
}
if n := testing.AllocsPerRun(100, func() { s.MatchAny("www.a.b.c.example.org") }); n != 0 {
t.Errorf("MatchAny allocates %v times", n)
}
}
@@ -4,6 +4,7 @@ import (
"math/rand" "math/rand"
"reflect" "reflect"
"slices" "slices"
"strings"
"testing" "testing"
"github.com/xtls/xray-core/common" "github.com/xtls/xray-core/common"
@@ -304,7 +305,7 @@ func TestMphMatcherGroupRandom(t *testing.T) {
domain["."+p] = append(domain["."+p], value) domain["."+p] = append(domain["."+p], value)
} }
} }
g.Build() common.Must(g.Build())
for _, input := range inputs { for _, input := range inputs {
keys := []string{input} // Whole input first, then "." suffixes from longest to shortest keys := []string{input} // Whole input first, then "." suffixes from longest to shortest
for i := range len(input) { for i := range len(input) {
@@ -316,7 +317,10 @@ func TestMphMatcherGroupRandom(t *testing.T) {
for _, k := range keys { for _, k := range keys {
want = append(append(want, full[k]...), domain[k]...) want = append(append(want, full[k]...), domain[k]...)
} }
if m := g.Match(input); !slices.Equal(m, want) { // Compared as sets: Match reports a value once per matching pattern, and orders them differently
// from want for patterns and inputs with a leading dot
m := g.Match(input)
if !slices.Equal(sortedSet(m), sortedSet(want)) {
t.Fatalf("seed %d: Match(%q) = %v, want %v", seed, input, m, want) t.Fatalf("seed %d: Match(%q) = %v, want %v", seed, input, m, want)
} }
if m := g.MatchAny(input); m != (len(want) > 0) { if m := g.MatchAny(input); m != (len(want) > 0) {
@@ -338,3 +342,79 @@ func TestMphMatcherGroupAppend(t *testing.T) {
t.Error("expect [2], but ", m) t.Error("expect [2], but ", m)
} }
} }
func sortedSet(v []uint32) []uint32 {
v = slices.Clone(v)
slices.Sort(v)
return slices.Compact(v)
}
func TestMphMatcherGroupLongPattern(t *testing.T) {
long := strings.Repeat("a", 300) + ".com"
for _, values := range [][4]uint32{{1, 2, 3, 4}, {7, 7, 7, 7}} {
g := NewMphMatcherGroup()
g.AddDomainMatcher(DomainMatcher(long), values[0])
g.AddFullMatcher(FullMatcher("x."+long), values[1])
g.AddFullMatcher(FullMatcher(long[:255]), values[2]) // the shortest pattern stored with a long length
g.AddFullMatcher(FullMatcher(long[:254]), values[3])
common.Must(g.Build())
cases := []struct {
input string
want []uint32
}{
{long, []uint32{values[0]}},
{"www." + long, []uint32{values[0]}},
{"x." + long, []uint32{values[1], values[0]}},
{long[1:], nil},
{"a" + long, nil},
{long[:255], []uint32{values[2]}},
{long[:254], []uint32{values[3]}},
{long[:256], nil},
{long[:253], nil},
}
for _, c := range cases {
if m := g.Match(c.input); !slices.Equal(m, c.want) {
t.Errorf("Match(%d bytes) = %v, want %v", len(c.input), m, c.want)
}
if m := g.MatchAny(c.input); m != (c.want != nil) {
t.Errorf("MatchAny(%d bytes) = %v", len(c.input), m)
}
}
}
// A pattern longer than 65535 bytes builds and matches: a record's length is a uvarint,
// so the only cap was the build-time length field, now widened to uint32.
huge := strings.Repeat("a", 70000)
g := NewMphMatcherGroup()
g.AddFullMatcher(FullMatcher(strings.Repeat("a", 65535)), 1)
g.AddDomainMatcher(DomainMatcher(huge+".com"), 2)
g.AddFullMatcher(FullMatcher("a.com"), 3)
common.Must(g.Build())
if !g.MatchAny(strings.Repeat("a", 65535)) || g.MatchAny(strings.Repeat("a", 65534)) {
t.Error("wrong answer for a 65535-byte pattern")
}
if m := g.Match(huge + ".com"); !slices.Equal(m, []uint32{2}) {
t.Errorf("Match(%d-byte input) = %v, want [2]", len(huge)+4, m)
}
if m := g.Match("x." + huge + ".com"); !slices.Equal(m, []uint32{2}) {
t.Errorf("Match(subdomain of a %d-byte pattern) = %v, want [2]", len(huge)+4, m)
}
if g.MatchAny(huge) { // the 70000-byte label on its own is not a rule
t.Error("unexpected match for the bare 70000-byte label")
}
}
func TestMphMatcherGroupBuildOnce(t *testing.T) {
g := NewMphMatcherGroup()
g.AddFullMatcher(FullMatcher("a.com"), 1)
common.Must(g.Build())
if err := g.Build(); err == nil || !g.MatchAny("a.com") {
t.Errorf("second Build() = %v, MatchAny(a.com) = %v", err, g.MatchAny("a.com"))
}
defer func() {
if recover() == nil {
t.Error("Add after Build did not panic")
}
}()
g.AddDomainMatcher(DomainMatcher("b.com"), 2)
}
+237 -1
View File
@@ -2,10 +2,12 @@ package strmatcher
import ( import (
"errors" "errors"
"math/bits"
"regexp" "regexp"
"regexp/syntax" "regexp/syntax"
"slices" "slices"
"strings" "strings"
"unicode"
"unicode/utf8" "unicode/utf8"
"golang.org/x/net/idna" "golang.org/x/net/idna"
@@ -75,7 +77,9 @@ func (m SubstrMatcher) Match(s string) bool {
// RegexMatcher is an implementation of Matcher. // RegexMatcher is an implementation of Matcher.
type RegexMatcher struct { type RegexMatcher struct {
pattern *regexp.Regexp pattern *regexp.Regexp
literals []string // every match contains all of them, longest first literals []string // every match contains all of them, longest first
tail []byteSet // tail[i] holds the bytes a matching input can have i bytes before its end
rest *byteSet // the bytes it can have further before, nil if any
} }
func newRegexMatcher(pattern string) (Matcher, error) { func newRegexMatcher(pattern string) (Matcher, error) {
@@ -87,10 +91,239 @@ func newRegexMatcher(pattern string) (Matcher, error) {
if re, err := syntax.Parse(pattern, syntax.Perl); err == nil { // same flags as regexp.Compile if re, err := syntax.Parse(pattern, syntax.Perl); err == nil { // same flags as regexp.Compile
m.literals = requiredLiterals(re, nil) m.literals = requiredLiterals(re, nil)
slices.SortStableFunc(m.literals, func(a, b string) int { return len(b) - len(a) }) slices.SortStableFunc(m.literals, func(a, b string) int { return len(b) - len(a) })
m.tail, m.rest = tailGuard(re)
} }
return m, nil return m, nil
} }
// byteSet is a set of bytes. The bytes >= 0x80 share one bit with 0x7f.
type byteSet [4]uint32
func (s *byteSet) add(c byte) { c = min(c, 0x7f); s[c>>5] |= 1 << (c & 31) }
func (s *byteSet) has(c byte) bool { c = min(c, 0x7f); return s[c>>5]&(1<<(c&31)) != 0 }
func (s *byteSet) or(t *byteSet) {
for i := range s {
s[i] |= t[i]
}
}
var allBytes = byteSet{^uint32(0), ^uint32(0), ^uint32(0), ^uint32(0)}
// tailLen is how many positions before the end of the input tailGuard tells apart.
const tailLen = 8
// tailBudget caps how many repetition steps tailGuard walks. Only nested repeats can make the
// walk explode, so only they are charged: a flat pattern, however long, is walked once and keeps
// its guard.
const tailBudget = 100000
// tailWalk is a set of positions in the input, counted in bytes before its end.
type tailWalk struct {
at uint32 // bit i: exactly i bytes before the end, for i < tailLen
far bool // tailLen or more bytes before the end
free bool // not tied to the end of the input yet
}
func (w tailWalk) union(v tailWalk) tailWalk {
return tailWalk{w.at | v.at, w.far || v.far, w.free || v.free}
}
type tailBuilder struct {
tail [tailLen]byteSet
rest byteSet
void bool
work int
}
// tailGuard walks re backwards from the end of the input and collects the bytes an input
// matching re can have at each position before its end. It returns nil, nil when a branch
// of re does not end with $ or when nested repeats push the walk past tailBudget.
func tailGuard(re *syntax.Regexp) ([]byteSet, *byteSet) {
var b tailBuilder
w := b.walk(re, tailWalk{free: true})
b.stop(w)
if b.void {
return nil, nil
}
if w.at != 0 { // a match can start here, so any bytes can come before
for i := bits.TrailingZeros32(w.at); i < tailLen; i++ {
b.tail[i] = allBytes
}
}
if w.at != 0 || w.far {
b.rest = allBytes
}
n := tailLen
for n > 0 && b.tail[n-1] == b.rest {
n--
}
var tail []byteSet
if n > 0 {
tail = slices.Clone(b.tail[:n])
}
if b.rest != allBytes {
rest := b.rest
return tail, &rest
}
return tail, nil
}
// stop ends the paths of w. One that never met $ lets its match be followed by anything.
func (b *tailBuilder) stop(w tailWalk) {
if w.free {
b.void = true
}
}
func (b *tailBuilder) walk(re *syntax.Regexp, w tailWalk) tailWalk {
if w == (tailWalk{}) || b.void {
return w
}
switch re.Op {
case syntax.OpNoMatch:
return tailWalk{}
case syntax.OpLiteral:
for i := len(re.Rune) - 1; i >= 0; i-- {
var set byteSet
set.add(byte(min(re.Rune[i], utf8.RuneSelf)))
if re.Flags&syntax.FoldCase != 0 {
for f := unicode.SimpleFold(re.Rune[i]); f != re.Rune[i]; f = unicode.SimpleFold(f) {
set.add(byte(min(f, utf8.RuneSelf)))
}
}
w = b.step(w, &set)
}
return w
case syntax.OpCharClass:
var set byteSet
for i := 0; i+1 < len(re.Rune); i += 2 {
for r := min(re.Rune[i], utf8.RuneSelf); r <= min(re.Rune[i+1], utf8.RuneSelf); r++ {
set.add(byte(r))
}
}
return b.step(w, &set)
case syntax.OpAnyChar, syntax.OpAnyCharNotNL: // a domain has no \n to reject
return b.step(w, &allBytes)
case syntax.OpBeginText: // nothing comes before
b.stop(w)
return tailWalk{}
case syntax.OpEndText:
out := tailWalk{at: w.at & 1}
if w.free {
out.at = 1
}
return out
case syntax.OpCapture:
return b.walk(re.Sub[0], w)
case syntax.OpConcat:
for i := len(re.Sub) - 1; i >= 0; i-- {
w = b.walk(re.Sub[i], w)
}
return w
case syntax.OpAlternate:
var out tailWalk
for _, sub := range re.Sub {
out = out.union(b.walk(sub, w))
}
return out
case syntax.OpQuest:
return b.repeat(re.Sub[0], w, 1)
case syntax.OpStar:
return b.repeat(re.Sub[0], w, -1)
case syntax.OpPlus:
return b.repeat(re.Sub[0], b.walk(re.Sub[0], w), -1)
case syntax.OpRepeat:
for i := 0; i < re.Min; i++ {
if b.charge() {
return w
}
w = b.walk(re.Sub[0], w)
}
if re.Max < 0 {
return b.repeat(re.Sub[0], w, -1)
}
return b.repeat(re.Sub[0], w, re.Max-re.Min)
}
return w // empty match, line and word boundaries: no constraint
}
// charge counts one repetition step and reports whether the walk has run out of budget. Only
// repeats re-walk their body, so charging them alone bounds the blow-up of nested repeats while
// leaving a single linear pass, of any length, free.
func (b *tailBuilder) charge() bool {
b.work++
if b.work > tailBudget {
b.void = true
}
return b.void
}
// repeat walks back over up to n more repetitions of re, any number if n < 0.
func (b *tailBuilder) repeat(re *syntax.Regexp, w tailWalk, n int) tailWalk {
for ; n != 0; n-- {
if b.charge() {
return w
}
next := w.union(b.walk(re, w))
if next == w {
break
}
w = next
}
return w
}
// step walks back over one character whose last byte is in set. A character that can be
// non-ASCII can take up to 4 bytes, all >= 0x80; regexp matches an invalid byte as U+FFFD.
func (b *tailBuilder) step(w tailWalk, set *byteSet) tailWalk {
out := tailWalk{far: w.far, free: w.free}
if w.far {
b.rest.or(set)
}
width := 1
if set.has(0x80) {
width = utf8.UTFMax
}
for i := 0; i < tailLen; i++ {
if w.at&(1<<i) == 0 {
continue
}
b.tail[i].or(set)
for n := 1; n <= width; n++ {
if j := i + n; j < tailLen {
out.at |= 1 << j
if n < width {
b.tail[j].add(0x80)
}
} else {
out.far = true
if n < width {
b.rest.add(0x80)
}
}
}
}
return out
}
// mayMatch reports whether s passes the tail guard.
func (m *RegexMatcher) mayMatch(s string) bool {
n := len(s)
if m.rest == nil {
n = min(n, len(m.tail))
}
for i := 0; i < n; i++ {
set := m.rest
if i < len(m.tail) {
set = &m.tail[i]
}
if !set.has(s[len(s)-1-i]) {
return false
}
}
return true
}
// requiredLiterals appends to dst the case-sensitive strings that every match of re contains. // requiredLiterals appends to dst the case-sensitive strings that every match of re contains.
func requiredLiterals(re *syntax.Regexp, dst []string) []string { func requiredLiterals(re *syntax.Regexp, dst []string) []string {
switch re.Op { switch re.Op {
@@ -126,6 +359,9 @@ func (m *RegexMatcher) String() string {
} }
func (m *RegexMatcher) Match(s string) bool { func (m *RegexMatcher) Match(s string) bool {
if !m.mayMatch(s) {
return false
}
for _, l := range m.literals { for _, l := range m.literals {
if !strings.Contains(s, l) { if !strings.Contains(s, l) {
return false return false
@@ -1,9 +1,16 @@
package strmatcher package strmatcher
import ( import (
"hash/fnv"
"math/rand/v2"
"regexp" "regexp"
"regexp/syntax"
"slices" "slices"
"strconv"
"strings"
"testing" "testing"
"unicode"
"unicode/utf8"
) )
var regexLiteralCases = []struct { var regexLiteralCases = []struct {
@@ -37,6 +44,147 @@ func TestRegexRequiredLiterals(t *testing.T) {
} }
} }
var regexTailCases = []struct {
pattern string
guard bool
match []string // inputs the pattern matches
reject []string // inputs the tail guard alone rejects
}{
{`^[a-z]([a-z0-9-]{0,61}[a-z0-9])?$`, true, []string{"a", "localhost", "x-1"}, []string{"www.example.com", "localhost.", "LOCALHOST", "a b"}},
{`(^|\.)[a-z][1-9][0-9][a-z]\.com$`, true, []string{"a12b.com", "x.q10z.com"}, []string{"google.com", "a12b.co", "a12b.com.", "ab12.com"}},
{`^hses[1-7]?\.akamaized\.net$`, true, []string{"hses.akamaized.net", "hses3.akamaized.net"}, []string{"xhses.akamaized.net", "www.hses.akamaized.net"}},
{`(?i)k\.net$`, true, []string{"k.net", "K.NET", "\u212a.net"}, []string{"x.net", "k.nex"}},
{`[^.]+\.cn$`, true, []string{"a.cn", "\xff.cn", "\u4e2d.cn"}, []string{"a.cnn", "a.c"}},
{`\x{FFFD}$`, true, []string{"\xff", "a\xc3", "\uFFFD"}, []string{"a", "\xff."}},
{`^.\.cn$`, true, []string{"a.cn", "\u4E2D.cn", "\xff.cn"}, []string{"ab.cn"}},
{`^$`, true, []string{""}, []string{"a"}},
{`(^|\.)youyuapi\..+$`, false, []string{"youyuapi.com"}, nil},
{`abc`, false, []string{"abc", "xabcx"}, nil},
{`^ab`, false, []string{"ab", "abc"}, nil},
{`a$|b`, false, []string{"a", "bx"}, nil},
{`(?m)a$`, false, []string{"a", "a\nb"}, nil},
{strings.Repeat(`(?:abcdefgh(?:a`, 20) + strings.Repeat(`)*)*`, 20) + `\.com$`, false, []string{".com", "abcdefgha.com"}, nil}, // over tailBudget
}
func TestRegexTailGuard(t *testing.T) {
for _, test := range regexTailCases {
m, err := newRegexMatcher(test.pattern)
if err != nil {
t.Fatal(err)
}
rm := m.(*RegexMatcher)
if guard := rm.tail != nil || rm.rest != nil; guard != test.guard {
t.Errorf("%s: guard %v, want %v", test.pattern, guard, test.guard)
}
for _, s := range test.match {
if !rm.pattern.MatchString(s) || !rm.Match(s) {
t.Errorf("%s: %q does not match", test.pattern, s)
}
}
for _, s := range test.reject {
if rm.pattern.MatchString(s) || rm.mayMatch(s) {
t.Errorf("%s: %q passes the guard", test.pattern, s)
}
}
}
}
// TestRegexTailGuardFlatAlternation checks that a long but non-recursive pattern keeps its
// guard. Only nested repeats are charged against tailBudget, so a flat alternation of many
// names, however large, is walked once and guarded; its guard is checked against regexp.
func TestRegexTailGuardFlatAlternation(t *testing.T) {
var sb strings.Builder
sb.WriteString("(?:")
for i := 0; i < 20000; i++ {
if i > 0 {
sb.WriteByte('|')
}
sb.WriteString("name")
sb.WriteString(strconv.Itoa(i))
}
sb.WriteString(`)\.example\.com$`)
m, err := newRegexMatcher(sb.String())
if err != nil {
t.Fatal(err)
}
rm := m.(*RegexMatcher)
if rm.tail == nil && rm.rest == nil {
t.Fatal("flat alternation of 20000 names lost its guard")
}
for _, s := range []string{"name0.example.com", "name19999.example.com", "x.name12345.example.com"} {
if !rm.pattern.MatchString(s) || !rm.Match(s) {
t.Errorf("%q should match", s)
}
}
for _, s := range []string{"name0.example.org", "name0.example.com.", "name0.example.con", "google.com"} {
if rm.pattern.MatchString(s) {
t.Fatalf("test bug: %q matches the pattern", s)
}
if rm.mayMatch(s) {
t.Errorf("%q should be rejected by the guard", s)
}
}
}
// sampleMatch appends a string that re matches, assertions aside, unless it runs out of
// budget, which it spends one per call so that nested repeats stay cheap.
func sampleMatch(sb *strings.Builder, re *syntax.Regexp, rnd *rand.Rand, budget *int) {
if *budget <= 0 {
return
}
*budget--
switch re.Op {
case syntax.OpLiteral:
for _, r := range re.Rune {
if re.Flags&syntax.FoldCase != 0 {
for n := rnd.IntN(4); n > 0; n-- {
r = unicode.SimpleFold(r)
}
}
sampleRune(sb, r, rnd)
}
case syntax.OpCharClass:
if len(re.Rune) > 0 {
i := rnd.IntN(len(re.Rune)/2) * 2
sampleRune(sb, re.Rune[i]+rnd.Int32N(min(re.Rune[i+1]-re.Rune[i]+1, 300)), rnd)
}
case syntax.OpAnyChar, syntax.OpAnyCharNotNL:
sampleRune(sb, []rune{'a', '.', '\n', 0xe9, 0x212a, utf8.RuneError}[rnd.IntN(6)], rnd)
case syntax.OpCapture:
sampleMatch(sb, re.Sub[0], rnd, budget)
case syntax.OpConcat:
for _, sub := range re.Sub {
sampleMatch(sb, sub, rnd, budget)
}
case syntax.OpAlternate:
sampleMatch(sb, re.Sub[rnd.IntN(len(re.Sub))], rnd, budget)
case syntax.OpQuest, syntax.OpStar, syntax.OpPlus, syntax.OpRepeat:
lo, hi := 0, 3
switch re.Op {
case syntax.OpQuest:
hi = 1
case syntax.OpPlus:
lo = 1
case syntax.OpRepeat:
lo, hi = re.Min, re.Min+3
if re.Max >= 0 {
hi = min(hi, re.Max)
}
}
for n := lo + rnd.IntN(hi-lo+1); n > 0; n-- {
sampleMatch(sb, re.Sub[0], rnd, budget)
}
}
}
func sampleRune(sb *strings.Builder, r rune, rnd *rand.Rand) {
if r == utf8.RuneError && rnd.IntN(2) == 0 {
sb.WriteByte(0x80 | byte(rnd.IntN(0x80))) // regexp matches an invalid byte as U+FFFD
return
}
sb.WriteRune(r)
}
func FuzzRegexMatcher(f *testing.F) { func FuzzRegexMatcher(f *testing.F) {
inputs := []string{ inputs := []string{
"", "x", "yy", "abd", "ccc", "ABC", "abCDef", "abcdef", "abababcc", "a.b", "a\xffb", "a\uFFFDb", "", "x", "yy", "abd", "ccc", "ABC", "abCDef", "abcdef", "abababcc", "a.b", "a\xffb", "a\uFFFDb",
@@ -47,14 +195,39 @@ func FuzzRegexMatcher(f *testing.F) {
f.Add(test.pattern, s) f.Add(test.pattern, s)
} }
} }
for _, test := range regexTailCases {
for _, s := range append(test.match, test.reject...) {
f.Add(test.pattern, s)
}
}
f.Fuzz(func(t *testing.T, pattern, s string) { f.Fuzz(func(t *testing.T, pattern, s string) {
re, err := regexp.Compile(pattern) re, err := regexp.Compile(pattern)
if err != nil { if err != nil {
return return
} }
m, _ := newRegexMatcher(pattern) m, _ := newRegexMatcher(pattern)
if got, want := m.Match(s), re.MatchString(s); got != want { check := func(s string) {
t.Errorf("pattern %q, input %q: got %v, want %v", pattern, s, got, want) if got, want := m.Match(s), re.MatchString(s); got != want {
t.Errorf("pattern %q, input %q: got %v, want %v", pattern, s, got, want)
}
}
check(s)
// random inputs seldom match, so also try strings built from the pattern
parsed, _ := syntax.Parse(pattern, syntax.Perl)
h := fnv.New64a()
h.Write([]byte(s))
rnd := rand.New(rand.NewPCG(h.Sum64(), 1))
for range 8 {
var sb strings.Builder
budget := 256
sampleMatch(&sb, parsed, rnd, &budget)
sample := sb.String()
check(sample)
check(s + sample)
if len(sample) > 0 && len(s) > 0 {
i := rnd.IntN(len(sample))
check(sample[:i] + s[:1] + sample[i+1:])
}
} }
}) })
} }
+67 -12
View File
@@ -46,7 +46,9 @@ func (g *MphValueMatcher) Add(matcher Matcher, value uint32) {
func (g *MphValueMatcher) Build() error { func (g *MphValueMatcher) Build() error {
if g.mph != nil { if g.mph != nil {
runtime.GC() // peak mem runtime.GC() // peak mem
g.mph.Build() if err := g.mph.Build(); err != nil {
return err
}
} }
runtime.GC() // peak mem runtime.GC() // peak mem
if g.ac != nil { if g.ac != nil {
@@ -58,23 +60,17 @@ func (g *MphValueMatcher) Build() error {
// Match implements ValueMatcher.Match. // Match implements ValueMatcher.Match.
func (g *MphValueMatcher) Match(input string) []uint32 { func (g *MphValueMatcher) Match(input string) []uint32 {
result := make([][]uint32, 0, 5) var result []uint32
if g.mph != nil { if g.mph != nil {
if matches := g.mph.Match(input); len(matches) > 0 { result = g.mph.Match(input) // a new slice, returned without another copy
result = append(result, matches)
}
} }
if g.ac != nil { if g.ac != nil {
if matches := g.ac.Match(input); len(matches) > 0 { result = append(result, g.ac.Match(input)...)
result = append(result, matches)
}
} }
if g.regex != nil { if g.regex != nil {
if matches := g.regex.Match(input); len(matches) > 0 { result = append(result, g.regex.Match(input)...)
result = append(result, matches)
}
} }
return CompositeMatches(result) return result
} }
// MatchAny implements ValueMatcher.MatchAny. // MatchAny implements ValueMatcher.MatchAny.
@@ -87,3 +83,62 @@ func (g *MphValueMatcher) MatchAny(input string) bool {
} }
return g.regex != nil && g.regex.MatchAny(input) return g.regex != nil && g.regex.MatchAny(input)
} }
func (g *MphValueMatcher) matchAnyHashed(input string, parents []mphSuffix, h, mul uint64) bool {
if g.mph != nil && g.mph.matchAnyHashed(input, parents, h, mul) {
return true
}
if g.ac != nil && g.ac.MatchAny(input) {
return true
}
return g.regex != nil && g.regex.MatchAny(input)
}
// MphValueMatcherCombiner combines several built MphValueMatchers, each bound to one value, and matches an input
// against them as their MatchAny would, hashing the input once for all of them.
type MphValueMatcherCombiner struct {
matchers []*MphValueMatcher
values []uint32
}
// Add adds a built matcher that stands for value.
func (s *MphValueMatcherCombiner) Add(m *MphValueMatcher, value uint32) {
s.matchers = append(s.matchers, m)
s.values = append(s.values, value)
}
// Match returns the values of the matchers that match input, in Add order.
func (s *MphValueMatcherCombiner) Match(input string) []uint32 {
if len(s.matchers) == 0 {
return nil
}
var stack [16]mphSuffix
mul := mphMultipliers[0]
parents, h := mphSuffixes(stack[:0], mul, input)
var result []uint32
for i, m := range s.matchers {
if m.matchAnyHashed(input, parents, h, mul) {
result = append(result, s.values[i])
}
}
return result
}
// MatchAny returns true as soon as one matcher matches input.
func (s *MphValueMatcherCombiner) MatchAny(input string) bool {
switch len(s.matchers) {
case 0:
return false
case 1:
return s.matchers[0].MatchAny(input) // nothing to share, and it stops at the first matching suffix
}
var stack [16]mphSuffix
mul := mphMultipliers[0]
parents, h := mphSuffixes(stack[:0], mul, input)
for _, m := range s.matchers {
if m.matchAnyHashed(input, parents, h, mul) {
return true
}
}
return false
}
+61
View File
@@ -0,0 +1,61 @@
package log
import (
"path/filepath"
"strings"
lua "github.com/yuin/gopher-lua"
)
// RegisterLua makes xray.log available to require in an LState.
func RegisterLua(L *lua.LState) {
L.PreloadModule("xray.log", func(L *lua.LState) int {
module := L.CreateTable(0, 4)
var source, prefix string // cache
for name, severity := range map[string]Severity{
"Debug": Severity_Debug,
"Info": Severity_Info,
"Warning": Severity_Warning,
"Error": Severity_Error,
} {
module.RawSetString(name, L.NewFunction(func(L *lua.LState) int {
if GetSeverity() < severity {
return 0
}
var content strings.Builder
// Prefix with the calling script's filename.
if caller, ok := L.GetStack(1); ok {
if _, err := L.GetInfo("S", caller, lua.LNil); err == nil && caller.Source != "" {
if caller.Source != source {
source = caller.Source
prefix = filepath.Base(strings.TrimPrefix(source, "@")) + ": "
}
content.WriteString(prefix)
}
}
for i := 1; i <= L.GetTop(); i++ {
content.WriteString(luaLogString(L, L.Get(i)))
}
Record(&GeneralMessage{
Severity: severity,
Content: content.String(),
})
return 0
}))
}
L.Push(module)
return 1
})
}
func luaLogString(L *lua.LState, value lua.LValue) string {
if ud, ok := value.(*lua.LUserData); ok {
if err, ok := ud.Value.(error); ok {
return err.Error()
}
}
if _, ok := L.GetMetaField(value, "__tostring").(*lua.LFunction); ok {
return L.ToStringMeta(value).String()
}
return value.String()
}
+213
View File
@@ -0,0 +1,213 @@
package log
import (
"errors"
"fmt"
"os"
"path/filepath"
"testing"
lua "github.com/yuin/gopher-lua"
)
type luaLogHandler struct {
messages []Message
}
func (h *luaLogHandler) Handle(msg Message) {
h.messages = append(h.messages, msg)
}
func TestLuaLog(t *testing.T) {
previous := logHandler.Load()
t.Cleanup(func() { logHandler.Store(previous) })
handler := &luaLogHandler{}
RegisterHandler(handler)
L := lua.NewState()
defer L.Close()
RegisterLua(L)
nativeError := L.NewUserData()
nativeError.Value = fmt.Errorf("lookup failed: %w", errors.New("upstream timeout"))
L.SetGlobal("nativeError", nativeError)
path := filepath.Join(t.TempDir(), "logging.lua")
if err := os.WriteFile(path, []byte(`
local log = require("xray.log")
assert(log == require("xray.log"))
log.Debug("query: ", "example.com")
log.Info("count=", 42, ", enabled=", true, ", value=", nil)
log.Warning(setmetatable({}, {
__tostring = function() return "fallback" end
}))
assert(select("#", log.Error("failed")) == 0)
log.Error("DNS failed: ", nativeError)
log.Warning(nativeError)
local ok, err = pcall(function() error("Lua failure", 0) end)
assert(not ok)
log.Error(err)
local calls = 0
local custom = setmetatable({}, {
__tostring = function() calls = calls + 1; return "custom" end
})
log.Info(custom, custom)
assert(calls == 2)
log.Info("a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l")
log.Info()
function logHook()
log.Info("hook")
end
`), 0o600); err != nil {
t.Fatal(err)
}
if err := L.DoFile(path); err != nil {
t.Fatal(err)
}
if err := L.DoString(`
logHook()
require("xray.log").Info("anonymous")
`); err != nil {
t.Fatal(err)
}
other := filepath.Join(t.TempDir(), "other.lua")
if err := os.WriteFile(other, []byte(`
local log = require("xray.log")
log.Info("other")
logHook()
log.Info("other again")
`), 0o600); err != nil {
t.Fatal(err)
}
if err := L.DoFile(other); err != nil {
t.Fatal(err)
}
want := []struct {
severity Severity
message string
}{
{Severity_Debug, "[Debug] logging.lua: query: example.com"},
{Severity_Info, "[Info] logging.lua: count=42, enabled=true, value=nil"},
{Severity_Warning, "[Warning] logging.lua: fallback"},
{Severity_Error, "[Error] logging.lua: failed"},
{Severity_Error, "[Error] logging.lua: DNS failed: lookup failed: upstream timeout"},
{Severity_Warning, "[Warning] logging.lua: lookup failed: upstream timeout"},
{Severity_Error, "[Error] logging.lua: Lua failure"},
{Severity_Info, "[Info] logging.lua: customcustom"},
{Severity_Info, "[Info] logging.lua: abcdefghijkl"},
{Severity_Info, "[Info] logging.lua: "},
{Severity_Info, "[Info] logging.lua: hook"},
{Severity_Info, "[Info] <string>: anonymous"},
{Severity_Info, "[Info] other.lua: other"},
{Severity_Info, "[Info] logging.lua: hook"},
{Severity_Info, "[Info] other.lua: other again"},
}
if len(handler.messages) != len(want) {
t.Fatalf("logged %d messages, want %d", len(handler.messages), len(want))
}
for i, expected := range want {
msg, ok := handler.messages[i].(*GeneralMessage)
if !ok {
t.Fatalf("message %d has type %T, want *GeneralMessage", i, handler.messages[i])
}
if msg.Severity != expected.severity || msg.String() != expected.message {
t.Errorf("message %d = %q with severity %v, want %q with severity %v", i, msg.String(), msg.Severity, expected.message, expected.severity)
}
}
}
type luaSeverityLogHandler struct {
luaLogHandler
level Severity
}
func (h *luaSeverityLogHandler) Severity() Severity { return h.level }
func TestLuaLogSeverity(t *testing.T) {
previous := logHandler.Load()
t.Cleanup(func() { logHandler.Store(previous) })
L := lua.NewState()
defer L.Close()
RegisterLua(L)
for _, level := range []Severity{Severity_Unknown, Severity_Error, Severity_Warning, Severity_Info, Severity_Debug, Severity_Warning} {
t.Run(level.String(), func(t *testing.T) {
handler := &luaSeverityLogHandler{level: level}
RegisterHandler(handler)
want := []Severity{}
for _, severity := range []Severity{Severity_Error, Severity_Warning, Severity_Info, Severity_Debug} {
if severity <= level {
want = append(want, severity)
}
}
if err := L.DoString(fmt.Sprintf(`
local log = require("xray.log")
local calls = 0
local value = setmetatable({}, {
__tostring = function() calls = calls + 1; return "message" end
})
for _, write in ipairs({log.Error, log.Warning, log.Info, log.Debug}) do
assert(select("#", write(value)) == 0)
end
assert(calls == %d)
`, len(want))); err != nil {
t.Fatal(err)
}
if len(handler.messages) != len(want) {
t.Fatalf("logged %d messages, want %d", len(handler.messages), len(want))
}
for i, severity := range want {
msg := handler.messages[i].(*GeneralMessage)
if msg.Severity != severity || msg.Content != "<string>: message" {
t.Errorf("message %d = %v, want severity %v and content %q", i, msg, severity, "<string>: message")
}
}
})
}
}
type luaDiscardLogHandler struct{ level Severity }
func (luaDiscardLogHandler) Handle(Message) {}
func (h luaDiscardLogHandler) Severity() Severity { return h.level }
func BenchmarkLuaLog(b *testing.B) {
benchmarkLuaLog(b, Severity_Debug)
}
func BenchmarkLuaLogFiltered(b *testing.B) {
benchmarkLuaLog(b, Severity_Warning)
}
func benchmarkLuaLog(b *testing.B, level Severity) {
previous := logHandler.Load()
b.Cleanup(func() { logHandler.Store(previous) })
RegisterHandler(luaDiscardLogHandler{level: level})
L := lua.NewState()
defer L.Close()
RegisterLua(L)
if err := L.DoString(`custom = setmetatable({}, {__tostring = function() return "custom" end})`); err != nil {
b.Fatal(err)
}
for _, benchmark := range []struct {
name, arguments string
}{
{"strings", `"query: ", "example.com"`},
{"mixed", `"count=", 42, ", enabled=", true, ", value=", nil`},
{"many_arguments", `"a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l"`},
{"tostring", "custom"},
} {
b.Run(benchmark.name, func(b *testing.B) {
if err := L.DoString(fmt.Sprintf(`local log = require("xray.log")
function benchmarkLog() log.Info(%s) end`, benchmark.arguments)); err != nil {
b.Fatal(err)
}
fn := L.GetGlobal("benchmarkLog")
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
if err := L.CallByParam(lua.P{Fn: fn, NRet: 0, Protect: true}); err != nil {
b.Fatal(err)
}
}
})
}
}
@@ -0,0 +1,19 @@
// DO NOT EDIT. Regression test for app/geodata's luaGuardHeader.
package lua
import (
"errors"
"strings"
"testing"
"github.com/yuin/gopher-lua/parse"
)
func TestGeodataLuaGuard(t *testing.T) {
_, err := parse.Parse(strings.NewReader("\x12\x01\x00function HandleRoute() end"), "geodata")
var syntaxErr *parse.Error
if !errors.As(err, &syntaxErr) || syntaxErr.Pos.Line != 1 || syntaxErr.Pos.Column != 1 {
t.Fatalf("want Lua rejection at the first byte, got %v", err)
}
}
+3
View File
@@ -0,0 +1,3 @@
// Package lua provides shared GopherLua programs, state management, and value
// conversion and validation helpers for Xray scripts.
package lua
+90
View File
@@ -0,0 +1,90 @@
package lua
import (
glua "github.com/yuin/gopher-lua"
luar "layeh.com/gopher-luar"
)
// NewSlicePusher captures luar's slice metatable during state initialization.
// The returned function wraps slices without reflection or metatable lookup,
// and pushes nil for nil slices. Use it with this state or its coroutines.
func NewSlicePusher[T any](L *glua.LState) func(*glua.LState, []T) {
metatable := luar.New(L, []T{}).(*glua.LUserData).Metatable
return func(L *glua.LState, values []T) {
if values == nil {
L.Push(glua.LNil)
return
}
userdata := L.NewUserData()
userdata.Value = values
userdata.Metatable = metatable
L.Push(userdata)
}
}
// DirectMethod handles a Lua call without luar's reflected method invocation.
// It returns the result count and whether it handled the arguments. On false,
// it must leave the stack unchanged for the original luar wrapper.
type DirectMethod func(L *glua.LState) (nresults int, handled bool)
// PushMethods pushes a luar userdata exposing only the named Go methods and
// their aliases through reflected calls. The method list is fixed by the first
// binding of that Go type in each LState.
func PushMethods(L *glua.LState, value any, methods ...string) {
PushWithDirectMethods(L, value, nil, methods...)
}
// PushWithDirectMethods pushes a luar userdata with typed Go method bindings.
// Handled calls bypass luar's argument conversion and reflect.Call; method lookup
// uses the methods table directly instead of luar's reflected __index handler.
// Only directMethods and additionalMethods are exposed, including their aliases.
// Bindings and their closures are installed once per Go type per LState, outside
// the method-call hot path.
func PushWithDirectMethods(L *glua.LState, value any, directMethods map[string]DirectMethod, additionalMethods ...string) {
userdata := luar.New(L, value).(*glua.LUserData)
metatable := userdata.Metatable.(*glua.LTable)
methods := metatable.RawGetString("methods").(*glua.LTable)
if metatable.RawGetString("__index") != methods {
// Keep all aliases for the explicitly exposed methods.
exposed := make(map[glua.LValue]bool, len(directMethods)+len(additionalMethods))
for name := range directMethods {
exposed[methods.RawGetString(name)] = true
}
for _, name := range additionalMethods {
exposed[methods.RawGetString(name)] = true
}
for key, method := methods.Next(glua.LNil); key != glua.LNil; key, method = methods.Next(key) {
if !exposed[method] {
methods.RawSet(key, glua.LNil)
}
}
for name, direct := range directMethods {
original := methods.RawGetString(name)
fn := L.NewFunction(func(L *glua.LState) int {
if nresults, handled := direct(L); handled {
return nresults
}
return callLuarMethod(L, original)
})
// Keep luar's method aliases on the same direct binding.
for key, method := methods.Next(glua.LNil); key != glua.LNil; key, method = methods.Next(key) {
if method == original {
methods.RawSet(key, fn)
}
}
}
// Pointer operations would bypass the method-only API.
metatable.RawSetString("__newindex", glua.LNil)
metatable.RawSetString("__pow", glua.LNil)
metatable.RawSetString("__unm", glua.LNil)
metatable.RawSetString("__index", methods)
}
L.Push(userdata)
}
func callLuarMethod(L *glua.LState, method glua.LValue) int {
nargs := L.GetTop()
L.Insert(method, 1)
L.Call(nargs, glua.MultRet)
return L.GetTop()
}
+84
View File
@@ -0,0 +1,84 @@
package lua
import (
"net"
"testing"
glua "github.com/yuin/gopher-lua"
luar "layeh.com/gopher-luar"
)
func TestSlicePusher(t *testing.T) {
L := glua.NewState()
defer L.Close()
push := NewSlicePusher[int](L)
values := []int{3, 5}
L.SetGlobal("getValues", L.NewFunction(func(L *glua.LState) int {
push(L, values)
return 1
}))
if err := L.DoString(`
local values = getValues()
assert(#values == 2 and values[1] == 3 and values[2] == 5)
values[2] = 7
local co = coroutine.create(function()
local values = getValues()
assert(#values == 2 and values[1] == 3 and values[2] == 7)
return true
end)
local ok, result = coroutine.resume(co)
assert(ok and result == true)
`); err != nil {
t.Fatal(err)
}
if values[1] != 7 {
t.Fatal("slice storage was copied")
}
push(L, nil)
if L.Get(-1) != glua.LNil {
t.Fatal("nil slice must push Lua nil")
}
L.Pop(1)
push(L, []int{})
L.SetGlobal("empty", L.Get(-1))
L.Pop(1)
if err := L.DoString(`assert(type(empty) == "userdata" and #empty == 0)`); err != nil {
t.Fatal(err)
}
}
func TestSlicePusherMetatablePerState(t *testing.T) {
first := glua.NewState()
defer first.Close()
second := glua.NewState()
defer second.Close()
NewSlicePusher[int](first)(first, []int{1})
NewSlicePusher[int](second)(second, []int{1})
if first.Get(-1).(*glua.LUserData).Metatable == second.Get(-1).(*glua.LUserData).Metatable {
t.Fatal("independent states share a slice metatable")
}
}
func BenchmarkSlicePusher(b *testing.B) {
L := glua.NewState()
defer L.Close()
ips := []net.IP{net.ParseIP("127.0.0.1")}
pushIPs := NewSlicePusher[net.IP](L)
for _, benchmark := range []struct {
name string
push func(*glua.LState, []net.IP)
}{
{"bare", func(L *glua.LState, ips []net.IP) { PushUserData(L, ips) }},
{"luar", func(L *glua.LState, ips []net.IP) { L.Push(luar.New(L, ips)) }},
{"cached", pushIPs},
} {
b.Run(benchmark.name, func(b *testing.B) {
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
benchmark.push(L, ips)
L.Pop(1)
}
})
}
}
+151
View File
@@ -0,0 +1,151 @@
package lua
import (
"context"
"errors"
"sync"
"time"
glua "github.com/yuin/gopher-lua"
)
const maxIdleStates = 16
// Pool lends each state to one caller at a time. It grows on contention and
// keeps up to maxIdleStates idle states until Close. Acquire/Release callers
// decide reusability; WithState uses its callback's error.
type Pool struct {
ctx context.Context
cancel context.CancelFunc
timeout time.Duration
factory LStateFactory
idle []*glua.LState
top int
mu sync.Mutex
active sync.WaitGroup
closed bool
}
// NewPool tests the factory by creating one state during initialization.
func NewPool(ctx context.Context, timeout time.Duration, factory LStateFactory) (*Pool, error) {
if timeout <= 0 {
return nil, errors.New("Lua pool timeout must be positive")
}
poolCtx, cancel := context.WithCancel(ctx)
state, err := factory(poolCtx)
if err != nil {
cancel()
return nil, err
}
return &Pool{ctx: poolCtx, cancel: cancel, timeout: timeout, factory: factory, idle: []*glua.LState{state}, top: state.GetTop()}, nil
}
// Acquire returns an initialized exclusive state, growing the pool if necessary.
// ctx is passed to the factory for state creation; nil uses the pool context.
func (p *Pool) Acquire(ctx context.Context) (*glua.LState, error) {
p.mu.Lock()
if p.closed {
p.mu.Unlock()
return nil, errors.New("Lua pool is closed")
}
if err := p.ctx.Err(); err != nil {
p.mu.Unlock()
return nil, err
}
if ctx == nil {
ctx = p.ctx
} else if err := ctx.Err(); err != nil {
p.mu.Unlock()
return nil, err
}
p.active.Add(1)
n := len(p.idle)
if n != 0 {
state := p.idle[n-1]
p.idle[n-1] = nil
p.idle = p.idle[:n-1]
p.mu.Unlock()
return state, nil
}
p.mu.Unlock()
// TODO: Limit the total number of states. When the limit is reached, wait
// for a Release instead of creating another state; allow the wait to be
// cancelled by the caller or by Close.
state, err := p.factory(ctx)
if err != nil {
p.active.Done()
return nil, err
}
return state, nil
}
// WithState runs work on an exclusive state and releases it afterward.
// Nil ctx and zero timeout use pool defaults. The timeout starts after acquisition.
func (p *Pool) WithState(ctx context.Context, timeout time.Duration, work func(*glua.LState) error) error {
state, err := p.Acquire(ctx)
if err != nil {
return err
}
if ctx == nil {
ctx = p.ctx
}
if timeout == 0 {
timeout = p.timeout
}
ctx, cancel := context.WithTimeout(ctx, timeout)
state.SetContext(ctx)
reusable := false
defer func() {
cancel()
p.Release(state, reusable)
}()
err = work(state)
reusable = err == nil
return err
}
// Release resets a state for reuse or closes it.
func (p *Pool) Release(state *glua.LState, reusable bool) {
if reusable {
state.RemoveContext()
state.SetTop(p.top)
p.mu.Lock()
if !p.closed && p.ctx.Err() == nil && len(p.idle) < maxIdleStates {
p.idle = append(p.idle, state)
} else {
reusable = false
}
p.mu.Unlock()
}
if !reusable {
state.Close()
}
p.active.Done()
}
// Close cancels the pool context, closes idle states, and waits for borrowed states.
func (p *Pool) Close() {
p.mu.Lock()
if !p.closed {
p.closed = true
p.cancel()
for _, state := range p.idle {
state.Close()
}
p.idle = nil
}
p.mu.Unlock()
p.active.Wait()
}
+466
View File
@@ -0,0 +1,466 @@
package lua
import (
"context"
"errors"
"testing"
"time"
glua "github.com/yuin/gopher-lua"
)
func newTestPool(t testing.TB, ctx context.Context, timeout time.Duration, factory LStateFactory) *Pool {
t.Helper()
pool, err := NewPool(ctx, timeout, factory)
if err != nil {
t.Fatal(err)
}
t.Cleanup(pool.Close)
return pool
}
func assertPoolCloseBlocked(t *testing.T, done <-chan struct{}) {
t.Helper()
select {
case <-done:
t.Fatal("Close returned while work was still active")
case <-time.After(20 * time.Millisecond):
}
}
func TestPoolTimeoutValidation(t *testing.T) {
for _, tc := range []struct {
name string
timeout time.Duration
wantErr bool
}{
{"zero", 0, true},
{"negative", -time.Nanosecond, true},
{"positive", time.Nanosecond, false},
} {
t.Run(tc.name, func(t *testing.T) {
called := false
pool, err := NewPool(context.Background(), tc.timeout, func(context.Context) (*glua.LState, error) {
called = true
return glua.NewState(), nil
})
if pool != nil {
t.Cleanup(pool.Close)
}
if (err != nil) != tc.wantErr {
t.Fatalf("NewPool error = %v, want error %t", err, tc.wantErr)
}
if tc.wantErr && (pool != nil || called) {
t.Fatal("invalid timeout created a pool or called the factory")
}
})
}
}
func TestPoolFactoryFailure(t *testing.T) {
failure := errors.New("factory failed")
_, err := NewPool(context.Background(), time.Second, func(context.Context) (*glua.LState, error) {
return nil, failure
})
if !errors.Is(err, failure) {
t.Fatalf("NewPool error = %v, want original factory error", err)
}
calls := 0
pool := newTestPool(t, context.Background(), time.Second, func(context.Context) (*glua.LState, error) {
calls++
if calls == 1 {
return glua.NewState(), nil
}
return nil, failure
})
state, err := pool.Acquire(nil)
if err != nil {
t.Fatal(err)
}
defer pool.Release(state, true)
err = pool.WithState(nil, 0, func(*glua.LState) error {
t.Error("work ran after factory failure")
return nil
})
if !errors.Is(err, failure) {
t.Fatalf("WithState error = %v, want original factory error", err)
}
}
func TestPoolReusesStatesAndLimitsIdle(t *testing.T) {
created := 0
pool := newTestPool(t, context.Background(), time.Second, func(context.Context) (*glua.LState, error) {
created++
return glua.NewState(), nil
})
var borrowed []*glua.LState
defer func() {
for _, state := range borrowed {
pool.Release(state, false)
}
}()
for range maxIdleStates + 3 {
state, err := pool.Acquire(nil)
if err != nil {
t.Fatal(err)
}
borrowed = append(borrowed, state)
state.SetContext(context.Background())
}
states := borrowed
for _, state := range states {
pool.Release(state, true)
}
borrowed = nil
open := 0
for _, state := range states {
if !state.IsClosed() {
if state.Context() != nil {
t.Fatal("Release left a context on a reusable state")
}
open++
}
}
if open != maxIdleStates {
t.Fatalf("retained %d states, want %d", open, maxIdleStates)
}
if err := pool.WithState(nil, 0, func(*glua.LState) error { return nil }); err != nil {
t.Fatal(err)
}
if created != len(states) {
t.Fatalf("created %d states, want %d", created, len(states))
}
pool.Close()
for _, state := range states {
if !state.IsClosed() {
t.Fatal("Close left an idle state open")
}
}
}
func TestPoolWithStateOptions(t *testing.T) {
key := struct{}{}
parent := context.WithValue(context.Background(), key, "pool")
caller := context.WithValue(context.Background(), key, "caller")
pool := newTestPool(t, parent, time.Second, func(context.Context) (*glua.LState, error) {
return glua.NewState(), nil
})
for _, tc := range []struct {
name string
ctx context.Context
timeout time.Duration
wantValue string
wantTimeout time.Duration
}{
{"defaults", nil, 0, "pool", time.Second},
{"context", caller, 0, "caller", time.Second},
{"timeout", nil, 2 * time.Second, "pool", 2 * time.Second},
{"both", caller, 2 * time.Second, "caller", 2 * time.Second},
} {
t.Run(tc.name, func(t *testing.T) {
started := time.Now()
err := pool.WithState(tc.ctx, tc.timeout, func(L *glua.LState) error {
ctx := L.Context()
if ctx.Value(key) != tc.wantValue {
t.Errorf("context value = %v, want %q", ctx.Value(key), tc.wantValue)
}
deadline, ok := ctx.Deadline()
if !ok || deadline.Before(started.Add(tc.wantTimeout)) || deadline.After(time.Now().Add(tc.wantTimeout)) {
t.Errorf("deadline = %v, want timeout %v", deadline, tc.wantTimeout)
}
return nil
})
if err != nil {
t.Fatal(err)
}
})
}
}
func TestPoolFactoryContext(t *testing.T) {
caller, cancel := context.WithTimeout(context.Background(), time.Minute)
defer cancel()
for _, tc := range []struct {
name string
ctx context.Context
}{
{"default", nil},
{"caller", caller},
} {
t.Run(tc.name, func(t *testing.T) {
var contexts []context.Context
pool := newTestPool(t, context.Background(), time.Second, func(ctx context.Context) (*glua.LState, error) {
contexts = append(contexts, ctx)
return glua.NewState(), nil
})
state, err := pool.Acquire(nil)
if err != nil {
t.Fatal(err)
}
defer pool.Release(state, true)
if err := pool.WithState(tc.ctx, 2*time.Second, func(*glua.LState) error { return nil }); err != nil {
t.Fatal(err)
}
want := tc.ctx
if want == nil {
want = pool.ctx
}
if len(contexts) != 2 || contexts[0] != pool.ctx || contexts[1] != want {
t.Fatal("factory did not receive the initialization and acquisition contexts unchanged")
}
})
}
}
func TestPoolWithStateLifecycle(t *testing.T) {
failure := errors.New("work failed")
for _, tc := range []struct {
name string
work func(*glua.LState, context.CancelFunc) error
reusable bool
wantPanic bool
wantErr error
}{
{"success", func(*glua.LState, context.CancelFunc) error { return nil }, true, false, nil},
{"canceled success", func(_ *glua.LState, cancel context.CancelFunc) error {
cancel()
return nil
}, true, false, nil},
{"error", func(*glua.LState, context.CancelFunc) error { return failure }, false, false, failure},
{"timeout", func(L *glua.LState, _ context.CancelFunc) error { return L.DoString("while true do end") }, false, false, nil},
{"panic", func(*glua.LState, context.CancelFunc) error { panic(failure) }, false, true, nil},
} {
t.Run(tc.name, func(t *testing.T) {
pool := newTestPool(t, context.Background(), 10*time.Millisecond, func(context.Context) (*glua.LState, error) {
state := glua.NewState()
state.Push(glua.LTrue)
return state, nil
})
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
var state *glua.LState
var workCtx context.Context
var recovered any
err := func() (err error) {
defer func() { recovered = recover() }()
return pool.WithState(ctx, 0, func(L *glua.LState) error {
state, workCtx = L, L.Context()
L.Push(glua.LFalse)
return tc.work(L, cancel)
})
}()
if tc.wantPanic {
if recovered != failure {
t.Fatalf("panic = %v, want original panic", recovered)
}
} else {
if recovered != nil || (err == nil) != tc.reusable {
t.Fatalf("WithState error = %v, panic = %v", err, recovered)
}
if tc.wantErr != nil && !errors.Is(err, tc.wantErr) {
t.Fatalf("WithState error = %v, want %v", err, tc.wantErr)
}
}
if workCtx.Err() == nil {
t.Fatal("WithState did not cancel the execution context")
}
if closed := state.IsClosed(); closed == tc.reusable {
t.Fatalf("state closed = %t, want %t", closed, !tc.reusable)
}
if tc.reusable && (state.Context() != nil || state.GetTop() != 1 || state.Get(1) != glua.LTrue) {
t.Fatal("WithState did not reset the state for reuse")
}
if err := pool.WithState(nil, 0, func(L *glua.LState) error {
if (L == state) != tc.reusable {
t.Error("unexpected state reuse")
}
return nil
}); err != nil {
t.Fatal(err)
}
})
}
}
func TestPoolClose(t *testing.T) {
pool := newTestPool(t, context.Background(), time.Second, func(context.Context) (*glua.LState, error) {
return glua.NewState(), nil
})
finishCtx, finish := context.WithCancel(context.Background())
t.Cleanup(finish)
started, done := make(chan *glua.LState, 1), make(chan error, 1)
var workCtx context.Context
go func() {
done <- pool.WithState(nil, 0, func(L *glua.LState) error {
workCtx = L.Context()
started <- L
<-finishCtx.Done()
return nil
})
}()
var state *glua.LState
select {
case state = <-started:
case <-time.After(time.Second):
t.Fatal("WithState did not start")
}
closed := make(chan struct{})
go func() {
pool.Close()
close(closed)
}()
select {
case <-workCtx.Done():
case <-time.After(time.Second):
t.Fatal("Close did not cancel work using the pool context")
}
if !errors.Is(workCtx.Err(), context.Canceled) {
t.Fatalf("work context error = %v, want context.Canceled", workCtx.Err())
}
assertPoolCloseBlocked(t, closed)
finish()
select {
case err := <-done:
if err != nil {
t.Fatalf("successful work returned an error: %v", err)
}
case <-time.After(time.Second):
t.Fatal("WithState did not finish")
}
select {
case <-closed:
case <-time.After(time.Second):
t.Fatal("Close did not finish after WithState")
}
if !state.IsClosed() {
t.Fatal("Release returned a state to a closed pool")
}
if state, err := pool.Acquire(nil); state != nil || err == nil || errors.Is(err, context.Canceled) {
t.Fatalf("Acquire after Close = %v, %v; want closed pool error", state, err)
}
pool.Close()
}
func TestPoolCloseWaitsForFactory(t *testing.T) {
finishCtx, finish := context.WithCancel(context.Background())
started, canceled := make(chan struct{}), make(chan struct{})
first := true
pool := newTestPool(t, context.Background(), time.Second, func(ctx context.Context) (*glua.LState, error) {
if first {
first = false
return glua.NewState(), nil
}
close(started)
<-ctx.Done()
close(canceled)
<-finishCtx.Done()
return nil, ctx.Err()
})
t.Cleanup(finish)
state, err := pool.Acquire(nil)
if err != nil {
t.Fatal(err)
}
pool.Release(state, false)
acquireDone := make(chan error, 1)
go func() {
_, err := pool.Acquire(nil)
acquireDone <- err
}()
select {
case <-started:
case <-time.After(time.Second):
t.Fatal("state creation did not start")
}
closed := make(chan struct{})
go func() {
pool.Close()
close(closed)
}()
select {
case <-canceled:
case <-time.After(time.Second):
t.Fatal("Close did not cancel state creation")
}
assertPoolCloseBlocked(t, closed)
finish()
select {
case err := <-acquireDone:
if !errors.Is(err, context.Canceled) {
t.Fatalf("Acquire error = %v, want context.Canceled", err)
}
case <-time.After(time.Second):
t.Fatal("state creation did not finish")
}
select {
case <-closed:
case <-time.After(time.Second):
t.Fatal("Close did not finish after state creation")
}
}
func TestPoolCloseWaitsForCallerContext(t *testing.T) {
pool := newTestPool(t, context.Background(), time.Minute, func(context.Context) (*glua.LState, error) {
return glua.NewState(), nil
})
ctx, cancel := context.WithCancel(context.Background())
t.Cleanup(cancel)
started, done := make(chan context.Context, 1), make(chan error, 1)
go func() {
done <- pool.WithState(ctx, 0, func(L *glua.LState) error {
started <- L.Context()
<-L.Context().Done()
return L.Context().Err()
})
}()
var workCtx context.Context
select {
case workCtx = <-started:
case <-time.After(time.Second):
t.Fatal("WithState did not start")
}
closed := make(chan struct{})
go func() {
pool.Close()
close(closed)
}()
select {
case <-pool.ctx.Done():
case <-time.After(time.Second):
t.Fatal("Close did not cancel the pool context")
}
assertPoolCloseBlocked(t, closed)
if workCtx.Err() != nil || ctx.Err() != nil {
t.Fatal("Close canceled the caller's execution context")
}
cancel()
select {
case err := <-done:
if !errors.Is(err, context.Canceled) {
t.Fatalf("WithState error = %v, want context.Canceled", err)
}
case <-time.After(time.Second):
t.Fatal("WithState did not stop after caller cancellation")
}
select {
case <-closed:
case <-time.After(time.Second):
t.Fatal("Close did not finish after WithState")
}
}
func BenchmarkPoolAcquireRelease(b *testing.B) {
pool := newTestPool(b, context.Background(), time.Second, func(context.Context) (*glua.LState, error) {
return glua.NewState(), nil
})
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
state, err := pool.Acquire(nil)
if err != nil {
b.Fatal(err)
}
pool.Release(state, true)
}
}
+117
View File
@@ -0,0 +1,117 @@
package lua
import (
"bytes"
"crypto/ed25519"
"crypto/sha256"
"encoding/base64"
"sync"
"github.com/xtls/xray-core/common/errors"
)
var privilegedKey struct {
sync.Once
publicKey ed25519.PublicKey
}
// SetPrivilegedPublicKey configures the trusted Ed25519 public key once at
// startup, before compiling scripts. An empty value disables authorization.
// Nonempty values must be unpadded Base64URL encodings of exactly 32 bytes.
func SetPrivilegedPublicKey(value string) error {
var publicKey ed25519.PublicKey
if value != "" {
decoded, err := decodePrivilegedValue(value, ed25519.PublicKeySize)
if err != nil {
return errors.New("invalid Lua privileged public key").Base(err)
}
publicKey = ed25519.PublicKey(decoded)
}
privilegedKey.Do(func() {
privilegedKey.publicKey = publicKey
})
return nil
}
func decodePrivilegedValue(value string, size int) ([]byte, error) {
encoding := base64.RawURLEncoding.Strict()
if len(value) != encoding.EncodedLen(size) {
return nil, errors.New("expected an unpadded Base64URL encoding of ", size, " bytes")
}
decoded, err := encoding.DecodeString(value)
if err != nil || len(decoded) != size || encoding.EncodeToString(decoded) != value {
return nil, errors.New("expected an unpadded Base64URL encoding of ", size, " bytes")
}
return decoded, nil
}
func verifyPrivilegedSignature(code []byte) (bool, error) {
_, end, value, found := privilegedSignatureLine(code)
if !found {
return false, nil
}
publicKey := privilegedKey.publicKey
if len(publicKey) == 0 {
return false, errors.New("Lua privileged signature requires --lua-privileged-pubkey")
}
signature, err := decodePrivilegedValue(string(bytes.TrimSpace(value)), ed25519.SignatureSize)
if err != nil {
return false, errors.New("invalid Lua privileged signature").Base(err)
}
digest := sha256.Sum256(bytes.TrimSpace(code[end:]))
if !ed25519.Verify(publicKey, digest[:], signature) {
return false, errors.New("Lua privileged signature verification failed")
}
return true, nil
}
// SignPrivilegedScript signs the script body, then prepends or replaces the signature line.
func SignPrivilegedScript(code []byte, seed string) ([]byte, error) {
decoded, err := decodePrivilegedValue(seed, ed25519.SeedSize)
if err != nil {
return nil, errors.New("invalid Ed25519 private key seed").Base(err)
}
start, end, _, found := privilegedSignatureLine(code)
body := code
if found {
body = code[end:]
}
digest := sha256.Sum256(bytes.TrimSpace(body))
signature := ed25519.Sign(ed25519.NewKeyFromSeed(decoded), digest[:])
header := []byte("-- @xray:privileged-signature=" + base64.RawURLEncoding.EncodeToString(signature))
if !found {
if newline := bytes.IndexByte(code, '\n'); newline > 0 && code[newline-1] == '\r' {
header = append(header, '\r')
}
header = append(header, '\n')
return append(header, code...), nil
}
signed := make([]byte, 0, start+len(header)+len(code)-end)
signed = append(signed, code[:start]...)
signed = append(signed, header...)
return append(signed, code[end:]...), nil
}
func privilegedSignatureLine(code []byte) (start, end int, value []byte, found bool) {
for rest := code; len(rest) != 0; {
start = len(code) - len(rest)
var line []byte
line, rest, _ = bytes.Cut(rest, []byte{'\n'})
line = bytes.TrimSuffix(line, []byte{'\r'})
if len(bytes.TrimSpace(line)) == 0 {
continue
}
end = start + len(line)
line, found := bytes.CutPrefix(bytes.TrimLeft(line, " \t"), []byte("--"))
if !found {
return 0, 0, nil, false
}
value, found = bytes.CutPrefix(bytes.TrimLeft(line, " \t"), []byte("@xray:privileged-signature="))
return start, end, value, found
}
return 0, 0, nil, false
}
+82
View File
@@ -0,0 +1,82 @@
package lua
import (
"bytes"
"crypto/ed25519"
"encoding/base64"
"encoding/hex"
"strings"
"testing"
)
func privilegedSeed(t *testing.T) string {
t.Helper()
seed, err := hex.DecodeString(privilegedTestSeed)
if err != nil {
t.Fatal(err)
}
return base64.RawURLEncoding.EncodeToString(seed)
}
func TestSignPrivilegedScriptPreservesSource(t *testing.T) {
setTestPrivilegedPublicKey(t, privilegedTestPublicKey)
seed := privilegedSeed(t)
for _, tc := range []struct {
name, prefix, header, body, newline string
}{
{name: "unsigned", body: privilegedFixtureBody, newline: "\n"},
{name: "unsigned boundary whitespace", body: " \t\n\n" + privilegedFixtureBody + " \t\n", newline: "\n"},
{name: "unsigned CRLF", body: strings.ReplaceAll(privilegedFixtureBody, "\n", "\r\n"), newline: "\r\n"},
{name: "replace invalid header", header: privilegedSignatureHeader + "%", body: "\n" + privilegedFixtureBody, newline: "\n"},
{name: "replace indented header", prefix: " \t\r\n\n", header: " \t--\t @xray:privileged-signature=% \t", body: "\r\n \t\r\n" + privilegedFixtureBody + "\r\n", newline: "\r\n"},
{name: "replace valid header", header: privilegedSignatureHeader + privilegedFixtureSignature, body: "\n" + privilegedFixtureBody, newline: "\n"},
{name: "empty script", newline: "\n"},
{name: "header without newline", header: privilegedSignatureHeader + "%"},
{name: "ordinary comment before marker", body: "-- ordinary comment\n" + privilegedSignatureHeader + "%\nvalue = 1", newline: "\n"},
} {
t.Run(tc.name, func(t *testing.T) {
code := []byte(tc.prefix + tc.header + tc.body)
original := bytes.Clone(code)
signed, err := SignPrivilegedScript(code, seed)
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(code, original) {
t.Fatal("signing modified the input buffer")
}
body := tc.body
if tc.header != "" {
body = strings.TrimPrefix(body, tc.newline)
}
header := privilegedSignatureHeader + testSignature(t, body)
want := header + tc.newline + tc.body
if tc.header != "" {
want = tc.prefix + header + tc.body
}
if string(signed) != want {
t.Fatalf("signed source = %q, want %q", signed, want)
}
program, err := Compile(writeTestScript(t, string(signed)))
if err != nil || program == nil || !program.privileged {
t.Fatalf("signed source Compile = %v, %v; want privileged program", program, err)
}
again, err := SignPrivilegedScript(signed, seed)
if err != nil || !bytes.Equal(again, signed) {
t.Fatalf("resigning changed the source: %q, %v", again, err)
}
})
}
}
func TestSignPrivilegedScriptRejectsInvalidSeed(t *testing.T) {
seed := privilegedSeed(t)
for _, value := range []string{
"", "%", seed + "=",
base64.RawURLEncoding.EncodeToString(make([]byte, ed25519.SeedSize-1)),
base64.RawURLEncoding.EncodeToString(make([]byte, ed25519.PrivateKeySize)),
} {
if signed, err := SignPrivilegedScript([]byte(privilegedFixtureBody), value); signed != nil || err == nil {
t.Fatalf("SignPrivilegedScript with invalid seed = %q, %v; want rejection", signed, err)
}
}
}
+195
View File
@@ -0,0 +1,195 @@
package lua
import (
"bytes"
"context"
"fmt"
"os"
"strings"
"time"
"github.com/xtls/xray-core/common/base64"
"github.com/xtls/xray-core/common/errors"
"github.com/xtls/xray-core/common/platform"
glua "github.com/yuin/gopher-lua"
"github.com/yuin/gopher-lua/parse"
)
// Program holds immutable bytecode that can be run by independent LStates.
type Program struct {
proto *glua.FunctionProto
privileged bool
}
// LStateFactory returns a fully initialized state or nil and an error.
// Implementations must close partial states on failure; callers own successful states.
type LStateFactory func(context.Context) (*glua.LState, error)
// Compile reads a Lua file or decodes an inline script and compiles it into an immutable program.
func Compile(script string) (*Program, error) {
var code []byte
var err error
name := "[inline]"
privileged := false
if strings.HasPrefix(script, base64.GzipPrefix) || strings.HasPrefix(script, base64.Prefix) {
// TODO: Restore inline script support and re-enable the related tests.
return nil, errors.New("currently unavailable")
code, err = base64.Decode(script)
} else if name, err = platform.ResolveLuaFile(script); err == nil {
if code, err = os.ReadFile(name); err == nil {
privileged, err = verifyPrivilegedSignature(code)
}
}
if err != nil {
return nil, err
}
chunk, err := parse.Parse(bytes.NewReader(code), name)
if err != nil {
return nil, err
}
proto, err := glua.Compile(chunk, name)
if err != nil {
return nil, err
}
return &Program{proto: proto, privileged: privileged}, nil
}
// NewState creates a state, runs register, executes the program under ctx, and
// runs validate. It removes the initialization context before returning a state
// owned by the caller.
func (p *Program) NewState(ctx context.Context, register func(*glua.LState), validate func(*glua.LState) error) (*glua.LState, error) {
var L *glua.LState
if p.privileged {
L = glua.NewState()
} else {
L = newSandboxedState()
}
valid := false
defer func() {
if !valid {
L.Close()
}
}()
L.SetContext(ctx)
defer L.RemoveContext()
if register != nil {
register(L)
}
if !p.privileged {
// Hide package internals to prevent data snooping and sandbox escape.
L.SetGlobal(glua.LoadLibName, glua.LNil)
}
L.Push(L.NewFunctionFromProto(p.proto))
// Execute the Lua script's top level.
if err := L.PCall(0, 0, nil); err != nil {
return nil, err
}
if validate != nil {
if err := validate(L); err != nil {
return nil, err
}
}
valid = true
return L, nil
}
func newSandboxedState() *glua.LState {
L := glua.NewState(glua.Options{SkipOpenLibs: true})
for _, lib := range []struct {
name string
open glua.LGFunction
}{
{glua.LoadLibName, glua.OpenPackage},
{glua.BaseLibName, glua.OpenBase},
{glua.TabLibName, glua.OpenTable},
{glua.StringLibName, glua.OpenString},
{glua.MathLibName, glua.OpenMath},
} {
L.Push(L.NewFunction(lib.open))
L.Push(glua.LString(lib.name))
L.Call(1, 0)
}
for _, name := range []string{
"dofile", "loadfile", "load", "loadstring", "getfenv", "setfenv",
"getmetatable", "setmetatable", "rawget", "rawset",
"collectgarbage", "module", "newproxy", "print", "_printregs",
} {
L.SetGlobal(name, glua.LNil)
}
// Prevent printing memory addresses that reveal the memory layout.
toString := func(value glua.LValue) glua.LString {
result := "<opaque>"
switch value.Type() {
case glua.LTNil, glua.LTBool, glua.LTNumber, glua.LTString:
result = value.String()
case glua.LTUserData:
switch native := value.(*glua.LUserData).Value.(type) {
case fmt.Stringer:
result = native.String()
case error:
result = native.Error()
}
}
return glua.LString(result)
}
L.SetGlobal("tostring", L.NewFunction(func(L *glua.LState) int {
L.Push(toString(L.CheckAny(1)))
return 1
}))
// Prevent printing memory addresses that reveal the memory layout.
stringLib := L.GetGlobal(glua.StringLibName).(*glua.LTable)
stringLib.RawSetString("dump", glua.LNil)
format := stringLib.RawGetString("format")
stringLib.RawSetString("format", L.NewFunction(func(L *glua.LState) int {
nargs := L.GetTop()
for i := 2; i <= nargs; i++ {
value := L.Get(i)
switch value.Type() {
case glua.LTBool, glua.LTNumber, glua.LTString:
continue
}
L.Replace(i, toString(value))
}
L.Insert(format, 1)
L.Call(nargs, 1)
return 1
}))
// Prevent seed output from compromising cryptographic security.
L.GetGlobal(glua.MathLibName).(*glua.LTable).RawSetString("randomseed", glua.LNil)
// Prevent module injection through the global package table.
module := L.GetGlobal(glua.LoadLibName).(*glua.LTable)
preload := module.RawGetString("preload").(*glua.LTable)
loaders := L.CreateTable(1, 0)
loaders.RawSetInt(1, L.NewFunction(func(L *glua.LState) int {
name := L.CheckString(1)
loader := preload.RawGetString(name)
if loader == glua.LNil {
loader = glua.LString("no preloaded module: " + name)
}
L.Push(loader)
return 1
}))
module.RawSetString("loaders", loaders)
L.SetField(L.Get(glua.RegistryIndex), "_LOADERS", loaders)
for _, name := range []string{"loadlib", "seeall", "path", "cpath", "config"} {
module.RawSetString(name, glua.LNil)
}
return L
}
// NewStateFactory returns a factory that gives each state an initialization timeout.
func (p *Program) NewStateFactory(initTimeout time.Duration, register func(*glua.LState), validate func(*glua.LState) error) LStateFactory {
return func(ctx context.Context) (*glua.LState, error) {
initCtx, cancel := context.WithTimeout(ctx, initTimeout)
defer cancel()
return p.NewState(initCtx, register, validate)
}
}
+975
View File
@@ -0,0 +1,975 @@
package lua
import (
"bytes"
"context"
"crypto/ed25519"
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"errors"
"net"
"os"
"path/filepath"
"runtime"
"strings"
"sync"
"testing"
"time"
"weak"
glua "github.com/yuin/gopher-lua"
luar "layeh.com/gopher-luar"
)
func TestProgramStatesAreIndependent(t *testing.T) {
path := writeTestScript(t, "value = (value or 0) + 1")
program, err := Compile(path)
if err != nil {
t.Fatal(err)
}
if program.proto.SourceName != path {
t.Fatalf("program source = %q, want %q", program.proto.SourceName, path)
}
first, err := program.NewState(context.Background(), nil, nil)
if err != nil {
t.Fatal(err)
}
defer first.Close()
first.SetGlobal("value", glua.LNumber(42))
second, err := program.NewState(context.Background(), nil, nil)
if err != nil {
t.Fatal(err)
}
defer second.Close()
if got := second.GetGlobal("value"); got != glua.LNumber(1) {
t.Fatalf("second state value = %v, want 1", got)
}
}
func TestProgramInitializationObservesCancellation(t *testing.T) {
path := filepath.Join(t.TempDir(), "loop.lua")
if err := os.WriteFile(path, []byte("while true do end"), 0o600); err != nil {
t.Fatal(err)
}
program, err := Compile(path)
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
state, err := program.NewState(ctx, nil, nil)
if err == nil || state != nil {
if state != nil {
state.Close()
}
t.Fatalf("NewState with canceled context = %v, %v; want nil state and error", state, err)
}
}
func TestNewStateClosesFailedValidation(t *testing.T) {
path := filepath.Join(t.TempDir(), "state.lua")
if err := os.WriteFile(path, []byte("value = 1"), 0o600); err != nil {
t.Fatal(err)
}
program, err := Compile(path)
if err != nil {
t.Fatal(err)
}
wantErr := errors.New("invalid script")
var checked *glua.LState
L, err := program.NewState(context.Background(), nil, func(L *glua.LState) error {
checked = L
return wantErr
})
if L != nil || !errors.Is(err, wantErr) || checked == nil || !checked.IsClosed() {
t.Fatalf("state = %v, error = %v, checked state closed = %t", L, err, checked != nil && checked.IsClosed())
}
}
// TODO: Re-enable these tests and their helper when Compile supports inline scripts again.
/*
func compressSource(t *testing.T, code string) []byte {
t.Helper()
var data bytes.Buffer
writer := gzip.NewWriter(&data)
if _, err := writer.Write([]byte(code)); err != nil {
t.Fatal(err)
}
if err := writer.Close(); err != nil {
t.Fatal(err)
}
return data.Bytes()
}
func TestCompileInline(t *testing.T) {
const code = `value = "你好"`
for _, tc := range []struct {
name, script string
value glua.LValue
}{
{"base64", xbase64.Prefix + base64.StdEncoding.EncodeToString([]byte(code)), glua.LString("你好")},
{"base64+gzip", xbase64.GzipPrefix + base64.StdEncoding.EncodeToString(compressSource(t, code)), glua.LString("你好")},
{"empty base64", xbase64.Prefix, glua.LNil},
} {
t.Run(tc.name, func(t *testing.T) {
program, err := Compile(tc.script)
if err != nil {
t.Fatal(err)
}
if program.proto.SourceName != "[inline]" {
t.Fatalf("program source = %q, want [inline]", program.proto.SourceName)
}
state, err := program.NewState(context.Background(), nil, nil)
if err != nil {
t.Fatal(err)
}
defer state.Close()
if got := state.GetGlobal("value"); got != tc.value {
t.Fatalf("value = %v, want %v", got, tc.value)
}
})
}
}
func TestCompileRejectsInvalidInline(t *testing.T) {
corrupt := compressSource(t, "value = 1")
corrupt[len(corrupt)-8] ^= 0xff
for _, tc := range []struct {
name, script string
wantErr error
wantMessage string
}{
{"base64", xbase64.Prefix + "%", base64.CorruptInputError(0), "failed to decode base64"},
{"gzip base64", xbase64.GzipPrefix + "%", base64.CorruptInputError(0), "failed to decode base64"},
{"gzip header", xbase64.GzipPrefix + base64.StdEncoding.EncodeToString([]byte("value = 'uncompressed'")), gzip.ErrHeader, "failed to decompress gzip"},
{"gzip checksum", xbase64.GzipPrefix + base64.StdEncoding.EncodeToString(corrupt), gzip.ErrChecksum, "failed to decompress gzip"},
} {
t.Run(tc.name, func(t *testing.T) {
program, err := Compile(tc.script)
if program != nil || !errors.Is(err, tc.wantErr) || !strings.Contains(err.Error(), tc.wantMessage) {
t.Fatalf("Compile = %v, %v; want nil program and %q error wrapping %v", program, err, tc.wantMessage, tc.wantErr)
}
})
}
}
func TestCompileInlineDiagnostics(t *testing.T) {
for _, tc := range []struct {
code string
compileError bool
wantMessage string
}{
{"function broken(", true, ""},
{`error("initialization failed")`, false, ""},
{"\n-- @xray:privileged-signature=invalid\n\nerror('failure')", false, "[inline]:4: failure"},
} {
program, err := Compile(xbase64.Prefix + base64.StdEncoding.EncodeToString([]byte(tc.code)))
if tc.compileError {
if program != nil || err == nil {
t.Fatalf("Compile = %v, %v; want nil program and syntax error", program, err)
}
} else {
if err != nil {
t.Fatal(err)
}
state, initErr := program.NewState(context.Background(), nil, nil)
if state != nil {
state.Close()
t.Fatal("NewState returned a state after failed initialization")
}
err = initErr
}
if err == nil || !strings.Contains(err.Error(), "[inline]") {
t.Fatalf("Lua error = %v, want [inline] diagnostic", err)
}
if tc.wantMessage != "" && !strings.Contains(err.Error(), tc.wantMessage) {
t.Fatalf("Lua error = %v, want %q", err, tc.wantMessage)
}
}
}
*/
func TestCompileFileExtension(t *testing.T) {
for _, name := range []string{"script", "script.dat", "script.lua.txt", "script.LUA"} {
path := filepath.Join(t.TempDir(), name)
if err := os.WriteFile(path, []byte("value = 42"), 0o600); err != nil {
t.Fatal(err)
}
if program, err := Compile(path); program != nil || err == nil || !strings.Contains(err.Error(), "Lua file extension must be .lua") {
t.Fatalf("Compile(%q) = %v, %v; want nil program and .lua extension error", path, program, err)
}
}
path := writeTestScript(t, "value = 42") + string(filepath.Separator) + "."
if program, err := Compile(path); program == nil || err != nil {
t.Fatalf("Compile(%q) = %v, %v; want acceptance of the resolved .lua path", path, program, err)
}
}
func TestCompileRejectsReadFailure(t *testing.T) {
path := filepath.Join(t.TempDir(), "unreadable.lua")
if err := os.WriteFile(path, []byte("value = 42"), 0o600); err != nil {
t.Fatal(err)
}
if err := os.Chmod(path, 0); err != nil {
t.Fatal(err)
}
t.Cleanup(func() { os.Chmod(path, 0o600) })
if _, err := os.ReadFile(path); !errors.Is(err, os.ErrPermission) {
t.Skip("file read permissions are not enforced")
}
if program, err := Compile(path); program != nil || !errors.Is(err, os.ErrPermission) {
t.Fatalf("Compile = %v, %v; want nil program and permission error", program, err)
}
}
func compileTestProgram(t *testing.T, code string) *Program {
t.Helper()
program, err := Compile(writeTestScript(t, code))
if err != nil {
t.Fatal(err)
}
return program
}
const privilegedSignatureHeader = "-- @xray:privileged-signature="
// RFC 8032 test-vector key. The fixture signature was independently computed
// with Node's OpenSSL-backed crypto.sign(null, SHA256(body), privateKey), rather
// than by the signing helper or the production verifier.
const (
privilegedTestSeed = "9d61b19deffd5a60ba844af492ec2cc44449c5697b326919703bac031cae7f60"
privilegedTestPublicKey = "11qYAYKxCrfVS_7TyWQHOg7hcvPapiMlrwIaaPcHURo"
privilegedFixtureSignature = "rFbZtaaphe_69UEgSvfYwZcLZYHTXLFxdcukfEuLq6pzvmj8qtw6r60fD-7L5dwqSvnE7F1oAV0G_vHP3rLVBA"
privilegedFixtureDigest = "edef639ae641bacc9b76cea27473eed11369fbad65e223a87990864e899e578a"
privilegedFixtureBody = `-- Ordinary comments participate in the signed digest.
value = 1
function HasPrivilegedLibraries()
return io ~= nil and os ~= nil and debug ~= nil and channel ~= nil
end`
)
func setTestPrivilegedPublicKey(t *testing.T, key string) {
t.Helper()
// Each test setup represents a new process with its own startup key.
privilegedKey.Once = sync.Once{}
privilegedKey.publicKey = nil
t.Cleanup(func() {
privilegedKey.Once = sync.Once{}
privilegedKey.publicKey = nil
})
if err := SetPrivilegedPublicKey(key); err != nil {
t.Fatal(err)
}
}
func writeTestScript(t *testing.T, code string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "script.lua")
if err := os.WriteFile(path, []byte(code), 0o600); err != nil {
t.Fatal(err)
}
path, err := filepath.EvalSymlinks(path)
if err != nil {
t.Fatal(err)
}
return path
}
func testSignature(t *testing.T, body string) string {
t.Helper()
seed, err := hex.DecodeString(privilegedTestSeed)
if err != nil {
t.Fatal(err)
}
digest := sha256.Sum256(bytes.TrimSpace([]byte(body)))
return base64.RawURLEncoding.EncodeToString(ed25519.Sign(ed25519.NewKeyFromSeed(seed), digest[:]))
}
func TestSetPrivilegedPublicKey(t *testing.T) {
setTestPrivilegedPublicKey(t, privilegedTestPublicKey)
path := writeTestScript(t, privilegedSignatureHeader+privilegedFixtureSignature+"\n"+privilegedFixtureBody)
for _, tc := range []struct{ name, key string }{
{"invalid base64", "%"},
{"padding", privilegedTestPublicKey + "="},
{"standard base64 alphabet", strings.ReplaceAll(privilegedTestPublicKey, "_", "/")},
{"short key", base64.RawURLEncoding.EncodeToString(make([]byte, ed25519.PublicKeySize-1))},
{"long key", base64.RawURLEncoding.EncodeToString(make([]byte, ed25519.PublicKeySize+1))},
{"leading whitespace", " " + privilegedTestPublicKey},
{"trailing whitespace", privilegedTestPublicKey + "\t"},
{"embedded newline", privilegedTestPublicKey[:8] + "\n" + privilegedTestPublicKey[8:]},
{"embedded carriage return", privilegedTestPublicKey[:8] + "\r" + privilegedTestPublicKey[8:]},
{"nonzero unused bits", privilegedTestPublicKey[:len(privilegedTestPublicKey)-1] + "p"},
} {
t.Run(tc.name, func(t *testing.T) {
if err := SetPrivilegedPublicKey(tc.key); err == nil {
t.Fatal("invalid public key was accepted")
}
program, err := Compile(path)
if err != nil || program == nil || !program.privileged {
t.Fatalf("invalid setter changed the trusted key: Compile = %v, %v", program, err)
}
})
}
setTestPrivilegedPublicKey(t, "")
if program, err := Compile(path); program != nil || err == nil {
t.Fatalf("signed script after disabling authorization = %v, %v; want rejection", program, err)
}
unsigned, err := Compile(writeTestScript(t, privilegedFixtureBody))
if err != nil || unsigned == nil || unsigned.privileged {
t.Fatalf("unsigned script with authorization disabled = %v, %v; want sandbox", unsigned, err)
}
}
func TestPrivilegedPublicKeyIsFixed(t *testing.T) {
wrongKey := base64.RawURLEncoding.EncodeToString(make([]byte, ed25519.PublicKeySize))
for _, key := range []string{"", privilegedTestPublicKey} {
name := "disabled"
if key != "" {
name = "configured"
}
t.Run(name, func(t *testing.T) {
setTestPrivilegedPublicKey(t, key)
path := writeTestScript(t, privilegedSignatureHeader+privilegedFixtureSignature+"\n"+privilegedFixtureBody)
for _, replacement := range []string{privilegedTestPublicKey, "", wrongKey} {
if err := SetPrivilegedPublicKey(replacement); err != nil {
t.Fatal(err)
}
program, err := Compile(path)
if key == "" {
if program != nil || err == nil {
t.Fatalf("disabled startup key changed: Compile = %v, %v", program, err)
}
} else if err != nil || program == nil || !program.privileged {
t.Fatalf("trusted startup key changed: Compile = %v, %v", program, err)
}
}
})
}
}
func TestProgramPrivilegedSignatureFixture(t *testing.T) {
setTestPrivilegedPublicKey(t, privilegedTestPublicKey)
digest := sha256.Sum256([]byte(privilegedFixtureBody))
if got := hex.EncodeToString(digest[:]); got != privilegedFixtureDigest {
t.Fatalf("fixture SHA-256 = %s, want %s", got, privilegedFixtureDigest)
}
for _, signed := range []bool{false, true} {
name, code := "unsigned", privilegedFixtureBody
if signed {
name, code = "signed", privilegedSignatureHeader+privilegedFixtureSignature+"\n"+code
}
t.Run(name, func(t *testing.T) {
program, err := Compile(writeTestScript(t, code))
if err != nil {
t.Fatal(err)
}
if program.privileged != signed {
t.Fatalf("privileged = %t, want %t", program.privileged, signed)
}
L, err := program.NewState(context.Background(), nil, nil)
if err != nil {
t.Fatal(err)
}
defer L.Close()
if err := L.CallByParam(glua.P{Fn: L.GetGlobal("HasPrivilegedLibraries"), NRet: 1, Protect: true}); err != nil {
t.Fatal(err)
}
if got := L.Get(-1); got != glua.LBool(signed) {
t.Fatalf("privileged libraries available = %v, want %t", got, signed)
}
})
}
}
func TestProgramPrivilegedSignatureHeader(t *testing.T) {
setTestPrivilegedPublicKey(t, privilegedTestPublicKey)
header := privilegedSignatureHeader + privilegedFixtureSignature
for _, tc := range []struct {
name, code string
privileged bool
}{
{"match", header + "\n" + privilegedFixtureBody, true},
{"no comment space", "--@xray:privileged-signature=" + privilegedFixtureSignature + "\n" + privilegedFixtureBody, true},
{"comment whitespace", "-- \t @xray:privileged-signature=" + privilegedFixtureSignature + "\n" + privilegedFixtureBody, true},
{"signature whitespace", privilegedSignatureHeader + " \t" + privilegedFixtureSignature + " \t\n" + privilegedFixtureBody, true},
{"blank lines and indentation", " \t\r\n\n \t" + header + "\n" + privilegedFixtureBody, true},
{"CRLF header only", "\r\n" + header + "\r\n" + privilegedFixtureBody, true},
{"body boundary whitespace", header + "\n \t\r\n\n" + privilegedFixtureBody + " \t\r\n\n", true},
{"missing header", privilegedFixtureBody, false},
{"after comment", "-- ordinary comment\n" + header + "\n" + privilegedFixtureBody, false},
{"after code", "value = 1\n" + header + "\n" + privilegedFixtureBody, false},
{"inside string", `value = "` + header + `"`, false},
{"wrong marker", "-- xray:privileged-signature=" + privilegedFixtureSignature + "\n" + privilegedFixtureBody, false},
{"invalid marker after comment", "-- ordinary comment\n" + privilegedSignatureHeader + "%\nvalue = 1", false},
} {
t.Run(tc.name, func(t *testing.T) {
program, err := Compile(writeTestScript(t, tc.code))
if err != nil {
t.Fatal(err)
}
if program.privileged != tc.privileged {
t.Fatalf("privileged = %t, want %t", program.privileged, tc.privileged)
}
})
}
}
func TestProgramRejectsInvalidPrivilegedSignatures(t *testing.T) {
seed, err := hex.DecodeString(privilegedTestSeed)
if err != nil {
t.Fatal(err)
}
wrongKey := base64.RawURLEncoding.EncodeToString(ed25519.NewKeyFromSeed(make([]byte, ed25519.SeedSize)).Public().(ed25519.PublicKey))
bodySignature := base64.RawURLEncoding.EncodeToString(ed25519.Sign(ed25519.NewKeyFromSeed(seed), []byte(privilegedFixtureBody)))
for _, tc := range []struct{ name, key, signature string }{
{"missing key", "", privilegedFixtureSignature},
{"wrong key", wrongKey, privilegedFixtureSignature},
{"empty signature", privilegedTestPublicKey, ""},
{"whitespace-only signature", privilegedTestPublicKey, " \t"},
{"invalid base64", privilegedTestPublicKey, "%"},
{"padding", privilegedTestPublicKey, privilegedFixtureSignature + "=="},
{"standard base64 alphabet", privilegedTestPublicKey, strings.ReplaceAll(privilegedFixtureSignature, "_", "/")},
{"short signature", privilegedTestPublicKey, base64.RawURLEncoding.EncodeToString(make([]byte, ed25519.SignatureSize-1))},
{"long signature", privilegedTestPublicKey, base64.RawURLEncoding.EncodeToString(make([]byte, ed25519.SignatureSize+1))},
{"embedded carriage return", privilegedTestPublicKey, privilegedFixtureSignature[:8] + "\r" + privilegedFixtureSignature[8:]},
{"nonzero unused bits", privilegedTestPublicKey, privilegedFixtureSignature[:len(privilegedFixtureSignature)-1] + "B"},
{"damaged signature", privilegedTestPublicKey, "s" + privilegedFixtureSignature[1:]},
{"signature over body instead of digest", privilegedTestPublicKey, bodySignature},
} {
t.Run(tc.name, func(t *testing.T) {
setTestPrivilegedPublicKey(t, tc.key)
program, err := Compile(writeTestScript(t, privilegedSignatureHeader+tc.signature+"\n"+privilegedFixtureBody))
if program != nil || err == nil {
t.Fatalf("Compile = %v, %v; want nil program and signature rejection", program, err)
}
})
}
setTestPrivilegedPublicKey(t, privilegedTestPublicKey)
code := privilegedSignatureHeader + "%\n" + privilegedSignatureHeader + privilegedFixtureSignature + "\n" + privilegedFixtureBody
if program, err := Compile(writeTestScript(t, code)); program != nil || err == nil {
t.Fatalf("later signature replaced invalid first signature: %v, %v", program, err)
}
}
func TestProgramSignedBodyTampering(t *testing.T) {
setTestPrivilegedPublicKey(t, privilegedTestPublicKey)
for _, tc := range []struct{ name, body string }{
{"statement", strings.Replace(privilegedFixtureBody, "value = 1", "value = 2", 1)},
{"comment", strings.Replace(privilegedFixtureBody, "Ordinary", "Modified", 1)},
{"added comment", "-- added comment\n" + privilegedFixtureBody},
{"indentation", strings.Replace(privilegedFixtureBody, " return", " return", 1)},
{"internal spaces", strings.Replace(privilegedFixtureBody, "value = 1", "value = 1", 1)},
{"internal trailing spaces", strings.Replace(privilegedFixtureBody, "value = 1\n", "value = 1 \t\n", 1)},
{"removed blank line", strings.Replace(privilegedFixtureBody, "\n\n", "\n", 1)},
{"added blank line", strings.Replace(privilegedFixtureBody, "\n\n", "\n\n\n", 1)},
{"LF to CRLF", strings.ReplaceAll(privilegedFixtureBody, "\n", "\r\n")},
} {
t.Run(tc.name, func(t *testing.T) {
code := privilegedSignatureHeader + privilegedFixtureSignature + "\n" + tc.body
if program, err := Compile(writeTestScript(t, code)); program != nil || err == nil {
t.Fatalf("tampered body Compile = %v, %v; want signature rejection", program, err)
}
})
}
crlfBody := strings.ReplaceAll(privilegedFixtureBody, "\n", "\r\n")
crlfSignature := testSignature(t, crlfBody)
program, err := Compile(writeTestScript(t, privilegedSignatureHeader+crlfSignature+"\r\n"+crlfBody))
if err != nil || program == nil || !program.privileged {
t.Fatalf("correctly signed CRLF body = %v, %v; want privileged program", program, err)
}
if program, err := Compile(writeTestScript(t, privilegedSignatureHeader+crlfSignature+"\n"+privilegedFixtureBody)); program != nil || err == nil {
t.Fatalf("CRLF to LF tampering = %v, %v; want signature rejection", program, err)
}
}
// TODO: Re-enable this test when Compile supports inline scripts again.
/*
func TestProgramInlineSignaturesCannotAuthorize(t *testing.T) {
for _, key := range []string{"", privilegedTestPublicKey} {
setTestPrivilegedPublicKey(t, key)
for _, signature := range []string{privilegedFixtureSignature, "%", ""} {
code := privilegedSignatureHeader + signature + "\n" + privilegedFixtureBody
for _, script := range []string{
xbase64.Prefix + base64.StdEncoding.EncodeToString([]byte(code)),
xbase64.GzipPrefix + base64.StdEncoding.EncodeToString(compressSource(t, code)),
} {
program, err := Compile(script)
if err != nil || program == nil || program.privileged {
t.Fatalf("inline signed source Compile = %v, %v; want sandbox without verification", program, err)
}
L, err := program.NewState(context.Background(), nil, nil)
if err != nil {
t.Fatal(err)
}
if got := L.GetGlobal("os"); got != glua.LNil {
L.Close()
t.Fatalf("inline os = %v, want nil", got)
}
L.Close()
}
}
}
}
*/
func TestProgramSignedDiagnostics(t *testing.T) {
setTestPrivilegedPublicKey(t, privilegedTestPublicKey)
const invalidBody = "\n\nlocal = 1"
invalidPath := writeTestScript(t, "\n"+privilegedSignatureHeader+"%\n"+invalidBody)
if program, err := Compile(invalidPath); program != nil || err == nil || !strings.Contains(err.Error(), "signature") {
t.Fatalf("invalid signature and invalid syntax = %v, %v; want signature error before parsing", program, err)
}
validPath := writeTestScript(t, "\n"+privilegedSignatureHeader+testSignature(t, invalidBody)+"\n"+invalidBody)
if program, err := Compile(validPath); program != nil || err == nil || !strings.Contains(err.Error(), validPath+" line:5(") {
t.Fatalf("verified invalid syntax = %v, %v; want original source line 5", program, err)
}
const body = "\n\nerror('signed failure')"
path := writeTestScript(t, "\n"+privilegedSignatureHeader+testSignature(t, body)+"\n"+body)
program, err := Compile(path)
if err != nil {
t.Fatal(err)
}
L, err := program.NewState(context.Background(), nil, nil)
if L != nil {
L.Close()
t.Fatal("failing signed script returned a state")
}
if err == nil || !strings.Contains(err.Error(), path+":5: signed failure") {
t.Fatalf("signed initialization error = %v; want original source line 5", err)
}
}
func TestProgramSandbox(t *testing.T) {
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "escape.lua"), []byte(`disk_loaded = true; return "escaped"`), 0o600); err != nil {
t.Fatal(err)
}
t.Setenv("LUA_PATH", filepath.ToSlash(filepath.Join(dir, "?.lua")))
program := compileTestProgram(t, `
local function check()
assert(package == nil)
assert(require("_G") == _G)
for _, name in ipairs({"io", "os", "debug", "channel", "coroutine", "dofile", "loadfile",
"load", "loadstring", "getfenv", "setfenv", "collectgarbage", "module",
"getmetatable", "setmetatable", "rawget", "rawset", "newproxy", "print", "_printregs"}) do
assert(_G[name] == nil, name)
end
for _, name in ipairs({"io", "os", "debug", "channel", "coroutine", "package", "escape"}) do
assert(not pcall(require, name), name)
end
for _, name in ipairs({"math", "string", "table"}) do
assert(require(name) == _G[name], name)
end
assert(string.dump == nil and ("").dump == nil and require("string").dump == nil)
assert(math.randomseed == nil and require("math").randomseed == nil)
local helper = require("xray.test")
assert(helper.value == "ok" and helper == require("xray.test"))
assert(math.abs(-2) == 2 and string.upper("ok") == "OK")
assert(table.concat({"a", "b"}) == "ab")
end
check()
function CheckSandbox()
check()
local injected = false
package = {
preload = {
escape = function() injected = true; return "escaped" end,
["xray.late"] = function() injected = true; return "injected" end,
},
loaded = {escape = "escaped", ["cached.only"] = "injected"},
loaders = {function() injected = true; return function() return "escaped" end end},
path = testPath,
cpath = testPath,
}
assert(not pcall(require, "escape") and disk_loaded == nil)
assert(not pcall(require, "cached.only") and not pcall(require, "package"))
assert(require("xray.late") == "host" and require("xray.late") == "host")
assert(require("xray.test").value == "ok" and not injected)
package = nil
end`)
var helperLoads, lateLoads int
L, err := program.NewState(context.Background(), func(L *glua.LState) {
if L.GetGlobal("io") != glua.LNil || L.GetGlobal("loadfile") != glua.LNil {
t.Fatal("sandbox was not configured before module registration")
}
L.SetGlobal("testPath", glua.LString(filepath.ToSlash(filepath.Join(dir, "?.lua"))))
L.PreloadModule("xray.test", func(L *glua.LState) int {
helperLoads++
module := L.NewTable()
module.RawSetString("value", glua.LString("ok"))
L.Push(module)
return 1
})
L.PreloadModule("xray.late", func(L *glua.LState) int {
lateLoads++
L.Push(glua.LString("host"))
return 1
})
}, nil)
if err != nil {
t.Fatal(err)
}
defer L.Close()
if L.GetTop() != 0 {
t.Fatalf("stack top = %d, want 0", L.GetTop())
}
if err := L.CallByParam(glua.P{Fn: L.GetGlobal("CheckSandbox"), Protect: true}); err != nil {
t.Fatal(err)
}
if helperLoads != 1 || lateLoads != 1 {
t.Fatalf("module initialization calls = (%d, %d), want (1, 1)", helperLoads, lateLoads)
}
}
func TestProgramSandboxToString(t *testing.T) {
program := compileTestProgram(t, `
assert(tostring == require("_G").tostring)
for _, convert in ipairs({tostring, require("_G").tostring}) do
assert(convert(nil) == "nil")
assert(convert(true) == "true" and convert(false) == "false")
assert(convert(0) == "0" and convert(-42) == "-42" and convert(1.25) == "1.25")
assert(convert("") == "" and convert("hello") == "hello")
assert(convert(42, {}) == "42")
assert(not pcall(convert))
assert(convert(testIPv4) == "192.0.2.1")
assert(convert(testIPv6) == "2001:db8::1")
assert(convert(testNativeIP) == "198.51.100.1")
assert(convert(testNativeStringer) == "0xdeadbeef")
assert(convert(testNativeError) == "lookup failed")
for _, value in ipairs({{}, function() end, convert, testUserdata, testThread, testChannel, testGoStruct, testIPs}) do
local ok, result = pcall(convert, value)
assert(ok and result == "<opaque>")
end
end
string.__tostring = function() error("string __tostring must not run") end
assert(tostring("hello") == "hello")
string.__tostring = nil
`)
var metamethodCalls int
L, err := program.NewState(context.Background(), func(L *glua.LState) {
L.SetGlobal("testIPv4", luar.New(L, net.ParseIP("192.0.2.1")))
L.SetGlobal("testIPv6", luar.New(L, net.ParseIP("2001:db8::1")))
PushUserData(L, net.ParseIP("198.51.100.1"))
L.SetGlobal("testNativeIP", L.Get(-1))
L.Pop(1)
PushUserData(L, bytes.NewBufferString("0xdeadbeef"))
L.SetGlobal("testNativeStringer", L.Get(-1))
L.Pop(1)
PushError(L, errors.New("lookup failed"))
L.SetGlobal("testNativeError", L.Get(-1))
L.Pop(1)
L.SetGlobal("testGoStruct", luar.New(L, &struct{ Value string }{"readable field"}))
NewSlicePusher[net.IP](L)(L, []net.IP{net.ParseIP("192.0.2.1")})
L.SetGlobal("testIPs", L.Get(-1))
L.Pop(1)
userdata := L.NewUserData()
metatable := L.NewTable()
metatable.RawSetString("__tostring", L.NewFunction(func(L *glua.LState) int {
metamethodCalls++
L.Push(glua.LString("custom"))
return 1
}))
L.SetMetatable(userdata, metatable)
L.SetGlobal("testUserdata", userdata)
thread, cancel := L.NewThread()
if cancel != nil {
t.Cleanup(cancel)
}
L.SetGlobal("testThread", thread)
L.SetGlobal("testChannel", glua.LChannel(make(chan glua.LValue)))
}, nil)
if err != nil {
t.Fatal(err)
}
defer L.Close()
if metamethodCalls != 0 {
t.Fatalf("__tostring calls = %d, want 0", metamethodCalls)
}
}
func TestProgramSandboxStringFormat(t *testing.T) {
program := compileTestProgram(t, `
assert(string.format == require("string").format and string.format == ("").format)
for _, format in ipairs({string.format, require("string").format, ("").format}) do
assert(format("%s %04d %.2f %t", "hello", 42, 1.25, true) == "hello 0042 1.25 true")
assert(format("%q", "hello") == '"hello"')
assert(format("%[2]s:%[1]d", 42, "hello") == "hello:42")
assert(format("%%p") == "%p")
assert(format("%%p", {}) == format("%%p", "<opaque>"))
assert(format("%s", 42) == "42")
assert(format("%s", nil) == "nil")
assert(format("%s", testIP) == "192.0.2.1")
assert(format("%#v", testIP) == '"192.0.2.1"')
assert(format("%p", testIP) == format("%p", "192.0.2.1"))
assert(format("%s", testNativeStringer) == "0xdeadbeef")
assert(format("%s", testNativeError) == "lookup failed")
assert(not pcall(format))
for _, value in ipairs({{}, function() end, format, testUserdata, testThread, testChannel}) do
assert(format("%s", value) == "<opaque>")
assert(format("%v", value) == "<opaque>")
assert(format("%q", value) == '"<opaque>"')
assert(format("%#v", value) == '"<opaque>"')
for _, pattern in ipairs({"%p", "%#p", "%+020p", "%[1]p", "%[1]p %#[1]v",
"%x", "%X", "%#x", "%d", "%t", "%w"}) do
local result = format(pattern, value)
assert(result == format(pattern, "<opaque>"), pattern .. ": " .. result)
end
end
assert(format("%p", nil) == format("%p", "nil"))
end
assert(("%s"):format({}) == "<opaque>")
`)
L, err := program.NewState(context.Background(), func(L *glua.LState) {
L.SetGlobal("testIP", luar.New(L, net.ParseIP("192.0.2.1")))
PushUserData(L, bytes.NewBufferString("0xdeadbeef"))
L.SetGlobal("testNativeStringer", L.Get(-1))
L.Pop(1)
PushError(L, errors.New("lookup failed"))
L.SetGlobal("testNativeError", L.Get(-1))
L.Pop(1)
userdata := L.NewUserData()
metatable := L.NewTable()
metatable.RawSetString("__tostring", L.NewFunction(func(L *glua.LState) int {
t.Error("string.format must not call __tostring")
L.Push(glua.LString("custom"))
return 1
}))
L.SetMetatable(userdata, metatable)
L.SetGlobal("testUserdata", userdata)
thread, cancel := L.NewThread()
if cancel != nil {
t.Cleanup(cancel)
}
L.SetGlobal("testThread", thread)
L.SetGlobal("testChannel", glua.LChannel(make(chan glua.LValue)))
}, nil)
if err != nil {
t.Fatal(err)
}
defer L.Close()
if L.GetTop() != 0 {
t.Fatalf("stack top = %d, want 0", L.GetTop())
}
}
func TestProgramSandboxModuleFailures(t *testing.T) {
program := compileTestProgram(t, `
assert(package == nil)
local ok, err = pcall(require, "xray.failure")
assert(not ok and string.find(err, "module initialization failed", 1, true))
package = {loaded = {["xray.failure"] = "injected"}}
ok, err = pcall(require, "xray.failure")
assert(not ok and string.find(err, "loop or previous error loading module", 1, true))
for i = 1, 2 do
ok, err = pcall(require, "xray.cycle")
assert(not ok and string.find(err, "loop or previous error loading module", 1, true))
end
package = nil
`)
var failedLoads, cycleLoads int
L, err := program.NewState(context.Background(), func(L *glua.LState) {
L.PreloadModule("xray.failure", func(L *glua.LState) int {
failedLoads++
L.RaiseError("module initialization failed")
return 0
})
L.PreloadModule("xray.cycle", func(L *glua.LState) int {
cycleLoads++
L.Push(L.GetGlobal("require"))
L.Push(glua.LString("xray.cycle"))
L.Call(1, 1)
return 1
})
}, nil)
if err != nil {
t.Fatal(err)
}
defer L.Close()
if failedLoads != 1 || cycleLoads != 1 {
t.Fatalf("module initialization calls = (%d, %d), want (1, 1)", failedLoads, cycleLoads)
}
if L.GetTop() != 0 {
t.Fatalf("stack top = %d, want 0", L.GetTop())
}
}
func TestProgramSandboxModulesAreCollected(t *testing.T) {
for _, fail := range []bool{false, true} {
name := "success"
if fail {
name = "failure"
}
t.Run(name, func(t *testing.T) {
ref := func() weak.Pointer[glua.LTable] {
program := compileTestProgram(t, `payload = string.rep("x", 1024 * 1024); assert(pcall(require, "xray.test") == expectedSuccess)`)
L, err := program.NewState(context.Background(), func(L *glua.LState) {
L.SetGlobal("expectedSuccess", glua.LBool(!fail))
L.PreloadModule("xray.test", func(L *glua.LState) int {
if fail {
L.RaiseError("module initialization failed")
}
L.Push(L.NewTable())
return 1
})
}, nil)
if err != nil {
t.Fatal(err)
}
ref := weak.Make(L.G.Global)
L.Close()
return ref
}()
runtime.GC()
runtime.GC()
if ref.Value() != nil {
t.Fatal("closed sandbox's global table was not collected")
}
})
}
}
func TestProgramSandboxClosesFailedInitialization(t *testing.T) {
program := compileTestProgram(t, `require("xray.failure")`)
var partial *glua.LState
L, err := program.NewState(context.Background(), func(L *glua.LState) {
partial = L
L.PreloadModule("xray.failure", func(L *glua.LState) int {
L.RaiseError("module initialization failed")
return 0
})
}, func(*glua.LState) error {
t.Error("validation ran after failed initialization")
return nil
})
if L != nil {
L.Close()
t.Fatal("sandbox initialization returned a state")
}
if err == nil || !strings.Contains(err.Error(), "module initialization failed") || partial == nil || !partial.IsClosed() {
t.Fatalf("error = %v, partial state closed = %t", err, partial != nil && partial.IsClosed())
}
}
func TestProgramPrivilegedLibraries(t *testing.T) {
setTestPrivilegedPublicKey(t, privilegedTestPublicKey)
const body = `assert(require("io") == io and require("os") == os)
assert(require("debug") == debug and require("channel") == channel)
assert(type(coroutine.create) == "function" and require("coroutine") == coroutine)
assert(type(dofile) == "function" and type(loadfile) == "function")
assert(type(getfenv) == "function" and type(setfenv) == "function")
assert(type(collectgarbage) == "function" and type(print) == "function")
assert(type(getmetatable) == "function" and type(setmetatable) == "function")
assert(type(rawget) == "function" and type(rawset) == "function")
assert(tostring(42) == "42" and type(tostring({})) == "string")
assert(tostring(setmetatable({}, {__tostring = function() return "custom" end})) == "custom")
assert(type(string.dump) == "function" and type(math.randomseed) == "function")
assert(type(package.loaders[2]) == "function" and type(package.path) == "string")
assert(type(debug.traceback()) == "string" and type(os.time()) == "number")
assert(assert(loadstring("return 42"))() == 42)
local file = assert(io.open(testPath, "w"))
file:write("privileged")
file:close()`
code := privilegedSignatureHeader + testSignature(t, body) + "\n" + body
path := filepath.Join(t.TempDir(), "privileged.lua")
if err := os.WriteFile(path, []byte(code), 0o600); err != nil {
t.Fatal(err)
}
for _, tc := range []struct {
name, script string
privileged bool
}{
{"file", path, true},
// TODO: Re-enable these cases when Compile supports inline scripts again.
// {"base64", xbase64.Prefix + base64.StdEncoding.EncodeToString([]byte(code)), false},
// {"gzip", xbase64.GzipPrefix + base64.StdEncoding.EncodeToString(compressSource(t, code)), false},
} {
t.Run(tc.name, func(t *testing.T) {
program, err := Compile(tc.script)
if err != nil {
t.Fatal(err)
}
output := filepath.Join(t.TempDir(), "privileged.txt")
L, err := program.NewState(context.Background(), func(L *glua.LState) {
L.SetGlobal("testPath", glua.LString(output))
}, nil)
if !tc.privileged {
if L != nil {
L.Close()
t.Fatal("inline script with a valid signature accessed privileged libraries")
}
if err == nil || !strings.Contains(err.Error(), "no preloaded module: io") {
t.Fatalf("NewState error = %v, want io module unavailable", err)
}
if _, err := os.Stat(output); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("inline script output stat error = %v, want file not found", err)
}
return
}
if err != nil {
t.Fatal(err)
}
defer L.Close()
data, err := os.ReadFile(output)
if err != nil || string(data) != "privileged" {
t.Fatalf("privileged file contents = %q, error = %v", data, err)
}
})
}
}
func TestProgramPrivilegesAreCompileTimeSnapshot(t *testing.T) {
setTestPrivilegedPublicKey(t, privilegedTestPublicKey)
for _, privileged := range []bool{false, true} {
name := "sandboxed"
if privileged {
name = "privileged"
}
t.Run(name, func(t *testing.T) {
const body = "value = 1"
code := body
if privileged {
code = privilegedSignatureHeader + testSignature(t, body) + "\n" + body
}
path := filepath.Join(t.TempDir(), "snapshot.lua")
if err := os.WriteFile(path, []byte(code), 0o600); err != nil {
t.Fatal(err)
}
program, err := Compile(path)
if err != nil {
t.Fatal(err)
}
// Source edits cannot change an already compiled Program's authorization.
changed := privilegedSignatureHeader + testSignature(t, body) + "\n" + body
if privileged {
changed = body
}
if err := os.WriteFile(path, []byte(changed), 0o600); err != nil {
t.Fatal(err)
}
check := func(L *glua.LState) {
t.Helper()
if got := L.GetGlobal("os").Type() == glua.LTTable; got != privileged {
t.Fatalf("os available = %t, want compile-time value %t", got, privileged)
}
}
factory := program.NewStateFactory(time.Second, nil, nil)
for range 2 {
L, err := factory(context.Background())
if err != nil {
t.Fatal(err)
}
defer L.Close()
check(L)
}
recompiled, err := Compile(path)
if err != nil {
t.Fatal(err)
}
if recompiled.privileged == privileged {
t.Fatal("recompilation did not observe the changed source")
}
})
}
}
+95
View File
@@ -0,0 +1,95 @@
package lua
import (
"math"
"github.com/xtls/xray-core/common/errors"
glua "github.com/yuin/gopher-lua"
)
type number interface {
~int | ~int8 | ~int16 | ~int32 | ~int64 |
~uint | ~uint8 | ~uint16 | ~uint32 | ~uint64 | ~uintptr |
~float32 | ~float64
}
// PushNumber converts a Go number to a Lua number and pushes it.
func PushNumber[T number](L *glua.LState, value T) {
L.Push(glua.LNumber(value))
}
// PushString converts a Go string to a Lua string and pushes it.
func PushString(L *glua.LState, value string) {
L.Push(glua.LString(value))
}
// PushNil pushes Lua nil.
func PushNil(L *glua.LState) {
L.Push(glua.LNil)
}
// PushUserData pushes a native Go value without copying it.
func PushUserData(L *glua.LState, value any) {
ud := L.NewUserData()
ud.Value = value
L.Push(ud)
}
// PushError pushes nil or the original Go error as userdata.
func PushError(L *glua.LState, err error) {
if err == nil {
L.Push(glua.LNil)
return
}
PushUserData(L, err)
}
// ReadUserData reads a native Go value of type T without copying it.
// Other Lua values or userdata containing a different type return invalidMessage.
func ReadUserData[T any](value glua.LValue, invalidMessage string) (T, error) {
if ud, ok := value.(*glua.LUserData); ok {
if result, ok := ud.Value.(T); ok {
return result, nil
}
}
var zero T
return zero, errors.New(invalidMessage)
}
// ReadError accepts nil, a native Go error, or a Lua string.
// Native errors retain their identity; other values return invalidMessage.
func ReadError(value glua.LValue, invalidMessage string) error {
if value == glua.LNil {
return nil
}
if ud, ok := value.(*glua.LUserData); ok {
if err, ok := ud.Value.(error); ok {
return err
}
}
if message, ok := value.(glua.LString); ok {
return errors.New(string(message))
}
return errors.New(invalidMessage)
}
// ReadUint32 accepts only integral Lua numbers in the uint32 range.
func ReadUint32(value glua.LValue, invalidMessage string) (uint32, error) {
number, ok := value.(glua.LNumber)
if !ok || number < 0 || number > math.MaxUint32 || math.Trunc(float64(number)) != float64(number) {
return 0, errors.New(invalidMessage)
}
return uint32(number), nil
}
// ReadOptionalString accepts a Lua string or nil, which becomes an empty string.
// It does not coerce other values to strings.
func ReadOptionalString(value glua.LValue, invalidMessage string) (string, error) {
if value == glua.LNil {
return "", nil
}
if result, ok := value.(glua.LString); ok {
return string(result), nil
}
return "", errors.New(invalidMessage)
}
+121
View File
@@ -0,0 +1,121 @@
package lua
import (
"errors"
"math"
"strings"
"testing"
glua "github.com/yuin/gopher-lua"
)
func TestReadUint32(t *testing.T) {
for _, tc := range []struct {
name string
value glua.LValue
want uint32
wantErr bool
}{
{name: "zero", value: glua.LNumber(0)},
{name: "integer", value: glua.LNumber(45), want: 45},
{name: "maximum", value: glua.LNumber(math.MaxUint32), want: math.MaxUint32},
{name: "fraction", value: glua.LNumber(1.5), wantErr: true},
{name: "negative", value: glua.LNumber(-1), wantErr: true},
{name: "overflow", value: glua.LNumber(math.MaxUint32 + 1), wantErr: true},
{name: "NaN", value: glua.LNumber(math.NaN()), wantErr: true},
{name: "positive infinity", value: glua.LNumber(math.Inf(1)), wantErr: true},
{name: "negative infinity", value: glua.LNumber(math.Inf(-1)), wantErr: true},
{name: "nil", value: glua.LNil, wantErr: true},
{name: "numeric string", value: glua.LString("45"), wantErr: true},
{name: "boolean", value: glua.LTrue, wantErr: true},
} {
t.Run(tc.name, func(t *testing.T) {
got, err := ReadUint32(tc.value, "invalid number")
if got != tc.want || (err != nil) != tc.wantErr {
t.Fatalf("ReadUint32() = %d, %v; want %d, error %t", got, err, tc.want, tc.wantErr)
}
if err != nil && !strings.Contains(err.Error(), "invalid number") {
t.Fatalf("error = %v, want invalid number", err)
}
})
}
}
func TestReadOptionalString(t *testing.T) {
for _, tc := range []struct {
name string
value glua.LValue
want string
wantErr bool
}{
{name: "nil", value: glua.LNil},
{name: "empty", value: glua.LString("")},
{name: "string", value: glua.LString("out"), want: "out"},
{name: "number", value: glua.LNumber(1), wantErr: true},
{name: "boolean", value: glua.LFalse, wantErr: true},
} {
t.Run(tc.name, func(t *testing.T) {
got, err := ReadOptionalString(tc.value, "invalid string")
if got != tc.want || (err != nil) != tc.wantErr {
t.Fatalf("ReadOptionalString() = %q, %v; want %q, error %t", got, err, tc.want, tc.wantErr)
}
if err != nil && !strings.Contains(err.Error(), "invalid string") {
t.Fatalf("error = %v, want invalid string", err)
}
})
}
}
func TestUserDataRoundTrip(t *testing.T) {
L := glua.NewState()
defer L.Close()
want := []int{1, 2}
PushUserData(L, want)
if L.GetTop() != 1 {
t.Fatalf("stack top = %d, want 1", L.GetTop())
}
got, err := ReadUserData[[]int](L.Get(-1), "invalid userdata")
if err != nil || len(got) != len(want) || &got[0] != &want[0] {
t.Fatalf("userdata = %v, %v; want original slice", got, err)
}
PushUserData(L, []int(nil))
if got, err := ReadUserData[[]int](L.Get(-1), "invalid userdata"); err != nil || got != nil {
t.Fatalf("nil slice userdata = %v, %v", got, err)
}
for _, value := range []glua.LValue{glua.LNil, glua.LString("1"), L.NewTable(), L.Get(1)} {
if got, err := ReadUserData[int](value, "invalid userdata"); got != 0 || err == nil || !strings.Contains(err.Error(), "invalid userdata") {
t.Fatalf("ReadUserData(%v) = %d, %v; want invalid userdata", value, got, err)
}
}
}
func TestErrorRoundTrip(t *testing.T) {
L := glua.NewState()
defer L.Close()
want := errors.New("upstream failed")
for _, err := range []error{nil, want} {
PushError(L, err)
if L.GetTop() != 1 {
t.Fatalf("stack top = %d, want 1", L.GetTop())
}
if err == nil && L.Get(-1) != glua.LNil {
t.Fatalf("nil error pushed as %v", L.Get(-1))
}
if got := ReadError(L.Get(-1), "invalid error"); got != err {
t.Fatalf("ReadError() = %v, want original error %v", got, err)
}
L.Pop(1)
}
for _, message := range []string{"script failed", ""} {
if err := ReadError(glua.LString(message), "invalid error"); err == nil || !strings.Contains(err.Error(), message) {
t.Fatalf("string error = %v, want %q", err, message)
}
}
wrong := L.NewUserData()
wrong.Value = "not a native error"
for _, value := range []glua.LValue{glua.LTrue, glua.LNumber(1), L.NewTable(), wrong, L.NewUserData()} {
if err := ReadError(value, "invalid error"); err == nil || !strings.Contains(err.Error(), "invalid error") {
t.Fatalf("ReadError(%v) = %v, want invalid error", value, err)
}
}
}
+1 -3
View File
@@ -314,12 +314,10 @@ func (m *ClientWorker) Dispatch(ctx context.Context, link *transport.Link) bool
} }
sm := m.sessionManager sm := m.sessionManager
s := sm.Allocate(&m.strategy) s := sm.Allocate(&m.strategy, link.Reader, link.Writer)
if s == nil { if s == nil {
return false return false
} }
s.input = link.Reader
s.output = link.Writer
go fetchInput(ctx, s, m.link.Writer) go fetchInput(ctx, s, m.link.Writer)
if _, ok := link.Reader.(*pipe.Reader); !ok { if _, ok := link.Reader.(*pipe.Reader); !ok {
select { select {
+47
View File
@@ -7,9 +7,11 @@ import (
"github.com/golang/mock/gomock" "github.com/golang/mock/gomock"
"github.com/xtls/xray-core/common" "github.com/xtls/xray-core/common"
"github.com/xtls/xray-core/common/buf"
"github.com/xtls/xray-core/common/errors" "github.com/xtls/xray-core/common/errors"
"github.com/xtls/xray-core/common/mux" "github.com/xtls/xray-core/common/mux"
"github.com/xtls/xray-core/common/net" "github.com/xtls/xray-core/common/net"
"github.com/xtls/xray-core/common/protocol"
"github.com/xtls/xray-core/common/session" "github.com/xtls/xray-core/common/session"
"github.com/xtls/xray-core/testing/mocks" "github.com/xtls/xray-core/testing/mocks"
"github.com/xtls/xray-core/transport" "github.com/xtls/xray-core/transport"
@@ -114,3 +116,48 @@ func TestClientWorkerClose(t *testing.T) {
common.Must(w2.Close()) common.Must(w2.Close())
} }
func TestClientWorkerUDPSource(t *testing.T) {
downR, downW := pipe.New(pipe.WithoutSizeLimit())
upR, upW := pipe.New(pipe.WithoutSizeLimit())
worker, err := mux.NewClientWorker(transport.Link{Reader: downR, Writer: upW}, mux.ClientStrategy{})
common.Must(err)
inR, inW := pipe.New(pipe.WithoutSizeLimit())
outR, outW := pipe.New(pipe.WithoutSizeLimit())
ctx := session.ContextWithOutbounds(context.Background(), []*session.Outbound{{
Target: net.UDPDestination(net.ParseAddress("8.8.8.8"), 53),
}})
if !worker.Dispatch(ctx, &transport.Link{Reader: inR, Writer: outW}) {
t.Fatal("failed to dispatch")
}
b := buf.New()
b.WriteString("query")
common.Must(inW.WriteMultiBuffer(buf.MultiBuffer{b}))
mb, err := upR.ReadMultiBuffer() // New frame, the session is UDP from now on
common.Must(err)
buf.ReleaseMulti(mb)
srcs := []net.Destination{
net.UDPDestination(net.ParseAddress("1.1.1.1"), 1111),
net.UDPDestination(net.DomainAddress("example.com"), 2222),
net.UDPDestination(net.ParseAddress("3.3.3.3"), 3333),
}
w := mux.NewResponseWriter(1, downW, protocol.TransferTypePacket)
var got buf.MultiBuffer
for i := range srcs {
b := buf.New()
b.WriteString("reply")
b.UDP = &srcs[i]
common.Must(w.WriteMultiBuffer(buf.MultiBuffer{b}))
// keep earlier replies around while the next frame is parsed
mb, err := outR.ReadMultiBuffer()
common.Must(err)
got = append(got, mb...)
}
for i, b := range got {
if b.UDP == nil || *b.UDP != srcs[i] {
t.Errorf("reply %d: source = %v, want %v", i, b.UDP, srcs[i])
}
}
}
+5 -4
View File
@@ -14,15 +14,16 @@ import (
type PacketReader struct { type PacketReader struct {
reader io.Reader reader io.Reader
eof bool eof bool
dest *net.Destination dest net.Destination
} }
// NewPacketReader creates a new PacketReader. // NewPacketReader creates a new PacketReader.
// dest is copied because the caller reuses it for the next frame.
func NewPacketReader(reader io.Reader, dest *net.Destination) *PacketReader { func NewPacketReader(reader io.Reader, dest *net.Destination) *PacketReader {
return &PacketReader{ return &PacketReader{
reader: reader, reader: reader,
eof: false, eof: false,
dest: dest, dest: *dest,
} }
} }
@@ -47,8 +48,8 @@ func (r *PacketReader) ReadMultiBuffer() (buf.MultiBuffer, error) {
return nil, err return nil, err
} }
r.eof = true r.eof = true
if r.dest != nil && r.dest.Network == net.Network_UDP { if r.dest.Network == net.Network_UDP {
b.UDP = r.dest b.UDP = &r.dest // only one packet is read, so b owns r.dest
} }
return buf.MultiBuffer{b}, nil return buf.MultiBuffer{b}, nil
} }
+3 -1
View File
@@ -51,7 +51,7 @@ func (m *SessionManager) Count() int {
return int(m.count) return int(m.count)
} }
func (m *SessionManager) Allocate(Strategy *ClientStrategy) *Session { func (m *SessionManager) Allocate(Strategy *ClientStrategy, input buf.Reader, output buf.Writer) *Session {
m.Lock() m.Lock()
defer m.Unlock() defer m.Unlock()
@@ -64,6 +64,8 @@ func (m *SessionManager) Allocate(Strategy *ClientStrategy) *Session {
m.count++ m.count++
s := &Session{ s := &Session{
input: input,
output: output,
ID: m.count, ID: m.count,
parent: m, parent: m,
done: done.New(), done: done.New(),
+3 -3
View File
@@ -9,7 +9,7 @@ import (
func TestSessionManagerAdd(t *testing.T) { func TestSessionManagerAdd(t *testing.T) {
m := NewSessionManager() m := NewSessionManager()
s := m.Allocate(&ClientStrategy{}) s := m.Allocate(&ClientStrategy{}, nil, nil)
if s.ID != 1 { if s.ID != 1 {
t.Error("id: ", s.ID) t.Error("id: ", s.ID)
} }
@@ -17,7 +17,7 @@ func TestSessionManagerAdd(t *testing.T) {
t.Error("size: ", m.Size()) t.Error("size: ", m.Size())
} }
s = m.Allocate(&ClientStrategy{}) s = m.Allocate(&ClientStrategy{}, nil, nil)
if s.ID != 2 { if s.ID != 2 {
t.Error("id: ", s.ID) t.Error("id: ", s.ID)
} }
@@ -39,7 +39,7 @@ func TestSessionManagerAdd(t *testing.T) {
func TestSessionManagerClose(t *testing.T) { func TestSessionManagerClose(t *testing.T) {
m := NewSessionManager() m := NewSessionManager()
s := m.Allocate(&ClientStrategy{}) s := m.Allocate(&ClientStrategy{}, nil, nil)
if m.CloseIfNoSessionAndIdle(m.Size(), m.Count()) { if m.CloseIfNoSessionAndIdle(m.Size(), m.Count()) {
t.Error("able to close") t.Error("able to close")
+20
View File
@@ -0,0 +1,20 @@
package net
// PacketConnWrapper wraps a PacketConn into a Conn with a fixed destination address.
type PacketConnWrapper struct {
PacketConn
Dest Addr
}
func (c *PacketConnWrapper) Read(p []byte) (int, error) {
n, _, err := c.PacketConn.ReadFrom(p)
return n, err
}
func (c *PacketConnWrapper) Write(p []byte) (int, error) {
return c.PacketConn.WriteTo(p, c.Dest)
}
func (c *PacketConnWrapper) RemoteAddr() Addr {
return c.Dest
}
+50
View File
@@ -0,0 +1,50 @@
package platform
import (
"io"
"os"
"path/filepath"
)
type configFile struct {
path string
info os.FileInfo
}
var configFiles []configFile
// ReadConfigFile reads a config source and records local regular files.
func ReadConfigFile(file *os.File) ([]byte, error) {
info, err := file.Stat()
if err != nil {
return nil, err
}
path := ""
if info.Mode().IsRegular() && file != os.Stdin {
path, err = filepath.Abs(file.Name())
if err != nil {
return nil, err
}
path, err = filepath.EvalSymlinks(path)
if err != nil {
return nil, err
}
}
data, err := io.ReadAll(file)
if err != nil {
return nil, err
}
if info.Mode().IsRegular() {
configFiles = append(configFiles, configFile{path: path, info: info})
}
return data, nil
}
func isConfigFile(path string, info os.FileInfo) bool {
for _, file := range configFiles {
if file.path == path || os.SameFile(file.info, info) {
return true
}
}
return false
}
+5 -4
View File
@@ -34,12 +34,13 @@ func ReadAsset(file string) ([]byte, error) {
return ReadFile(path) return ReadFile(path)
} }
func OpenAsset(file string) (io.ReadCloser, error) { func OpenAsset(file string) (io.ReadCloser, int64, error) {
path, _, err := getAssetFileLocation(file) path, info, err := getAssetFileLocation(file)
if err != nil { if err != nil {
return nil, err return nil, 0, err
} }
return NewFileReader(path) r, err := NewFileReader(path)
return r, info.Size(), err
} }
func StatAsset(file string) (os.FileInfo, error) { func StatAsset(file string) (os.FileInfo, error) {
+70
View File
@@ -1,6 +1,8 @@
package platform // import "github.com/xtls/xray-core/common/platform" package platform // import "github.com/xtls/xray-core/common/platform"
import ( import (
"errors"
"fmt"
"os" "os"
"path/filepath" "path/filepath"
"strconv" "strconv"
@@ -90,3 +92,71 @@ func GetConfDirPath() string {
configPath := NewEnvFlag(ConfdirLocation).GetValue(func() string { return "" }) configPath := NewEnvFlag(ConfdirLocation).GetValue(func() string { return "" })
return configPath return configPath
} }
// ResolveLuaFile finds a local .lua script and returns its resolved absolute path.
// Relative paths: XRAY_LOCATION_CONFDIR > XRAY_LOCATION_CONFIG > working dir > executable dir.
func ResolveLuaFile(path string) (string, error) {
if path == "" {
return "", errors.New("Lua file path is empty")
}
paths := []string{path}
if !filepath.IsAbs(path) {
paths = nil
for _, dir := range []string{
GetConfDirPath(),
NewEnvFlag(ConfigLocation).GetValue(func() string { return "" }),
".",
getExecutableDir(),
} {
if dir != "" {
paths = append(paths, filepath.Join(dir, path))
}
}
}
path, info, err := resolveFile(paths)
if err != nil {
return "", err
}
path, err = resolveLuaTarget(path)
if err != nil {
return "", err
}
if isConfigFile(path, info) {
return "", fmt.Errorf("Lua file was loaded as a configuration file: %s", path)
}
return path, nil
}
func resolveLuaTarget(path string) (string, error) {
path, err := filepath.EvalSymlinks(path)
if err != nil {
return "", fmt.Errorf("failed to resolve Lua file symlinks: %w", err)
}
if filepath.Ext(path) != ".lua" {
return "", errors.New("Lua file extension must be .lua")
}
return path, nil
}
func resolveFile(paths []string) (string, os.FileInfo, error) {
var tried []string
for _, path := range paths {
path, err := filepath.Abs(path)
if err != nil {
return "", nil, fmt.Errorf("failed to resolve file path: %w", err)
}
tried = append(tried, path)
info, err := os.Stat(path)
if errors.Is(err, os.ErrNotExist) {
continue
}
if err != nil {
return "", nil, fmt.Errorf("failed to inspect file %q: %w", path, err)
}
if !info.Mode().IsRegular() {
return "", nil, fmt.Errorf("file is not a regular file: %s", path)
}
return path, info, nil
}
return "", nil, fmt.Errorf("file not found; tried %q: %w", tried, os.ErrNotExist)
}
+88
View File
@@ -1,9 +1,11 @@
package platform_test package platform_test
import ( import (
"errors"
"os" "os"
"path/filepath" "path/filepath"
"runtime" "runtime"
"strings"
"testing" "testing"
"github.com/xtls/xray-core/common" "github.com/xtls/xray-core/common"
@@ -64,3 +66,89 @@ func TestGetAssetLocation(t *testing.T) {
} }
} }
} }
func TestResolveLuaFile(t *testing.T) {
workingDir := t.TempDir()
t.Chdir(workingDir)
executable, err := os.Executable()
common.Must(err)
file, err := os.CreateTemp(filepath.Dir(executable), "lua-*.lua")
common.Must(err)
common.Must(file.Close())
defer os.Remove(file.Name())
name := filepath.Base(file.Name())
paths := []string{
filepath.Join(t.TempDir(), name),
filepath.Join(t.TempDir(), name),
filepath.Join(workingDir, name),
file.Name(),
}
t.Setenv(ConfdirLocation, filepath.Dir(paths[0]))
t.Setenv(ConfigLocation, filepath.Dir(paths[1]))
for _, path := range paths[:3] {
common.Must(os.WriteFile(path, nil, 0o600))
}
for i, path := range paths {
paths[i], err = filepath.EvalSymlinks(path)
common.Must(err)
}
if got, err := ResolveLuaFile(paths[2]); err != nil || got != paths[2] {
t.Fatalf("absolute path = %q, %v; want %q", got, err, paths[2])
}
for i, want := range paths {
if i == 2 {
t.Setenv(ConfdirLocation, "")
t.Setenv(ConfigLocation, "")
}
if got, err := ResolveLuaFile(name); err != nil || got != want {
t.Fatalf("resolved path = %q, %v; want %q", got, err, want)
}
common.Must(os.Remove(want))
}
if _, err := ResolveLuaFile(name); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("missing file error = %v", err)
}
t.Setenv(ConfdirLocation, filepath.Dir(paths[0]))
t.Setenv(ConfigLocation, filepath.Dir(paths[1]))
common.Must(os.Mkdir(paths[0], 0o700))
common.Must(os.WriteFile(paths[1], nil, 0o600))
for _, path := range []string{"", name, filepath.Join(t.TempDir(), name)} {
if _, err := ResolveLuaFile(path); err == nil {
t.Fatalf("accepted invalid path %q", path)
}
}
}
func TestResolveLuaFileSymlink(t *testing.T) {
for _, tc := range []struct {
link, target string
valid bool
}{
{"alias.lua", "data.dat", false},
{"alias.dat", "script.lua", true},
} {
t.Run(tc.link, func(t *testing.T) {
dir, err := filepath.EvalSymlinks(t.TempDir())
common.Must(err)
target := filepath.Join(dir, tc.target)
common.Must(os.WriteFile(target, nil, 0o600))
link := filepath.Join(dir, tc.link)
if err := os.Symlink(tc.target, link); err != nil {
if runtime.GOOS == "windows" {
t.Skipf("cannot create symlinks: %v", err)
}
t.Fatal(err)
}
got, err := ResolveLuaFile(link)
if tc.valid {
if err != nil || got != target {
t.Fatalf("ResolveLuaFile(%q) = %q, %v; want %q", link, got, err, target)
}
} else if err == nil || got != "" || !strings.Contains(err.Error(), "Lua file extension must be .lua") {
t.Fatalf("ResolveLuaFile(%q) = %q, %v; want .lua extension error", link, got, err)
}
})
}
}
+4
View File
@@ -146,6 +146,10 @@ func SniffQUIC(b []byte) (*SniffHeader, error) {
} }
restPayload := b[hdrLen+int(packetLen):] restPayload := b[hdrLen+int(packetLen):]
// cachedReader can concatenate zero-padded UDP datagrams.
for len(restPayload) > 0 && restPayload[0] == 0 {
restPayload = restPayload[1:]
}
if !isQUICInitial { // Skip this packet if it's not initial packet if !isQUICInitial { // Skip this packet if it's not initial packet
b = restPayload b = restPayload
continue continue
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -62,7 +62,7 @@ func (s *Service) Cleanup() error {
} }
for name, subs := range s.subs { for name, subs := range s.subs {
newSub := make([]*Subscriber, 0, len(s.subs)) newSub := make([]*Subscriber, 0, len(subs))
for _, sub := range subs { for _, sub := range subs {
if !sub.IsClosed() { if !sub.IsClosed() {
newSub = append(newSub, sub) newSub = append(newSub, sub)
+2 -2
View File
@@ -19,8 +19,8 @@ import (
var ( var (
Version_x byte = 26 Version_x byte = 26
Version_y byte = 9 Version_y byte = 10
Version_z byte = 9 Version_z byte = 10
) )
var ( var (
+3 -1
View File
@@ -92,6 +92,8 @@ type Instance struct {
// Instance state // Instance state
func (server *Instance) IsRunning() bool { func (server *Instance) IsRunning() bool {
server.statusLock.Lock()
defer server.statusLock.Unlock()
return server.running return server.running
} }
@@ -320,7 +322,7 @@ func (s *Instance) RequireFeatures(callback interface{}, optional bool) error {
// AddFeature registers a feature into current Instance. // AddFeature registers a feature into current Instance.
func (s *Instance) AddFeature(feature features.Feature) error { func (s *Instance) AddFeature(feature features.Feature) error {
if s.running { if s.IsRunning() {
if err := feature.Start(); err != nil { if err := feature.Start(); err != nil {
errors.LogInfoInner(s.ctx, err, "failed to start feature") errors.LogInfoInner(s.ctx, err, "failed to start feature")
} }
+4 -1
View File
@@ -21,10 +21,12 @@ require (
github.com/stretchr/testify v1.12.1 github.com/stretchr/testify v1.12.1
github.com/vishvananda/netlink v1.3.1 github.com/vishvananda/netlink v1.3.1
github.com/xtls/reality v0.0.0-20260908062103-8cdf7bf9c7f0 github.com/xtls/reality v0.0.0-20260908062103-8cdf7bf9c7f0
github.com/yuin/gopher-lua v1.1.2
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba go4.org/netipx v0.0.0-20231129151722-fdeea329fbba
golang.org/x/crypto v0.57.0 golang.org/x/crypto v0.57.0
golang.org/x/crypto/x509roots/fallback v0.0.0-20261005185213-c3db4df58582
golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842 golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842
golang.org/x/net v0.59.0 golang.org/x/net v0.60.0
golang.org/x/sync v0.23.0 golang.org/x/sync v0.23.0
golang.org/x/sys v0.48.0 golang.org/x/sys v0.48.0
golang.zx2c4.com/wintun v0.0.0-20230126152724-0fa3db229ce2 golang.zx2c4.com/wintun v0.0.0-20230126152724-0fa3db229ce2
@@ -34,6 +36,7 @@ require (
google.golang.org/protobuf v1.36.12 google.golang.org/protobuf v1.36.12
gvisor.dev/gvisor v0.0.0-20260122175437-89a5d21be8f0 gvisor.dev/gvisor v0.0.0-20260122175437-89a5d21be8f0
h12.io/socks v1.0.3 h12.io/socks v1.0.3
layeh.com/gopher-luar v1.0.11
lukechampine.com/blake3 v1.4.1 lukechampine.com/blake3 v1.4.1
mvdan.cc/gofumpt v0.12.0 mvdan.cc/gofumpt v0.12.0
) )
+13 -2
View File
@@ -2,6 +2,9 @@ github.com/andybalholm/brotli v1.0.6 h1:Yf9fFpf49Zrxb9NlQaluyE92/+X7UVHlhMNJN2sx
github.com/andybalholm/brotli v1.0.6/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig= github.com/andybalholm/brotli v1.0.6/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig=
github.com/apernet/quic-go v0.61.1-0.20260806010916-184d081eef3e h1:5mgtR5gwIgBKMiGI1QdXldZZ+SNor06Nbu1wCBulQBg= github.com/apernet/quic-go v0.61.1-0.20260806010916-184d081eef3e h1:5mgtR5gwIgBKMiGI1QdXldZZ+SNor06Nbu1wCBulQBg=
github.com/apernet/quic-go v0.61.1-0.20260806010916-184d081eef3e/go.mod h1:x7qxEvX6MCVtDuBKHj3E+88+BtrbEMuAL5qGUKItjW8= github.com/apernet/quic-go v0.61.1-0.20260806010916-184d081eef3e/go.mod h1:x7qxEvX6MCVtDuBKHj3E+88+BtrbEMuAL5qGUKItjW8=
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
github.com/cloudflare/circl v1.6.5 h1:O64F26HEqNhznd/hrC5KZXVKYuKM2rx4deZDTc4ihQA= github.com/cloudflare/circl v1.6.5 h1:O64F26HEqNhznd/hrC5KZXVKYuKM2rx4deZDTc4ihQA=
github.com/cloudflare/circl v1.6.5/go.mod h1:h5LNyxAc5nTue9DS5jT+48en2PSDYt3zdGnz5OstK6c= github.com/cloudflare/circl v1.6.5/go.mod h1:h5LNyxAc5nTue9DS5jT+48en2PSDYt3zdGnz5OstK6c=
github.com/ghodss/yaml v1.0.1-0.20220118164431-d8423dcdf344 h1:Arcl6UOIS/kgO2nW3A65HN+7CMjSDP/gofXL4CZt1V4= github.com/ghodss/yaml v1.0.1-0.20220118164431-d8423dcdf344 h1:Arcl6UOIS/kgO2nW3A65HN+7CMjSDP/gofXL4CZt1V4=
@@ -81,6 +84,9 @@ github.com/wlynxg/anet v0.0.5/go.mod h1:eay5PRQr7fIVAMbTbchTnO9gG65Hg/uYGdc7mguH
github.com/xtls/reality v0.0.0-20260908062103-8cdf7bf9c7f0 h1:rb+fKQFhz+5I2PPuQsNYxI5mUU840XWYtRF0ZBjvkws= github.com/xtls/reality v0.0.0-20260908062103-8cdf7bf9c7f0 h1:rb+fKQFhz+5I2PPuQsNYxI5mUU840XWYtRF0ZBjvkws=
github.com/xtls/reality v0.0.0-20260908062103-8cdf7bf9c7f0/go.mod h1:DsJblcWDGt76+FVqBVwbwRhxyyNJsGV48gJLch0OOWI= github.com/xtls/reality v0.0.0-20260908062103-8cdf7bf9c7f0/go.mod h1:DsJblcWDGt76+FVqBVwbwRhxyyNJsGV48gJLch0OOWI=
github.com/yuin/goldmark v1.4.1/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= github.com/yuin/goldmark v1.4.1/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
github.com/yuin/gopher-lua v0.0.0-20190206043414-8bfc7677f583/go.mod h1:gqRgreBUhTSL0GeU64rtZ3Uq3wtjOa/TB2YfrtkCbVQ=
github.com/yuin/gopher-lua v1.1.2 h1:yF/FjE3hD65tBbt0VXLE13HWS9h34fdzJmrWRXwobGA=
github.com/yuin/gopher-lua v1.1.2/go.mod h1:7aRmXIWl37SqRf0koeyylBEzJ+aPt8A+mmkQ4f1ntR8=
go.uber.org/mock v0.5.2 h1:LbtPTcP8A5k9WPXj54PPPbjcI4Y6lhyOZXn+VS7wNko= go.uber.org/mock v0.5.2 h1:LbtPTcP8A5k9WPXj54PPPbjcI4Y6lhyOZXn+VS7wNko=
go.uber.org/mock v0.5.2/go.mod h1:wLlUxC2vVTPTaE3UD51E0BGOAElKrILxhVSDYQLld5o= go.uber.org/mock v0.5.2/go.mod h1:wLlUxC2vVTPTaE3UD51E0BGOAElKrILxhVSDYQLld5o=
go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw= go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
@@ -91,6 +97,8 @@ golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACk
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M= golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA= golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
golang.org/x/crypto/x509roots/fallback v0.0.0-20261005185213-c3db4df58582 h1:wjDBrGbfLuifgrVLFEWUBJYAfh5Q1wkMc5t0FY0tCbs=
golang.org/x/crypto/x509roots/fallback v0.0.0-20261005185213-c3db4df58582/go.mod h1:HPze8vhfG6fO06AM+VSvxRm4E3+5Yk375mgrJ5M2z1E=
golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842 h1:vr/HnozRka3pE4EsMEg1lgkXJkTFJCVUX+S/ZT6wYzM= golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842 h1:vr/HnozRka3pE4EsMEg1lgkXJkTFJCVUX+S/ZT6wYzM=
golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842/go.mod h1:XtvwrStGgqGPLc4cjQfWqZHG1YFdYs6swckp8vpsjnc= golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842/go.mod h1:XtvwrStGgqGPLc4cjQfWqZHG1YFdYs6swckp8vpsjnc=
golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
@@ -99,12 +107,13 @@ golang.org/x/mod v0.5.1/go.mod h1:5OXOZSfqPIIbmVBIIKWRFfZjPR0E5r58TLhUjH0a2Ro=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20211015210444-4f30a5c0130f/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20211015210444-4f30a5c0130f/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.59.0 h1:5zfYln+w5XCxwrnMMJPufRgNoXEaGxl0wo5GqPXyues= golang.org/x/net v0.60.0 h1:79p50tfZlm0J9YfoDsSi639qSXNGVwEzOPLCxM2FsYU=
golang.org/x/net v0.59.0/go.mod h1:2DA/G1UfVbCpQPeWTmMPGY7Cs2PkBkwu743bVX5PIVg= golang.org/x/net v0.60.0/go.mod h1:2DA/G1UfVbCpQPeWTmMPGY7Cs2PkBkwu743bVX5PIVg=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk= golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk=
golang.org/x/sync v0.23.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0= golang.org/x/sync v0.23.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
golang.org/x/sys v0.0.0-20190204203706-41f3e6584952/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -155,6 +164,8 @@ gvisor.dev/gvisor v0.0.0-20260122175437-89a5d21be8f0 h1:Lk6hARj5UPY47dBep70OD/TI
gvisor.dev/gvisor v0.0.0-20260122175437-89a5d21be8f0/go.mod h1:QkHjoMIBaYtpVufgwv3keYAbln78mBoCuShZrPrer1Q= gvisor.dev/gvisor v0.0.0-20260122175437-89a5d21be8f0/go.mod h1:QkHjoMIBaYtpVufgwv3keYAbln78mBoCuShZrPrer1Q=
h12.io/socks v1.0.3 h1:Ka3qaQewws4j4/eDQnOdpr4wXsC//dXtWvftlIcCQUo= h12.io/socks v1.0.3 h1:Ka3qaQewws4j4/eDQnOdpr4wXsC//dXtWvftlIcCQUo=
h12.io/socks v1.0.3/go.mod h1:AIhxy1jOId/XCz9BO+EIgNL2rQiPTBNnOfnVnQ+3Eck= h12.io/socks v1.0.3/go.mod h1:AIhxy1jOId/XCz9BO+EIgNL2rQiPTBNnOfnVnQ+3Eck=
layeh.com/gopher-luar v1.0.11 h1:8zJudpKI6HWkoh9eyyNFaTM79PY6CAPcIr6X/KTiliw=
layeh.com/gopher-luar v1.0.11/go.mod h1:TPnIVCZ2RJBndm7ohXyaqfhzjlZ+OA2SZR/YwL8tECk=
lukechampine.com/blake3 v1.4.1 h1:I3Smz7gso8w4/TunLKec6K2fn+kyKtDxr/xcQEN84Wg= lukechampine.com/blake3 v1.4.1 h1:I3Smz7gso8w4/TunLKec6K2fn+kyKtDxr/xcQEN84Wg=
lukechampine.com/blake3 v1.4.1/go.mod h1:QFosUxmjB8mnrWFSNwKmvxHpfY72bmD2tQ0kBMM3kwo= lukechampine.com/blake3 v1.4.1/go.mod h1:QFosUxmjB8mnrWFSNwKmvxHpfY72bmD2tQ0kBMM3kwo=
mvdan.cc/gofumpt v0.12.0 h1:1Lbudkz2kpM9Cjz2pL4M19u7q+GaEhCTNf7N9mfpcho= mvdan.cc/gofumpt v0.12.0 h1:1Lbudkz2kpM9Cjz2pL4M19u7q+GaEhCTNf7N9mfpcho=
+17
View File
@@ -17,6 +17,7 @@ import (
) )
type NameServerConfig struct { type NameServerConfig struct {
ID string `json:"id"`
Address *Address `json:"address"` Address *Address `json:"address"`
ClientIP *Address `json:"clientIp"` ClientIP *Address `json:"clientIp"`
Port uint16 `json:"port"` Port uint16 `json:"port"`
@@ -43,6 +44,7 @@ func (c *NameServerConfig) UnmarshalJSON(data []byte) error {
} }
var advanced struct { var advanced struct {
ID string `json:"id"`
Address *Address `json:"address"` Address *Address `json:"address"`
ClientIP *Address `json:"clientIp"` ClientIP *Address `json:"clientIp"`
Port uint16 `json:"port"` Port uint16 `json:"port"`
@@ -60,6 +62,7 @@ func (c *NameServerConfig) UnmarshalJSON(data []byte) error {
UnexpectedIPs StringList `json:"unexpectedIPs"` UnexpectedIPs StringList `json:"unexpectedIPs"`
} }
if err := json.Unmarshal(data, &advanced); err == nil { if err := json.Unmarshal(data, &advanced); err == nil {
c.ID = advanced.ID
c.Address = advanced.Address c.Address = advanced.Address
c.ClientIP = advanced.ClientIP c.ClientIP = advanced.ClientIP
c.Port = advanced.Port c.Port = advanced.Port
@@ -134,6 +137,7 @@ func (c *NameServerConfig) Build() (*dns.NameServer, error) {
} }
return &dns.NameServer{ return &dns.NameServer{
Id: c.ID,
Address: &net.Endpoint{ Address: &net.Endpoint{
Network: net.Network_UDP, Network: net.Network_UDP,
Address: c.Address.Build(), Address: c.Address.Build(),
@@ -159,6 +163,7 @@ func (c *NameServerConfig) Build() (*dns.NameServer, error) {
// DNSConfig is a JSON serializable object for dns.Config // DNSConfig is a JSON serializable object for dns.Config
type DNSConfig struct { type DNSConfig struct {
Servers []*NameServerConfig `json:"servers"` Servers []*NameServerConfig `json:"servers"`
Script string `json:"script"`
Hosts *HostsWrapper `json:"hosts"` Hosts *HostsWrapper `json:"hosts"`
ClientIP *Address `json:"clientIp"` ClientIP *Address `json:"clientIp"`
Tag string `json:"tag"` Tag string `json:"tag"`
@@ -267,6 +272,17 @@ func (m *HostsWrapper) Build() ([]*dns.Config_HostMapping, error) {
// Build implements Buildable // Build implements Buildable
func (c *DNSConfig) Build() (*dns.Config, error) { func (c *DNSConfig) Build() (*dns.Config, error) {
if c.Script != "" {
if c.DisableFallback || c.DisableFallbackIfMatch || c.EnableParallelQuery {
return nil, errors.New("DNS script cannot be used with disableFallback, disableFallbackIfMatch or enableParallelQuery")
}
for i, server := range c.Servers {
if len(server.Domains) > 0 || server.SkipFallback || server.FinalQuery {
return nil, errors.New("DNS script cannot be used with domains, skipFallback or finalQuery in server ", i)
}
}
}
config := &dns.Config{ config := &dns.Config{
Tag: c.Tag, Tag: c.Tag,
DisableCache: c.DisableCache, DisableCache: c.DisableCache,
@@ -276,6 +292,7 @@ func (c *DNSConfig) Build() (*dns.Config, error) {
DisableFallbackIfMatch: c.DisableFallbackIfMatch, DisableFallbackIfMatch: c.DisableFallbackIfMatch,
EnableParallelQuery: c.EnableParallelQuery, EnableParallelQuery: c.EnableParallelQuery,
QueryStrategy: resolveQueryStrategy(c.QueryStrategy), QueryStrategy: resolveQueryStrategy(c.QueryStrategy),
Script: c.Script,
} }
if c.ClientIP != nil { if c.ClientIP != nil {
+55
View File
@@ -2,6 +2,7 @@ package conf_test
import ( import (
"encoding/json" "encoding/json"
"strings"
"testing" "testing"
"github.com/google/go-cmp/cmp" "github.com/google/go-cmp/cmp"
@@ -122,3 +123,57 @@ func TestDNSConfigParsing(t *testing.T) {
} }
} }
} }
func TestDNSNameServerID(t *testing.T) {
var parsed DNSConfig
if err := json.Unmarshal([]byte(`{"servers":[{"id":"primary","address":"1.1.1.1"}]}`), &parsed); err != nil {
t.Fatal(err)
}
built, err := parsed.Build()
if err != nil {
t.Fatal(err)
}
if len(built.NameServer) != 1 || built.NameServer[0].Id != "primary" {
t.Fatalf("nameserver IDs = %v, want primary", built.NameServer)
}
}
func TestDNSScriptConfig(t *testing.T) {
for _, test := range []struct {
name string
input string
wantError string
}{
{name: "script only", input: `{"script":"script.lua"}`},
{name: "compatible fields", input: `{"script":"script.lua","servers":["1.1.1.1",{"id":"primary","address":"8.8.8.8","expectedIPs":["8.8.8.8"],"queryStrategy":"UseIPv4","disableCache":true}],"hosts":{"example.com":"127.0.0.1"},"disableCache":true}`},
{name: "disabled options", input: `{"script":"script.lua","disableFallback":false,"disableFallbackIfMatch":false,"enableParallelQuery":false,"servers":[{"address":"1.1.1.1","domains":[],"skipFallback":false,"finalQuery":false}]}`},
{name: "domains", input: `{"script":"script.lua","servers":[{"address":"1.1.1.1","domains":["domain:example.com"]}]}`, wantError: "domains"},
{name: "skip fallback", input: `{"script":"script.lua","servers":[{"address":"1.1.1.1","skipFallback":true}]}`, wantError: "skipFallback"},
{name: "final query", input: `{"script":"script.lua","servers":[{"address":"1.1.1.1","finalQuery":true}]}`, wantError: "finalQuery"},
{name: "later server", input: `{"script":"script.lua","servers":["1.1.1.1",{"address":"8.8.8.8","finalQuery":true}]}`, wantError: "server 1"},
{name: "disable fallback", input: `{"script":"script.lua","disableFallback":true}`, wantError: "disableFallback"},
{name: "disable fallback if match", input: `{"script":"script.lua","disableFallbackIfMatch":true}`, wantError: "disableFallbackIfMatch"},
{name: "parallel query", input: `{"script":"script.lua","enableParallelQuery":true}`, wantError: "enableParallelQuery"},
{name: "empty script", input: `{"script":"","disableFallback":true,"disableFallbackIfMatch":true,"enableParallelQuery":true,"servers":[{"address":"1.1.1.1","domains":["example.com"],"skipFallback":true,"finalQuery":true}]}`},
} {
t.Run(test.name, func(t *testing.T) {
var config DNSConfig
if err := json.Unmarshal([]byte(test.input), &config); err != nil {
t.Fatal(err)
}
built, err := config.Build()
if test.wantError != "" {
if built != nil || err == nil || !strings.Contains(err.Error(), "DNS script cannot be used with ") || !strings.Contains(err.Error(), test.wantError) {
t.Fatalf("Build() = %v, %v; want nil config and script conflict error mentioning %s", built, err, test.wantError)
}
return
}
if err != nil {
t.Fatal(err)
}
if built.Script != config.Script || len(built.NameServer) != len(config.Servers) {
t.Fatalf("Build() did not preserve script or servers: %v", built)
}
})
}
}
+190 -11
View File
@@ -1,7 +1,14 @@
package conf_test package conf_test
import ( import (
"crypto/ecdsa"
"crypto/ed25519"
"crypto/elliptic"
"crypto/rand"
"crypto/x509"
"encoding/base64"
"encoding/json" "encoding/json"
"encoding/pem"
"testing" "testing"
"github.com/xtls/xray-core/common/protocol" "github.com/xtls/xray-core/common/protocol"
@@ -9,8 +16,61 @@ import (
. "github.com/xtls/xray-core/infra/conf" . "github.com/xtls/xray-core/infra/conf"
masqueproxy "github.com/xtls/xray-core/proxy/masque" masqueproxy "github.com/xtls/xray-core/proxy/masque"
"github.com/xtls/xray-core/transport/internet/masque" "github.com/xtls/xray-core/transport/internet/masque"
"github.com/xtls/xray-core/transport/internet/splithttp"
) )
func defaultMasqueXmux() *splithttp.XmuxConfig {
return &splithttp.XmuxConfig{
MaxConcurrency: &splithttp.RangeConfig{},
MaxConnections: &splithttp.RangeConfig{From: 1, To: 1},
CMaxReuseTimes: &splithttp.RangeConfig{},
HMaxRequestTimes: &splithttp.RangeConfig{},
HMaxReusableSecs: &splithttp.RangeConfig{From: 1800, To: 3000},
}
}
func TestMasqueXmuxConfig(t *testing.T) {
creator := func() Buildable {
return new(MasqueConfig)
}
runMultiTestCase(t, []TestCase{
{
Input: `{"xmux": {"maxConnections": "2-4", "hMaxReusableSecs": 60, "hKeepAlivePeriod": -1}}`,
Parser: loadJSON(creator),
Output: &masque.Config{
Path: "/.well-known/masque/ip/*/*/",
Xmux: &splithttp.XmuxConfig{
MaxConcurrency: &splithttp.RangeConfig{},
MaxConnections: &splithttp.RangeConfig{From: 2, To: 4},
CMaxReuseTimes: &splithttp.RangeConfig{},
HMaxRequestTimes: &splithttp.RangeConfig{},
HMaxReusableSecs: &splithttp.RangeConfig{From: 60, To: 60},
HKeepAlivePeriod: -1,
},
},
},
{
Input: `{"xmux": {"maxConcurrency": 8}}`,
Parser: loadJSON(creator),
Output: &masque.Config{
Path: "/.well-known/masque/ip/*/*/",
Xmux: &splithttp.XmuxConfig{
MaxConcurrency: &splithttp.RangeConfig{From: 8, To: 8},
MaxConnections: &splithttp.RangeConfig{},
CMaxReuseTimes: &splithttp.RangeConfig{},
HMaxRequestTimes: &splithttp.RangeConfig{},
HMaxReusableSecs: &splithttp.RangeConfig{},
},
},
},
})
if _, err := loadJSON(creator)(`{"xmux": {"maxConnections": 2, "maxConcurrency": 8}}`); err == nil {
t.Error("expected an error for maxConnections with maxConcurrency")
}
}
func TestMasqueConfig(t *testing.T) { func TestMasqueConfig(t *testing.T) {
creator := func() Buildable { creator := func() Buildable {
return new(MasqueConfig) return new(MasqueConfig)
@@ -20,7 +80,7 @@ func TestMasqueConfig(t *testing.T) {
{ {
Input: `{}`, Input: `{}`,
Parser: loadJSON(creator), Parser: loadJSON(creator),
Output: &masque.Config{Path: "/.well-known/masque/ip/*/*/"}, Output: &masque.Config{Path: "/.well-known/masque/ip/*/*/", Xmux: defaultMasqueXmux()},
}, },
{ {
Input: `{ Input: `{
@@ -33,12 +93,13 @@ func TestMasqueConfig(t *testing.T) {
Host: "example.com:8443", Host: "example.com:8443",
Path: "/.well-known/masque/ip/*/*/", Path: "/.well-known/masque/ip/*/*/",
Headers: map[string]string{"Authorization": "Basic dTpw"}, Headers: map[string]string{"Authorization": "Basic dTpw"},
Xmux: defaultMasqueXmux(),
}, },
}, },
{ {
Input: `{"path": "/masque/ip{?target,ipproto}"}`, Input: `{"path": "/masque/ip{?target,ipproto}"}`,
Parser: loadJSON(creator), Parser: loadJSON(creator),
Output: &masque.Config{Path: "/masque/ip?target=*&ipproto=*"}, Output: &masque.Config{Path: "/masque/ip?target=*&ipproto=*", Xmux: defaultMasqueXmux()},
}, },
{ {
Input: `{"user": "u", "pass": "p:q", "headers": {"X-Token": "a"}}`, Input: `{"user": "u", "pass": "p:q", "headers": {"X-Token": "a"}}`,
@@ -46,6 +107,7 @@ func TestMasqueConfig(t *testing.T) {
Output: &masque.Config{ Output: &masque.Config{
Path: "/.well-known/masque/ip/*/*/", Path: "/.well-known/masque/ip/*/*/",
Headers: map[string]string{"Authorization": "Basic dTpwOnE=", "X-Token": "a"}, Headers: map[string]string{"Authorization": "Basic dTpwOnE=", "X-Token": "a"},
Xmux: defaultMasqueXmux(),
}, },
}, },
}) })
@@ -67,6 +129,124 @@ func TestMasqueConfig(t *testing.T) {
} }
} }
func TestMasqueWarpConfig(t *testing.T) {
creator := func() Buildable {
return new(MasqueConfig)
}
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
t.Fatal(err)
}
pkcs8, err := x509.MarshalPKCS8PrivateKey(key)
if err != nil {
t.Fatal(err)
}
sec1, err := x509.MarshalECPrivateKey(key)
if err != nil {
t.Fatal(err)
}
quote := func(s string) string {
b, _ := json.Marshal(s)
return string(b)
}
server, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
t.Fatal(err)
}
publicKey, err := x509.MarshalPKIXPublicKey(&server.PublicKey)
if err != nil {
t.Fatal(err)
}
publicPEM := string(pem.EncodeToMemory(&pem.Block{Type: "PUBLIC KEY", Bytes: publicKey}))
warpInput := func(key string, extra string) string {
return `{` + extra + `"warp": {"privateKey": ` + quote(key) + `, "publicKey": ` + quote(publicPEM) + `, "address": ["172.16.0.2", "2606:4700:110:8a36::2/128"]}}`
}
address := []string{"172.16.0.2/32", "2606:4700:110:8a36::2/128"}
for _, input := range []string{
string(pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: pkcs8})),
string(pem.EncodeToMemory(&pem.Block{Type: "EC PRIVATE KEY", Bytes: sec1})),
base64.StdEncoding.EncodeToString(pkcs8),
base64.StdEncoding.EncodeToString(sec1),
} {
runMultiTestCase(t, []TestCase{
{
Input: warpInput(input, ""),
Parser: loadJSON(creator),
Output: &masque.Config{
Host: "cloudflareaccess.com",
Path: "/",
Warp: &masque.Warp{PrivateKey: pkcs8, PublicKey: publicKey, Address: address},
Xmux: defaultMasqueXmux(),
},
},
})
}
runMultiTestCase(t, []TestCase{
{
Input: warpInput(base64.StdEncoding.EncodeToString(sec1), `"host": "example.com", "path": "/warp", `),
Parser: loadJSON(creator),
Output: &masque.Config{
Host: "example.com",
Path: "/warp",
Warp: &masque.Warp{PrivateKey: pkcs8, PublicKey: publicKey, Address: address},
Xmux: defaultMasqueXmux(),
},
},
})
p384, err := ecdsa.GenerateKey(elliptic.P384(), rand.Reader)
if err != nil {
t.Fatal(err)
}
p384DER, err := x509.MarshalPKCS8PrivateKey(p384)
if err != nil {
t.Fatal(err)
}
ed, err := x509.MarshalPKCS8PrivateKey(ed25519.NewKeyFromSeed(make([]byte, ed25519.SeedSize)))
if err != nil {
t.Fatal(err)
}
withAddress := func(address string) string {
return `{"warp": {"privateKey": ` + quote(base64.StdEncoding.EncodeToString(pkcs8)) + `, "publicKey": ` + quote(publicPEM) + `, "address": ` + address + `}}`
}
withPublicKey := func(key string) string {
return `{"warp": {"privateKey": ` + quote(base64.StdEncoding.EncodeToString(pkcs8)) + `, "publicKey": ` + quote(key) + `, "address": ["172.16.0.2"]}}`
}
runMultiTestCase(t, []TestCase{
{
Input: withPublicKey(base64.StdEncoding.EncodeToString(publicKey)),
Parser: loadJSON(creator),
Output: &masque.Config{
Host: "cloudflareaccess.com",
Path: "/",
Warp: &masque.Warp{PrivateKey: pkcs8, PublicKey: publicKey, Address: []string{"172.16.0.2/32"}},
Xmux: defaultMasqueXmux(),
},
},
})
for _, input := range []string{
`{"warp": {}}`,
withAddress(`[]`),
withPublicKey(""),
withPublicKey("not a key"),
withPublicKey(base64.StdEncoding.EncodeToString([]byte("not a key"))),
withPublicKey(base64.StdEncoding.EncodeToString(pkcs8)),
withAddress(`["172.16.0"]`),
withAddress(`["172.16.0.2", "172.16.0.3"]`),
withAddress(`["2606:4700::1", "2606:4700::2/128"]`),
warpInput("not a key", ""),
warpInput(base64.StdEncoding.EncodeToString([]byte("not a key")), ""),
warpInput(base64.StdEncoding.EncodeToString(p384DER), ""),
warpInput(base64.StdEncoding.EncodeToString(ed), ""),
warpInput(base64.StdEncoding.EncodeToString(pkcs8), `"user": "u", "pass": "p", `),
} {
if _, err := loadJSON(creator)(input); err == nil {
t.Errorf("expected an error for %s", input)
}
}
}
func TestMasqueOutboundConfig(t *testing.T) { func TestMasqueOutboundConfig(t *testing.T) {
build := func(s string) error { build := func(s string) error {
c := new(OutboundDetourConfig) c := new(OutboundDetourConfig)
@@ -77,18 +257,17 @@ func TestMasqueOutboundConfig(t *testing.T) {
return err return err
} }
if err := build(`{ for _, input := range []string{
"protocol": "masque", `{"protocol": "masque", "settings": {"address": "example.com", "port": 443}, "streamSettings": {"network": "masque", "security": "tls"}, "mux": {"enabled": false, "concurrency": -1}}`,
"settings": {"address": "example.com", "port": 443}, `{"protocol": "masque", "settings": {"address": "example.com", "port": 443}, "streamSettings": {"network": "masque", "security": "tls"}, "mux": {"enabled": true}}`,
"streamSettings": {"network": "masque", "security": "tls"}, `{"protocol": "masque", "settings": {"address": "example.com", "port": 443}, "streamSettings": {"network": "masque", "security": "tls"}, "mux": {"enabled": true, "concurrency": -1, "xudpConcurrency": 8}}`,
"mux": {"enabled": false, "concurrency": -1} } {
}`); err != nil { if err := build(input); err != nil {
t.Error(err) t.Errorf("%s: %v", input, err)
}
} }
for _, input := range []string{ for _, input := range []string{
`{"protocol": "masque", "settings": {"address": "example.com"}, "streamSettings": {"network": "masque", "security": "tls"}}`, `{"protocol": "masque", "settings": {"address": "example.com"}, "streamSettings": {"network": "masque", "security": "tls"}}`,
`{"protocol": "masque", "settings": {"address": "example.com", "port": 443}, "streamSettings": {"network": "masque", "security": "tls"}, "mux": {"enabled": true}}`,
`{"protocol": "masque", "settings": {"address": "example.com", "port": 443}, "streamSettings": {"network": "masque", "security": "tls"}, "mux": {"enabled": true, "concurrency": -1}}`,
`{"protocol": "freedom", "streamSettings": {"network": "masque", "security": "tls"}}`, `{"protocol": "freedom", "streamSettings": {"network": "masque", "security": "tls"}}`,
} { } {
if err := build(input); err == nil { if err := build(input); err == nil {
+7
View File
@@ -72,6 +72,7 @@ type RouterConfig struct {
RuleList []json.RawMessage `json:"rules"` RuleList []json.RawMessage `json:"rules"`
DomainStrategy *string `json:"domainStrategy"` DomainStrategy *string `json:"domainStrategy"`
Balancers []*BalancingRule `json:"balancers"` Balancers []*BalancingRule `json:"balancers"`
Script string `json:"script"`
} }
func (c *RouterConfig) getDomainStrategy() router.Config_DomainStrategy { func (c *RouterConfig) getDomainStrategy() router.Config_DomainStrategy {
@@ -92,6 +93,12 @@ func (c *RouterConfig) getDomainStrategy() router.Config_DomainStrategy {
func (c *RouterConfig) Build() (*router.Config, error) { func (c *RouterConfig) Build() (*router.Config, error) {
config := new(router.Config) config := new(router.Config)
if c.Script != "" && (c.DomainStrategy != nil || len(c.RuleList) > 0) {
return nil, errors.New("routing script cannot be used with domainStrategy or rules")
}
config.Script = c.Script
config.DomainStrategy = c.getDomainStrategy() config.DomainStrategy = c.getDomainStrategy()
var rawRuleList []json.RawMessage var rawRuleList []json.RawMessage
+38
View File
@@ -2,6 +2,7 @@ package conf_test
import ( import (
"encoding/json" "encoding/json"
"strings"
"testing" "testing"
"time" "time"
_ "unsafe" _ "unsafe"
@@ -236,3 +237,40 @@ func TestRouterConfig(t *testing.T) {
}, },
}) })
} }
func TestRouterScriptConfig(t *testing.T) {
for _, test := range []struct {
name string
input string
wantError bool
}{
{name: "script only", input: `{"script":"missing.lua"}`},
{name: "empty rules", input: `{"script":"missing.lua","rules":[]}`},
{name: "balancers", input: `{"script":"missing.lua","balancers":[{"tag":"b1","selector":["direct"]}]}`},
{name: "default domain strategy", input: `{"script":"missing.lua","domainStrategy":"AsIs"}`, wantError: true},
{name: "IP domain strategy", input: `{"script":"missing.lua","domainStrategy":"IPIfNonMatch"}`, wantError: true},
{name: "empty domain strategy", input: `{"script":"missing.lua","domainStrategy":""}`, wantError: true},
{name: "rules", input: `{"script":"missing.lua","rules":[{"outboundTag":"direct","domain":["example.com"]}]}`, wantError: true},
{name: "empty script", input: `{"script":"","domainStrategy":"IPOnDemand","rules":[{"outboundTag":"direct","domain":["example.com"]}]}`},
} {
t.Run(test.name, func(t *testing.T) {
var config RouterConfig
if err := json.Unmarshal([]byte(test.input), &config); err != nil {
t.Fatal(err)
}
built, err := config.Build()
if test.wantError {
if built != nil || err == nil || !strings.Contains(err.Error(), "routing script cannot be used with domainStrategy or rules") {
t.Fatalf("Build() = %v, %v; want nil config and script conflict error", built, err)
}
return
}
if err != nil {
t.Fatal(err)
}
if built.Script != config.Script || len(built.Rule) != len(config.RuleList) || len(built.BalancingRule) != len(config.Balancers) {
t.Fatalf("Build() did not preserve script, rules or balancers: %v", built)
}
})
}
}
+194
View File
@@ -0,0 +1,194 @@
package serial_test
import (
"context"
"io"
"os"
"path/filepath"
"strings"
"testing"
"time"
xlua "github.com/xtls/xray-core/common/lua"
"github.com/xtls/xray-core/core"
"github.com/xtls/xray-core/infra/conf"
"github.com/xtls/xray-core/infra/conf/serial"
_ "github.com/xtls/xray-core/main/confloader/external"
glua "github.com/yuin/gopher-lua"
)
func TestConfigSourceSecurity(t *testing.T) {
// Keep registered files alive until every file identity check finishes.
dir := t.TempDir()
t.Run("polyglots", testConfigLuaPolyglots)
t.Run("merged", func(t *testing.T) { testMergedConfigCannotRunAsLua(t, dir) })
t.Run("hardlink", func(t *testing.T) { testConfigHardLinkCannotRunAsLua(t, dir) })
t.Run("stdin", func(t *testing.T) { testStdinConfigCannotRunAsLua(t, dir) })
}
func testConfigLuaPolyglots(t *testing.T) {
const luaBody = "polyglotExecuted = true\nfunction HandleRoute() return 'direct' end\n"
for _, tc := range []struct {
name string
decode func(io.Reader) (*conf.Config, error)
source string
script string
}{
{"JSON", serial.DecodeJSONConfig, "null = 0\n" + luaBody, ""},
{"YAML", serial.DecodeYAMLConfig, `--[[: ignored
routing:
script: payload.lua
--]] --: ignored
_G: assert((function() polyglotExecuted = true; function HandleRoute() return 'direct' end end)())
`, "payload.lua"},
{"TOML", serial.DecodeTOMLConfig, `routing = { script = "payload.lua" }
-- = """
polyglotExecuted = true
function HandleRoute() return 'direct' end
-- """
`, "payload.lua"},
} {
t.Run(tc.name, func(t *testing.T) {
config, err := tc.decode(strings.NewReader(tc.source))
if err != nil {
t.Fatal(err)
}
script := ""
if config.RouterConfig != nil {
script = config.RouterConfig.Script
}
if script != tc.script {
t.Fatalf("routing.script = %q, want %q", script, tc.script)
}
if built, err := config.Build(); err != nil || built == nil {
t.Fatalf("config Build failed: %v", err)
}
if _, err := serial.DecodeJSONConfigStrict(strings.NewReader(tc.source)); err == nil {
t.Fatal("strict JSON accepted the polyglot")
}
dir := t.TempDir()
t.Setenv("xray.location.confdir", dir)
if err := os.WriteFile(filepath.Join(dir, "payload.lua"), []byte(tc.source), 0o600); err != nil {
t.Fatal(err)
}
configPath := filepath.Join(dir, "payload."+strings.ToLower(tc.name))
if err := os.WriteFile(configPath, []byte(tc.source), 0o600); err != nil {
t.Fatal(err)
}
if program, err := xlua.Compile(configPath); program != nil || err == nil || !strings.Contains(err.Error(), "Lua file extension must be .lua") {
t.Fatalf("Compile(%q) = %v, %v; want .lua extension rejection", configPath, program, err)
}
program, err := xlua.Compile("payload.lua")
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
state, err := program.NewState(ctx, nil, nil)
if err != nil {
t.Fatal(err)
}
defer state.Close()
if state.GetGlobal("polyglotExecuted") != glua.LTrue {
t.Fatal("Lua top-level code did not execute")
}
if err := state.CallByParam(glua.P{Fn: state.GetGlobal("HandleRoute"), NRet: 1, Protect: true}); err != nil {
t.Fatal(err)
}
if got := state.Get(-1); got != glua.LString("direct") {
t.Fatalf("HandleRoute() = %v, want direct", got)
}
t.Logf("config built with routing.script=%q; the same bytes executed as Lua", script)
})
}
}
func testMergedConfigCannotRunAsLua(t *testing.T, dir string) {
t.Setenv("xray.location.confdir", dir)
base := filepath.Join(dir, "base.lua")
for format, body := range map[string]string{
"json": "null = 0\nfunction HandleRoute() return 'direct' end\n",
"yaml": "_G: assert((function() function HandleRoute() return 'direct' end end)())\n",
"toml": "-- = \"\"\"\nfunction HandleRoute() return 'direct' end\n-- \"\"\"\n",
} {
t.Run(format, func(t *testing.T) {
path := filepath.Join(dir, format+".lua")
if err := os.WriteFile(base, []byte(`{"routing":{"script":"`+format+`.lua"}}`), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte(body), 0o600); err != nil {
t.Fatal(err)
}
if program, err := xlua.Compile(path); err != nil || program == nil {
t.Fatalf("fixture must compile before config loading: %v", err)
}
if _, err := serial.BuildConfig([]*core.ConfigSource{
{Name: base, Format: "json"},
{Name: path, Format: format},
}); err != nil {
t.Fatal(err)
}
for _, source := range []string{base, path} {
if program, err := xlua.Compile(source); program != nil || err == nil || !strings.Contains(err.Error(), "loaded as a configuration file") {
t.Fatalf("Compile(%q) = %v, %v; want config source rejection", source, program, err)
}
}
if err := os.Rename(path, path+".old"); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte("function HandleRoute() end"), 0o600); err != nil {
t.Fatal(err)
}
if program, err := xlua.Compile(path); program != nil || err == nil || !strings.Contains(err.Error(), "loaded as a configuration file") {
t.Fatalf("Compile = %v, %v; want replaced config path rejection", program, err)
}
})
}
path := filepath.Join(dir, "ordinary.lua")
if err := os.WriteFile(path, []byte("function HandleRoute() end"), 0o600); err != nil {
t.Fatal(err)
}
if program, err := xlua.Compile(path); err != nil || program == nil {
t.Fatalf("ordinary script in the config directory must remain usable: %v", err)
}
}
func testConfigHardLinkCannotRunAsLua(t *testing.T, dir string) {
path := filepath.Join(dir, "config.json")
if err := os.WriteFile(path, []byte("null = 0\nfunction HandleRoute() end"), 0o600); err != nil {
t.Fatal(err)
}
alias := filepath.Join(dir, "alias.lua")
if err := os.Link(path, alias); err != nil {
t.Fatal(err)
}
if _, err := serial.BuildConfig([]*core.ConfigSource{{Name: path, Format: "json"}}); err != nil {
t.Fatal(err)
}
if program, err := xlua.Compile(alias); program != nil || err == nil || !strings.Contains(err.Error(), "loaded as a configuration file") {
t.Fatalf("Compile = %v, %v; want config hard link rejection", program, err)
}
}
func testStdinConfigCannotRunAsLua(t *testing.T, dir string) {
path := filepath.Join(dir, "config.lua")
if err := os.WriteFile(path, []byte("null = 0\nfunction HandleRoute() end"), 0o600); err != nil {
t.Fatal(err)
}
file, err := os.Open(path)
if err != nil {
t.Fatal(err)
}
originalStdin := os.Stdin
os.Stdin = file
t.Cleanup(func() {
os.Stdin = originalStdin
file.Close()
})
if _, err := serial.BuildConfig([]*core.ConfigSource{{Name: "stdin:", Format: "json"}}); err != nil {
t.Fatal(err)
}
if program, err := xlua.Compile(path); program != nil || err == nil || !strings.Contains(err.Error(), "loaded as a configuration file") {
t.Fatalf("Compile = %v, %v; want redirected config rejection", program, err)
}
}
+168 -25
View File
@@ -1,6 +1,7 @@
package conf package conf
import ( import (
"context"
"crypto/x509" "crypto/x509"
"encoding/base64" "encoding/base64"
"encoding/hex" "encoding/hex"
@@ -81,7 +82,7 @@ var (
"noise": func() interface{} { return new(NoiseMask) }, "noise": func() interface{} { return new(NoiseMask) },
"salamander": func() interface{} { return new(Salamander) }, "salamander": func() interface{} { return new(Salamander) },
"sudoku": func() interface{} { return new(Sudoku) }, "sudoku": func() interface{} { return new(Sudoku) },
"xdns": func() interface{} { return new(Xdns) }, "xdns": func() interface{} { return new(XDNS) },
"xicmp": func() interface{} { return new(Xicmp) }, "xicmp": func() interface{} { return new(Xicmp) },
"realm": func() interface{} { return new(Realm) }, "realm": func() interface{} { return new(Realm) },
"udphop": func() interface{} { return new(UDPHop) }, "udphop": func() interface{} { return new(UDPHop) },
@@ -308,14 +309,27 @@ type NoiseMask struct {
} }
func (c *NoiseMask) Build() (proto.Message, error) { func (c *NoiseMask) Build() (proto.Message, error) {
noiseSlice := make([]*noise.Item, 0, len(c.Noise))
for _, item := range c.Noise { for _, item := range c.Noise {
if len(item.Packet) > 0 && item.Rand.To > 0 { if len(item.Packet) > 0 && item.Rand.To > 0 {
return nil, errors.New("len(item.Packet) > 0 && item.Rand.To > 0") return nil, errors.New("len(item.Packet) > 0 && item.Rand.To > 0")
} }
} if strings.ToLower(item.Type) == "exp" {
var exp string
noiseSlice := make([]*noise.Item, 0, len(c.Noise)) if err := json.Unmarshal(item.Packet, &exp); err != nil {
for _, item := range c.Noise { return nil, errors.New(`"packet" of noise "type": "exp" must be a string`).Base(err)
}
segments, err := parseNoiseExp(exp)
if err != nil {
return nil, err
}
noiseSlice = append(noiseSlice, &noise.Item{
Segments: segments,
DelayMin: int64(item.Delay.From),
DelayMax: int64(item.Delay.To),
})
continue
}
if item.RandRange == nil { if item.RandRange == nil {
item.RandRange = &Int32Range{From: 0, To: 255} item.RandRange = &Int32Range{From: 0, To: 255}
} }
@@ -344,6 +358,88 @@ func (c *NoiseMask) Build() (proto.Message, error) {
}, nil }, nil
} }
var noiseExpPattern = regexp.MustCompile(`<\s*([a-z]+)(?:\s+([^>]*?))?\s*>`)
func parseNoiseExp(exp string) ([]*noise.Segment, error) {
var segments []*noise.Segment
matches := noiseExpPattern.FindAllStringSubmatchIndex(exp, -1)
last := 0
for _, m := range matches {
if strings.TrimSpace(exp[last:m[0]]) != "" {
return nil, errors.New("invalid noise exp near ", exp[last:m[0]])
}
last = m[1]
key := exp[m[2]:m[3]]
arg := ""
if m[4] >= 0 {
arg = exp[m[4]:m[5]]
}
segment, err := buildNoiseSegment(key, arg)
if err != nil {
return nil, err
}
segments = append(segments, segment)
}
if strings.TrimSpace(exp[last:]) != "" {
return nil, errors.New("invalid noise exp near ", exp[last:])
}
if len(segments) == 0 {
return nil, errors.New("empty noise exp: ", exp)
}
return segments, nil
}
func buildNoiseSegment(key, arg string) (*noise.Segment, error) {
sizeSegment := func(kind noise.Segment_Kind) (*noise.Segment, error) {
if arg == "" {
return nil, errors.New("<", key, "> in noise exp needs a size")
}
lo, hi, err := ParseRangeString(arg)
if err != nil {
return nil, err
}
if lo < 0 || hi < lo || hi > 65535 {
return nil, errors.New("invalid size in noise exp: ", arg)
}
return &noise.Segment{Kind: kind, MinSize: int64(lo), MaxSize: int64(hi)}, nil
}
switch key {
case "b":
hexStr := strings.TrimPrefix(strings.TrimPrefix(strings.Join(strings.Fields(arg), ""), "0x"), "0X")
if len(hexStr) == 0 {
return nil, errors.New("empty bytes in noise exp")
}
raw, err := hex.DecodeString(hexStr)
if err != nil {
return nil, errors.New("invalid hex in noise exp: ", arg).Base(err)
}
return &noise.Segment{Kind: noise.Segment_BYTES, Bytes: raw}, nil
case "r":
return sizeSegment(noise.Segment_RANDOM)
case "rc":
return sizeSegment(noise.Segment_RANDOM_ASCII)
case "rd":
return sizeSegment(noise.Segment_RANDOM_DIGIT)
case "t":
if arg != "" {
return nil, errors.New("<t> in noise exp takes no argument")
}
return &noise.Segment{Kind: noise.Segment_TIMESTAMP}, nil
case "c":
if arg != "" {
return nil, errors.New("<c> in noise exp takes no argument")
}
return &noise.Segment{Kind: noise.Segment_COUNTER}, nil
case "n":
if arg != "" {
return nil, errors.New("<n> in noise exp takes no argument")
}
return &noise.Segment{Kind: noise.Segment_NONCE}, nil
default:
return nil, errors.New("unknown <", key, "> in noise exp")
}
}
type UDPItem struct { type UDPItem struct {
Rand int32 `json:"rand"` Rand int32 `json:"rand"`
RandRange *Int32Range `json:"randRange"` RandRange *Int32Range `json:"randRange"`
@@ -694,32 +790,79 @@ func (c *Sudoku) Build() (proto.Message, error) {
}, nil }, nil
} }
type Xdns struct { type XDNSDomain struct {
Domain json.RawMessage `json:"domain"` Names []string `json:"names"`
LenLimit int32 `json:"lenLimit"`
Domains []string `json:"domains"` LabelLimit int32 `json:"labelLimit"`
Resolvers []string `json:"resolvers"` Types []int32 `json:"types"`
Edns0 int32 `json:"edns0"`
} }
func (c *Xdns) Build() (proto.Message, error) { type XDNSResolver struct {
if c.Domain != nil { Addrs []string `json:"addrs"`
return nil, errors.PrintRemovedFeatureError("domain", "domains(server) & resolvers(client)") }
}
if len(c.Domains) == 0 && len(c.Resolvers) == 0 { type XDNS struct {
return nil, errors.New("empty domains & empty resolvers") Domains []XDNSDomain `json:"domains"`
} Resolvers []XDNSResolver `json:"resolvers"`
ExtraPoll int32 `json:"extraPoll"`
}
for _, r := range c.Resolvers { func (c *XDNS) Build() (proto.Message, error) {
if !strings.Contains(r, "+udp://") { var domains []*xdns.DomainProto
return nil, errors.New("invalid resolver ", r) var resolvers []*xdns.ResolverProto
for i := range c.Domains {
if c.Domains[i].LenLimit == 0 {
c.Domains[i].LenLimit = 255
}
if c.Domains[i].LabelLimit == 0 {
c.Domains[i].LabelLimit = 63
}
for j := range c.Domains[i].Names {
domain, err := xdns.NewDomain(c.Domains[i].Names[j], int(c.Domains[i].LenLimit), int(c.Domains[i].LabelLimit), []uint16{1, 5, 16, 28}, uint16(c.Domains[i].Edns0))
if err != nil {
return nil, err
}
errors.LogInfo(context.Background(), domain.Show())
domains = append(domains, &xdns.DomainProto{
Name: c.Domains[i].Names[j],
LenLimit: c.Domains[i].LenLimit,
LabelLimit: c.Domains[i].LabelLimit,
Types: c.Domains[i].Types,
Edns0: c.Domains[i].Edns0,
})
} }
} }
for i := range c.Resolvers {
return &xdns.Config{ for j := range c.Resolvers[i].Addrs {
Domains: c.Domains, var u *url.URL
Resolvers: c.Resolvers, var e error
}, nil if !strings.Contains(c.Resolvers[i].Addrs[j], "://") {
u, e = url.Parse("udp://" + c.Resolvers[i].Addrs[j])
} else {
u, e = url.Parse(c.Resolvers[i].Addrs[j])
}
if e != nil {
return nil, e
}
switch u.Scheme {
case "tcp", "udp":
default:
return nil, errors.New("invalid protocol")
}
var host, port string
host = u.Hostname()
port = u.Port()
if port == "" {
port = "53"
}
resolvers = append(resolvers, &xdns.ResolverProto{Type: u.Scheme, Addr: net.JoinHostPort(host, port)})
}
}
if c.ExtraPoll < 0 || c.ExtraPoll > 3 {
return nil, errors.New("c.ExtraPoll < 0 || c.ExtraPoll > 3")
}
return &xdns.Config{Domains: domains, Resolvers: resolvers, ExtraPoll: c.ExtraPoll}, nil
} }
type XMC struct { type XMC struct {
@@ -0,0 +1,136 @@
package conf
import (
"encoding/json"
"testing"
"github.com/xtls/xray-core/transport/internet/finalmask/noise"
)
func expPacket(exp string) json.RawMessage {
b, _ := json.Marshal(exp)
return b
}
func buildNoiseExp(exp string) (*noise.Config, error) {
msg, err := (&NoiseMask{Noise: []NoiseItem{{Type: "exp", Packet: expPacket(exp)}}}).Build()
if err != nil {
return nil, err
}
return msg.(*noise.Config), nil
}
func TestNoiseExp(t *testing.T) {
cfg, err := buildNoiseExp("<b 0d0a0d0a><t><r 24><rc 20-40><rd 8><c><n>")
if err != nil {
t.Fatal(err)
}
segments := cfg.Items[0].Segments
if len(segments) != 7 {
t.Fatalf("got %d segments, want 7", len(segments))
}
want := []struct {
kind noise.Segment_Kind
bytes []byte
min, max int64
}{
{noise.Segment_BYTES, []byte{0x0d, 0x0a, 0x0d, 0x0a}, 0, 0},
{noise.Segment_TIMESTAMP, nil, 0, 0},
{noise.Segment_RANDOM, nil, 24, 24},
{noise.Segment_RANDOM_ASCII, nil, 20, 40},
{noise.Segment_RANDOM_DIGIT, nil, 8, 8},
{noise.Segment_COUNTER, nil, 0, 0},
{noise.Segment_NONCE, nil, 0, 0},
}
for i, w := range want {
s := segments[i]
if s.Kind != w.kind || s.MinSize != w.min || s.MaxSize != w.max || string(s.Bytes) != string(w.bytes) {
t.Errorf("segment %d = %+v, want %+v", i, s, w)
}
}
}
func TestNoiseExpStripsHexPrefix(t *testing.T) {
cfg, err := buildNoiseExp("<b 0x16030100>")
if err != nil {
t.Fatal(err)
}
if got := cfg.Items[0].Segments[0].Bytes; string(got) != string([]byte{0x16, 0x03, 0x01, 0x00}) {
t.Errorf("got %x", got)
}
}
func TestNoiseExpWhitespace(t *testing.T) {
if _, err := buildNoiseExp(" <b 00> <t> "); err != nil {
t.Errorf("surrounding whitespace should be allowed: %v", err)
}
cfg, err := buildNoiseExp("<b 0d 0a 0d 0a>")
if err != nil {
t.Fatal(err)
}
if got := cfg.Items[0].Segments[0].Bytes; string(got) != "\r\n\r\n" {
t.Errorf("got %x", got)
}
}
func TestNoiseExpRejects(t *testing.T) {
for _, exp := range []string{
"<x 1>",
"<b>",
"<b zz>",
"<b 0d0>",
"<r>",
"<r -1>",
"<r 40-20>",
"<r 70000>",
"<t 5>",
"<n 5>",
"garbage<t>",
"<t> tail",
"<t><b>",
} {
if _, err := buildNoiseExp(exp); err == nil {
t.Errorf("expected an error for %q", exp)
}
}
}
func TestNoiseExpConflicts(t *testing.T) {
if _, err := (&NoiseMask{Noise: []NoiseItem{{Type: "exp", Packet: expPacket("<t>"), Rand: Int32Range{From: 10, To: 20}}}}).Build(); err == nil {
t.Error("exp with rand should be rejected")
}
for _, packet := range []string{``, `[1, 2]`, `5`} {
if _, err := (&NoiseMask{Noise: []NoiseItem{{Type: "exp", Packet: json.RawMessage(packet)}}}).Build(); err == nil {
t.Errorf("expected an error for packet %q", packet)
}
}
}
func TestNoiseExpFromJSON(t *testing.T) {
var mask NoiseMask
if err := json.Unmarshal([]byte(`{"noise": [
{"type": "exp", "packet": "<b 504f5354><rd 10-20>", "delay": "1-3"},
{"type": "EXP", "packet": "<t>"},
{"type": "str", "packet": "<t>"},
{"rand": "10-20"}
]}`), &mask); err != nil {
t.Fatal(err)
}
msg, err := mask.Build()
if err != nil {
t.Fatal(err)
}
items := msg.(*noise.Config).Items
if len(items[0].Segments) != 2 || items[0].DelayMin != 1 || items[0].DelayMax != 3 {
t.Errorf("item 0 = %+v", items[0])
}
if len(items[1].Segments) != 1 || items[1].Segments[0].Kind != noise.Segment_TIMESTAMP {
t.Errorf("item 1 = %+v", items[1])
}
if len(items[2].Segments) != 0 || string(items[2].Packet) != "<t>" {
t.Errorf("item 2 = %+v", items[2])
}
if len(items[3].Segments) != 0 || items[3].RandMin != 10 || items[3].RandMax != 20 {
t.Errorf("item 3 = %+v", items[3])
}
}
+131 -4
View File
@@ -1,10 +1,15 @@
package conf package conf
import ( import (
"crypto/ecdsa"
"crypto/elliptic"
"crypto/x509"
"encoding/base64" "encoding/base64"
"encoding/json" "encoding/json"
"encoding/pem"
"maps" "maps"
"math/big" "math/big"
"net/netip"
"net/url" "net/url"
"sort" "sort"
"strconv" "strconv"
@@ -790,16 +795,50 @@ func (c *HysteriaConfig) Build() (proto.Message, error) {
return config, nil return config, nil
} }
type MasqueWarpConfig struct {
PrivateKey string `json:"privateKey"`
PublicKey string `json:"publicKey"`
Address []string `json:"address"`
}
type MasqueConfig struct { type MasqueConfig struct {
Host string `json:"host"` Host string `json:"host"`
Path string `json:"path"` Path string `json:"path"`
User string `json:"user"` User string `json:"user"`
Pass string `json:"pass"` Pass string `json:"pass"`
Headers map[string]string `json:"headers"` Headers map[string]string `json:"headers"`
Warp *MasqueWarpConfig `json:"warp"`
Xmux XmuxConfig `json:"xmux"`
} }
func (c *MasqueConfig) Build() (proto.Message, error) { func (c *MasqueConfig) Build() (proto.Message, error) {
var warp *masque.Warp
host := c.Host
path := c.Path path := c.Path
if c.Warp != nil {
if c.User != "" || c.Pass != "" {
return nil, errors.New(`"user" and "pass" can't be used with "warp"`)
}
key, err := parseWarpPrivateKey(c.Warp.PrivateKey)
if err != nil {
return nil, errors.New(`invalid "privateKey" in "warp"`).Base(err)
}
publicKey, err := parseWarpPublicKey(c.Warp.PublicKey)
if err != nil {
return nil, errors.New(`invalid "publicKey" in "warp"`).Base(err)
}
address, err := parseWarpAddress(c.Warp.Address)
if err != nil {
return nil, err
}
warp = &masque.Warp{PrivateKey: key, PublicKey: publicKey, Address: address}
if host == "" {
host = masque.WarpHost
}
if path == "" {
path = masque.WarpPath
}
}
if path == "" { if path == "" {
path = masque.DefaultPath path = masque.DefaultPath
} }
@@ -811,9 +850,9 @@ func (c *MasqueConfig) Build() (proto.Message, error) {
if !strings.HasPrefix(path, "/") || strings.ContainsAny(path, "{}") { if !strings.HasPrefix(path, "/") || strings.ContainsAny(path, "{}") {
return nil, errors.New(`invalid "path": `, path, `, only the variables {target} and {ipproto} are supported`) return nil, errors.New(`invalid "path": `, path, `, only the variables {target} and {ipproto} are supported`)
} }
if c.Host != "" { if host != "" {
if u, err := url.Parse("https://" + c.Host); err != nil || u.Host != c.Host { if u, err := url.Parse("https://" + host); err != nil || u.Host != host {
return nil, errors.New(`invalid "host": `, c.Host) return nil, errors.New(`invalid "host": `, host)
} }
} }
for k, v := range c.Headers { for k, v := range c.Headers {
@@ -840,13 +879,101 @@ func (c *MasqueConfig) Build() (proto.Message, error) {
} }
headers["Authorization"] = "Basic " + base64.StdEncoding.EncodeToString([]byte(c.User+":"+c.Pass)) headers["Authorization"] = "Basic " + base64.StdEncoding.EncodeToString([]byte(c.User+":"+c.Pass))
} }
if c.Xmux.MaxConnections.To > 0 && c.Xmux.MaxConcurrency.To > 0 {
return nil, errors.New("maxConnections cannot be specified together with maxConcurrency")
}
if c.Xmux == (XmuxConfig{}) {
c.Xmux.MaxConnections.From = 1
c.Xmux.MaxConnections.To = 1
c.Xmux.HMaxReusableSecs.From = 1800
c.Xmux.HMaxReusableSecs.To = 3000
}
return &masque.Config{ return &masque.Config{
Host: c.Host, Host: host,
Path: path, Path: path,
Headers: headers, Headers: headers,
Warp: warp,
Xmux: &splithttp.XmuxConfig{
MaxConcurrency: newRangeConfig(c.Xmux.MaxConcurrency),
MaxConnections: newRangeConfig(c.Xmux.MaxConnections),
CMaxReuseTimes: newRangeConfig(c.Xmux.CMaxReuseTimes),
HMaxRequestTimes: newRangeConfig(c.Xmux.HMaxRequestTimes),
HMaxReusableSecs: newRangeConfig(c.Xmux.HMaxReusableSecs),
HKeepAlivePeriod: c.Xmux.HKeepAlivePeriod,
},
}, nil }, nil
} }
func parseWarpAddress(list []string) ([]string, error) {
if len(list) == 0 {
return nil, errors.New(`"address" in "warp" is not set`)
}
var v4, v6 bool
address := make([]string, 0, len(list))
for _, s := range list {
prefix, err := netip.ParsePrefix(s)
if err != nil {
addr, err := netip.ParseAddr(s)
if err != nil {
return nil, errors.New(`invalid "address" in "warp": `, s)
}
prefix = netip.PrefixFrom(addr, addr.BitLen())
}
if prefix.Addr().Is4() && v4 || prefix.Addr().Is6() && v6 {
return nil, errors.New(`"address" in "warp" takes at most one IPv4 and one IPv6 address`)
}
v4 = v4 || prefix.Addr().Is4()
v6 = v6 || prefix.Addr().Is6()
address = append(address, prefix.String())
}
return address, nil
}
func decodeWarpKey(s string) ([]byte, error) {
s = strings.TrimSpace(s)
if s == "" {
return nil, errors.New("empty key")
}
if block, _ := pem.Decode([]byte(s)); block != nil {
return block.Bytes, nil
}
der, err := base64.StdEncoding.DecodeString(s)
if err != nil {
return nil, errors.New("neither PEM nor base64").Base(err)
}
return der, nil
}
func parseWarpPublicKey(s string) ([]byte, error) {
der, err := decodeWarpKey(s)
if err != nil {
return nil, err
}
if _, err := x509.ParsePKIXPublicKey(der); err != nil {
return nil, errors.New("not a PKIX public key").Base(err)
}
return der, nil
}
func parseWarpPrivateKey(s string) ([]byte, error) {
der, err := decodeWarpKey(s)
if err != nil {
return nil, err
}
var key any
key, err = x509.ParsePKCS8PrivateKey(der)
if err != nil {
if key, err = x509.ParseECPrivateKey(der); err != nil {
return nil, errors.New("neither a PKCS #8 nor a SEC 1 private key")
}
}
ecKey, ok := key.(*ecdsa.PrivateKey)
if !ok || ecKey.Curve != elliptic.P256() {
return nil, errors.New("not an ECDSA P-256 key")
}
return x509.MarshalPKCS8PrivateKey(ecKey)
}
func readFileOrString(f string, s []string) ([]byte, error) { func readFileOrString(f string, s []string) ([]byte, error) {
if len(f) > 0 { if len(f) > 0 {
return filesystem.ReadCert(f) return filesystem.ReadCert(f)
+2
View File
@@ -316,6 +316,7 @@ type TLSConfig struct {
ECHServerKeys string `json:"echServerKeys"` ECHServerKeys string `json:"echServerKeys"`
ECHConfigList string `json:"echConfigList"` ECHConfigList string `json:"echConfigList"`
ECHSocketSettings *SocketConfig `json:"echSockopt"` ECHSocketSettings *SocketConfig `json:"echSockopt"`
UseSystemCA bool `json:"useSystemCA"`
} }
// Build implements Buildable. // Build implements Buildable.
@@ -403,6 +404,7 @@ func (c *TLSConfig) Build() (proto.Message, error) {
} }
config.EchSocketSettings = ss config.EchSocketSettings = ss
} }
config.UseSystemCa = c.UseSystemCA
return config, nil return config, nil
} }
+1 -1
View File
@@ -59,7 +59,7 @@ func (v *TunConfig) Build() (proto.Message, error) {
if slices.Contains(config.AutoSystemWfpBlockLeak, "dns") && len(v.DNS) == 0 { if slices.Contains(config.AutoSystemWfpBlockLeak, "dns") && len(v.DNS) == 0 {
return nil, errors.New(`autoSystemWfpBlockLeak "dns" needs dns to be set`) return nil, errors.New(`autoSystemWfpBlockLeak "dns" needs dns to be set`)
} }
case "linux": case "linux", "freebsd":
if v.AutoSystemDnsToGateway && len(v.Gateway) == 0 { if v.AutoSystemDnsToGateway && len(v.Gateway) == 0 {
return nil, errors.New("autoSystemDnsToGateway needs gateway to be set") return nil, errors.New("autoSystemDnsToGateway needs gateway to be set")
} }
+7 -6
View File
@@ -3,6 +3,7 @@ package conf_test
import ( import (
"encoding/json" "encoding/json"
"runtime" "runtime"
"slices"
"testing" "testing"
. "github.com/xtls/xray-core/infra/conf" . "github.com/xtls/xray-core/infra/conf"
@@ -43,18 +44,18 @@ func TestTunConfigAutoSystem(t *testing.T) {
func TestTunConfigAutoSystemNeeds(t *testing.T) { func TestTunConfigAutoSystemNeeds(t *testing.T) {
for _, c := range []struct { for _, c := range []struct {
input string input string
goos string // where it is rejected goos []string // where it is rejected
}{ }{
{`{"name": "xray0", "autoSystemWfpBlockLeak": ["misconfigtun"]}`, "windows"}, {`{"name": "xray0", "autoSystemWfpBlockLeak": ["misconfigtun"]}`, []string{"windows"}},
{`{"name": "xray0", "autoSystemRoutingTable": ["0.0.0.0/0"], "autoSystemWfpBlockLeak": ["misconfigtun"]}`, ""}, {`{"name": "xray0", "autoSystemRoutingTable": ["0.0.0.0/0"], "autoSystemWfpBlockLeak": ["misconfigtun"]}`, nil},
{`{"name": "xray0", "autoSystemRoutingTable": ["0.0.0.0/0"], "autoSystemWfpBlockLeak": ["dns"]}`, "windows"}, {`{"name": "xray0", "autoSystemRoutingTable": ["0.0.0.0/0"], "autoSystemWfpBlockLeak": ["dns"]}`, []string{"windows"}},
{`{"name": "xray0", "autoSystemDnsToGateway": true}`, "linux"}, {`{"name": "xray0", "autoSystemDnsToGateway": true}`, []string{"linux", "freebsd"}},
} { } {
config := new(TunConfig) config := new(TunConfig)
if err := json.Unmarshal([]byte(c.input), config); err != nil { if err := json.Unmarshal([]byte(c.input), config); err != nil {
t.Fatal(err) t.Fatal(err)
} }
if _, err := config.Build(); (err != nil) != (runtime.GOOS == c.goos) { if _, err := config.Build(); (err != nil) != slices.Contains(c.goos, runtime.GOOS) {
t.Errorf("%s on %s: error = %v", c.input, runtime.GOOS, err) t.Errorf("%s on %s: error = %v", c.input, runtime.GOOS, err)
} }
} }
+1 -5
View File
@@ -344,11 +344,7 @@ func (c *OutboundDetourConfig) Build() (*core.OutboundHandlerConfig, error) {
return nil, err return nil, err
} }
if _, ok := ts.(*masque.ClientConfig); ok { if _, ok := ts.(*masque.ClientConfig); !ok && senderSettings.StreamSettings != nil && senderSettings.StreamSettings.ProtocolName == "masque" {
if ms := senderSettings.MultiplexSettings; ms != nil && ms.Enabled {
return nil, errors.New(`masque outbound does not support "mux"`)
}
} else if senderSettings.StreamSettings != nil && senderSettings.StreamSettings.ProtocolName == "masque" {
return nil, errors.New("the masque transport can only be used by the masque outbound") return nil, errors.New("the masque transport can only be used by the masque outbound")
} }
@@ -12,6 +12,7 @@ import (
"github.com/xtls/xray-core/core" "github.com/xtls/xray-core/core"
"github.com/xtls/xray-core/infra/conf" "github.com/xtls/xray-core/infra/conf"
"github.com/xtls/xray-core/infra/conf/serial" "github.com/xtls/xray-core/infra/conf/serial"
"github.com/xtls/xray-core/proxy/hysteria"
"github.com/xtls/xray-core/proxy/masque" "github.com/xtls/xray-core/proxy/masque"
"github.com/xtls/xray-core/proxy/shadowsocks" "github.com/xtls/xray-core/proxy/shadowsocks"
"github.com/xtls/xray-core/proxy/shadowsocks_2022" "github.com/xtls/xray-core/proxy/shadowsocks_2022"
@@ -91,6 +92,8 @@ func extractInboundUsers(inb *core.InboundHandlerConfig) []*protocol.User {
return ty.Users return ty.Users
case *masque.ServerConfig: case *masque.ServerConfig:
return ty.Users return ty.Users
case *hysteria.ServerConfig:
return ty.Users
default: default:
fmt.Println("unsupported inbound type") fmt.Println("unsupported inbound type")
} }
+14
View File
@@ -0,0 +1,14 @@
package base64
import "github.com/xtls/xray-core/main/commands/base"
// CmdBase64 holds all Base64 subcommands.
var CmdBase64 = &base.Command{
UsageLine: "{{.Exec}} base64",
Short: "Encode and decode Base64",
Long: "{{.Exec}} {{.LongName}} provides tools to encode and decode arbitrary bytes.",
Commands: []*base.Command{
cmdEncode,
cmdDecode,
},
}
+66
View File
@@ -0,0 +1,66 @@
package base64
import (
"fmt"
"io"
"os"
"strings"
xbase64 "github.com/xtls/xray-core/common/base64"
"github.com/xtls/xray-core/main/commands/base"
)
var cmdDecode = &base.Command{
UsageLine: "{{.Exec}} base64 decode [-gzip] <value | - | stdin:> [output-file]",
Short: "Decode Base64",
Long: fmt.Sprintf(`
Decode a Base64 value, or pass "-" or "stdin:" to read stdin.
Write to output-file, or stdout if omitted.
Accepts %s, %s, or no prefix; gzip is enabled by %s or -gzip.
Examples:
{{.Exec}} base64 decode eA== out.bin
{{.Exec}} base64 decode -gzip H4sIAAAAAAAC/6sAAIMW3IwBAAAA out.bin
`, xbase64.Prefix, xbase64.GzipPrefix, xbase64.GzipPrefix),
}
var decodeGzipEnabled = cmdDecode.Flag.Bool("gzip", false, "Decompress the decoded bytes with gzip.")
func init() {
cmdDecode.Run = executeDecode
}
func executeDecode(cmd *base.Command, args []string) {
if len(args) == 0 || len(args) > 2 {
cmd.Usage()
}
value := args[0]
if value == "-" || value == "stdin:" {
input, err := io.ReadAll(os.Stdin)
if err != nil {
base.Fatalf("failed to read input: %s", err)
}
value = string(input)
}
value = strings.TrimSpace(value)
if *decodeGzipEnabled && !strings.HasPrefix(value, xbase64.GzipPrefix) {
value = xbase64.GzipPrefix + strings.TrimPrefix(value, xbase64.Prefix)
}
data, err := xbase64.Decode(value)
if err != nil {
base.Fatalf("failed to decode input: %s", err)
}
if len(args) == 2 && args[1] != "-" {
err = os.WriteFile(args[1], data, 0o666)
} else {
_, err = os.Stdout.Write(data)
}
if err != nil {
base.Fatalf("failed to write decoded output: %s", err)
}
}
+57
View File
@@ -0,0 +1,57 @@
package base64
import (
"fmt"
"io"
"os"
xbase64 "github.com/xtls/xray-core/common/base64"
"github.com/xtls/xray-core/main/commands/base"
)
var cmdEncode = &base.Command{
UsageLine: "{{.Exec}} base64 encode [-gzip] <input-file | - | stdin:>",
Short: "Encode as Base64",
Long: fmt.Sprintf(`
Encode input-file as Base64 with the %s prefix.
Pass "-" or "stdin:" to read stdin.
Use -gzip to compress and output %s.
Examples:
{{.Exec}} base64 encode input.bin
{{.Exec}} base64 encode -gzip input.bin
`, xbase64.Prefix, xbase64.GzipPrefix),
}
var gzipEnabled = cmdEncode.Flag.Bool("gzip", false, "Compress the input with gzip.")
func init() {
cmdEncode.Run = executeEncode
}
func executeEncode(cmd *base.Command, args []string) {
if len(args) != 1 {
cmd.Usage()
}
var data []byte
var err error
if args[0] == "-" || args[0] == "stdin:" {
data, err = io.ReadAll(os.Stdin)
} else {
data, err = os.ReadFile(args[0])
}
if err != nil {
base.Fatalf("failed to read input: %s", err)
}
value, err := xbase64.Encode(data, *gzipEnabled)
if err != nil {
base.Fatalf("failed to encode input: %s", err)
}
if _, err := fmt.Fprintln(os.Stdout, value); err != nil {
base.Fatalf("failed to write encoded output: %s", err)
}
}
+5
View File
@@ -2,7 +2,9 @@ package all
import ( import (
"github.com/xtls/xray-core/main/commands/all/api" "github.com/xtls/xray-core/main/commands/all/api"
"github.com/xtls/xray-core/main/commands/all/base64"
"github.com/xtls/xray-core/main/commands/all/convert" "github.com/xtls/xray-core/main/commands/all/convert"
"github.com/xtls/xray-core/main/commands/all/lua"
"github.com/xtls/xray-core/main/commands/all/tls" "github.com/xtls/xray-core/main/commands/all/tls"
"github.com/xtls/xray-core/main/commands/base" "github.com/xtls/xray-core/main/commands/base"
) )
@@ -11,10 +13,13 @@ func init() {
base.RootCommand.Commands = append( base.RootCommand.Commands = append(
base.RootCommand.Commands, base.RootCommand.Commands,
api.CmdAPI, api.CmdAPI,
base64.CmdBase64,
convert.CmdConvert, convert.CmdConvert,
lua.CmdLua,
tls.CmdTLS, tls.CmdTLS,
cmdUUID, cmdUUID,
cmdX25519, cmdX25519,
cmdEd25519,
cmdWG, cmdWG,
cmdMLDSA65, cmdMLDSA65,
cmdMLKEM768, cmdMLKEM768,
+53
View File
@@ -0,0 +1,53 @@
package all
import (
"crypto/ed25519"
"crypto/rand"
"encoding/base64"
"fmt"
"github.com/xtls/xray-core/main/commands/base"
)
var cmdEd25519 = &base.Command{
UsageLine: `{{.Exec}} ed25519 [-i "private key seed (Base64URL)"]`,
Short: "Generate an Ed25519 key pair",
Long: `
Generate an Ed25519 key pair, or restore it from a 32-byte private key seed
with -i. PrivateKey (seed) and PublicKey use unpadded Base64URL.
`,
}
var inputEd25519 = cmdEd25519.Flag.String("i", "", "Ed25519 private key seed in unpadded Base64URL.")
func init() {
cmdEd25519.Run = executeEd25519
}
func executeEd25519(cmd *base.Command, args []string) {
if len(args) != 0 {
cmd.Usage()
}
var privateKey ed25519.PrivateKey
if *inputEd25519 == "" {
_, key, err := ed25519.GenerateKey(rand.Reader)
if err != nil {
base.Fatalf("failed to generate Ed25519 key: %s", err)
}
privateKey = key
} else {
encoding := base64.RawURLEncoding.Strict()
seed, err := encoding.DecodeString(*inputEd25519)
if err != nil || len(seed) != ed25519.SeedSize || encoding.EncodeToString(seed) != *inputEd25519 {
base.Fatalf("invalid Ed25519 private key seed: expected 32 bytes in unpadded Base64URL")
}
privateKey = ed25519.NewKeyFromSeed(seed)
}
if _, err := fmt.Printf("PrivateKey: %s\nPublicKey: %s\n",
base64.RawURLEncoding.EncodeToString(privateKey.Seed()),
base64.RawURLEncoding.EncodeToString(privateKey.Public().(ed25519.PublicKey))); err != nil {
base.Fatalf("failed to write Ed25519 keys: %s", err)
}
}
+13
View File
@@ -0,0 +1,13 @@
package lua
import "github.com/xtls/xray-core/main/commands/base"
// CmdLua holds the Lua script commands.
var CmdLua = &base.Command{
UsageLine: "{{.Exec}} lua",
Short: "Manage Lua scripts",
Long: "{{.Exec}} {{.LongName}} provides tools for local Lua scripts.",
Commands: []*base.Command{
cmdSign,
},
}
+98
View File
@@ -0,0 +1,98 @@
package lua
import (
"os"
"path/filepath"
xlua "github.com/xtls/xray-core/common/lua"
"github.com/xtls/xray-core/main/commands/base"
)
var cmdSign = &base.Command{
UsageLine: "{{.Exec}} lua sign [-key seed] [-o output-file] <input-file>",
Short: "Sign a local Lua script for privileged execution",
Long: `
Sign a local Lua file using a 32-byte Ed25519 private key seed in unpadded
Base64URL. Use {{.Exec}} ed25519 to generate a key pair.
Write back to the input file, or use -o to write another file (- for stdout).
`,
}
var (
signKey = cmdSign.Flag.String("key", "", "Ed25519 private key seed in unpadded Base64URL.")
signOutput = cmdSign.Flag.String("o", "", "Output file, or - for stdout. Default: input file.")
)
func init() {
cmdSign.Run = executeSign
}
func executeSign(cmd *base.Command, args []string) {
if len(args) != 1 {
cmd.Usage()
}
info, err := os.Stat(args[0])
if err != nil {
base.Fatalf("failed to inspect Lua file: %s", err)
}
if !info.Mode().IsRegular() {
base.Fatalf("Lua input is not a regular file: %s", args[0])
}
code, err := os.ReadFile(args[0])
if err != nil {
base.Fatalf("failed to read Lua file: %s", err)
}
signed, err := xlua.SignPrivilegedScript(code, *signKey)
if err != nil {
base.Fatalf("failed to sign Lua file: %s", err)
}
if *signOutput == "-" {
if _, err := os.Stdout.Write(signed); err != nil {
base.Fatalf("failed to write signed Lua file: %s", err)
}
return
}
output := *signOutput
if output == "" {
output = args[0]
}
if err := writeSignedFile(output, signed, info.Mode().Perm()); err != nil {
base.Fatalf("failed to write signed Lua file: %s", err)
}
}
func writeSignedFile(path string, code []byte, mode os.FileMode) error {
if info, err := os.Stat(path); err == nil {
if !info.Mode().IsRegular() {
return &os.PathError{Op: "write", Path: path, Err: os.ErrInvalid}
}
mode = info.Mode().Perm()
path, err = filepath.EvalSymlinks(path)
if err != nil {
return err
}
} else if !os.IsNotExist(err) {
return err
}
file, err := os.CreateTemp(filepath.Dir(path), ".xray-lua-sign-*")
if err != nil {
return err
}
defer os.Remove(file.Name())
defer file.Close()
if _, err := file.Write(code); err != nil {
return err
}
if err := file.Chmod(mode); err != nil {
return err
}
if err := file.Close(); err != nil {
return err
}
return os.Rename(file.Name(), path)
}
+8 -2
View File
@@ -13,6 +13,7 @@ import (
"github.com/xtls/xray-core/common/buf" "github.com/xtls/xray-core/common/buf"
"github.com/xtls/xray-core/common/errors" "github.com/xtls/xray-core/common/errors"
"github.com/xtls/xray-core/common/platform"
"github.com/xtls/xray-core/common/utils" "github.com/xtls/xray-core/common/utils"
"github.com/xtls/xray-core/main/confloader" "github.com/xtls/xray-core/main/confloader"
) )
@@ -28,10 +29,15 @@ func ConfigLoader(arg string) (out io.Reader, err error) {
data, err = FetchHTTPContent(arg) data, err = FetchHTTPContent(arg)
case arg == "stdin:": case arg == "stdin:":
data, err = io.ReadAll(os.Stdin) data, err = platform.ReadConfigFile(os.Stdin)
default: default:
data, err = os.ReadFile(arg) file, openErr := os.Open(arg)
if openErr != nil {
return nil, openErr
}
defer file.Close()
data, err = platform.ReadConfigFile(file)
} }
if err != nil { if err != nil {
+13 -8
View File
@@ -17,6 +17,7 @@ import (
"github.com/xtls/xray-core/common/cmdarg" "github.com/xtls/xray-core/common/cmdarg"
"github.com/xtls/xray-core/common/errors" "github.com/xtls/xray-core/common/errors"
clog "github.com/xtls/xray-core/common/log" clog "github.com/xtls/xray-core/common/log"
"github.com/xtls/xray-core/common/lua"
"github.com/xtls/xray-core/common/platform" "github.com/xtls/xray-core/common/platform"
"github.com/xtls/xray-core/core" "github.com/xtls/xray-core/core"
"github.com/xtls/xray-core/main/commands/base" "github.com/xtls/xray-core/main/commands/base"
@@ -36,9 +37,8 @@ The -confdir=dir flag sets a dir with multiple json config
The -format=json flag sets the format of config files. The -format=json flag sets the format of config files.
Default "auto". Default "auto".
The config root env object sets process environment variables after all config The --lua-privileged-pubkey=key flag sets the trusted Ed25519 public key
files are parsed. Variables needed to locate or parse config files must be set for privileged local Lua scripts, encoded as unpadded Base64URL.
in the process environment before Xray starts.
The -test flag tells Xray to test config files only, The -test flag tells Xray to test config files only,
without launching the server. without launching the server.
@@ -53,11 +53,12 @@ func init() {
} }
var ( var (
configFiles cmdarg.Arg // "Config file for Xray.", the option is customed type, parse in main configFiles cmdarg.Arg // "Config file for Xray.", the option is customed type, parse in main
configDir string configDir string
dump = cmdRun.Flag.Bool("dump", false, "Dump merged config only, without launching Xray server.") dump = cmdRun.Flag.Bool("dump", false, "Dump merged config only, without launching Xray server.")
test = cmdRun.Flag.Bool("test", false, "Test config file only, without launching Xray server.") test = cmdRun.Flag.Bool("test", false, "Test config file only, without launching Xray server.")
format = cmdRun.Flag.String("format", "auto", "Format of input file.") format = cmdRun.Flag.String("format", "auto", "Format of input file.")
luaPrivilegedPublicKey = cmdRun.Flag.String("lua-privileged-pubkey", "", "Unpadded Base64URL Ed25519 public key for privileged local Lua scripts.")
/* We have to do this here because Golang's Test will also need to parse flag, before /* We have to do this here because Golang's Test will also need to parse flag, before
* main func in this file is run. * main func in this file is run.
@@ -214,6 +215,10 @@ func getConfigFormat() string {
} }
func startXray() (core.Server, error) { func startXray() (core.Server, error) {
if err := lua.SetPrivilegedPublicKey(*luaPrivilegedPublicKey); err != nil {
return nil, errors.New("failed to configure Lua privileged authorization").Base(err)
}
configFiles := getConfigFilePath(true) configFiles := getConfigFilePath(true)
c, err := core.LoadConfig(getConfigFormat(), configFiles) c, err := core.LoadConfig(getConfigFormat(), configFiles)
+4 -4
View File
@@ -467,7 +467,7 @@ func NewPacketReader(conn net.Conn, h *Handler, defaultRule *FinalRule, UDPOverr
if statConn != nil { if statConn != nil {
counter = statConn.ReadCounter counter = statConn.ReadCounter
} }
if c, ok := iConn.(*internet.PacketConnWrapper); ok { if c, ok := iConn.(*net.PacketConnWrapper); ok {
isOverridden := false isOverridden := false
if UDPOverride.Address != nil || UDPOverride.Port != 0 { if UDPOverride.Address != nil || UDPOverride.Port != 0 {
isOverridden = true isOverridden = true
@@ -487,7 +487,7 @@ func NewPacketReader(conn net.Conn, h *Handler, defaultRule *FinalRule, UDPOverr
} }
type PacketReader struct { type PacketReader struct {
*internet.PacketConnWrapper *net.PacketConnWrapper
stats.Counter stats.Counter
Handler *Handler Handler *Handler
DefaultRule *FinalRule DefaultRule *FinalRule
@@ -542,7 +542,7 @@ func NewPacketWriter(conn net.Conn, h *Handler, defaultRule *FinalRule, UDPOverr
if statConn != nil { if statConn != nil {
counter = statConn.WriteCounter counter = statConn.WriteCounter
} }
if c, ok := iConn.(*internet.PacketConnWrapper); ok { if c, ok := iConn.(*net.PacketConnWrapper); ok {
// If DialDest is a domain, it will be resolved in dialer // If DialDest is a domain, it will be resolved in dialer
// check this behavior and add it to map // check this behavior and add it to map
resolvedUDPAddr := utils.NewTypedSyncMap[string, net.Address]() resolvedUDPAddr := utils.NewTypedSyncMap[string, net.Address]()
@@ -563,7 +563,7 @@ func NewPacketWriter(conn net.Conn, h *Handler, defaultRule *FinalRule, UDPOverr
} }
type PacketWriter struct { type PacketWriter struct {
*internet.PacketConnWrapper *net.PacketConnWrapper
stats.Counter stats.Counter
*Handler *Handler
DefaultRule *FinalRule DefaultRule *FinalRule
+104 -35
View File
@@ -7,7 +7,6 @@ import (
"net/netip" "net/netip"
"slices" "slices"
"sync" "sync"
"sync/atomic"
"time" "time"
"golang.zx2c4.com/wireguard/tun" "golang.zx2c4.com/wireguard/tun"
@@ -26,6 +25,7 @@ import (
"github.com/xtls/xray-core/transport" "github.com/xtls/xray-core/transport"
"github.com/xtls/xray-core/transport/internet" "github.com/xtls/xray-core/transport/internet"
"github.com/xtls/xray-core/transport/internet/masque" "github.com/xtls/xray-core/transport/internet/masque"
"github.com/xtls/xray-core/transport/internet/splithttp"
"github.com/xtls/xray-core/transport/internet/stat" "github.com/xtls/xray-core/transport/internet/stat"
"github.com/xtls/xray-core/transport/internet/tls" "github.com/xtls/xray-core/transport/internet/tls"
) )
@@ -43,11 +43,15 @@ type Client struct {
ctx context.Context ctx context.Context
cancel context.CancelFunc cancel context.CancelFunc
tunnel atomic.Pointer[tunnel]
mu sync.Mutex mu sync.Mutex
xmux *splithttp.XmuxManager
dialCtx context.Context
dialer internet.Dialer
lastErr error lastErr error
lastErrAt time.Time lastErrAt time.Time
tunnelsMu sync.Mutex
tunnels map[*tunnel]struct{}
} }
func NewClient(ctx context.Context, config *ClientConfig) (*Client, error) { func NewClient(ctx context.Context, config *ClientConfig) (*Client, error) {
@@ -55,7 +59,8 @@ func NewClient(ctx context.Context, config *ClientConfig) (*Client, error) {
p := v.GetFeature(policy.ManagerType()).(policy.Manager) p := v.GetFeature(policy.ManagerType()).(policy.Manager)
streamSettings := session.StreamSettingsFromContext(ctx).(*internet.MemoryStreamConfig) streamSettings := session.StreamSettingsFromContext(ctx).(*internet.MemoryStreamConfig)
if _, ok := streamSettings.ProtocolSettings.(*masque.Config); !ok { transportConfig, ok := streamSettings.ProtocolSettings.(*masque.Config)
if !ok {
return nil, errors.New("not masque transport") return nil, errors.New("not masque transport")
} }
if tls.ConfigFromStreamSettings(streamSettings) == nil { if tls.ConfigFromStreamSettings(streamSettings) == nil {
@@ -86,7 +91,13 @@ func NewClient(ctx context.Context, config *ClientConfig) (*Client, error) {
server: server, server: server,
policyManager: p, policyManager: p,
remoteDNS: remoteDNS, remoteDNS: remoteDNS,
tunnels: make(map[*tunnel]struct{}),
} }
xmuxConfig := transportConfig.Xmux
if xmuxConfig == nil {
xmuxConfig = &splithttp.XmuxConfig{}
}
c.xmux = splithttp.NewXmuxManager(*xmuxConfig, c.newXmuxConn)
c.ctx, c.cancel = context.WithCancel(context.Background()) c.ctx, c.cancel = context.WithCancel(context.Background())
return c, nil return c, nil
} }
@@ -100,10 +111,11 @@ func (c *Client) Process(ctx context.Context, link *transport.Link, dialer inter
ob.Name = "masque" ob.Name = "masque"
ob.CanSpliceCopy = 3 ob.CanSpliceCopy = 3
t, err := c.getTunnel(ctx, dialer) xmuxClient, t, err := c.getTunnel(ctx, dialer)
if err != nil { if err != nil {
return errors.New("failed to establish CONNECT-IP tunnel").Base(err) return errors.New("failed to establish CONNECT-IP tunnel").Base(err)
} }
defer xmuxClient.DoneRunning()
var newCtx context.Context var newCtx context.Context
var newCancel context.CancelFunc var newCancel context.CancelFunc
@@ -151,7 +163,7 @@ func (c *Client) Process(ctx context.Context, link *transport.Link, dialer inter
} }
defer conn.Close() defer conn.Close()
uc := &wireguard.UDPConnClient{ uc := &wireguard.UDPConnClient{
PacketConn: conn.(*internet.PacketConnWrapper).PacketConn, PacketConn: conn.(*net.PacketConnWrapper).PacketConn,
Dest: conn.RemoteAddr().(*net.UDPAddr), Dest: conn.RemoteAddr().(*net.UDPAddr),
} }
reader = uc reader = uc
@@ -180,40 +192,63 @@ func (c *Client) Process(ctx context.Context, link *transport.Link, dialer inter
return nil return nil
} }
func (c *Client) getTunnel(ctx context.Context, dialer internet.Dialer) (*tunnel, error) { func (c *Client) getTunnel(ctx context.Context, dialer internet.Dialer) (*splithttp.XmuxClient, *tunnel, error) {
c.mu.Lock() c.mu.Lock()
defer c.mu.Unlock() defer c.mu.Unlock()
if c.ctx.Err() != nil { if c.ctx.Err() != nil {
return nil, errors.New("closed") return nil, nil, errors.New("closed")
}
if t := c.tunnel.Load(); t != nil {
select {
case <-t.done:
default:
return t, nil
}
} }
if err := ctx.Err(); err != nil { if err := ctx.Err(); err != nil {
return nil, err return nil, nil, err
} }
var xmuxClient *splithttp.XmuxClient
if c.lastErr != nil && time.Since(c.lastErrAt) < retryInterval { if c.lastErr != nil && time.Since(c.lastErrAt) < retryInterval {
return nil, c.lastErr if xmuxClient = c.xmux.GetExistingXmuxClient(ctx); xmuxClient == nil {
return nil, nil, c.lastErr
}
} else {
c.dialCtx, c.dialer = ctx, dialer
xmuxClient = c.xmux.GetXmuxClient(ctx)
c.dialCtx, c.dialer = nil, nil
if err := xmuxClient.XmuxConn.(*tunnel).err; err != nil {
if xmuxClient = c.xmux.GetExistingXmuxClient(ctx); xmuxClient == nil {
return nil, nil, err
}
}
} }
t := xmuxClient.XmuxConn.(*tunnel)
if c.ctx.Err() != nil {
return nil, nil, errors.New("closed")
}
xmuxClient.AddRunning()
return xmuxClient, t, nil
}
t, err := c.establish(ctx, dialer) func (c *Client) newXmuxConn() splithttp.XmuxConn {
t, err := c.establish(c.dialCtx, c.dialer)
if err != nil { if err != nil {
c.lastErr, c.lastErrAt = err, time.Now() c.lastErr, c.lastErrAt = err, time.Now()
return nil, err return failedTunnel(err)
} }
c.lastErr = nil c.lastErr = nil
c.tunnel.Store(t) c.tunnelsMu.Lock()
if c.ctx.Err() != nil { if c.tunnels == nil {
if c.tunnel.CompareAndSwap(t, nil) { c.tunnelsMu.Unlock()
t.close() t.close()
} return failedTunnel(errors.New("closed"))
return nil, errors.New("closed")
} }
return t, nil c.tunnels[t] = struct{}{}
c.tunnelsMu.Unlock()
if t.IsClosed() {
c.removeTunnel(t)
}
return t
}
func (c *Client) removeTunnel(t *tunnel) {
c.tunnelsMu.Lock()
delete(c.tunnels, t)
c.tunnelsMu.Unlock()
} }
func (c *Client) establish(ctx context.Context, dialer internet.Dialer) (*tunnel, error) { func (c *Client) establish(ctx context.Context, dialer internet.Dialer) (*tunnel, error) {
@@ -229,7 +264,7 @@ func (c *Client) establish(ctx context.Context, dialer internet.Dialer) (*tunnel
conn.Close() conn.Close()
return nil, errors.New("not a CONNECT-IP connection") return nil, errors.New("not a CONNECT-IP connection")
} }
t, err := newTunnel(conn, mconn.LocalAddrs(), c.remoteDNS) t, err := newTunnel(conn, mconn.LocalAddrs(), c.remoteDNS, c.removeTunnel)
if err != nil { if err != nil {
conn.Close() conn.Close()
return nil, err return nil, err
@@ -240,7 +275,11 @@ func (c *Client) establish(ctx context.Context, dialer internet.Dialer) (*tunnel
func (c *Client) Close() error { func (c *Client) Close() error {
c.cancel() c.cancel()
if t := c.tunnel.Swap(nil); t != nil { c.tunnelsMu.Lock()
tunnels := c.tunnels
c.tunnels = nil
c.tunnelsMu.Unlock()
for t := range tunnels {
t.close() t.close()
} }
return nil return nil
@@ -252,9 +291,31 @@ type tunnel struct {
tnet *wireguard.Net tnet *wireguard.Net
done chan struct{} done chan struct{}
closeOnce sync.Once closeOnce sync.Once
onClose func(*tunnel)
err error
} }
func newTunnel(conn stat.Connection, local []netip.Addr, remoteDNS []netip.Addr) (*tunnel, error) { func failedTunnel(err error) *tunnel {
t := &tunnel{err: err, done: make(chan struct{})}
t.close()
return t
}
func (t *tunnel) IsClosed() bool {
select {
case <-t.done:
return true
default:
return false
}
}
func (t *tunnel) Close() error {
t.close()
return nil
}
func newTunnel(conn stat.Connection, local []netip.Addr, remoteDNS []netip.Addr, onClose func(*tunnel)) (*tunnel, error) {
var dns []netip.Addr var dns []netip.Addr
for _, addr := range remoteDNS { for _, addr := range remoteDNS {
if slices.ContainsFunc(local, func(l netip.Addr) bool { return l.Is4() == addr.Is4() }) { if slices.ContainsFunc(local, func(l netip.Addr) bool { return l.Is4() == addr.Is4() }) {
@@ -271,10 +332,11 @@ func newTunnel(conn stat.Connection, local []netip.Addr, remoteDNS []netip.Addr)
return nil, err return nil, err
} }
t := &tunnel{ t := &tunnel{
conn: conn, conn: conn,
dev: dev, dev: dev,
tnet: tnet, tnet: tnet,
done: make(chan struct{}), done: make(chan struct{}),
onClose: onClose,
} }
go t.readFromTunnel() go t.readFromTunnel()
go t.writeToTunnel() go t.writeToTunnel()
@@ -316,8 +378,15 @@ func (t *tunnel) writeToTunnel() {
func (t *tunnel) close() { func (t *tunnel) close() {
t.closeOnce.Do(func() { t.closeOnce.Do(func() {
close(t.done) close(t.done)
t.conn.Close() if t.conn != nil {
t.dev.Close() t.conn.Close()
}
if t.dev != nil {
t.dev.Close()
}
if t.onClose != nil {
t.onClose(t)
}
}) })
} }
+54
View File
@@ -0,0 +1,54 @@
package masque
import (
"context"
"errors"
"testing"
"time"
"github.com/xtls/xray-core/transport/internet/splithttp"
)
func newTestClient(newConn func(c *Client) splithttp.XmuxConn) *Client {
c := &Client{ctx: context.Background()}
c.xmux = splithttp.NewXmuxManager(splithttp.XmuxConfig{
MaxConnections: &splithttp.RangeConfig{From: 3, To: 3},
}, func() splithttp.XmuxConn {
return newConn(c)
})
return c
}
func TestGetTunnelFallsBack(t *testing.T) {
refused := errors.New("refused")
healthy := &tunnel{done: make(chan struct{})}
dials := 0
c := newTestClient(func(c *Client) splithttp.XmuxConn {
if dials++; dials == 1 {
return healthy
}
c.lastErr, c.lastErrAt = refused, time.Now()
return failedTunnel(refused)
})
for i := 0; i < 5; i++ {
if _, got, err := c.getTunnel(context.Background(), nil); err != nil || got != healthy {
t.Fatalf("flow %d: got %p, %v instead of the healthy tunnel", i, got, err)
}
}
if dials != 2 {
t.Errorf("dialed %d times, a failed tunnel should not be retried within %v", dials, retryInterval)
}
}
func TestGetTunnelFailsWithoutTunnels(t *testing.T) {
refused := errors.New("refused")
c := newTestClient(func(c *Client) splithttp.XmuxConn {
c.lastErr, c.lastErrAt = refused, time.Now()
return failedTunnel(refused)
})
for i := 0; i < 2; i++ {
if _, _, err := c.getTunnel(context.Background(), nil); err != refused {
t.Fatalf("flow %d: got %v instead of %v", i, err, refused)
}
}
}
+15
View File
@@ -277,6 +277,7 @@ func (w *VisionReader) ReadMultiBuffer() (buf.MultiBuffer, error) {
w.ob.CanSpliceCopy = 1 w.ob.CanSpliceCopy = 1
} }
} }
SuppressOuterCloseNotify(w.conn)
readerConn, readCounter, _ := UnwrapRawConn(w.conn) readerConn, readCounter, _ := UnwrapRawConn(w.conn)
w.directReadCounter = readCounter w.directReadCounter = readCounter
w.Reader = buf.NewReader(readerConn) w.Reader = buf.NewReader(readerConn)
@@ -340,6 +341,7 @@ func (w *VisionWriter) WriteMultiBuffer(mb buf.MultiBuffer) error {
// w.ob.CanSpliceCopy = 1 // w.ob.CanSpliceCopy = 1
// } // }
} }
SuppressOuterCloseNotify(w.conn)
rawConn, _, writerCounter := UnwrapRawConn(w.conn) rawConn, _, writerCounter := UnwrapRawConn(w.conn)
w.Writer = buf.NewWriter(rawConn) w.Writer = buf.NewWriter(rawConn)
w.directWriteCounter = writerCounter w.directWriteCounter = writerCounter
@@ -669,6 +671,19 @@ func XtlsFilterTls(buffer buf.MultiBuffer, trafficState *TrafficState, ctx conte
} }
} }
type CloseNotifySuppressor interface {
SuppressCloseNotify()
}
// Close our local TLS conn instance might send a incorrect close_notify alert
// if the XTLS direct copy mode is enabled and cause TLS BAD_RECORD_MAC on users' browser
// Close the underlying connection directly to avoid this issue.
func SuppressOuterCloseNotify(conn net.Conn) {
if suppressor, ok := stat.TryUnwrapStatsConn(conn).(CloseNotifySuppressor); ok {
suppressor.SuppressCloseNotify()
}
}
// UnwrapRawConn support unwrap encryption, stats, mask wrappers, tls, utls, reality, proxyproto, uds-wrapper conn and get raw tcp/uds conn from it // UnwrapRawConn support unwrap encryption, stats, mask wrappers, tls, utls, reality, proxyproto, uds-wrapper conn and get raw tcp/uds conn from it
func UnwrapRawConn(conn net.Conn) (net.Conn, stats.Counter, stats.Counter) { func UnwrapRawConn(conn net.Conn) (net.Conn, stats.Counter, stats.Counter) {
var readCounter, writerCounter stats.Counter var readCounter, writerCounter stats.Counter
+34 -114
View File
@@ -2,7 +2,6 @@ package shadowsocks_2022
import ( import (
"context" "context"
"io"
"time" "time"
"github.com/xtls/xray-core/common" "github.com/xtls/xray-core/common"
@@ -13,9 +12,6 @@ import (
"github.com/xtls/xray-core/common/net" "github.com/xtls/xray-core/common/net"
"github.com/xtls/xray-core/common/protocol" "github.com/xtls/xray-core/common/protocol"
"github.com/xtls/xray-core/common/session" "github.com/xtls/xray-core/common/session"
"github.com/xtls/xray-core/common/signal"
"github.com/xtls/xray-core/common/task"
"github.com/xtls/xray-core/common/utils"
"github.com/xtls/xray-core/core" "github.com/xtls/xray-core/core"
"github.com/xtls/xray-core/features/policy" "github.com/xtls/xray-core/features/policy"
"github.com/xtls/xray-core/features/routing" "github.com/xtls/xray-core/features/routing"
@@ -101,35 +97,29 @@ func (i *Inbound) processTCP(ctx context.Context, conn net.Conn, dispatcher rout
return errors.New("unable to set read deadline").Base(err) return errors.New("unable to set read deadline").Base(err)
} }
// 1. Single read call for Salt + Fixed-length header chunk per SIP022 §3.1.4
headerLen := i.method.KeySaltLength + RequestHeaderFixedChunkLength + AEADTagSize
headerBuf := make([]byte, headerLen)
n, err := conn.Read(headerBuf)
if err != nil || n < headerLen {
ResetTCPConn(conn)
return errors.New("failed to read complete handshake header")
}
var salt [32]byte var salt [32]byte
copy(salt[:i.method.KeySaltLength], headerBuf[:i.method.KeySaltLength])
saltSlice := salt[:i.method.KeySaltLength] saltSlice := salt[:i.method.KeySaltLength]
if _, err := io.ReadFull(conn, saltSlice); err != nil { fixedChunk := headerBuf[i.method.KeySaltLength:]
return err
}
if !i.saltFilter.Check(salt) { reader, reqHeader, err := InitServerStream(conn, i.method, i.psk, saltSlice, salt, fixedChunk, i.saltFilter)
return ErrSaltNotUnique
}
sessionKey := DeriveSessionSubKey(i.psk, saltSlice, i.method.KeySaltLength)
aead, err := i.method.NewAEAD(sessionKey)
if err != nil { if err != nil {
ResetTCPConn(conn)
return err return err
} }
reader := NewStreamReader(conn, aead)
reqHeader, err := ReadClientRequestHeader(conn, reader)
if err != nil {
return err
}
conn.SetReadDeadline(time.Time{})
dest := reqHeader.Destination dest := reqHeader.Destination
writer, err := WriteTCPResponse(conn, i.method, i.psk, saltSlice, nil) writer := NewServerStreamWriter(conn, i.method, i.psk, saltSlice)
if err != nil {
return err
}
ctx = log.ContextWithAccessMessage(ctx, &log.AccessMessage{ ctx = log.ContextWithAccessMessage(ctx, &log.AccessMessage{
From: conn.RemoteAddr(), From: conn.RemoteAddr(),
@@ -146,42 +136,17 @@ func (i *Inbound) processTCP(ctx context.Context, conn net.Conn, dispatcher rout
} }
if len(reqHeader.EarlyData) > 0 { if len(reqHeader.EarlyData) > 0 {
earlyBuf := buf.New() mb := buf.MergeBytes(nil, reqHeader.EarlyData)
earlyBuf.Write(reqHeader.EarlyData) if err := link.Writer.WriteMultiBuffer(mb); err != nil {
if err := link.Writer.WriteMultiBuffer(buf.MultiBuffer{earlyBuf}); err != nil {
return err return err
} }
} }
sessionPolicy = i.policyManager.ForLevel(uint32(i.user.Level)) return TransportTCP(ctx, i.policyManager.ForLevel(uint32(i.user.Level)), reader, writer, link)
ctx, cancel := context.WithCancel(ctx)
timer := signal.CancelAfterInactivity(ctx, cancel, sessionPolicy.Timeouts.ConnectionIdle)
ctx = policy.ContextWithBufferPolicy(ctx, sessionPolicy.Buffer)
requestDone := func() error {
defer timer.SetTimeout(sessionPolicy.Timeouts.DownlinkOnly)
return buf.Copy(reader, link.Writer, buf.UpdateActivity(timer))
}
responseDone := func() error {
defer timer.SetTimeout(sessionPolicy.Timeouts.UplinkOnly)
return buf.Copy(link.Reader, writer, buf.UpdateActivity(timer))
}
responseDoneAndCloseWriter := task.OnSuccess(responseDone, task.Close(link.Writer))
return task.Run(ctx, requestDone, responseDoneAndCloseWriter)
} }
func (i *Inbound) processUDP(ctx context.Context, conn stat.Connection, dispatcher routing.Dispatcher) error { func (i *Inbound) processUDP(ctx context.Context, conn stat.Connection, dispatcher routing.Dispatcher) error {
udpConns := utils.NewTypedSyncMap[uint64, *udpConnEntry]() reader := buf.NewPacketReader(conn)
defer func() {
udpConns.Range(func(key uint64, entry *udpConnEntry) bool {
entry.timer.SetTimeout(0)
return true
})
}()
reader := buf.NewReader(conn)
for { for {
mb, err := reader.ReadMultiBuffer() mb, err := reader.ReadMultiBuffer()
if err != nil { if err != nil {
@@ -191,75 +156,30 @@ func (i *Inbound) processUDP(ctx context.Context, conn stat.Connection, dispatch
for _, b := range mb { for _, b := range mb {
decoded, err := i.udpCodec.DecodePacket(b.Bytes()) decoded, err := i.udpCodec.DecodePacket(b.Bytes())
if err != nil { b.Release()
b.Release() if err != nil || decoded.HeaderType != HeaderTypeClient {
continue continue
} }
entry, ok := udpConns.Load(decoded.SessionID) sessionItem := i.udpCodec.GetSession(decoded.SessionID)
if !ok { if sessionItem.User == nil {
sessCtx, cancel := context.WithCancel(ctx) sessionItem.Lock()
sessCtx = log.ContextWithAccessMessage(sessCtx, &log.AccessMessage{ if sessionItem.User == nil {
From: conn.RemoteAddr(), sessionItem.User = i.user
To: decoded.Destination,
Status: log.AccessAccepted,
Email: i.user.Email,
})
link, err := dispatcher.Dispatch(sessCtx, decoded.Destination)
if err != nil {
cancel()
b.Release()
continue
}
newEntry := &udpConnEntry{
link: link,
cancel: cancel,
}
sessionPolicy := i.policyManager.ForLevel(uint32(i.user.Level))
newEntry.timer = signal.CancelAfterInactivity(sessCtx, func() {
udpConns.Delete(decoded.SessionID)
common.Interrupt(link.Reader)
common.Interrupt(link.Writer)
cancel()
}, sessionPolicy.Timeouts.ConnectionIdle)
actual, loaded := udpConns.LoadOrStore(decoded.SessionID, newEntry)
if loaded {
// Another goroutine/packet beat us to storing, terminate our redundant link
newEntry.timer.SetTimeout(0)
entry = actual
} else {
entry = newEntry
go func(sessID uint64, dest net.Destination, cEntry *udpConnEntry) {
defer func() {
cEntry.timer.SetTimeout(0)
}()
for {
resMb, err := cEntry.link.Reader.ReadMultiBuffer()
if err != nil {
return
}
cEntry.timer.Update()
for _, rb := range resMb {
encPacket, err := i.udpCodec.EncodeServerPacket(sessID, dest, rb.Bytes())
rb.Release()
if err != nil {
continue
}
_, _ = conn.Write(encPacket)
}
}
}(decoded.SessionID, decoded.Destination, entry)
} }
sessionItem.Unlock()
}
link, err := sessionItem.EnsureLink(ctx, conn, decoded.Destination, dispatcher, i.policyManager, func(dest net.Destination, payload []byte) ([]byte, error) {
return i.udpCodec.EncodeServerPacket(decoded.SessionID, dest, payload)
})
if err != nil {
continue
} }
entry.timer.Update()
payloadBuf := buf.New() payloadBuf := buf.New()
payloadBuf.Write(decoded.Payload) payloadBuf.Write(decoded.Payload)
b.Release() payloadBuf.UDP = &decoded.Destination
_ = entry.link.Writer.WriteMultiBuffer(buf.MultiBuffer{payloadBuf}) _ = link.Writer.WriteMultiBuffer(buf.MultiBuffer{payloadBuf})
} }
} }
} }

Some files were not shown because too many files have changed in this diff Show More