What are the possible issues adding https://github.com/gwarser/filter-lists/blob/master/lan-block.txt to uBO's default lists? (maybe even importing it in the project as a uBlock list).

||::1^$third-party needs to become ||[::1]^$third-party though.

I imported the list and I see no impact on performance as per built-in benchmark.

I did see a request from the benchmark which is being blocked by that list and which was not blocked:

url=http://127.0.0.1:3033/__webpack_hmr
docOrigin=https://hltv.gainskins.com

What are the possible issues adding https://github.com/gwarser/filter-lists/blob/master/lan-block.txt to uBO's default lists? (maybe even importing it in the project as a uBlock list).

||::1^$third-party needs to become ||[::1]^$third-party though.

I imported the list and I see no impact on performance as per built-in benchmark.

I did see a request from the benchmark which is being blocked by that list and which was not blocked:

url=http://127.0.0.1:3033/__webpack_hmr
docOrigin=https://hltv.gainskins.com

@gwarser could probably comment...

But what if a site assigns a hostname to one of these IP addresses? In such a case, would these IP blocks by regex still catch those? If not we may need to think about that too...

Well it's possible to block at onHeadersReceived() time and the ip address information is available at this point, so theoretically uBO could ensure the ip address is not a local one. In that case that would not be a matter of filter list, but maybe a per-site switch.

The amazing thing is that this is apparently a feature and not a bug in most browsers...

So I guess the question is whether it would be worthwhile to just use the list, or as your suggested per-site switch...

The author of this article, which discusses and shows some of the more malicious possibilities with a stock Firefox:

https://medium.com/@stestagg/stealing-secrets-from-developers-using-websockets-254f98d577a0

Has provided a websocket portscan test here, but it uses IP and not hostnames...still, useful for peace of mind on that front...minus domains:

http://frontend-overflowstack.com/

NoScript had "ABE (Application Boundaries Enforcer") enabled by default which basically blocked access from non-local domains to local domains (so opening 127.0.0.1 could still access 127.0.0.1, but internet websites could not). This helped against attackes like https://chaoswebs.net/blog/timebleed-breaking-privacy-with-a-simple-timing-attack.html (using pictures that are by default never blocked in uMatrix).

Since NS removed this feature I had no use for it anymore and switched to uMatrix with a blocklist like this: https://www.privacy-handbuch.de/handbuch_21g.htm
For a long time I wondered why uMatrix (or uBlock) does not come with this feature by default & enabled since it's just a few rules that need to be added.

Are those giant regex even necessary @gwarser ? The 127.xxx one for example is written in this guide only as ||127.0.*^$important,third-party whereas you have some giant regex that is probably much slower? Same with 172 where it might be faster just adding those 16 prefixes separately?
I once coded some script that used similar giant regex - it was painfully slow. I speed it up immensely by breaking the regex apart in smaller ones (not possible here of course) and then even more by replacing some regex with simpler string matchers.

As what does a WSS register in uMatrix? Fetch/XHR?

Web devs could get mad if this is implemented. Third-party requests to localhost are often used to detect the presence of an application on a user's computer, which can be malicious, but also serves a purpose. For example, discord uses websockets on localhost to determine if a user already has the discord application installed, and communicate between the website and application. This happens on invite pages like this: https://discord.com/invite/valorant

Another example - website-application communication on localhost also occurs on https://screencast-o-matic.com/screen-recorder. (click the Launch Free Recorder button)

https://screencast-o-matic.com/screen-recorder

Interesting, they seem to actually use the TimeBleed attack vector whereas discord uses some XHR according to uMatrix.
Firefox network inspector for the site does not show anything. That really sucks.

Are those giant regex even necessary @gwarser ?

Yes. You can have local server hosted on any IP. Does not matter if Ebay is scanning only 127.0.0.1. Other pages can scan for something else.

The 127.xxx one for example is written in this guide only as ||127.0.*^$important,third-party whereas you have some giant regex

This filter is not valid - IP is not equal domain. It will match foo.127.0.com for example.

Replacing || by // will reduce false positives a bit, but can still match in query parameters. [even using start anchor with protocol (|http://127.0.*) will not prevent matching http://127.0.com for example]. Similarly filters like this ||*.local (wildcard next to domain anchor) will match any address with .local in query. ||*.fritz.box^? Why not ||fritz.box^?

that is probably much slower? Same with 172 where it might be faster just adding those 16 prefixes separately?

In uBO these Regular Expressions are tokenized - plain strings are extracted from them. In these filters it will be first octet (172 for example). uBO will use these regexes only when URL contains this token.

Same with 172 where it might be faster just adding those 16 prefixes separately?

@gorhill exposing second octet will help with tokenization?

I'm pretty sure ipv6 address filters from https://www.privacy-handbuch.de/handbuch_21g.htm are also wrong. For example fe80::/10 is not [fe80::*] because it ranges from fe80:0:0:0:0:0:0:0 to febf:ffff:ffff:ffff:ffff:ffff:ffff:ffff.

exposing second octet will help with tokenization?

172 is a good token, it's not a common occurrence in URLs, and if ever it occurs, best to execute a single regex than 16 separate ones.

This has now even made it to Schneier:
https://www.schneier.com/blog/archives/2020/05/websites_conduc.html

After doing some research, I've found that these fingerprint scripts are all hosted via CNAME on the same domain, which uBO already blocks when CNAME checking is on:

online-metrix.net

Beyond that, it usually is served up as one ore more of these:

• clear.png
• check.js
• fp.swf

It's used by a lot of sites:

https://src.ebay-us.com/fp/check.js?org_id=usllpic0&session_id=1
https://src.ebay-us.com/fp/fp.swf?org_id=usllpic0&session_id=1

https://content.mercadopago.com/fp/clear.png

https://b177.kabbage.com/fp/check.js?org_id=rxemj1ou&pageid=1&session_id=1

https://content3.risk.lexisnexis.com/tags.js?org_id=baa3flhp&allow_reprofile=1&session_id=1

And even more:
https://www.google.com/search?hl=en&q=inurl%3Acheck.js%3Forg_id%3D

Someone also has built a payload decryption tool:

https://gist.github.com/nemec/ea6b21bcd027b81ac1e3fbcfeb01db3e

https://news.ycombinator.com/item?id=23361823

https://news.ycombinator.com/item?id=23361823

The linked spreadsheet in that article...those domains should probably make it into the blocklists.

https://docs.google.com/spreadsheets/u/1/d/1Nu4lpyZ5PQUIpiLJBddXnr67t5-1y0u40dzyzSYj1gc/preview

No need to add any of those hostnames, online-metrix.net is already blocked by both EasyPrivacy and Peter Lowe's.

What is still missing in my list is ipv6 handling https://en.wikipedia.org/wiki/Private_network#Private_IPv6_addresses. No one will scan these from a to z, but some software can still be listening on one [address, so specialized fingerprinting scripts may look only for these specific addresses].

Calculator https://www.vultr.com/resources/subnet-calculator-ipv6/

• fc00::/7 rages from fc00:0000:0000:0000:0000:0000:0000:0000 to fdff:ffff:ffff:ffff:ffff:ffff:ffff:ffff

  ||[fc*]^
||[fd*]^

• fd00::/8 should be included in above - fd00:0000:0000:0000:0000:0000:0000:0000 to fdff:ffff:ffff:ffff:ffff:ffff:ffff:ffff

• legacy "site-local" fec0::/10 - fec0:0000:0000:0000:0000:0000:0000:0000 to feff:ffff:ffff:ffff:ffff:ffff:ffff:ffff

  ||[fec*]^
||[fed*]^
||[fee*]^
||[fef*]^

• autoconfiguration? fe80::/10 - fe80:0000:0000:0000:0000:0000:0000:0000 to febf:ffff:ffff:ffff:ffff:ffff:ffff:ffff

  ||[fe8*]^
||[fe9*]^
||[fea*]^
||[feb*]^

I'm not sure my filters above are actually valid or if these addresses can be so easily matched by simple text comparison with wildcard. Is [ and ] invalid in normal domain name? Someone with better knowledge about ipv6 address formatting will need to say. It may be good based on this: https://en.wikipedia.org/wiki/IPv6_address#Representation

And dang :/ - http://[::ffff:127.0.0.1]:8080/ is also valid https://en.wikipedia.org/wiki/IPv6_address#Transition_from_IPv4. Browser converts this to http://[::ffff:7f00:1]:8080/. It will be ::ffff:0000:0000 to ::ffff:ffff:ffff. Another regex?

It seem unique (has unique application), so maybe ||[::ffff:*]^ will be ok. Maybe not, but this is simple dec to hex translation - should not be hard to create good filter.

10.*
||[::ffff:0a*]^ this is bad, with short syntax leading 0 is ommited
172.16-31.*
||[::ffff:ac1*]^
192.168.*
||[::ffff:c0a8:*]^
127.*
||[::ffff:7f*]^

I will add these filters if they look good for you (anyone?).

How to efficiently add ws: protocol regex filters?
Copy-paste, replace https? by wss?, append ,websocket to options? (http|ws)s? will not be tokenized.

Testing tokenization: https://regex101.com/r/gNYvlE/2 [rev. 2 start and end of string anchors must be removed]

Hmmm, /^\w+://... seem to pass test? And even ^\w{2,5}://.... I can also use two versions with http...$~websocket and ws...$websocket.

I think @gwarser 's list is worth being added to stock lists but with default-disabled until/unless we gather enough evidences that it doesn't cause FP as warned by @llacb47 . I have been blocking these local accesses, though with problematic rules until I see it, and want to think there was no serious FP but IIRC some videos made requests to these local addresses which I haven't investigated.

Security-conscious people have long been trying to block these accesses but unfortunately rules shared on Wilders or Malwaretips are wrong and don't work. I know nothing can stop those people from importing wrong or poorly written filters but making a proper list included in stock lists will mitigate this.

(Off-topic) @ilikenwf
online-metrix.net is nothing new, they've been around for years and included in Firefox' Tracking Protection and all the major hosts files except for Dan Pollock's. Unlike their early days now they're only on payment sites because their supposed objective is preventing XSS/fraud, this appears to be the reason Malwarebytes removed it from their blacklist. Blocking them apparently caused an error on a shopping site but payment was succeeded so I didn't care much, but EasyPrivacy has related whitelists such as
@@||online-metrix.net/fp/tags.js$domain=donorschoose.org
so it and its CNAME counterparts may cause FP for some sites.
Note even without CNAME uncloaking EasyPrivacy blocks many of them by generic filters like /fp/clear.png?.

There's a problem.

I'm not sure my filters above are actually valid or if these addresses can be so easily matched by simple text comparison with wildcard. Is [ and ] invalid in normal domain name?

||[fc*]^ will match [fc00:..., [fc0:... (false positive), [fc:... (false positive).

Regex filters will not be tokenized because of [.

And if someone will like to use IP scanning maliciously, it's very easy to convert ipv4 to ipv6 gwarser/filter-lists#6

I think special syntax for IP addresses is needed.

You all realize we're the first ones to actually even address this issue? Not even mozilla has commented..."it's a feature, not a bug..."

@ilikenwf No this has been known for years now, especially since NS lost ABE, with several "exploits" using this "bug". Unfortunately nobody really cares.
See for example arkenfox/user.js#319

I still think uB and uM should come with a default enabled blocklist that works like ABE.

||[fc*]^ will match [fc00:..., [fc0:... (false positive), [fc:... (false positive).

Regex filters will not be tokenized because of [.

Well ||[fc*]^ is already untokenizable so using a regex instead won't make the filter worst.

@gorhill gwarser/filter-lists@eebda2a

Given I see no impact on benchmarking results, should I offer this list as an option in stock filter lists? Under "Privacy"? (Not enabled by default)

You can halve the number of regex filters to maintain by simply replacing ^https?:... with ^\w+:... and dropping the ~websocket option and removing the ^wss?:... version of the filter -- the resulting filters will still be tokenizable.

should I offer this list as an option in stock filter lists? Under "Privacy"? (Not enabled by default)

+1 but another option will be to put under Malware domains (better to rename Security if possible). These rules will prevent CSRF (and DNS binding, if anyone care about) to routers, essentially the superior alternative of now-gone Noscript's ABE. You can see how many home routers have suffered CSRF vulnerability nowadays by just searching in CVE database.

and DNS binding

This wouldn't take care of DNS rebinding, for this uBO would need to filter according to IP address information, not request URL. IP address information become available at webRequest.onHeadersReceived() time in Firefox, later at webRequest.onResponseStarted() time with Chromium.

@gorhill Ah, you're right, as it makes the victim to access local IP as evil.com. Thx for correction.

@gorhill

IP address information become available at webRequest.onHeadersReceived() time in Firefox

A bit off-topic but uBlock tries to uncloak CNAME before actual HTTP request is fired, right?

uBlock tries to uncloak CNAME before actual HTTP request is fired, right?

Yes.

Enabling Block Outsider Intrusion into LAN list is the solution for this issue.