simple-adblock: racially-neutral names
Signed-off-by: Stan Grishin <stangri@melmac.net>
This commit is contained in:
Stan Grishin
parent
435e5077f8
commit
3167c3c4c0
|
|
@ -6,7 +6,7 @@ include $(TOPDIR)/rules.mk
|
|||
|
||||
PKG_NAME:=simple-adblock
|
||||
PKG_VERSION:=1.8.3
|
||||
PKG_RELEASE:=11
|
||||
PKG_RELEASE:=13
|
||||
PKG_MAINTAINER:=Stan Grishin <stangri@melmac.net>
|
||||
PKG_LICENSE:=GPL-3.0-or-later
|
||||
|
||||
|
|
|
|||
|
|
@ -4,13 +4,13 @@ A simple DNSMASQ/Unbound-based AdBlocking service for OpenWrt/LEDE Project.
|
|||
|
||||
## Features
|
||||
|
||||
- Super-fast due to the nature of supported block lists and parallel downloading/processing of the blacklists.
|
||||
- Super-fast due to the nature of supported block-lists and parallel downloading/processing of the block-lists.
|
||||
- Supports hosts files and domains lists for blocking.
|
||||
- Everything is configurable from Web UI.
|
||||
- Allows you to easily add your own domains to whitelist or blacklist.
|
||||
- Allows you to easily add URLs to your own blocked hosts or domains lists to block/whitelist (just put whitelisted domains one per line in the file you're linking).
|
||||
- Allows you to easily add your own domains to allow-list or block-list.
|
||||
- Allows you to easily add URLs to your own blocked hosts or domains lists to allow/block-list (just put allowed domains one per line in the file you're linking).
|
||||
- Supports multiple modes of AdBlocking implementations with DNSMASQ and Unbound.
|
||||
- Doesn't stay in memory -- creates the list of blocked domains and then uses DNSMASQ/Unbound and firewall rules to serve NXDOMAIN or 127.0.0.1 reply or to reject access (depending on settings) for blocked domains.
|
||||
- Doesn't stay in memory -- creates the list of blocked domains and then uses DNSMASQ/Unbound and firewall rules to serve NXDOMAIN or 127.0.0.1 reply or to reject access (depending on settings) for blocked domains.
|
||||
- As some of the default lists are using https, reliably works with either wget/libopenssl, uclient-fetch/libustream-mbedtls or curl.
|
||||
- Very lightweight and easily hackable, the whole script is just one ```/etc/init.d/simple-adblock``` file.
|
||||
- Retains the downloaded/sorted AdBlocking list on service stop and reuses it on service start (use ```dl``` command if you want to force re-download of the list).
|
||||
|
|
@ -19,7 +19,7 @@ A simple DNSMASQ/Unbound-based AdBlocking service for OpenWrt/LEDE Project.
|
|||
- Blocks ads inside browsers with [DNS-over-HTTPS proxy](https://en.wikipedia.org/wiki/DNS_over_HTTPS) built-in, like [Mozilla Firefox](https://support.mozilla.org/en-US/kb/firefox-dns-over-https#w_about-dns-over-https) or [Google Chrome/Chromium](https://blog.chromium.org/2019/09/experimenting-with-same-provider-dns.html) -- with the ```dnsmasq.ipset``` option.
|
||||
- Proudly made in Canada, using locally-sourced electrons.
|
||||
|
||||
If you want a more robust AdBlocking, supporting free memory detection and complex block lists, supporting IDN, check out ```net/adblock``` on [GitHub](https://github.com/openwrt/packages/tree/master/net/adblock/files)/[jsDelivr](https://cdn.jsdelivr.net/gh/openwrt/packages@master/net/adblock/files/README.md).
|
||||
If you want a more robust AdBlocking, supporting free memory detection and complex block-lists, supporting IDN, check out ```net/adblock``` on [GitHub](https://github.com/openwrt/packages/tree/master/net/adblock/files)/[jsDelivr](https://cdn.jsdelivr.net/gh/openwrt/packages@master/net/adblock/files/README.md).
|
||||
|
||||
## Screenshots (luci-app-simple-adblock)
|
||||
|
||||
|
|
@ -35,9 +35,9 @@ Configuration - Advanced Configuration
|
|||
|
||||
|
||||
|
||||
Whitelist and Blocklist Management
|
||||
Allowed and Blocked Lists Management
|
||||
|
||||
|
||||
|
||||
|
||||
## Requirements
|
||||
|
||||
|
|
@ -51,7 +51,7 @@ opkg install uclient-fetch libustream-mbedtls ca-bundle ca-certificates; fi
|
|||
|
||||
### Requirements for file:// Scheme
|
||||
|
||||
If you want to include some local files as the blocklists or whitelists, you need to install ```curl```:
|
||||
If you want to include some local files as the allow-lists or block-lists, you need to install ```curl```:
|
||||
|
||||
```sh
|
||||
opkg update; opkg install curl;
|
||||
|
|
@ -76,9 +76,9 @@ For IPv6 support additionally install ```ip6tables-mod-nat``` and ```kmod-ipt-na
|
|||
opkg update; opkg install ip6tables-mod-nat kmod-ipt-nat6;
|
||||
```
|
||||
|
||||
### Requirements for Faster Blocklist Processing
|
||||
### Requirements for Faster Block-list Processing
|
||||
|
||||
The ```coreutils-sort``` is an optional, but recommended package as it speeds up sorting and removing duplicates from the merged list dramatically. If opkg complains that it can't install ```coreutils-sort``` because /usr/bin/sort is already provided by busybox, you can run the following command:
|
||||
The ```coreutils-sort``` is an optional, but recommended package as it speeds up sorting and removing duplicates from the merged list dramatically. If opkg complains that it can't install ```coreutils-sort``` because /usr/bin/sort is already provided by busybox, you can run the following command:
|
||||
|
||||
```sh
|
||||
opkg --force-overwrite install coreutils-sort
|
||||
|
|
@ -110,11 +110,11 @@ You can use Web UI (found in Services/Simple AdBlock) to add/remove/edit links t
|
|||
|
||||
- [hosts files](https://en.wikipedia.org/wiki/Hosts_(file)) (127.0.0.1 or 0.0.0.0 followed by space and domain name per line) to be blocked.
|
||||
- domains lists (one domain name per line) to be blocked.
|
||||
- domains lists (one domain name per line) to be whitelisted. It is useful if you want to run ```simple-adblock``` on multiple routers and maintain one centralized whitelist which you can publish on a web-server.
|
||||
- domains lists (one domain name per line) to be allowed. It is useful if you want to run ```simple-adblock``` on multiple routers and maintain one centralized allow-list which you can publish on a web-server.
|
||||
|
||||
Please note that these lists **must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix. Some of the top block lists (both hosts files and domains lists) suitable for routers with at least 8MB RAM are used in the default ```simple-adblock``` installation.
|
||||
Please note that these lists **must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix. Some of the top block-lists (both hosts files and domains lists) suitable for routers with at least 8MB RAM are used in the default ```simple-adblock``` installation.
|
||||
|
||||
You can also use Web UI to add individual domains to be blocked or whitelisted.
|
||||
You can also use Web UI to add individual domains to be blocked or allowed.
|
||||
|
||||
If you want to use CLI to customize ```simple-adblock``` config, refer to the [Customization Settings](#customization-settings) section.
|
||||
|
||||
|
|
@ -146,10 +146,11 @@ In the Web UI the ```simple-adblock``` settings are split into ```basic``` and `
|
|||
|Advanced|debug|boolean|0|If enabled, output service full debug to ```/tmp/simple-adblock.log```. Please note that the debug file may clog up the router's RAM on some devices. Use with caution.|
|
||||
|Advanced|allow_non_ascii|boolean|0|Enable support for non-ASCII characters in the final AdBlocking file. Only enable if your target service supports non-ASCII characters. If you enable this on the system where DNS resolver doesn't support non-ASCII characters, it will crash. Use with caution.|
|
||||
|Advanced|compressed_cache|boolean|0|Create compressed cache of the AdBlocking file in router's persistent memory. Only recommended to be used on routers with large ROM and/or routers with metered/flaky internet connection.|
|
||||
||whitelist_domain|list/string||List of white-listed domains.|
|
||||
||whitelist_domains_url|list/string||List of URL(s) to text files containing white-listed domains. **Must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix. Useful if you want to keep/publish a single white-list for multiple routers.|
|
||||
||blacklist_domains_url|list/string||List of URL(s) to text files containing black-listed domains. **Must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix.|
|
||||
||blacklist_hosts_url|list/string||List of URL(s) to [hosts files](https://en.wikipedia.org/wiki/Hosts_(file)) containing black-listed domains. **Must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix.|
|
||||
||allowed_domain|list/string||List of allowed domains.|
|
||||
||allowed_domains_url|list/string||List of URL(s) to text files containing allowed domains. **Must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix. Useful if you want to keep/publish a single allow-list for multiple routers.|
|
||||
||blocked_domain|list/string||List of blocked domains.|
|
||||
||blocked_domains_url|list/string||List of URL(s) to text files containing blocked domains. **Must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix.|
|
||||
||blocked_hosts_url|list/string||List of URL(s) to [hosts files](https://en.wikipedia.org/wiki/Hosts_(file)) containing block-listed domains. **Must** include either ```http://``` or ```https://``` (or, if ```curl``` is installed the ```file://```) prefix.|
|
||||
|
||||
### DNS Resolution Option
|
||||
|
||||
|
|
@ -165,11 +166,11 @@ Currently supported options are:
|
|||
|
||||
## How Does It Work
|
||||
|
||||
This service downloads (and processes in the background, removing comments and other useless data) lists of hosts and domains to be blocked, combines those lists into one big block list, removes duplicates and sorts it and then removes your whitelisted domains from the block list before converting to to DNSMASQ/Unbound-compatible file and restarting DNSMASQ/Unbound if needed. The result of the process is that DNSMASQ/Unbound return NXDOMAIN or 127.0.0.1 (depending on settings) for the blocked domains.
|
||||
This service downloads (and processes in the background, removing comments and other useless data) lists of hosts and domains to be blocked, combines those lists into one big block-list, removes duplicates and sorts it and then removes your allowed domains from the block-list before converting to to DNSMASQ/Unbound-compatible file and restarting DNSMASQ/Unbound if needed. The result of the process is that DNSMASQ/Unbound return NXDOMAIN or 127.0.0.1 (depending on settings) for the blocked domains.
|
||||
|
||||
If you specify ```google.com``` as a domain to be whitelisted, you will have access to ```google.com```, ```www.google.com```, ```analytics.google.com```, but not fake domains like ```email-google.com``` or ```drive.google.com.verify.signin.normandeassociation.com``` for example. If you only want to allow ```www.google.com``` while blocking all other ```google.com``` subdomains, just specify ```www.google.com``` as domain to be whitelisted.
|
||||
If you specify ```google.com``` as a domain to be allowed, you will have access to ```google.com```, ```www.google.com```, ```analytics.google.com```, but not fake domains like ```email-google.com``` or ```drive.google.com.verify.signin.normandeassociation.com``` for example. If you only want to allow ```www.google.com``` while blocking all other ```google.com``` subdomains, just specify ```www.google.com``` as domain to be allowed.
|
||||
|
||||
In general, whatever domain is specified to be whitelisted; it, along with with its subdomains will be whitelisted, but not any fake domains containing it.
|
||||
In general, whatever domain is specified to be allowed; it, along with with its subdomains will be allowed, but not any fake domains containing it.
|
||||
|
||||
## How It Does Not Work
|
||||
|
||||
|
|
|
|||
|
|
@ -11,73 +11,65 @@ config simple-adblock 'config'
|
|||
option parallel_downloads '1'
|
||||
option debug '0'
|
||||
option compressed_cache '0'
|
||||
list whitelist_domain 'cdn.jsdelivr.net'
|
||||
list allowed_domain 'cdn.jsdelivr.net'
|
||||
|
||||
# Thu Oct 3 17:54:04 PDT 2019
|
||||
# File size: 4.0K
|
||||
list blacklist_domains_url 'https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt'
|
||||
list blocked_domains_url 'https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt'
|
||||
|
||||
# File size: 12.0K
|
||||
list blacklist_domains_url 'https://ssl.bblck.me/blacklists/domain-list.txt'
|
||||
list blocked_domains_url 'https://ssl.bblck.me/blacklists/domain-list.txt'
|
||||
|
||||
# File size: 44.0K
|
||||
list blacklist_domains_url 'https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt'
|
||||
list blocked_domains_url 'https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt'
|
||||
|
||||
# File size: 44.0K
|
||||
list blacklist_domains_url 'https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt'
|
||||
|
||||
# File size: 60.0K
|
||||
# use just one of the www.dshield.org blocklists
|
||||
list blacklist_domains_url 'https://www.dshield.org/feeds/suspiciousdomains_Medium.txt'
|
||||
|
||||
# File size: 64.0K
|
||||
# use just one of the www.dshield.org blocklists
|
||||
# list blacklist_domains_url 'https://www.dshield.org/feeds/suspiciousdomains_Low.txt'
|
||||
list blocked_domains_url 'https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt'
|
||||
|
||||
# File size: 584.0K
|
||||
# blocklist too big for most routers
|
||||
# list blacklist_domains_url 'https://mirror1.malwaredomains.com/files/justdomains'
|
||||
# block-list too big for most routers
|
||||
# list blocked_domains_url 'https://mirror1.malwaredomains.com/files/justdomains'
|
||||
|
||||
# File size: 16.0K
|
||||
list blacklist_hosts_url 'https://adaway.org/hosts.txt'
|
||||
list blocked_hosts_url 'https://adaway.org/hosts.txt'
|
||||
|
||||
# File size: 20.0K
|
||||
list blacklist_hosts_url 'https://cdn.jsdelivr.net/gh/hoshsadiq/adblock-nocoin-list@master/hosts.txt'
|
||||
list blocked_hosts_url 'https://cdn.jsdelivr.net/gh/hoshsadiq/adblock-nocoin-list@master/hosts.txt'
|
||||
|
||||
# File size: 36.0K
|
||||
list blacklist_hosts_url 'https://www.malwaredomainlist.com/hostslist/hosts.txt'
|
||||
list blocked_hosts_url 'https://www.malwaredomainlist.com/hostslist/hosts.txt'
|
||||
|
||||
# File size: 80.0K
|
||||
list blacklist_hosts_url 'https://pgl.yoyo.org/as/serverlist.php?hostformat=hosts&showintro=1&mimetype=plaintext'
|
||||
list blocked_hosts_url 'https://pgl.yoyo.org/as/serverlist.php?hostformat=hosts&showintro=1&mimetype=plaintext'
|
||||
|
||||
# File size: 388.0K
|
||||
# blocklist may be too big for some routers
|
||||
# blocklist may block some video-streaming content
|
||||
# list blacklist_hosts_url 'https://cdn.jsdelivr.net/gh/jawz101/MobileAdTrackers@master/hosts'
|
||||
# block-list may be too big for some routers
|
||||
# block-list may block some video-streaming content
|
||||
# list blocked_hosts_url 'https://cdn.jsdelivr.net/gh/jawz101/MobileAdTrackers@master/hosts'
|
||||
|
||||
# File size: 424.0K
|
||||
# blocklist may be too big for some routers
|
||||
list blacklist_hosts_url 'http://winhelp2002.mvps.org/hosts.txt'
|
||||
# block-list may be too big for some routers
|
||||
list blocked_hosts_url 'http://winhelp2002.mvps.org/hosts.txt'
|
||||
|
||||
# File size: 432.0K
|
||||
# blocklist may be too big for some routers
|
||||
list blacklist_hosts_url 'https://someonewhocares.org/hosts/hosts'
|
||||
# block-list may be too big for some routers
|
||||
list blocked_hosts_url 'https://someonewhocares.org/hosts/hosts'
|
||||
|
||||
# File size: 624.0K
|
||||
# blocklist too big for most routers
|
||||
# list blacklist_hosts_url 'http://sysctl.org/cameleon/hosts'
|
||||
# block-list too big for most routers
|
||||
# list blocked_hosts_url 'http://sysctl.org/cameleon/hosts'
|
||||
|
||||
# File size: 1.6M
|
||||
# blocklist too big for most routers
|
||||
# list blacklist_hosts_url 'https://cdn.jsdelivr.net/gh/StevenBlack/hosts@master/hosts'
|
||||
# block-list too big for most routers
|
||||
# list blocked_hosts_url 'https://cdn.jsdelivr.net/gh/StevenBlack/hosts@master/hosts'
|
||||
|
||||
# File size: 3.1M
|
||||
# blocklist too big for most routers
|
||||
# list blacklist_hosts_url 'https://hostsfile.mine.nu/Hosts'
|
||||
# block-list too big for most routers
|
||||
# list blocked_hosts_url 'https://hostsfile.mine.nu/Hosts'
|
||||
|
||||
# File size: 23.0M
|
||||
# blocklist too big for most routers
|
||||
# list blacklist_hosts_url 'https://hosts.oisd.nl/'
|
||||
# block-list too big for most routers
|
||||
# list blocked_hosts_url 'https://hosts.oisd.nl/'
|
||||
|
||||
# site was down on last check
|
||||
# list blacklist_domains_url 'http://support.it-mate.co.uk/downloads/hosts.txt'
|
||||
# list blocked_domains_url 'http://support.it-mate.co.uk/downloads/hosts.txt'
|
||||
|
|
|
|||
|
|
@ -1,3 +1,8 @@
|
|||
s|whitelist_domains_url|allowed_domains_url|g
|
||||
s|whitelist_domain|allowed_domain|g
|
||||
s|blacklist_hosts_url|blocked_hosts_url|g
|
||||
s|blacklist_domains_url|blocked_domains_url|g
|
||||
s|blacklist_domain|blocked_domain|g
|
||||
s|dbl.oisd.nl|hosts.oisd.nl|g
|
||||
s|raw.githubusercontent.com/StevenBlack/hosts/|cdn.jsdelivr.net/gh/StevenBlack/hosts@|g
|
||||
s|raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/|cdn.jsdelivr.net/gh/hoshsadiq/adblock-nocoin-list@|g
|
||||
|
|
|
|||
|
|
@ -3,12 +3,16 @@
|
|||
# shellcheck disable=SC2039,SC1091
|
||||
PKG_VERSION='dev-test'
|
||||
|
||||
export START=94
|
||||
export USE_PROCD=1
|
||||
export LC_ALL=C
|
||||
# shellcheck disable=SC2034
|
||||
START=94
|
||||
# shellcheck disable=SC2034
|
||||
USE_PROCD=1
|
||||
LC_ALL=C
|
||||
|
||||
export EXTRA_COMMANDS='check dl killcache sizes show'
|
||||
export EXTRA_HELP=' check Checks if specified domain is found in current blacklist
|
||||
# shellcheck disable=SC2034
|
||||
EXTRA_COMMANDS='check dl killcache sizes show'
|
||||
# shellcheck disable=SC2034
|
||||
EXTRA_HELP=' check Checks if specified domain is found in current block-list
|
||||
dl Force-downloads all enabled block-list
|
||||
sizes Displays the file-sizes of enabled block-lists
|
||||
show Shows the service last-run status'
|
||||
|
|
@ -44,23 +48,12 @@ readonly sharedMemoryError="/dev/shm/$packageName-error"
|
|||
readonly sharedMemoryOutput="/dev/shm/$packageName-output"
|
||||
readonly hostsFilter='/localhost/d;/^#/d;/^[^0-9]/d;s/^0\.0\.0\.0.//;s/^127\.0\.0\.1.//;s/[[:space:]]*#.*$//;s/[[:cntrl:]]$//;s/[[:space:]]//g;/[`~!@#\$%\^&\*()=+;:"'\'',<>?/\|[{}]/d;/]/d;/\./!d;/^$/d;/[^[:alnum:]_.-]/d;'
|
||||
readonly domainsFilter='/^#/d;s/[[:space:]]*#.*$//;s/[[:space:]]*$//;s/[[:cntrl:]]$//;/[[:space:]]/d;/[`~!@#\$%\^&\*()=+;:"'\'',<>?/\|[{}]/d;/]/d;/\./!d;/^$/d;/[^[:alnum:]_.-]/d;'
|
||||
readonly checkmark='\xe2\x9c\x93'
|
||||
readonly xmark='\xe2\x9c\x97'
|
||||
readonly _OK_='\033[0;32m\xe2\x9c\x93\033[0m'
|
||||
readonly _FAIL_='\033[0;31m\xe2\x9c\x97\033[0m'
|
||||
readonly __OK__='\033[0;32m[\xe2\x9c\x93]\033[0m'
|
||||
readonly __FAIL__='\033[0;31m[\xe2\x9c\x97]\033[0m'
|
||||
readonly _ERROR_='\033[0;31mERROR\033[0m'
|
||||
|
||||
readonly messageSuccess='Success'
|
||||
readonly messageFail='Fail'
|
||||
readonly messageDownloading='Downloading'
|
||||
readonly messageReloading='Reloading'
|
||||
readonly messageRestarting='Restarting'
|
||||
readonly messageStarting='Starting'
|
||||
readonly messageForceReloading='Force-Reloading'
|
||||
readonly messageProcessing='Processing'
|
||||
readonly messageStopped='Stopped'
|
||||
getStatusText() {
|
||||
local _ret
|
||||
case "$1" in
|
||||
|
|
@ -70,6 +63,7 @@ getStatusText() {
|
|||
statusRestarting) _ret="Restarting";;
|
||||
statusForceReloading) _ret="Force Reloading";;
|
||||
statusDownloading) _ret="Downloading";;
|
||||
statusProcessing) _ret="Processing";;
|
||||
statusError) _ret="Error";;
|
||||
statusWarning) _ret="Warning";;
|
||||
statusFail) _ret="Fail";;
|
||||
|
|
@ -85,14 +79,14 @@ getErrorText() {
|
|||
errorSharedMemory) _ret="failed to access shared memory";;
|
||||
errorSorting) _ret="failed to sort data file";;
|
||||
errorOptimization) _ret="failed to optimize data file";;
|
||||
errorWhitelistProcessing) _ret="failed to process whitelist";;
|
||||
errorAllowListProcessing) _ret="failed to process allow-list";;
|
||||
errorDataFileFormatting) _ret="failed to format data file";;
|
||||
errorMovingDataFile) _ret="failed to move data file '${A_TMP}' to '${outputFile}'";;
|
||||
errorCreatingCompressedCache) _ret="failed to create compressed cache";;
|
||||
errorRemovingTempFiles) _ret="failed to remove temporary files";;
|
||||
errorRestoreCompressedCache) _ret="failed to unpack compressed cache";;
|
||||
errorRestoreCache) _ret="failed to move '$outputCache' to '$outputFile'";;
|
||||
errorOhSnap) _ret="failed to create blocklist or restart DNS resolver";;
|
||||
errorOhSnap) _ret="failed to create block-list or restart DNS resolver";;
|
||||
errorStopping) _ret="failed to stop $serviceName";;
|
||||
errorDNSReload) _ret="failed to reload/restart DNS resolver";;
|
||||
errorDownloadingList) _ret="failed to download";;
|
||||
|
|
@ -108,8 +102,8 @@ output_ok() { output 1 "$_OK_"; output 2 "$__OK__\\n"; }
|
|||
output_okn() { output 1 "$_OK_\\n"; output 2 "$__OK__\\n"; }
|
||||
output_fail() { output 1 "$_FAIL_"; output 2 "$__FAIL__\\n"; }
|
||||
output_failn() { output 1 "$_FAIL_\\n"; output 2 "$__FAIL__\\n"; }
|
||||
# str_replace() { printf "%b" "$1" | sed -e "s/$(printf "%b" "$2")/$(printf "%b" "$3")/g"; }
|
||||
# str_contains() { test "$1" != "$(str_replace "$1" "$2" '')"; }
|
||||
str_replace() { printf "%b" "$1" | sed -e "s/$(printf "%b" "$2")/$(printf "%b" "$3")/g"; }
|
||||
str_contains() { test "$1" != "$(str_replace "$1" "$2" '')"; }
|
||||
compare_versions() { test "$(printf '%s\n' "$@" | sort -V | head -n 1)" != "$1"; }
|
||||
is_chaos_calmer() { ubus -S call system board | grep -q 'Chaos Calmer'; }
|
||||
is_ipset_procd() { compare_versions "$(sed -ne 's/^Version: //p' /usr/lib/opkg/info/firewall.control)" "2019-09-18"; }
|
||||
|
|
@ -139,13 +133,33 @@ output() {
|
|||
fi
|
||||
}
|
||||
|
||||
export serviceEnabled forceDNS parallelDL debug allowIDN compressedCache
|
||||
export targetDNS bootDelay dlTimeout curlRetry verbosity=1 led dnsInstance
|
||||
export whitelist_domains blacklist_domains
|
||||
export whitelist_domains_urls blacklist_domains_urls blacklist_hosts_urls
|
||||
export wan_if wan_gw wanphysdev dl_command serviceStatus dl_flag
|
||||
export outputFilter outputFilterIPv6 outputFile outputGzip outputCache ipv6Enabled
|
||||
export is_ssl_supported
|
||||
serviceEnabled=1
|
||||
forceDNS=1
|
||||
parallelDL=1
|
||||
debug=0
|
||||
compressedCache=0
|
||||
ipv6Enabled=0
|
||||
bootDelay=120
|
||||
dlTimeout=20
|
||||
curlRetry=3
|
||||
verbosity=2
|
||||
led=''
|
||||
targetDNS=dnsmasq.servers
|
||||
dnsInstance=0
|
||||
allowed_domains=''
|
||||
blocked_domains=''
|
||||
allowed_domains_urls=''
|
||||
blocked_domains_urls=''
|
||||
blocked_hosts_urls=''
|
||||
dl_command=''
|
||||
dl_flag=''
|
||||
outputFilter=''
|
||||
outputFilterIPv6=''
|
||||
outputFile=''
|
||||
outputGzip=''
|
||||
outputCache=''
|
||||
isSSLSupported=''
|
||||
allowIDN=0
|
||||
|
||||
load_package_config() {
|
||||
config_load "$packageName"
|
||||
|
|
@ -162,11 +176,11 @@ load_package_config() {
|
|||
config_get led 'config' 'led'
|
||||
config_get targetDNS 'config' 'dns' 'dnsmasq.servers'
|
||||
config_get dnsInstance 'config' 'dns_instance' '0'
|
||||
config_get whitelist_domains 'config' 'whitelist_domain'
|
||||
config_get blacklist_domains 'config' 'blacklist_domain'
|
||||
config_get whitelist_domains_urls 'config' 'whitelist_domains_url'
|
||||
config_get blacklist_domains_urls 'config' 'blacklist_domains_url'
|
||||
config_get blacklist_hosts_urls 'config' 'blacklist_hosts_url'
|
||||
config_get allowed_domains 'config' 'allowed_domain'
|
||||
config_get allowed_domains_urls 'config' 'allowed_domains_url'
|
||||
config_get blocked_domains 'config' 'blocked_domain'
|
||||
config_get blocked_domains_urls 'config' 'blocked_domains_url'
|
||||
config_get blocked_hosts_urls 'config' 'blocked_hosts_url'
|
||||
|
||||
if [ "$targetDNS" != 'dnsmasq.addnhosts' ] && [ "$targetDNS" != 'dnsmasq.conf' ] && \
|
||||
[ "$targetDNS" != 'dnsmasq.servers' ] && [ "$targetDNS" != 'unbound.adb_list' ] && \
|
||||
|
|
@ -249,13 +263,14 @@ load_package_config() {
|
|||
|| grep -q "libustream-mbedtls" /usr/lib/opkg/status \
|
||||
|| grep -q "libustream-openssl" /usr/lib/opkg/status \
|
||||
|| grep -q "libustream-wolfssl" /usr/lib/opkg/status; then
|
||||
is_ssl_supported=1
|
||||
isSSLSupported=1
|
||||
else
|
||||
unset is_ssl_supported
|
||||
unset isSSLSupported
|
||||
fi
|
||||
}
|
||||
|
||||
is_enabled() {
|
||||
wan_if=''; wan_gw='';
|
||||
load_package_config
|
||||
|
||||
if [ "$debug" -ne 0 ]; then
|
||||
|
|
@ -457,7 +472,7 @@ tmpfs() {
|
|||
stats)
|
||||
printf "%b" "$stats"; return;;
|
||||
triggers)
|
||||
curReload="$parallelDL $debug $dlTimeout $whitelist_domains $blacklist_domains $whitelist_domains_urls $blacklist_domains_urls $blacklist_hosts_urls $targetDNS"
|
||||
curReload="$parallelDL $debug $dlTimeout $allowed_domains $blocked_domains $allowed_domains_urls $blocked_domains_urls $blocked_hosts_urls $targetDNS"
|
||||
curRestart="$compressedCache $forceDNS $led"
|
||||
if [ ! -s "$jsonFile" ]; then
|
||||
ret='on_boot'
|
||||
|
|
@ -513,7 +528,7 @@ tmpfs() {
|
|||
stats)
|
||||
stats="$value";;
|
||||
triggers)
|
||||
readReload="$parallelDL $debug $dlTimeout $whitelist_domains $blacklist_domains $whitelist_domains_urls $blacklist_domains_urls $blacklist_hosts_urls $targetDNS"
|
||||
readReload="$parallelDL $debug $dlTimeout $allowed_domains $blocked_domains $allowed_domains_urls $blocked_domains_urls $blocked_hosts_urls $targetDNS"
|
||||
readRestart="$compressedCache $forceDNS $led"
|
||||
;;
|
||||
esac
|
||||
|
|
@ -678,7 +693,7 @@ process_url() {
|
|||
else
|
||||
type='Allowed'; D_TMP="$A_TMP";
|
||||
fi
|
||||
if [ "${1:0:5}" == "https" ] && [ -z "$is_ssl_supported" ]; then
|
||||
if [ "${1:0:5}" == "https" ] && [ -z "$isSSLSupported" ]; then
|
||||
output 1 "$_FAIL_"
|
||||
output 2 "[DL] $type $label $__FAIL__\\n"
|
||||
echo "errorNoSSLSupport|${1}" >> "$sharedMemoryError"
|
||||
|
|
@ -710,7 +725,7 @@ process_url() {
|
|||
download_lists() {
|
||||
local hf w_filter j=0 R_TMP
|
||||
|
||||
tmpfs set message "${messageDownloading}..."
|
||||
tmpfs set message "$(getStatusText "statusDownloading")..."
|
||||
tmpfs set status "statusDownloading"
|
||||
|
||||
rm -f "$A_TMP" "$B_TMP" "$outputFile" "$outputCache" "$outputGzip"
|
||||
|
|
@ -725,8 +740,8 @@ download_lists() {
|
|||
touch $A_TMP; touch $B_TMP;
|
||||
output 1 'Downloading lists '
|
||||
rm -f "$sharedMemoryError"
|
||||
if [ -n "$blacklist_hosts_urls" ]; then
|
||||
for hf in ${blacklist_hosts_urls}; do
|
||||
if [ -n "$blocked_hosts_urls" ]; then
|
||||
for hf in ${blocked_hosts_urls}; do
|
||||
if [ "$parallelDL" -gt 0 ]; then
|
||||
process_url "$hf" 'hosts' 'blocked' &
|
||||
else
|
||||
|
|
@ -734,8 +749,8 @@ download_lists() {
|
|||
fi
|
||||
done
|
||||
fi
|
||||
if [ -n "$blacklist_domains_urls" ]; then
|
||||
for hf in ${blacklist_domains_urls}; do
|
||||
if [ -n "$blocked_domains_urls" ]; then
|
||||
for hf in ${blocked_domains_urls}; do
|
||||
if [ "$parallelDL" -gt 0 ]; then
|
||||
process_url "$hf" 'domains' 'blocked' &
|
||||
else
|
||||
|
|
@ -743,8 +758,8 @@ download_lists() {
|
|||
fi
|
||||
done
|
||||
fi
|
||||
if [ -n "$whitelist_domains_urls" ]; then
|
||||
for hf in ${whitelist_domains_urls}; do
|
||||
if [ -n "$allowed_domains_urls" ]; then
|
||||
for hf in ${allowed_domains_urls}; do
|
||||
if [ "$parallelDL" -gt 0 ]; then
|
||||
process_url "$hf" 'domains' 'allowed' &
|
||||
else
|
||||
|
|
@ -761,16 +776,16 @@ download_lists() {
|
|||
rm -f "$sharedMemoryError"
|
||||
fi
|
||||
|
||||
[ -n "$blacklist_domains" ] && for hf in ${blacklist_domains}; do echo "$hf" | sed "$domainsFilter" >> $B_TMP; done
|
||||
whitelist_domains="${whitelist_domains}
|
||||
[ -n "$blocked_domains" ] && for hf in ${blocked_domains}; do echo "$hf" | sed "$domainsFilter" >> $B_TMP; done
|
||||
allowed_domains="${allowed_domains}
|
||||
$(cat $A_TMP)"
|
||||
[ -n "$whitelist_domains" ] && for hf in ${whitelist_domains}; do hf="$(echo "$hf" | sed 's/\./\\./g')"; w_filter="$w_filter/^${hf}$/d;/\\.${hf}$/d;"; done
|
||||
[ -n "$allowed_domains" ] && for hf in ${allowed_domains}; do hf="$(echo "$hf" | sed 's/\./\\./g')"; w_filter="$w_filter/^${hf}$/d;/\\.${hf}$/d;"; done
|
||||
|
||||
[ ! -s "$B_TMP" ] && return 1
|
||||
|
||||
output 1 'Processing downloads '
|
||||
output 2 'Sorting combined list '
|
||||
tmpfs set message "$messageProcessing: sorting combined list"
|
||||
tmpfs set message "$(getStatusText "statusProcessing"): sorting combined list"
|
||||
if [ "$allowIDN" -gt 0 ]; then
|
||||
if sort -u "$B_TMP" > "$A_TMP"; then
|
||||
output_ok
|
||||
|
|
@ -793,7 +808,7 @@ $(cat $A_TMP)"
|
|||
[ "$targetDNS" = 'unbound.adb_list' ]; then
|
||||
# TLD optimization written by Dirk Brenken (dev@brenken.org)
|
||||
output 2 'Optimizing combined list '
|
||||
tmpfs set message "$messageProcessing: optimizing combined list"
|
||||
tmpfs set message "$(getStatusText "statusProcessing"): optimizing combined list"
|
||||
# sed -E 'G;:t;s/(.*)(\.)(.*)(\n)(.*)/\1\4\5\2\3/;tt;s/(.*)\n(\.)(.*)/\3\2\1/' is actually slower than awk
|
||||
if awk -F "." '{for(f=NF;f>1;f--)printf "%s.",$f;print $1}' "$A_TMP" > "$B_TMP"; then
|
||||
if sort "$B_TMP" > "$A_TMP"; then
|
||||
|
|
@ -828,17 +843,17 @@ $(cat $A_TMP)"
|
|||
mv "$A_TMP" "$B_TMP"
|
||||
fi
|
||||
|
||||
output 2 'Whitelisting domains '
|
||||
tmpfs set message "$messageProcessing: whitelisting domains"
|
||||
output 2 'Allowing domains '
|
||||
tmpfs set message "$(getStatusText "statusProcessing"): allowing domains"
|
||||
if sed -i "$w_filter" "$B_TMP"; then
|
||||
output_ok
|
||||
else
|
||||
output_failn
|
||||
tmpfs add error "errorWhitelistProcessing"
|
||||
tmpfs add error "errorAllowListProcessing"
|
||||
fi
|
||||
|
||||
output 2 'Formatting merged file '
|
||||
tmpfs set message "$messageProcessing: formatting merged file"
|
||||
tmpfs set message "$(getStatusText "statusProcessing"): formatting merged file"
|
||||
if [ -z "$outputFilterIPv6" ]; then
|
||||
if sed "$outputFilter" "$B_TMP" > "$A_TMP"; then
|
||||
output_ok
|
||||
|
|
@ -863,23 +878,23 @@ $(cat $A_TMP)"
|
|||
case "$targetDNS" in
|
||||
dnsmasq.addnhosts)
|
||||
output 2 'Creating DNSMASQ addnhosts file '
|
||||
tmpfs set message "$messageProcessing: creating DNSMASQ addnhosts file"
|
||||
tmpfs set message "$(getStatusText "statusProcessing"): creating DNSMASQ addnhosts file"
|
||||
;;
|
||||
dnsmasq.conf)
|
||||
output 2 'Creating DNSMASQ config file '
|
||||
tmpfs set message "$messageProcessing: creating DNSMASQ config file"
|
||||
tmpfs set message "$(getStatusText "statusProcessing"): creating DNSMASQ config file"
|
||||
;;
|
||||
dnsmasq.ipset)
|
||||
output 2 'Creating DNSMASQ ipset file '
|
||||
tmpfs set message "$messageProcessing: creating DNSMASQ ipset file"
|
||||
tmpfs set message "$(getStatusText "statusProcessing"): creating DNSMASQ ipset file"
|
||||
;;
|
||||
dnsmasq.servers)
|
||||
output 2 'Creating DNSMASQ servers file '
|
||||
tmpfs set message "$messageProcessing: creating DNSMASQ servers file"
|
||||
tmpfs set message "$(getStatusText "statusProcessing"): creating DNSMASQ servers file"
|
||||
;;
|
||||
unbound.adb_list)
|
||||
output 2 'Creating Unbound adb_list file '
|
||||
tmpfs set message "$messageProcessing: creating Unbound adb_list file"
|
||||
tmpfs set message "$(getStatusText "statusProcessing"): creating Unbound adb_list file"
|
||||
;;
|
||||
esac
|
||||
if mv "$A_TMP" "$outputFile"; then
|
||||
|
|
@ -890,7 +905,7 @@ $(cat $A_TMP)"
|
|||
fi
|
||||
if [ "$compressedCache" -gt 0 ]; then
|
||||
output 2 'Creating compressed cache '
|
||||
tmpfs set message "$messageProcessing: creating compressed cache"
|
||||
tmpfs set message "$(getStatusText "statusProcessing"): creating compressed cache"
|
||||
if cacheOps 'createGzip'; then
|
||||
output_ok
|
||||
else
|
||||
|
|
@ -901,7 +916,7 @@ $(cat $A_TMP)"
|
|||
rm -f "$outputGzip"
|
||||
fi
|
||||
output 2 'Removing temporary files '
|
||||
tmpfs set message "$messageProcessing: removing temporary files"
|
||||
tmpfs set message "$(getStatusText "statusProcessing"): removing temporary files"
|
||||
rm -f "/tmp/${packageName}_tmp.*" "$A_TMP" "$B_TMP" "$outputCache" || j=1
|
||||
if [ $j -eq 0 ]; then
|
||||
output_ok
|
||||
|
|
@ -1173,7 +1188,7 @@ check() {
|
|||
local string="$1"
|
||||
local c="$(grep -c "$string" "$outputFile")"
|
||||
if [ ! -s "$outputFile" ]; then
|
||||
echo "No blacklist ('$outputFile') found."
|
||||
echo "No block-list ('$outputFile') found."
|
||||
elif [ -z "$string" ]; then
|
||||
echo "Usage: /etc/init.d/${packageName} check string"
|
||||
elif [ "$c" -gt 0 ]; then
|
||||
|
|
@ -1195,7 +1210,7 @@ check() {
|
|||
grep "$string" "$outputFile" | sed 's|^local-zone: "||;s|" static$||;';;
|
||||
esac
|
||||
else
|
||||
echo "The $string is not found in current blacklist ('$outputFile')."
|
||||
echo "The $string is not found in current block-list ('$outputFile')."
|
||||
fi
|
||||
}
|
||||
|
||||
|
|
@ -1204,39 +1219,39 @@ sizes() {
|
|||
load_package_config
|
||||
echo "# $(date)"
|
||||
|
||||
for i in $blacklist_domains_urls; do
|
||||
for i in $blocked_domains_urls; do
|
||||
[ "${i//melmac}" != "$i" ] && continue
|
||||
if $dl_command "$i" $dl_flag /tmp/sast 2>/dev/null && [ -s /tmp/sast ]; then
|
||||
echo "# File size: $(du -sh /tmp/sast | awk '{print $1}')"
|
||||
if compare_versions "$(du -sk /tmp/sast)" "500"; then
|
||||
echo "# blocklist too big for most routers"
|
||||
echo "# block-list too big for most routers"
|
||||
elif compare_versions "$(du -sk /tmp/sast)" "100"; then
|
||||
echo "# blocklist may be too big for some routers"
|
||||
echo "# block-list may be too big for some routers"
|
||||
fi
|
||||
rm -rf /tmp/sast
|
||||
echo " list blacklist_domains_url '$i'"
|
||||
echo " list blocked_domains_url '$i'"
|
||||
echo ""
|
||||
else
|
||||
echo "# site was down on last check"
|
||||
echo "# list blacklist_domains_url '$i'"
|
||||
echo "# list blocked_domains_url '$i'"
|
||||
echo ""
|
||||
fi
|
||||
done
|
||||
|
||||
for i in $blacklist_hosts_urls; do
|
||||
for i in $blocked_hosts_urls; do
|
||||
if $dl_command "$i" $dl_flag /tmp/sast 2>/dev/null && [ -s /tmp/sast ]; then
|
||||
echo "# File size: $(du -sh /tmp/sast | awk '{print $1}')"
|
||||
if compare_versions "$(du -sk /tmp/sast)" "500"; then
|
||||
echo "# blocklist too big for most routers"
|
||||
echo "# block-list too big for most routers"
|
||||
elif compare_versions "$(du -sk /tmp/sast)" "100"; then
|
||||
echo "# blocklist may be too big for some routers"
|
||||
echo "# block-list may be too big for some routers"
|
||||
fi
|
||||
rm -rf /tmp/sast
|
||||
echo " list blacklist_hosts_url '$i'"
|
||||
echo " list blocked_hosts_url '$i'"
|
||||
echo ""
|
||||
else
|
||||
echo "# site was down on last check"
|
||||
echo "# list blacklist_hosts_url '$i'"
|
||||
echo "# list blocked_hosts_url '$i'"
|
||||
echo ""
|
||||
fi
|
||||
done
|
||||
|
|
|
|||
Loading…
Reference in New Issue