Ubuntu 18.04LTS 下编译安装最新版wget2

听说wget2增加了多线程,所以来编译安装使用下。
https://lists.gnu.org/archive/html/info-gnu/2021-09/msg00011.html

GNU Wget2 2.0.0 released
From:   Tim Rühsen
Subject:    GNU Wget2 2.0.0 released
Date:   Sun, 26 Sep 2021 12:40:37 +0200
User-agent:     Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0

Hi,

we are happy to announce the release 2.0.0 of GNU Wget2.

Wget2 is the successor of GNU Wget, a file and recursive website
downloader.

Designed and written from scratch it wraps around libwget, that provides
the basic functions needed by a web client.

Wget2 works multi-threaded and uses "modern" features to allow fast operation.

In many cases Wget2 downloads much faster than Wget due to HTTP2,

HTTP compression, parallel connections, use of If-Modified-Since HTTP header and more.

Wget2 has several new command-line options, see the wiki page for a list and comparison with Wget.

Wget will be maintained further. The idea is that breaking changes and new functionalities go into Wget2 / libwget.

Except for WARC and FTP, Wget2 is a drop-in replacement for Wget in most cases. Of course there may be subtle differences, so make sure to test well before replacing Wget by Wget2.

GNU Wget2 is licensed under GPLv3+. Libwget is licensed under LGPLv3+.

**Noteworthy changes since the last release (see also the NEWS file):**

  * OpenSSL: Fix CRL checking
  * OpenSSL: Implement ALPN
  * OpenSSL: Fix memory leaks
  * Update license information
  * Fix for reproducible builds
  * Add Accept-Encoding: lzip
  * Rename --retry-on-http-status to --retry-on-http-error
  * Limit to page requisites only for leaf pages
  * Allow list of tokens for Connection: header
  * Fix directory clash with --no-clobber
  * Fix NULL dereference with --convert-links
  * Using --robots=off downloads robots.txt
  * Add pkg-config support for GPGME
  * Fix conversion (-k) in combination with -E
  * Fix cookie file header to be recognized by 'file' command
  * Fix loading CA certs when 'system' is not supported
  * Add option --method, improving backward compatibility
  * Add option --body-data, improving backward compatibility
  * Add option --body-file, improving backward compatibility
  * Add option --ignore-length, improving backward compatibility
  * Add option --convert-file-only, improving backward compatibility

* Add option --download-attr to make use of the HTML5 'download' attribute

  * Support terminal hyperlinks in output
  * Configure switch --disable-manylibs to disable building small libraries
  * Support --background on Windows
  * Add option --bind-interface
  * Add HTTP2 uploading
  * Support the HTML download attribute (for a and area tags)

* Add option --download-attr=[strippath|usepath] to control download attribute support

  * OpenSSL: Add OCSP support
  * OpenSSL: Implement OCSP stapling
  * Support data: URL in srcset attribute
  * Fixed diverse issues
  * Improved code, documentation, building, testing, CI, ...

**Download**

  https://ftp.gnu.org/gnu/wget/wget2-2.0.0.tar.gz

  https://ftp.gnu.org/gnu/wget/wget2-2.0.0.tar.lz

**Signatures (GPG key 0x08302DB6A2670428)**

  https://ftp.gnu.org/gnu/wget/wget2-2.0.0.tar.gz.sig

  https://ftp.gnu.org/gnu/wget/wget2-2.0.0.tar.lz.sig

**Project Website**

  https://gitlab.com/gnuwget/wget2

**Online Documentation**

  https://gnuwget.gitlab.io/wget2/reference/

**Wiki**

  https://gitlab.com/gnuwget/wget2/-/wikis/Home

**Mailing List**

  https://savannah.gnu.org/mail/?group=wget

**Bug Tracker**

  https://gitlab.com/gnuwget/wget2/issues

**Getting the Source Code (Development)**

  git clone https://gitlab.com/gnuwget/wget2.git

**Source Code Test Coverage**

  https://gnuwget.gitlab.io/wget2/coverage/

**Fuzz Code Coverage**

  https://gnuwget.gitlab.io/wget2/fuzz-coverage/

With Best Regards, the maintainers.

0.安装依赖

apt update && apt install -y libzstd-dev lzip libpsl-dev libnghttp2-14 libnghttp2-dev

1.下载源码

wget https://gnuwget.gitlab.io/wget2/wget2-latest.tar.gz

2.解压

tar -xzvf wget2-latest.tar.gz

3.进入目录

cd wget2-2.0.0/

4.配置&检查依赖

./configure

5.编译

make

6.测试

make check

结果显示通过

PASS: test
PASS: test-parse-html
PASS: test-cond
PASS: test-decompress
PASS: test-dl
============================================================================
Testsuite summary for wget2 2.0.0
============================================================================
# TOTAL: 5
# PASS:  5
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================

6.安装

make install

7.查看版本

wget2 -V
GNU Wget2 2.0.0 - multithreaded metalink/file/website downloader

+digest +https +ssl/gnutls +ipv6 +iri +large-file +nls -ntlm -opie +psl -hsts
+iconv +idn2 +zlib +lzma +brotlidec +zstd +bzip2 -lzip +http2 +gpgme

Copyright (C) 2012-2015 Tim Ruehsen
Copyright (C) 2015-2021 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later
<http://www.gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Please send bug reports and questions to <[email protected]>.

8.使用参数

GNU Wget2 V2.0.0 - multithreaded metalink/file/website downloader

Usage: wget [options...] <url>...

Startup:
  -a  --append-output       File where messages are appended to, '-' for STDOUT
  -b  --background          Go to background immediately after startup. If no
                              output file is specified via the -o, output is redirected to wget-log
  -B  --base                Base for relative URLs read from input-file
                              or from command line
      --config              Path to initialization file (default: ~/.config/wget/wget2rc)
  -d  --debug               Print debugging messages.(default: off)
  -e  --execute             Wget compatibility option, not needed for Wget
      --force-atom          Treat input file as Atom Feed. (default: off) (NEW!)
      --force-css           Treat input file as CSS. (default: off) (NEW!)
  -F  --force-html          Treat input file as HTML. (default: off)
      --force-metalink      Treat input file as Metalink. (default: off) (NEW!)
      --force-rss           Treat input file as RSS Feed. (default: off) (NEW!)
      --force-sitemap       Treat input file as Sitemap. (default: off) (NEW!)
      --fsync-policy        Use fsync() to wait for data being written to
                              the pysical layer. (default: off) (NEW!)
  -h  --help                Print this help.
      --hyperlink           Enable terminal hyperlink support
      --input-encoding      Character encoding of the file contents read with
                              --input-file. (default: local encoding)
  -i  --input-file          File where URLs are read from, - for STDIN.
      --local-db            Read or load databases
  -o  --output-file         File where messages are printed to,
                              '-' for STDOUT.
  -q  --quiet               Print no messages except debugging messages.
                              (default: off)
      --stats-dns           Print DNS stats. (default: off)
                              Additional format supported:
                              --stats-dns=[FORMAT:]FILE
      --stats-ocsp          Print OCSP stats. (default: off)
                              Additional format supported:
                              --stats-ocsp=[FORMAT:]FILE
      --stats-server        Print server stats. (default: off)
                              Additional format supported:
                              --stats-server=[FORMAT:]FILE
      --stats-site          Print site stats. (default: off)
                              Additional format supported:
                              --stats-site=[FORMAT:]FILE
      --stats-tls           Print TLS stats. (default: off)
                              Additional format supported:
                              --stats-tls=[FORMAT:]FILE
      --unlink              Remove files before clobbering. (default: off)
  -v  --verbose             Print more messages. (default: on)
  -V  --version             Display the version of Wget and exit.

Download:
  -A  --accept              Comma-separated list of file name suffixes or
                              patterns.
      --accept-regex        Regex matching accepted URLs.
      --ask-password        Print prompt for password
      --backups             Make backups instead of overwriting/increasing
                              number. (default: 0)
      --bind-address        Bind to sockets to local address.
                              (default: automatic)
      --bind-interface      Bind sockets to the input Network Interface.
                              (default: automatic)
      --body-data           Data to be sent in a request.
      --body-file           File with data to be sent in a request.
      --cache               Enabled using of server cache. (default: on)
      --chunk-size          Download large files in multithreaded chunks.
                              (default: 0 (=off)) Example:
                              wget --chunk-size=1M
      --clobber             Enable file clobbering. (default: on)
      --connect-timeout     Connect timeout in seconds.
      --content-on-error    Save response body even on error status.
                              (default: off)
  -c  --continue            Continue download for given files. (default: off)
      --convert-file-only   Convert only filename part of embedded URLs.
                              (default: off)
  -k  --convert-links       Convert embedded URLs to local URLs.
                              (default: off)
      --cut-file-get-vars   Cut HTTP GET vars from file names. (default: off)
      --cut-url-get-vars    Cut HTTP GET vars from URLs. (default: off)
      --delete-after        Don't save downloaded files. (default: off)
      --dns-cache           Caching of domain name lookups. (default: on)
      --dns-cache-preload   File to be used to preload the DNS cache.
                              Format is like /etc/hosts (IP<whitespace>hostname).
      --dns-timeout         DNS lookup timeout in seconds.
  -D  --domains             Comma-separated list of domains to follow.
      --download-attr       Recognize HTML5 download attributes.
                              'strippath' strips the path to be more secure.
'usepath' uses the path as is (this can be extremely dangerous !).
(default: strippath)
  -X  --exclude-directories
                            Comma-separated list of directories NOT to download.
                              Wildcards are allowed.
      --exclude-domains     Comma-separated list of domains NOT to follow.
      --filter-mime-type    Specify a list of mime types to be saved or ignored
      --filter-urls         Apply the accept and reject filters on the URL
                              before starting a download. (default: off)
      --follow-tags         Scan additional tag/attributes for URLs,
                              e.g. --follow-tags="img/data-500px,img/data-hires
      --force-progress      Force progress bar.
                              (default: off)
      --http2-request-window
                            Max. number of parallel streams per HTTP/2
                              connection. (default: 30)
      --if-modified-since   Do not send If-Modified-Since header in -N mode.
Send preliminary HEAD request instead. This has only
                              effect in -N mode.
      --ignore-case         Ignore case when matching files. (default: off)
      --ignore-length       Ignore content-length header field
                              (default: off)
      --ignore-tags         Ignore tag/attributes for URL scanning,
                              e.g. --ignore-tags="img,a/href
  -I  --include-directories
                            Comma-separated list of directories TO download.
                              Wildcards are allowed.
  -4  --inet4-only          Use IPv4 connections only. (default: off)
  -6  --inet6-only          Use IPv6 connections only. (default: off)
      --iri                 Wget dummy option, you can't switch off
                              international support
      --keep-extension      If file exists: Use pattern 'basename_N.ext'
                              instead of 'filename.N'. (default: off)
  -l  --level               Maximum recursion depth. (default: 5)
      --local-encoding      Character encoding of environment and filenames.
      --max-redirect        Max. number of redirections to follow.
                              (default: 20)
      --max-threads         Max. concurrent download threads.
                              (default: 5) (NEW!)
  -m  --mirror              Turn on mirroring options -r -N -l inf
      --netrc               Load credentials from ~/.netrc if not given.
                              (default: on)
  -O  --output-document     File where downloaded content is written to,
                              '-O'  for STDOUT.
  -p  --page-requisites     Download all necessary files to display a
                              HTML page
      --parent              Ascend above parent directory. (default: on)
      --password            Password for Authentication.
                              (default: empty password)
      --post-data           Data to be sent in a POST request.
      --post-file           File with data to be sent in a POST request.
      --prefer-family       Prefer IPv4 or IPv6. (default: none)
      --progress            Type of progress bar (bar, none).
                              (default: none)
      --proxy               Enable support for *_proxy environment variables.
                              (default: on)
      --random-wait         Wait 0.5 up to 1.5*<--wait> seconds between
                              downloads (per thread). (default: off)
      --read-timeout        Read and write timeout in seconds.
  -r  --recursive           Recursive download. (default: off)
      --regex-type          Regular expression type. Possible types are
                              posix or pcre. (default: posix)
  -R  --reject              Comma-separated list of file name suffixes or
                              patterns.
      --reject-regex        Regex matching rejected URLs.
      --remote-encoding     Character encoding of remote files
                              (if not specified in Content-Type HTTP header
                              or in document itself)
      --report-speed        Output bandwidth as TYPE. TYPE can be bytes
                              or bits. --progress MUST be used.
      --restrict-file-names
                            unix, windows, nocontrol, ascii, lowercase,
                              uppercase, none
      --retry-on-http-error
                            Specify a list of http statuses in which the download will be retried
      --robots              Respect robots.txt standard for recursive
                              downloads. (default: on)
      --save-content-on     Specify a list of response codes that requires it's
                              response body to be saved on error status
  -S  --server-response     Print the server response headers. (default: off)
  -H  --span-hosts          Span hosts that were not given on the
                              command line. (default: off)
      --spider              Enable web spider mode. (default: off)
      --start-pos           Start downloading at zero-based position, 0 = option disabled. (default: 0)
      --strict-comments     A dummy option. Parsing always works non-strict.
      --tcp-fastopen        Enable TCP Fast Open (TFO). (default: on)
  -T  --timeout             General network timeout in seconds.
  -N  --timestamping        Just retrieve younger files than the local ones.
                              (default: off)
  -t  --tries               Number of tries for each download. (default 20)
      --trust-server-names  On redirection use the server's filename.
                              (default: off)
      --use-askpass         Prompt for a user and password using
                              the specified command.
      --use-server-timestamps
                            Set local file's timestamp to server's timestamp.
                              (default: on)
      --user                Username for Authentication.
                              (default: empty username)
  -w  --wait                Wait number of seconds between downloads
                              (per thread). (default: 0)
      --waitretry           Wait up to number of seconds after error
                              (per thread). (default: 10)
      --xattr               Save extended file attributes. (default: off)

HTTP related options:
  -E  --adjust-extension    Append extension to saved file (.html or .css).
                              (default: off)
      --auth-no-challenge   send Basic HTTP Authentication before challenge
  -K  --backup-converted    When converting, keep the original file with
                              a .orig suffix. (default: off)
      --compression         Customize Accept-Encoding with
                              identity, gzip, deflate, xz, lzma, br, bzip2, zstd, lzip
                              and any combination of it
                              no-compression means no Accept-Encoding
      --content-disposition
                            Take filename from Content-Disposition.
                              (default: off)
      --cookie-suffixes     Load public suffixes from file. 
                              They prevent 'supercookie' vulnerabilities.
                              See https://publicsuffix.org/ for details.
      --cookies             Enable use of cookies. (default: on)
      --default-http-port   Set default port for HTTP. (default: 80)
      --default-https-port  Set default port for HTTPS. (default: 443)
      --default-page        Default file name. (default: index.html)
      --header              Insert input string as a HTTP header in
                              all requests
      --html-extension      Obsoleted by --adjust-extension
      --http-keep-alive     Keep connection open for further requests.
                              (default: on)
      --http-password       Password for HTTP Authentication.
                              (default: empty password)
      --http-proxy          Set HTTP proxy/proxies, overriding environment
                              variables. Use comma to separate proxies.
      --http-proxy-password
                            Password for HTTP Proxy Authentication.
                              (default: empty password)
      --http-proxy-user     Username for HTTP Proxy Authentication.
                              (default: empty username)
      --http-user           Username for HTTP Authentication.
                              (default: empty username)
      --keep-session-cookies
                            Also save session cookies. (default: off)
      --limit-rate          Limit rate of download per second, 0 = no limit. (default: 0)
      --load-cookies        Load cookies from file.
      --metalink            Follow a metalink file instead of storing it
                              (default: on)
      --method              HTTP method to use for request.
      --netrc-file          Set file for login/password to use instead of
                              ~/.netrc. (default: ~/.netrc)
  -Q  --quota               Download quota, 0 = no quota. (default: 0)
      --referer             Include Referer: url in HTTP request.
                              (default: off)
      --retry-connrefused   Consider "connection refused" a transient error.
                               (default: off)
      --save-cookies        Save cookies to file.
      --save-headers        Save the response headers in front of the response
                              data. (default: off)
  -U  --user-agent          HTTP User Agent string.
                              (default: wget)

HTTPS (SSL/TLS) related options:
      --ca-certificate      File with bundle of PEM CA certificates.
      --ca-directory        Directory with PEM CA certificates.
      --certificate         File with client certificate.
      --certificate-type    Certificate type: PEM or DER (known as ASN1).
                              (default: PEM)
      --check-certificate   Check the server's certificate. (default: on)
      --check-hostname      Check the server's certificate's hostname.
                              (default: on)
      --crl-file            File with PEM CRL certificates.
      --egd-file            File to be used as socket for random data from
                              Entropy Gathering Daemon.
      --hpkp                Use HTTP Public Key Pinning (HPKP). (default: on)
      --hpkp-file           Set file for storing HPKP data
                              (default: ~/.wget-hpkp)
      --hsts                Use HTTP Strict Transport Security (HSTS).
                              (default: on)
      --hsts-file           Set file for HSTS caching. (default: ~/.wget-hsts)
      --hsts-preload        Use HTTP Strict Transport Security (HSTS).
                              [Not built with libhsts, so not functional]
      --hsts-preload-file   Set name for the HSTS Preload file (DAFSA format).
                              [Not built with libhsts, so not functional]
      --http2               Use HTTP/2 protocol if possible. (default: on)
      --http2-only          Only use HTTP/2 protocol, error if server doesn't offer it. (default: off)
      --https-enforce       Use secure HTTPS instead of HTTP. Legal types are
                              'hard', 'soft' and 'none'.
                              If --https-only is enabled,
                              this option has no effect. (default: none)
      --https-only          Do not follow non-secure URLs. (default: off).
      --https-proxy         Set HTTPS proxy/proxies, overriding environment
                              variables. Use comma to separate proxies.
      --ocsp                Use OCSP server access to verify server's
                              certificate. (default: on)
      --ocsp-date           Check if OCSP response date is too old.
                              (default: on)
      --ocsp-file           Set file for OCSP chaching.
                              (default: ~/.wget-ocsp)
      --ocsp-nonce          Allow nonce checking when verifying OCSP
                              response. (default: on)
      --ocsp-server         Set OCSP server address.
                              (default: OCSP server given in certificate)
      --ocsp-stapling       Use OCSP stapling to verify the server's
                              certificate. (default: on)
      --private-key         File with private key.
      --private-key-type    Type of the private key (PEM or DER).
                              (default: PEM)
      --random-file         File to be used as source of random data.
      --secure-protocol     Set protocol to be used (auto, SSLv3, TLSv1, TLSv1_1, TLSv1_2, TLS1_3, PFS).
                              (default: auto). Or use GnuTLS priority
                              strings, e.g. NORMAL:-VERS-SSL3.0:-RSA
      --tls-false-start     Enable TLS False Start (needs GnuTLS 3.5+).
                              (default: on)
      --tls-resume          Enable TLS Session Resumption. (default: off)
      --tls-session-file    Set file for TLS Session caching.
                              (default: ~/.wget-session)

Directory options:
      --cut-dirs            Skip creating given number of directory
                              components. (default: 0)
      --directories         Create hierarchy of directories when retrieving
                              recursively. (default: on)
  -P  --directory-prefix    Set directory prefix.
  -x  --force-directories   Create hierarchy of directories when not
                              retrieving recursively. (default: off)
      --host-directories    Create host directories when retrieving
                              recursively. (default: on)
      --protocol-directories
                            Force creating protocol directories.
                              (default: off)

GPG related options:
      --gnupg-homedir       Specify a directory to use as the GnuPG home
                               directory. (default: gnupg default homedir)
      --signature-extensions
                            The extension of the signature file which should be
                              downloaded. (default: sig)
      --verify-save-failed  Save target files even when their signatures fail
                              GPG validation. (default: off)
  -s  --verify-sig          Download .sig file and verify. (default: off)

Plugin options:
      --list-plugins        Lists all the plugins in the plugin search paths.
      --local-plugin        Loads a plugin with a given path.
      --plugin              Load a plugin with a given name.
      --plugin-dirs         Specify alternative directories to look
                              for plugins, separated by ','
      --plugin-help         Print help message for all loaded plugins
      --plugin-opt          Forward an option to a loaded plugin.
                              The option should be in format:
                              <plugin_name>.<option>[=value]

Example boolean option:
 --quiet=no is the same as --no-quiet or --quiet=off or --quiet off
Example string option:
 --user-agent=SpecialAgent/1.3.5 or --user-agent "SpecialAgent/1.3.5"

To reset string options use --[no-]option

9.使用

wget2下载时默认是5个线程

wget2 --progress=bar --report-speed=bits https://releases.ubuntu.com/21.04/ubuntu-21.04-live-server-amd64.iso

输出如下

[email protected]:~# wget2 --progress=bar --report-speed=bits https://releases.ubuntu.com/21.04/ubuntu-21.04-live-server-amd64.iso
ubuntu-21.04-live-se 100% [========================================================================================================>]    1.09G  800.41Kb/s
                          [Files: 1  Bytes: 1.09G [759.92Mb/s] Redirects: 0  Todo: 0  Errors: 0                                     ]

8线程

wget2 --max-threads=8 --progress=bar --report-speed=bits https://releases.ubuntu.com/21.04/ubuntu-21.04-live-server-amd64.iso

输出如下

ubuntu-21.04-live-se 100% [========================================================================================================>]    1.09G  623.13Kb/s
                          [Files: 1  Bytes: 1.09G [600.33Mb/s] Redirects: 0  Todo: 0  Errors: 0                                     ]

这个网络大概也就这样了

这篇文章有用吗?

点击星号为它评分!

平均评分 0 / 5. 投票数: 0

到目前为止还没有投票!成为第一位评论此文章。

发表评论

您的电子邮箱地址不会被公开。 必填项已用*标注

4 × 2 =

一条评论 “Ubuntu 18.04LTS 下编译安装最新版wget2”