wget
Last updated
Was this helpful?
Last updated
Was this helpful?
The wget command, which we will use extensively throughout this aide-memoire, downloads files using the HTTP/HTTPS and FTP protocols. The example below shows the use of wget along with the -O switch to save the destination file with a different name on the local machine:
wget has some other handy options that you should be familiar with.
We can specify a directory to download a file to with the -P option:
If you want to download multiple files at once, use the -i option, followed by the path to a local or external file containing a list of the URLs to be downloaded. Each URL needs to be on a separate line.
The following example shows how to download the Arch Linux, Debian, and Fedora iso files using the URLs specified in the linux-distros.txt file:
We can also resume a download that has dropped off with -c:
We can also run downloads in the background with the -b option:
We can also manually change the User-Agent string so wget can appear as any browser when downloading a file via the --user-agent= option:
To create a mirror of a website with wget, use the -m option. This will create a complete local copy of the website by following and downloading all internal links as well as the website resources (JavaScript, CSS, Images):
If you want to use the downloaded website for local browsing, you will need to pass a few extra arguments to the command above:
The -k option will cause wget to convert the links in the downloaded documents to make them suitable for local viewing. The -p option will tell wget to download all necessary files for displaying the HTML page.
If you want to download a file over HTTPS from a host that has an invalid SSL certificate, use the --no-check-certificate option:
In the following example, wget will quietly (-q) download and output the latest WordPress version to STDOUT(-O -) and pipe it to the tar utility, which will extract the archive to the /var/www directory.
