What is wget?

Wget is a free GNU command-line utility tool used to download files from the internet. It retrieves files using HTTP, HTTPS, and FTP protocols.

It serves as a tool to sustain unstable and slow network connections. If a network problem occurs during a download, this helpful software can resume retrieving the files without starting from scratch.

Another important aspect is its capability of recursive downloads, with which it mirrors websites. It transfers parts of a website by following links and directory structure, thus creating local versions of webpages.

The wget command is also highly flexible and can be used in terminals, scripts, and cron jobs. During download, the user does not have to be active nor logged in. As wget is non-interactive, it can independently run in the background.

Read this article to learn how to use some of the most common wget commands.

how to use wget command with examples

How to Check if wget is Installed?

Most likely, the wget package is already on your system as it now comes pre-installed on most Linux distributions.

To check, open the terminal window and type in:

wget

If you have the wget software, the output will tell you that the wget command is missing a URL, as shown in the image below:

wget-command-missing-url

wget Command Not Found

If the wget command returns an output that says “wget command not found” you will have to download and install the tool manually. Below are installation instructions for Ubuntu/Debian, CentOS, and Windows.

How to Install wget on Ubuntu/Debian?

To install wget on Ubuntu and Debian releases, use the command:

sudo apt-get install wget

How to Install wget on CentOS/Fedora?

To install wget on CentOS and Fedora releases, type the following command:

sudo yum install wget

How to Install wget on Windows?

To install and configure wget for Windows:

  1. Download wget for Windows and install.
  2. Add the wget bin path to environment variables (optional). Configuring this removes the need for full paths, and makes it a lot easier to run wget from the command prompt:
      • Open the Start menu and search for “environment.”
      • Select the Edit the system environment variables result.
      • Select the Advanced tab and click the Environment Variables button.
      • Select the Path variable under System Variables.
      • Click Edit.
      • In the Variable value field add the path to the wget bin directory preceded with a semicolon (;). If installed in the default path, add C:\Program Files (x86)\GnuWin32\bin.
  3. Run the command prompt (cmd.exe) and start running wget commands.

Introduction to wget Syntax

Wget syntax has the following pattern:

wget [option][URL]

Each [option] has its long and short form that are conveniently interchangeable. It specifies what to do with the URL that follows.

[URL] is the address of the file or directory you wish to download.

wget Command Examples

Download File from Web

wget [URL]

For example, to install Tomcat 9, first you need to download the package with wget using the command:

wget http://apache.cs.utah.edu/tomcat/tomcat-9/v9.0.20/bin/apache-tomcat-9.0.20.tar.gz

Download File and Save Under Specific Name

wget –O [file_name] [URL]

The wget command allows you to rename files prior to downloading them on your computer. For instance, you may want to install Terraform. To download its package and rename it terraform.zip use the following command:

wget –O terraform.zip https://releases.hashicorp.com/terraform/0.12.2/terraform_0.12.2_linux_amd64.zip

Download File to Specific Directory

wget –P [wanted_directory] [URL]

By default wget downloads a file in the directory the user is in. To save the file in a different location, use the –P option.

For example, while installing Git on Ubuntu, you may want to download the package in the /temp directory with the command:

wget –P /temp https://github.com/git/git/archive/master.zip

Set Download Speed

wget --limit-rate [wanted_speed] [URL]

The speed is defined in kilobytes (k) and megabytes (m). Use this option when downloading a big file, so it does not use the full available bandwidth.

If you are installing NVIDIA TESLA drivers on Linux and want to limit the download speed to 1 megabyte, use the command:

wget --limit-rate 1m http://us.download.nvidia.com/tesla/396.37/nvidia-diag-driver-local-repo-ubuntu1710-396.37_1.0-1_amd64.deb 

You can also set the speed rate to 50 kilobytes with the command:

wget --limit-rate 50k http://us.download.nvidia.com/tesla/396.37/nvidia-diag-driver-local-repo-ubuntu1710-396.37_1.0-1_amd64.deb 

Continue Download After Interruption

wget –c [URL]

This is a useful option if you lose connection while downloading a file. Instead of having to start from scratch, wget can resume downloading where it stopped before the interruption.

For instance, you may want to install a Mumble Server on Linux and suddenly lose internet connection while downloading the installation file. To continue downloading, type in the command:

wget –c https://github.com/mumble-voip/mumble/releases/download/1.2.19/murmur-static_x86-1.2.19.tar.bz2

Download Multiple Files

wget –i [file_name]

wget allows downloading multiple files at the same time in a couple of steps:

1. First, create and open a text file under the name txt (or any name you choose), using a text editor. In this case, we used Nano:

nano MultipleDownloads.txt

2. In the editor, add the URLs of the packages you want to download, one per line.

download-multiple-files-using-wget

3. Save and exit the file.

4. Type in the following wget command in the terminal window:

wget –i MultipleDownloads.txt

This prompts wget to download from each URL in the text file.

Download Web page (Mirror Web page)

wget –m [URL]

The –m command represents creating a mirror of a webpage. With wget you can download an entire website from the internet.

For example:

wget –m https://phoenixnap.com

Download via FTP

wget --ftp-user=[ftp_username] --ftp-password=[ftp_password] ftp://...

Type in the username and password of the FTP serve, followed by the ftp address. For instance:

wget --ftp-user=sofiftp --ftp-password=TopSecretPassword ftp://123.456.7890

Download in Background

wget –b [URL]

This feature is practical when downloading a large file.

To download the RPM package manager in the background while working on something else, type:

wget –b http://some_website/sample_file.rpm

You can check the status of the download with the command:

tail –f wget –log

Increase Retry Attempts

wget --tries=[number_of_tries] [URL]

Set how many times wget attempts to download a file after being interrupted by a bad network. By default, the number of retry attempts is set to 20.

You can also set the number to infinity with the values 0 or inf, as in the following example:

wget --tries=inf [ http://download.virtualbox.org/virtualbox/rpm/rhel/virtualbox.repo

Skip Certificate Check

wget --no-check-certificate [URL]

By default, wget checks whether the server has a valid SSL/TLS certificate. If it does not identify an authentic certificate, it will refuse to download.

The --no-check-certificate option is used to avoid certificate authorities checking for a server certificate. However, use it only if you are sure of the website’s credibility or are not worried about security issues it may cause.

If http://enteratonerisk.com has an untrusted certificate, but will not harm the system, download it with:

wget --no-check-certificate http://enteratonerisk.com

Change User Agent

wget --user-agent=”User Agent Here” “[URL]”

or

wget –U ”User Agent Here” “[URL]”

When downloading a webpage, wget essentially emulates a browser. In some cases, the output might say you don’t have permission to access the server, or that the connection is forbidden. This may be due to a website blocking client browsers that have a specific “User-Agent.”

“User-Agent” is a header field that the browser sends to the server it wants to access. Therefore, to download from a server that is refusing to connect, try to modify the user agent.

Find a database of all user agents online and search for the one you need. For example, to emulate Chrome (version 74), you would change the user agent with the command:

wget --user-agent=” Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36” “https://phoenixnap.com”

Conclusion

This article sums up why wget is such a powerful tool for downloading files over the internet. It also serves as a good reference for beginners with its list of 12 essential wget commands and examples.