How to use the curl command for uploading and downloading files without user interaction

Автор: | 25.05.2018

The curl command is quite useful and flexible. The intent of the tool is to transfer data, without user interaction, to or from a server, using one of the many supported protocols. It is that list of protocols that helps curl manage to be so flexible, as the command supports:

  • DICT
  • FILE
  • FTP
  • FTPS
  • GOPHER
  • HTTP
  • HTTPS
  • IMAP
  • IMAPS
  • LDAP
  • LDAPS
  • POP3
  • POP3S
  • RTMP
  • RTSP
  • SCP
  • SMB
  • SMBS
  • SMTP
  • SMTPS
  • TELNET
  • TFTP

That’s a healthy list of possibilities. It also means curl can transfer just about any type of data. Curl can even display the source code of a URL. All of this without requiring user interaction (a crucial feature for scripting).

I want to show you how to make use of this command (one you’ll frequently find in Linux howtos and scripts). I’ll be demonstrating on Elementary OS, but the command works on nearly every available Linux distribution. Curl is also available on macOS and Windows.

SEE: 20 quick tips to make Linux networking easier (free PDF) (TechRepublic)

Installing curl

Your Linux distribution should have curl installed by default. If not, the installation is quite simple. On a Debian derivative, the installation command would be:

sudo apt install curl

You can check the installation by issuing the command curl —version. You should see the version number of the installed application (on my Elementary OS machine, it’s 7.47.0).

Using curl

Let’s first see how curl can be used at its simplest. Say you want to view the source of a web site. We’ll use the curl site as an example. Issue the command:

curl https://curl.haxx.se | less

You can now scroll through the HTML for that site (Figure A).

Figure A

Figure A

Viewing HTML source via the curl command.

This is a great way to figure out how a site has been created or even troubleshooting your own sites.

Of course, that example is quite limited. Let’s use curl to pull down a file from a site. Let’s stick with the same example. Say you want to download the HTML for the curl site to view later. For this, we’ll use the -o switch like so:

curl https://curl.haxx.se -o curl.html

The above command would download the HTML code from the curl site and save it as curl.html. Of course, curl isn’t only capable of downloading source HTML. Say you have a file you want to download from a site. Curl can handle this like so:

curl http://SERVER_ADDRESS/FILENAME -o FILENAME

Where SERVER_ADDRESS is the URL of the server and FILENAME is the name of the file to be downloaded. Say for example, you want to download the latest release of Ubuntu Server. That can be done like so:

curl http://releases.ubuntu.com/18.04/ubuntu-18.04-live-server-amd64.iso -o ubuntu-server-18.04.iso

If that file is password protected, curl can handle that like so:

curl -u USERNAME:PASSWORD http://SERVER_ADDRESS -o FILENAME

Where:

  • USERNAME is the username on the server.
  • PASSWORD is the password for the user on the server.
  • FILENAME is the file to be downloaded.
  • SERVER_ADDRESS is the direct link to the file.

You can also use curl with an FTP server. Say you need to download a file from an FTP server that happens to be password protected. The command for this would be:

curl ftp://SERVER_ADDRESS/FILENAME -user USERNAME:PASSWORD -o FILENAME

Where:

  • SERVER_ADDRESS is the address of the FTP server.
  • FILENAME is the name of the file to be downloaded.
  • USERNAME is the username on the FTP server.
  • PASSWORD is the password for the user on the FTP server.

To upload a file to an FTP server, the command would be:

curl -T FILENAME SERVER_ADDRESS -user USERNAME:PASSWORD

Again where:

  • SERVER_ADDRESS is the address of the FTP server.
  • FILENAME is the name of the file to be downloaded.
  • USERNAME is the username on the FTP server.
  • PASSWORD is the password for the user on the FTP server.

At some point, the curl developers removed SFTP support from the libcurl. If you need to download a file, via SSH, you’ll need to use the sftp command.

Why use curl

One of the biggest benefits of curl is that it can be used without interaction. Because of that, it’s perfectly suited for scripting. You’ll run into many instances of Linux installer scripts that make use of curl. As an installer script runs, you might well see curl do its thing by downloading necessary dependencies. So if you’re writing a Linux bash (or installer) script, curl will most certainly be your friend. For more information on curl (and there’s a lot of it), issue the command man curl and read through the manual page.

Also see:

networkhero.jpg

Source

Добавить комментарий