Pages

Sunday, March 31, 2013

LFTP - Multi-Segmented Linux FTP


Disclaimer: The process in this article may seem complicated and slow, but it is designed for you to learn the basics first. The final section of the article is dedicated to making things easier and faster for everyday use. So, please don't give up early.

Installing

Windows

Install lftp, OpenSSH, and OpenSSL from the Net category in Cygwin

Linux

Install lftp with your package manager, something like: sudo apt-get install lftp
Note: lftp is installed on your Whatbox slot by default. You can use it to transfer between your slot and other ftp servers (e.g. another seedbox). Just connect to your slot using SSH and use it as normal.

Connecting to Whatbox

  1. Open your terminal (Cygwin Terminal for Windows users). You will see a window that looks similar to below.
  2. Start lftp and connect to Whatbox. Type lftp sftp://user@server.whatbox.ca. Enter your password at the prompt. lftp keeps your password hidden, so it's easiest to paste with the right-click menu.
  3. Now type ls in the terminal to show a listing of your home directory. Since most users will want to start in their files directory, type cd files to navigate there for the next step.
  4. (optional) If you want your password to be saved when you create a bookmark, use the command set bmk:save-passwords true now.
    bookmark add Whatbox will save a bookmark to your slot named Whatbox. You can use bookmark list to make sure it saved properly. This allows you to use lftp Whatbox to open a connection to your slot instead of having to type out the address.

Local and Remote Navigation

Note: I have set up directories and files for demonstration. You can set things up like I have for practice, if you like, or replace them in the commands with your own.
  1. You've used cd to go down one level into a directory. You can use cd .. to go up one level. Typels to see that you're back in your home directory.
  2. Next, you'll use cd to navigate directly to a directory more than one level away. cd ~/files/Video brings me into the directory I set up.
  3. Now to navigate to directories on your computer with the lcd(read: local cd) command. The directory structure will be different on Linux than with Cygwin. On Linux, simply type lcd /path/to/directory. With Cygwin, if you want C:\Download you would type lcd /cygdrive/c/Download. To list what's in the current local directory, add an exclamation point to the ls command like !ls.

Transfers

  1. There are two main commands used for downloading: pget for files and mirror for directories.
  2. There are two main commands used for uploading: put for files and mirror -R for directories.
  3. To cancel a transfer, use Ctrl+c
  4. To resume partially downloaded files/directories, you will use the -c switch: mirror -c Directory pget -c Video1.mp4 put -c Video1.mp4

Segmented Downloading

Note: This guide uses 5 segments as an example. You may use as many as needed or desired.
  1. A pget command using segmentation is pget -n 5 Video2.mp4, where 5 is the number of segments.
  2. A mirror command using segmentation is mirror --use-pget-n=5 Directory, where 5 is the number of segments.
  3. You can use jobs -v to see the speeds of the individual segments as well as the total speed.

Queues and Jobs

  1. The queue command will allow you to transfer files/directories while still being able to browse. The basic command to queue a file transfer will look like queue pget Video1.mp4. To transfer entire directories, queue mirror Directory. This adds them to your transfer queue and automatically starts the transfer if ready.
  2. Use queue alone to display your transfer queue. The jobs command displays your current transfers and transfer queue with more detail.
  3. You can stop your queue from automatically transferring by using queue stop. This can be used before adding anything to your queue to prevent immediate transfer. queue start will start your queued transfers. To delete a transfer from your queue, add the -d switch like queue -d #, where # is the queue number listed by jobs or queue
  4. To stop and delete a running transfer, use the kill command. If you set lftp to allow multiple transfers simultaneously, kill #, where # is the number listed by jobs, will stop and delete the specified transfer. kill all will stop and delete all transfers.

Configuration and Aliases

Now to make it quick and easy, we'll set up some aliases. Aliases allow you to make shortcuts to commands.
  • In Linux, find your lftp.conf file. It should be in /etc.
  • In Windows, open the etc directory where you installed Cygwin (C:\Cygwin by default) and open lftp.conf.
  • Go to the end of the file and add some or all of these example aliases. Modify them to fit your needs.
Some examples from this guide with those aliases applied:
lcd /cygdrive/c/Download becomes down
queue pget -cn 5 Video1.mp4 becomes q p Video1.mp4
queue mirror -c --use-pget-n=5 Directory becomes q m Directory
For full command help and settings, see the lftp man page.

Tips and Tricks

  • The Tab key autocompletes file and directory names. It also autocompletes commands.
  • Files and directories with spaces or special characters need special handling. The easiest method is to put quotes around the file/directory names.
  • You can cycle through peviously used commands with the Up and Down arrow keys.
  • To quit lftp, type exit. Then type exit again to close the terminal.

at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)