Useful Linux Commands when Writing Data Loader and Training Code — Part 2

Managing Background Processes (tools — nohup, screen, tmux)

Content in this article

1. Run processes in background with nohup
2. nohup background process -> change output file name from nohup.out
3. nohup background process with CUDA_VISIBLE_DEVICES
4. scp deep copy in background with nohup
5. Download from http using wget and run it in background
6. Run process in background and also redirect stdout and stderror to file
7. List processes running with nohup
8. Background process with screen command— Basic example
9. Background process with tmux command — Basic example

You can use this article like a cheat sheet since I have listed the most commonly required commands along with references.

I will be talking mostly about using the command “nohup”.

What other commands are there?

Some other commands are screen and tmux. I have a given a few examples using screen and tmux at the very end of this article. I am not sure how to save the terminal output to a file when using screen and tmux. So, I don’t use them. But, I have been able to find tmux commands for all of my use cases.

1. Run processes in background with nohup

nohup command &

The command output is redirected to the nohup.out file.

Reference : Link

2. nohup background process -> change output file name from nohup.out

nohup some_command &> nohup2.out &

Reference : Link

3. nohup background process with CUDA_VISIBLE_DEVICES

CUDA_VISIBLE_DEVICES=0,1 nohup python train.py &> nohup_train.out &

4. scp deep copy in background with nohup

nohup scp -r  <folder_you_want_to_copy> username@servername:/<path>/ > nohup.out 2>&1 &

References : Link , Link (when you have to provide a password)

5. Download from http using wget and run it in background

nohup wget --user=<username> --password=<password> https://h2odataset.ethz.ch/data/dataset/subject3_v1_1.tar.gz &

6. Run process in background and also redirect stdout and stderror to file

nohup 2>&1 python H2O_trainer_mano.py &

Reference: Link

7. List processes running with nohup

ps ax | grep <pid>

top

8. Background process (with screen command)

• In a new screen, type “screen” and press enter on the page shown after that. Then, in the screen that appears after (looks like a normal screen, but it’s attached to the screen command), you can type the required commands.
• To show how many processes are running with “screen”, type “screen -ls”
• To resume other activities while the background process (with screen) runs, close the teminal.
• If you press “exit”, it will kill the screen terminal and also the background process which you started.

1. First type “screen -ls”.

2.1. If the process is attached, first detach it , then reattach it to view it in the current terminal

2.2. If the process is already detached, then just type “screen -r <id>”

 screen -r <id>

9. Background process with tmux

tmux new  -s <name>