Send commands to Linux screen

GNU Screen is very powerful piece of software that lets you multiplex virtual consoles. I personally use it for running jobs & servers on background. I also use irssi trough screen.

Name your screens!

When you run stuff at screen its recommended to set name for that session using -S switch.

For example with irssi i would specify irssi also as session name.

 m@srv:~$ screen -S irssi irssi  
 m@srv:~$ screen -list  
 There are screens on:  
     16425.irssi   (09/13/2012 01:28:36 PM)    (Detached)  
     16323.mc    (09/13/2012 01:24:43 PM)    (Detached)  
 2 Sockets in /var/run/screen/S-m.  

ID in front of your session name is PID of virtual screen console.

 m@srv:~$ ps -A | grep 16425  
 16425 ?    00:00:00 screen  

As you can see there is now two screens running, irssi and mc (mc is my minecraft server :-)). Why I'm telling this is simply because this way its bit easier to track what is running in your screen. Its not mandatory.

You can resume this sessions now with name, for example

 screen -r irssi  

If you are new to screen here is a good tip for you, you can detach from screen by doing ctrl+a+d

How to send commands to screen?

Especially with Minecraft server its handy to communicate with your server without attaching your screen session. For automation this is good, you can do daily backups, restart your server and before all that, you can alert your users!

This doesn't work if your screen is password protected.

 screen -S mc -X stuff 'say Server going down for maintenance in 20 seconds'$'\012'  

First you specify session with -S, then you need to do -X stuff '<message here>'

"stuff" needs to be there because.. you are sending stuff to it. Sorry i couldn't find any details about this.

\012 is a newline character, if you don't pass this, screen will input your message but wont "press enter" for you, so your message will remain in the console input screen.

OMG! Server going down

Replicate linux data using rsync

I¨ve always had multiple Linux boxes at my disposal, usually they are just old laptops or desktop that i have planned to use for something but never had time to do anything useful with them, so i have them just lying around.

They never had any kind of raid or redundancy and my data has been "kept save" just by holy spirit :-)

Easy way to sleep without worrying is to replicate your data between servers. Easy way to achieve this is by using rsync and cron. In this example we don't use rsync protocol but we are using SSH connection for the transfer.

First you should create SSH keys to your server to make them access each other without need to enter credentials.

In this example i will create rsync from my home server to external server, in this case i have only 1 account for both servers that i will be using.

1. Create SSH Keys

Im using ubuntu servers so first i need to edit file /etc/ssh/sshd_config and uncomment one line

#AuthorizedKeysFile     %h/.ssh/authorized_keys
(remove # from the start)

Then i will login to my home server and create SSH key with command ssh-keygen -t rsa

I didn't use any passphrase

 m@homeserver:~$ ssh-keygen -t rsa  
 Generating public/private rsa key pair.  
 Enter file in which to save the key (/home/m/.ssh/id_rsa):   
 Enter passphrase (empty for no passphrase):   
 Enter same passphrase again:   
 Your identification has been saved in /home/m/.ssh/id_rsa.  
 Your public key has been saved in /home/m/.ssh/id_rsa.pub.  

Now we need to open this public key in /home/profile/.ssh/ that we just created, and copy paste all of its contents to external server.

login to external server
Authorized keys will be stored at file "authorized_keys" in /home/profile/.ssh

File didn't exist so i had to create it

 m@external:~/.ssh$ touch authorized_keys  

Open file with your favourite editor and paste contents of your public key into it + save.

We should be all done, time to test

 m@homeserver:~/.ssh$ ssh m@extserver  

Works! (at least for me), no password prompted, yippee.

2. rsync

I want to keep contents between my servers always identical, so i will use following rsync command:

 rsync -avz --delete /home/m/samba m@external:/home/m/backup/  

-a is for archive mode, its very good for backups it will preserve when file was modified and who is the owner and so on

-v is for verbose, so you will get some output about transfer itself

-z is for compress, will save some bandwidth

--delete will remove files on extserver that don't exist on sender, in my case homeserver

If you need to specify alternate port for your connection, add --rsh='ssh -p1234' modify 1234 to your port

3. Cronjob

Now we create cron in order to make this rsync run automatically. I want to replicate all changes once per day, there might be some huge changes in my samba directory so i want to give some time for this process. 24 hours should be enough..

Access crontab with:

 crontab -e  

In my case this will be easy, there is some predefined scheduling definitions that will work for me.
@daily will run job everyday at midnight. Please see more at Cron wiki page

So my crontab looks like this:

 @daily rsync -avz --delete /home/m/samba m@external:/home/m/backup/  

Some final words

Actually this is kind of bad backup, since one human error may compromise all of my data. So if i delete a very important file.txt from my samba share, rsync will destroy it from external server also.

There are many ways to overcome this, for example you could zip backup folder everyday for last x days, in my case this is not possible since i don't have enough disk space :-(

Anyway, be careful !

Experimenting with Python subprocess Popen

Process control and management with Python is pure greatness. Parallel tasks can be easily launched and monitored, output can be captured and used later on if needed.

You can communicate with your processes by sending data to stdin. You can also send in signals and all processes will give you return code after their execution.

Official subprocess documentation can be found here

I made small script to demonstrate subprocess.Popen

 import random  
 import subprocess  
 import time
  
 # Amount of spawned processes  
 amount = 10  
 # Process time  
 min = 5  
 max = 20  
 print " "  
 print "Subprocess example starting"  
 print "....................................."  
 time.sleep(1)  
 print " "  
 print "Spawning " + str(amount) + " processes"  
 # Store subprocesses in this array  
 process_list=[]  
 # Spawn all processes  
 for i in range (1, amount + 1):  
      timer = random.randint(min, max)       
      sp = subprocess.Popen("sleep " + str(timer), shell=True)  
      print "Spawning process nro " + str(i) + " | PID: " + str(sp.pid) + " | Timer set to: " + str(timer)  
      process_list.append(sp)  
 print "All processes spawned"  
 print " "  
 print "Waiting.."  
 print " "  
 time.sleep(5)  
 # Loop for processes, wait for completion  
 stop = 0  
 while stop != 1:  
      for sp in process_list:  
           sp.poll()  
           if str(sp.returncode) != "None":  
                print "Process (PID: " + str(sp.pid) + ") finished working! returncode: " + str(sp.returncode)  
                process_list.remove(sp)  
           time.sleep(0.1)  
      if len(process_list) <= 0:  
           stop = 1  
 print " "  
 print "....................................."  
 print "All subprocesses completed!"  

In this example script, first loop spawns 10 shells that will execute sleep timer randomly between 5 and 20 seconds. Second loop will just wait until every process has finished. poll() will set return code for process so it must be done once in a loop, so my application will notice finished processes.

Its handy to store your processes in a list!

Like in this case, if shell argument is set to true in Popen() .pid() (process ID) will belong to spawned shell. Alternatively you could do this:

 sp = subprocess.Popen(["./sleep_subprocess.sh", str(timer)])  

Here is the output:

 Subprocess engine starting  
 .....................................  
 Spawning 10 processes  
 Spawning process nro 1 | PID: 14644 | Timer set to: 19  
 Spawning process nro 2 | PID: 14645 | Timer set to: 5  
 Spawning process nro 3 | PID: 14647 | Timer set to: 10  
 Spawning process nro 4 | PID: 14649 | Timer set to: 12  
 Spawning process nro 5 | PID: 14651 | Timer set to: 6  
 Spawning process nro 6 | PID: 14653 | Timer set to: 17  
 Spawning process nro 7 | PID: 14655 | Timer set to: 7  
 Spawning process nro 8 | PID: 14657 | Timer set to: 17  
 Spawning process nro 9 | PID: 14659 | Timer set to: 19  
 Spawning process nro 10 | PID: 14661 | Timer set to: 17  
 All processes spawned  
 Waiting..  
 Process (PID: 14645) finished working! returncode: 0  
 Process (PID: 14651) finished working! returncode: 0  
 Process (PID: 14655) finished working! returncode: 0  
 Process (PID: 14647) finished working! returncode: 0  
 Process (PID: 14649) finished working! returncode: 0  
 Process (PID: 14653) finished working! returncode: 0  
 Process (PID: 14661) finished working! returncode: 0  
 Process (PID: 14657) finished working! returncode: 0  
 Process (PID: 14644) finished working! returncode: 0  
 Process (PID: 14659) finished working! returncode: 0  
 .....................................  
 All subprocesses completed!  

If your process stays too long you can terminate it with .terminate() or if its stuck you can .kill() it!