Pages

27.7.12

Automated Rotation Backup Script

Hi guys!

In this post we'll se something useful about backup your personal data!

Backup data is very important, especially for people that use computer for work. The loss of important data can be a nefarious event.

What we need to run this script is:

- Personal Computer with a distribution of Linux Installed
- About 5 minutes of patience :-)

-------------------------------------------------------------------------------------------------------------------------

UPDATE

The original script has been uploaded by zyphlar on a gitHub repository. The code has been improved in order to perofm security checks. Indeed, when not invoked properly, the script could have deleted data.

https://github.com/zyphlar/backup-rotation-script/blob/master/rotate.sh

Thanks to Zyphlar for the improvements

-------------------------------------------------------------------------------------------------------------------------

Description: This is the script that will makes backup of a specified directory and implements rotation. Rotation in backups means that some old backup files will be deleted automatically and newer will be shifted to replace them. You can set the number of days after which the old backup folders will be deleted. This can be done through changing the NUMDAYS variable. This variable is set up to 7 days by default. Once you have pasted script into a file, save it as "backup_script.sh"

1 - The script must obtain the "x" flag because it's necessary to make possible to eXecute it. So open linux terminal and go to the directory where the script is and type


chmod +x backup_script.sh

2 -  To run this script you must use terminal, cd to the directory where the script is and type


./backup_script.sh what where

Where:
  • the parameter <what> must defines the ABSOLUTE path of the directory that must be "backupped".
  • the parameter <where> must defines the ABSOLUTE path of the directory where the backup will be stored
This script generates also a log text. In fact if you run it from terminal you'll see a list of log lines talking about the backup operations. If you want to store them into a separate file you can execute the script in this way:

./backup_script.sh what where > backup_log.log 

so in this way the log lines will be stored in backup_log.log file.

3 - We need to execute the last step, the automation of this process. To automate the backup process and schedule it we can use cron. Cron is a daemon that can execute scheduled commands for every user of a machine. You can schedule a command to be executed specifying the minutes of a day when execute it, the hours, the days etc like shown here below.

.---------------- [m]inute (0 - 59) 
|  .------------- [h]our (0 - 23)
|  |  .---------- [d]ay [o]f [m]onth (1 - 31)
|  |  |  .------- [mon]th (1 - 12) OR jan,feb,mar,apr... 
|  |  |  |  .---- [d]ay [o]f [w]eek (0 - 6) (sunday=0 o 7)  OR sun,mon,tue,wed,thu,fri,sat 
|  |  |  |  |

*  *  *  *  *  comand to be executed


Now we have got to add a scheduled execution of our backup script. Here is an example that you can personalize at your own. For first thing open linux terminal and type

crontab -e


In this way you will edit your cron scheduled commands. (-e option means edit). Now we insert the scheduled execution of our script. To do this, before you can edit the content you must press the key "I" that switches the editor mode into "INSERT" mode. Now it's possible to enter data. Type something like

45 15 * * * /path/to/backup_script.sh > /path/to/backup_log.log


You have to define the absolute path to your script, so replace /path/to/ with the exactly absolute path where the script is placed. Once finished to edit the content press the button "ESC" that switches from "INSERT" mode to "VIEW" mode and type ":wq" and hit return. "wq" means "write changes and quit". Now we have added a scheduled execution of the script to the cron daemon. We can verify this typing


crontab -l


that means to "list" all the cron entries for this user. In the example above we have scheduled the execution of the script every month, every day of the week and the month, at 15.45 (in 24h notation, else 3.45PM in 12h notation). In this way your backups will be automated and the old backups will be automatically deleted and replaced with the shifting logic introduced below.

21.7.12

My First Post on Happy Engineer - SSH Tunneling

A very powerfull tool is SSH. SSH is the acronym of secure shell and is a kind of prehistoric version of Remote Desktop Connection or VNC. The main difference between VNC, RDC and SSH is that SSH make possible to manage a remote host using without using GUIs, but only through a prompt.

You can think "Bleah, only a poor command prompt?  Where the way to unpack tarball is made of a long sequence of chars??? WTF????"

Answer: "Yes".

Trust me, if you begin to use linux's terminal it will be the END. You'll not use linux without using terminal! It's a drug!

But, wait a second, try to think about the idea of sending only few bytes to manage remote hosts instead of large quantities of bytes for send screenshots of VNC and RDC. It sounds good!!

Leaving behind these considerations, let's talk technically.

If you want to connect to a remote host using ssh you have to be sure of two things:

1 - The remote host must have installed a SSH daemon (OpenSSH for example) and must be correctly configured.

In Ubuntu distributions is very simple to install OpenSSH, just open terminal and type:

    sudo apt-get install openssh-server

Once installed, the daemon will startup automatically when the host starts up, and all the user of the host will be enabled to use ssh service, so if you want to access to your machine you can use the credentials you normally use to access the host.

2- You must know a pair o username-password enabled on the remote host.

Try to connect to a remote host using ssh. Open the terminal and type:

    ssh username@hostip



Where:

    Username is the username on the remote host
    hostip defines the remote host ip you want to connect using SSH

 If you want to test if OpenSSH is working on your local host you can run:

    ssh yourusername@localhost



After the execution of this command you'll be asked to accept a key and you'll need to answer "yes".

    What's the meaning of this question?



SSH bases its operation on the use on a public key cryptography system and the first time you make a connection to a remote host you must accept its public key to start the conversation.

    Wow... Now i'm connected to a remote host.. And now?



The main advantage of using SSH for administrate remote hosts is the limitate usage of bandwidth. It seems a nonsense, but there are a lot of situations where the optimization of bandwidth usage can make the difference.



    Yeah. .. Boring Things...



Wait!

There's a little funcionality of SSH that is very powerfull. The Tunneling. Tunneling defines the possibility to make a tunnel between two hosts and make possible to exchange data between them.



Imagine to have three hosts:

    host A, connected to the internet, placed in Australia
    host B and C in the same subnet but only B is directly connected to the Internet. These two hosts are in Norway.

Imagine also that host A must access to C for some reasons. There are a lot of ways to do this for example VPNs (maybe we talk about this another time).. Another way can be tunneling.

Idea!

Using host B like a gateway for exchange data with C. Great! How?

SSH

There's a little tiny argument that you can pass to SSH command before execute it. The -L argument. Let's see a usage for this argument.

    ssh johndoe@hostB -L 5000:hostC:21



In this example i'm connecting to the host B placed in Norway, using the username johndoe. Passing -L argument i'm saying to ssh to bind port 21 of hostC (hostC must be the subnet's ip of host C) to the port 5000 of my host. So if i connect using a FTP client to port 5000 of my localhost, the communication will be routed through the tunnel to host C using host B like a gateway.

So if you have the possibility to access a host of a subnet using SSH, you'll be able to access on every host of that subnet.

I wish that you will appreciate this post. I'm also happy if you want to correct me in contents or in grammar (i'm not so trained about writing english :) )

Yhank you, see you soon

18.7.12

Concatenate arrays with different sizes in Matlab

Sometimes you want to concatenate arrays with different sizes, and Matlab doesn't allow you to. This makes sense, of course, but still you just want to put all that stuff together for some reason. Then this function may help you. As any other functions I've wrote, I'm sure it an be optimized, so I'll be happy to hear suggestions.
You can also download it from my matlab file exchange and rate it :-)



function [catmat]=padconcatenation(a,b,c)
%[catmat]=padconcatenation(a,b,c)
%concatenates arrays with different sizes and pads with NaN.
%a and b are two arrays (one or two-dimensional) to be concatenated, c must be 1 for
%vertical concatenation ([a;b]) and 2 for horizontal concatenation ([a b])
%
% a=rand(3,4)
% b=rand(5,2)
% a =
%
%     0.8423    0.8809    0.7773    0.3531
%     0.2230    0.9365    0.1575    0.3072
%     0.4320    0.4889    0.1650    0.9846
% b =
%
%     0.6506    0.8854
%     0.8269    0.0527
%     0.4742    0.3516
%     0.4826    0.2625
%     0.6184    0.5161
%
% PADab=padconcatenation(a,b,1)
% PADab =
%
%     0.8423    0.8809    0.7773    0.3531
%     0.2230    0.9365    0.1575    0.3072
%     0.4320    0.4889    0.1650    0.9846
%     0.6506    0.8854       NaN       NaN
%     0.8269    0.0527       NaN       NaN
%     0.4742    0.3516       NaN       NaN
%     0.4826    0.2625       NaN       NaN
%     0.6184    0.5161       NaN       NaN
%
% PADab=padconcatenation(a,b,2)
%
% PADab =
%
%     0.8423    0.8809    0.7773    0.3531    0.6506    0.8854
%     0.2230    0.9365    0.1575    0.3072    0.8269    0.0527
%     0.4320    0.4889    0.1650    0.9846    0.4742    0.3516
%        NaN       NaN       NaN       NaN    0.4826    0.2625
%        NaN       NaN       NaN       NaN    0.6184    0.5161

sa=size(a);
sb=size(b);

switch c
    case 1
        tempmat=NaN(sa(1)+sb(1),max([sa(2) sb(2)]));
        tempmat(1:sa(1),1:sa(2))=a;
        tempmat(sa(1)+1:end,1:sb(2))=b;
       
    case 2
        tempmat=NaN(max([sa(1) sb(1)]),sa(2)+sb(2));
        tempmat(1:sa(1),1:sa(2))=a;
        tempmat(1:sb(1),sa(2)+1:end)=b;
end

catmat=tempmat;
end