Labels

Friday, July 27, 2012

Automated Rotation Backup Script

Hi guys!

In this post we'll se something useful about backup your personal data!

Backup data is very important, especially for people that use computer for work. The loss of important data can be a nefarious event.

What we need to run this script is:

- Personal Computer with a distribution of Linux Installed
- About 5 minutes of patience :-)

Follow the step-by-step guide here below:

1 - Open a text editor like Vi, gedit, emacs or others and create an empty file.

2 - Copy and paste this code:


#!/bin/bash

# Author: Marcello Torchio

# Set the directory name that you wish to backup
DIRTOBACKUP=$1

# Set up the backup destination directory
BACKUPDIRDEST=$2

# Set up the number of days for backup rotation
NUMDAYS=7;

# Start logging
echo "["$(date)"] - Started Backup"
# Check if the destination dir exists. If not create it
if [ ! -d "$BACKUPDIRDEST/"$(date +"%m-%d-%Y")"/" ]; then
    # Creates directory for today's backup
    mkdir $BACKUPDIR/"$(date +"%m-%d-%Y")"
fi

# Echo the log line.
echo "["$(date)"] - Backup directory $DIRTOBACKUP in $BACKUPDIRDEST/"$(date +"%m-%d-%Y")"/"
# Execute the backup command. This backup consist in a recursive copy of the source dir into destination dir
cp -R $DIRTOBACKUP $BACKUPDIRDEST/"$(date +"%m-%d-%Y")"/

TOTMB=$(du $BACKUPDIRDEST/"$(date +"%m-%d-%Y")"/ -hs)
echo "["$(date)"] - Finished Backup - $TOTMB Copied"

# Search for folders older than NUMDAYS days and remove them. This makes possible rotation of backups
for i in `find $BACKUPDIRDEST/ -maxdepth 1 -type d -mtime +$NUMDAYS -print`;
do
        echo -e "["$(date)"] - Found outdated backup folder! Deleting directory $i";
        \rm -rf $i;
done

This is the script that will makes backup of a specified directory and implements rotation. Rotation in backups means that some old backup files will be deleted automatically and newer will be shifted to replace them. You can set the number of days after which the old backup folders will be deleted. This can be done through changing the NUMDAYS variable. This variable is set up to 7 days by default. Once you have pasted script into a file, save it as "backup_script.sh"

3 - The script must obtain the "x" flag because it's necessary to make possible to eXecute it. So open linux terminal and go to the directory where the script is and type


chmod +x backup_script.sh

4 -  To run this script you must use terminal, cd to the directory where the script is and type


./backup_script.sh what where

Where:
  • the parameter <what> must defines the ABSOLUTE path of the directory that must be "backupped".
  • the parameter <where> must defines the ABSOLUTE path of the directory where the backup will be stored
This script generates also a log text. In fact if you run it from terminal you'll see a list of log lines talking about the backup operations. If you want to store them into a separate file you can execute the script in this way:

./backup_script.sh what where > backup_log.log 

so in this way the log lines will be stored in backup_log.log file.

5 - We need to execute the last step, the automation of this process. To automate the backup process and schedule it we can use cron. Cron is a daemon that can execute scheduled commands for every user of a machine. You can schedule a command to be executed specifying the minutes of a day when execute it, the hours, the days etc like shown here below.

.---------------- [m]inute (0 - 59) 
|  .------------- [h]our (0 - 23)
|  |  .---------- [d]ay [o]f [m]onth (1 - 31)
|  |  |  .------- [mon]th (1 - 12) OR jan,feb,mar,apr... 
|  |  |  |  .---- [d]ay [o]f [w]eek (0 - 6) (sunday=0 o 7)  OR sun,mon,tue,wed,thu,fri,sat 
|  |  |  |  |

*  *  *  *  *  comand to be executed


Now we have got to add a scheduled execution of our backup script. Here is an example that you can personalize at your own. For first thing open linux terminal and type

crontab -e


In this way you will edit your cron scheduled commands. (-e option means edit). Now we insert the scheduled execution of our script. To do this, before you can edit the content you must press the key "I" that switches the editor mode into "INSERT" mode. Now it's possible to enter data. Type something like

45 15 * * * /path/to/backup_script.sh > /path/to/backup_log.log


You have to define the absolute path to your script, so replace /path/to/ with the exactly absolute path where the script is placed. Once finished to edit the content press the button "ESC" that switches from "INSERT" mode to "VIEW" mode and type ":wq" and hit return. "wq" means "write changes and quit". Now we have added a scheduled execution of the script to the cron daemon. We can verify this typing


crontab -l


that means to "list" all the cron entries for this user. In the example above we have scheduled the execution of the script every month, every day of the week and the month, at 15.45 (in 24h notation, else 3.45PM in 12h notation). In this way your backups will be automated and the old backups will be automatically deleted and replaced with the shifting logic introduced below.

Bye

Saturday, July 21, 2012

My First Post on Happy Engineer - SSH Tunneling

Hi everyone,

my name is Marcello and i'm a junior engineer ( i'm about to graduate ). I have a lot of interests, the two principals are Music and Computer Science. I'm also a drum player (i'm very louder :) ). I don't play drums only for myself, in fact i have a band named Apocalypse Please (quote Muse) and we play Metal music (our songs, few covers).

this is the first time i write on this blog and i'm very proud to do this and I thank Andres for the opportunity he gave me.



The first thing i want to explain in this post is the powerful tool named "SSH". Today i'm a totally Linux lover. When i was young, at the beginning of my computer science experience, i was not very practical of linux and my little knowledge about roots of operating systems doesn't helped me to understand better the potentiality of the little Penguin.

A very powerfull tool is SSH. SSH is the acronym of secure shell and is a kind of prehistoric version of Remote Desktop Connection or VNC. The main difference between VNC, RDC and SSH is that SSH make possible to manage a remote host using without using GUIs, but only through a prompt.

You can think "Bleah, only a poor command prompt?  Where the way to unpack tarball is made of a long sequence of chars??? WTF????"

Answer: "Yes".

Trust me, if you begin to use linux's terminal it will be the END. You'll not use linux without using terminal! It's a drug!

But, wait a second, try to think about the idea of sending only few bytes to manage remote hosts instead of large quantities of bytes for send screenshots of VNC and RDC. It sounds good!!

Leaving behind these considerations, let's talk technically.

If you want to connect to a remote host using ssh you have to be sure of two things:

1 - The remote host must have installed a SSH daemon (OpenSSH for example) and must be correctly configured.

In Ubuntu distributions is very simple to install OpenSSH, just open terminal and type:

    sudo apt-get install openssh-server

Once installed, the daemon will startup automatically when the host starts up, and all the user of the host will be enabled to use ssh service, so if you want to access to your machine you can use the credentials you normally use to access the host.

2- You must know a pair o username-password enabled on the remote host.

Try to connect to a remote host using ssh. Open the terminal and type:

    ssh username@hostip



Where:

    Username is the username on the remote host
    hostip defines the remote host ip you want to connect using SSH

 If you want to test if OpenSSH is working on your local host you can run:

    ssh yourusername@localhost



After the execution of this command you'll be asked to accept a key and you'll need to answer "yes".

    What's the meaning of this question?



SSH bases its operation on the use on a public key cryptography system and the first time you make a connection to a remote host you must accept its public key to start the conversation.

    Wow... Now i'm connected to a remote host.. And now?



The main advantage of using SSH for administrate remote hosts is the limitate usage of bandwidth. It seems a nonsense, but there are a lot of situations where the optimization of bandwidth usage can make the difference.



    Yeah. .. Boring Things...



Wait!

There's a little funcionality of SSH that is very powerfull. The Tunneling. Tunneling defines the possibility to make a tunnel between two hosts and make possible to exchange data between them.



Imagine to have three hosts:

    host A, connected to the internet, placed in Australia
    host B and C in the same subnet but only B is directly connected to the Internet. These two hosts are in Norway.

Imagine also that host A must access to C for some reasons. There are a lot of ways to do this for example VPNs (maybe we talk about this another time).. Another way can be tunneling.

Idea!

Using host B like a gateway for exchange data with C. Great! How?

SSH

There's a little tiny argument that you can pass to SSH command before execute it. The -L argument. Let's see a usage for this argument.

    ssh johndoe@hostB -L 5000:hostC:21



In this example i'm connecting to the host B placed in Norway, using the username johndoe. Passing -L argument i'm saying to ssh to bind port 21 of hostC (hostC must be the subnet's ip of host C) to the port 5000 of my host. So if i connect using a FTP client to port 5000 of my localhost, the communication will be routed through the tunnel to host C using host B like a gateway.

So if you have the possibility to access a host of a subnet using SSH, you'll be able to access on every host of that subnet.

I wish that you will appreciate this post. I'm also happy if you want to correct me in contents or in grammar (i'm not so trained about writing english :) )

Yhank you, see you soon

Wednesday, July 18, 2012

Concatenate arrays with different sizes in Matlab

Sometimes you want to concatenate arrays with different sizes, and Matlab doesn't allow you to. This makes sense, of course, but still you just want to put all that stuff together for some reason. Then this function may help you. As any other functions I've wrote, I'm sure it an be optimized, so I'll be happy to hear suggestions.
You can also download it from my matlab file exchange and rate it :-)



function [catmat]=padconcatenation(a,b,c)
%[catmat]=padconcatenation(a,b,c)
%concatenates arrays with different sizes and pads with NaN.
%a and b are two arrays (one or two-dimensional) to be concatenated, c must be 1 for
%vertical concatenation ([a;b]) and 2 for horizontal concatenation ([a b])
%
% a=rand(3,4)
% b=rand(5,2)
% a =
%
%     0.8423    0.8809    0.7773    0.3531
%     0.2230    0.9365    0.1575    0.3072
%     0.4320    0.4889    0.1650    0.9846
% b =
%
%     0.6506    0.8854
%     0.8269    0.0527
%     0.4742    0.3516
%     0.4826    0.2625
%     0.6184    0.5161
%
% PADab=padconcatenation(a,b,1)
% PADab =
%
%     0.8423    0.8809    0.7773    0.3531
%     0.2230    0.9365    0.1575    0.3072
%     0.4320    0.4889    0.1650    0.9846
%     0.6506    0.8854       NaN       NaN
%     0.8269    0.0527       NaN       NaN
%     0.4742    0.3516       NaN       NaN
%     0.4826    0.2625       NaN       NaN
%     0.6184    0.5161       NaN       NaN
%
% PADab=padconcatenation(a,b,2)
%
% PADab =
%
%     0.8423    0.8809    0.7773    0.3531    0.6506    0.8854
%     0.2230    0.9365    0.1575    0.3072    0.8269    0.0527
%     0.4320    0.4889    0.1650    0.9846    0.4742    0.3516
%        NaN       NaN       NaN       NaN    0.4826    0.2625
%        NaN       NaN       NaN       NaN    0.6184    0.5161

sa=size(a);
sb=size(b);

switch c
    case 1
        tempmat=NaN(sa(1)+sb(1),max([sa(2) sb(2)]));
        tempmat(1:sa(1),1:sa(2))=a;
        tempmat(sa(1)+1:end,1:sb(2))=b;
       
    case 2
        tempmat=NaN(max([sa(1) sb(1)]),sa(2)+sb(2));
        tempmat(1:sa(1),1:sa(2))=a;
        tempmat(1:sb(1),sa(2)+1:end)=b;
end

catmat=tempmat;
end

Thursday, May 3, 2012

Custom Colors for Matlab Plots


This is a(nother) simple function I wrote for matlab, it can also be downloaded from my file exchange at mathworks:
http://www.mathworks.com/matlabcentral/fileexchange/36514-custom-colors-for-plots

In plot graphs, often it's needed to use a several colors that can be easily differentiated on a first look on the graphic. With this function you can call easily (by name or by code) a selection of 15 colors.
It's very simple and there are more elegant ways to do it, but I think it's a very functional solution.


function ccol=CustomColors(ct1)
%This is a function to create some easy to differentiate colors, very
%useful when plotting many things in the same graphic
%The RGB coordinates and names where taken from the following page:
%http://web.njit.edu/~kevin/rgb.txt.html
%
%Examples:
%for ccc=1:15
%plot(ccc*ones(1,10),'Linewidth',4,'Color', CustomColors(ccc))
%end
%
%plot(ones(1,10),'Linewidth',4,'Color', CustomColors('Coral'))
%plot(ones(1,10),'Linewidth',4,'Color', CustomColors('DeepSkyBlue4'))
%
%Copyright: Andres Gonzalez. 2012.

CColors={
    [   [0,0.407843137254902,0.545098039215686] ];% DeepSkyBlue4
    [   [0.545098039215686,0.270588235294118,0.0745098039215686]    ];% SaddleBrown
    [   [0,0.803921568627451,0] ];% green3
    [   [0,0.498039215686275,1] ];% SlateBlue
    [   [0.956862745098039,0.643137254901961,0.376470588235294] ];% SandyBrown
    [   [0.419607843137255,0.556862745098039,0.137254901960784] ];% OliveDrab
    [   [0.800000000000000,0.196078431372549,0.600000000000000] ];% Violet,Red
    [   [0.556862745098039,0.419607843137255,0.137254901960784] ];% Sienna
    [   [0.803921568627451,0.678431372549020,0] ];% gold3
    [   [0.545098039215686,0,0.545098039215686] ];% magenta4
    [   [1,0.498039215686275,0] ];% coral
    [   [1,0.843137254901961,0] ];% gold1
    [   [0.600000000000000,0.196078431372549,0.800000000000000] ];% DarkOrchid
    [   [0.941176470588235,0.501960784313726,0.501960784313726] ];% LightCoral
    [   [0.635294117647059,0.803921568627451,0.352941176470588] ];% DarkOliveGreen3
   
    };
NColors={
    'DeepSkyBlue4';
    'SaddleBrown';
    'Green3';
    'SlateBlue';
    'SandyBrown';
    'OliveDrab';
    'VioletRed';
    'Sienna';
    'Gold3';
    'Magenta';
    'Coral';
    'Gold1';
    'DarkOrchid';
    'LightCoral';
    'DarkOliveGreen3';
    };

switch class(ct1)
    case 'double'
        if ct1<1
            ct1=1
            disp(sprintf('Color index must be between 1 and %d, taking closest value [%d]',length(CColors),ct1))
        elseif ct1>length(CColors)
            ct1=length(CColors)
            disp(sprintf('Color index must be between 1 and %d, taking closest value [%d]',length(CColors),ct1))
        end
       
       
       
    case 'char'
        ctfound=0
        for ct2=1:length(NColors)
            if (strcmp(NColors{ct2},ct1))
                ctfound=ct2
            end
        end
        if ctfound==0
            disp(sprintf('Color not found taking default value [%d]',1))
        else
            ct1=ctfound
        end
       
    otherwise
        disp(sprintf('Color not valid taking default value [%d]',1))
       
       
end





ccol=CColors{ct1}
end






Tuesday, April 17, 2012

Save Workspace to Struct

This was a nice function that I created today, and it allows to save all the variables from the "base" workspace to a single struct, in order to have them all nicely packed in case you want to create mat file with them and later reload it without having so much worry of overwriting the variables in the other workspace. I gratefully accept suggestions for better options.
And now it's also available at the Matlab Central (if you want to rate it, which is always welcome):  http://www.mathworks.com/matlabcentral/fileexchange/36257-save-workspace-to-struct

%This Script saves all the variables from the current workspace into a
%single structure array.
%Created by Andres Gonzalez. 2012

function WStruct=ws2struct()

WSVARS = evalin('base', 'who');
for wscon=1:size(WSVARS,1)
    thisvar=evalin('base', WSVARS{wscon});
    eval(strcat('THEWORKSPACE.(WSVARS{wscon})=thisvar;'))
end

WStruct=THEWORKSPACE;

Monday, February 27, 2012

Brain Registration with ANTS Toolkit

In most neuro imaging research it is needed to study specific areas of the brain instead of the wholevolume. For me, as many other researchers, it is very difficult or impossible to locate by hand some specific brain areas. And it is not desired to do conclusions over an area that is not correctly located, because this conclusions will end up corresponding to some other part of the brain. So, is very important to use a good tool for registration. The following papers explain why ANTs is one of the best tools for Brain Registration: 
http://www.mindboggle.info/papers/evaluation_NeuroImage2010/Evaluation_Klein_NeuroImage2010.pdf
http://www.mindboggle.info/papers/evaluation_NeuroImage2009/Evaluation_Klein_NeuroImage2009.pdf
There are no “best parameters”for every registration, it mostly depends on the resolution of the images and the moving and fix spaces as well as many other characteristics. However, I think that there should be some basic recommendations for running a correct corregister, because it would be a waste of time to try on with hundred possibilities for each parameter. As a new ANTs user I struggled a lot trying to find out which could be the best options to be used, because every paper I read would say that the best parameters were different depending each specific situation. But still, for a new user is difficult to guess from zero the best options. So finally, after reading ANTs manual and with personal communication with Dr. Klein, I did a resume that I hope can help as a basis for other people that are learning about ANTs as well.  
The file ants.h that can be found on my web page contains my recommended parameters for doing brain registration from an atlas to a subject's space. It can be executed directly from the command line if you have ANTs installed in your computer. You can find there as well an explanation on how the parameters were chosen. 
Visit www.monicagiraldochica.com (go to 'My PhD Project' > Project Progress > Files)

Wednesday, February 22, 2012

Touchpad not working AT ALL (Not even in BIOS)

Ok... not having time to post more elaborated stuff... I will go on to give talk about this apparently simple problem...

Today I found out my touchpad was not working. Ok I said... It must be something disabled on the device setup... Nop, it's not. Well... yep, it must be the driver. Uninstall, reboot, re-install, reboot... nop. Ok, it must at any rate, be a Windows problem... let's try Linux... reboot, come on Linux darling... nop, still not working... what the....????

Well, it should work in the BIOS, or there is maybe some option disabled... checking the BIOS... nop, no way... is the hardware broken? did I spill some liquid? did I hit it? no... no that I remember.

Let's google it... config? check, drivers? check, bios? check... ok let's see this post by whiteblade here and... wowwww that actually worked!

The solution: DISCONNECT THE LAPTOP AND TAKE OUT THE BATTERY FOR SOME MINUTES. It should gave time to some part of the memory to discharge, so when out turn it back on, your touchpad will be alive.

Yep, this might not work for everyone... but it did for me and for other people... so this post should help those who are about to start tearing their shirts apart and yelling at the laptop for being so fragile.