Linux Geek


This is my repository for computer stuff, where I will "try" to leave interesting things and reminders to myself of the programming/computing world... Feel free to help me out, of course.




Moved to Blogger

posted Dec 3, 2009 2:18 AM by Bruno Braga   [ updated Jan 9, 2010 8:22 AM ]

Yeah, well.... since Google Sites are not meant to be a blog (at least until this moment), I have decided to move to a blogging services, where I can at least allow collaboration from friends and colleagues to my experiences...

Check it out here: http://blog.brunobraga.net/


Latest Posts .:. Linux Geek


[bash] Using file descriptors

posted Nov 18, 2009 1:13 AM by Bruno Braga   [ updated Nov 18, 2009 1:27 AM ]

 
Working on a script for linscripts project (something I am working on my spare time), I encountered the following issue: I could not update a file info (such as a simple setting value) on a single line. I will explain in details:

I wanted to, by script, update a single value on a configuration file called apport, to enable crash reports on Ubuntu.
It is basically simple to do that such as:

    sed -e 's/enabled\=0/enabled\=1/' /etc/default/apport

However, the command below does not work:

    sed -e 's/enabled\=0/enabled\=1/' /etc/default/apport > /etc/default/apport

or worse, it removes all contents from the original file (do not try to do this!). So it means (I did not know) that stdin a file to a command and using the stdout directly does not work. Ok, you could do a temporary file and rename it back, of course, but I have found a more elegant solution in terms of scripting: to open a file for read/write using file descriptors:

# backup current file (always nice thing to do)
cp /etc/default/apport /etc/default/apport.old

# open the file for read/write into file descriptor 3
# (0, 1 and 2 are already taken by stdin, stdout and stderr respectively)
exec 3<> /etc/default/apport

# read the file, update and write it back
cat <&3 | sed -e 's/enabled\=0/enabled\=1/' >&3

# save/close
exec 3>&-

Nice!

References

    [1] http://tldp.org/LDP/abs/html/io-redirection.html

Ubuntu 9.10 (ext4) Backup with FSArchiver

posted Nov 9, 2009 6:45 PM by Bruno Braga   [ updated Nov 9, 2009 7:04 PM ]

Some time ago I wrote about creating image backups (at "Ghost" Image backup), which worked pretty well for me, until now. Just recently installed the new Ubuntu 9.10 Karmic Koala version, which comes with ext4 file system by default, and that is, the partimage application does not support it.

By looking to some forums (here) and the partimage application main page, I decided to give it a try to fsarchiver application, and seems to be working just great.

So, here is what I did (see also my previous post "Ghost" Image backup):

1) Mount the external storage

This thing you need to do manually, but it is no trouble at all, if you know what you need to do... since I was in this group, it took me a while, hehe...

In my case I am mounting a NTFS drive, so that I can transport files to multiple OS without problems (and over 4GB). Just Mac has issues with NTFS, as the drives are read-only.

# Create mounting directory
mkdir /mnt/{your drive name}

# Mount the storage
mount.ntfs-3g /dev/{device code} /mnt/{your drive name}

2) Backup your data with fsarchiver application

# substitute the drive name by the mounted name in (1)
# check the device that represents your hard drive to backup
fsarchiver -o /mnt/{your drive name}/backup.fsa
/dev/{device code} -v -j2 -A


Done!

More detailed instructions on how to use fsarchiver at: http://www.fsarchiver.org/QuickStart

[BASH] Read command for paths

posted Nov 3, 2009 8:55 AM by Bruno Braga   [ updated Nov 3, 2009 9:12 AM ]

Today I was struggling to prepare some easy ways for converting video files (will post that later on), and came up with lots of interesting things on bash scripting. One of them is the READ command.

Besides what you can find in the man pages (note: Ubuntu 9.10 does not have the man page for this command), here are some things you may need to look into when using that:

Input paths

To have the easiness of the "tab" feature to auto-complete the path name of a file or directory, you need to set the option "-e". Strange is that you can not fully understand the power of this option just by reading the manual. Example:

    read -e -p "Type the file path: " file

Tilde on input paths

Another thing that bothered me a lot is the fact that the input value, even though is coming from the auto-complete feature from the terminal interface, is not fully understood as a path, but as a string. That means, for instance, if the path contains a tilde (~) character, the path will not be valid for a serious of operations.

For example, from the previous command (above):

    read -e -p "Type the file path: " file
    [ -f $file ] && echo "This is a file!"

If you try this with a string "/home/user/", it will work, but if you try with "~/", it will fail :-(.

The solution for that is to implement the EVAL command:

    read -e -p "Type the file path: " temp
    eval file=$temp
    [ -f $file ] && echo "This is a file!"

And voila, now it works!

Github Offline

posted Oct 30, 2009 8:10 AM by Bruno Braga   [ updated Oct 30, 2009 9:08 AM ]


I just finished upgrading my Ubuntu to the newest release (Karmic Koala - 9.10), and went to my repository GitHub to install my preferred applications, at: ubuntu-fresh-install... To my surprise, it was off-line!!!


I heard a lot about distributed source control, blah blah, but this time I really needed the server there (rather than my local repository). Had to find it from my last backup (good to do that once in a while). 

Update (2009/10/31 1:08 AM JST):

The problem continues...


Copy command progress from Command Line

posted Oct 30, 2009 5:15 AM by Bruno Braga   [ updated Oct 30, 2009 6:00 AM ]


Never thought about using rsync for purposes different than synchronization. Yeah, well.... Recently, I ran into a situation where I had to copy a bunch of stuff from one place to another, but the size of the contents were matter of GBs, and I kind'a needed to know the progress of it, like when you so it from GUI...

Searching a bit, I found that you could use rsync for that (impressively, "cp" or "mv" commands do not have such feature). So here it is:

rsync --archive --verbose --human-readable --progress /path/to/origin /path/to/destination

Note: for those not familiar with rsync, read about the "--archive" option before using it.

$ rsync -av --human-readable --progress --stats /path/to/origin /path/to/destination
sending incremental file list
video1.avi
       2.96G 100%    2.03MB/s    0:23:11 (xfer#1, to-check=4/5)
video2.avi
     288.24M 100%    1.90MB/s    0:02:24 (xfer#2, to-check=3/5)
video3.avi
     302.28M 100%    1.89MB/s    0:02:32 (xfer#3, to-check=2/5)
video4.avi
     576.69M 100%    2.02MB/s    0:04:32 (xfer#4, to-check=1/5)
video5.avi
     182.67M 100%    1.84MB/s    0:01:34 (xfer#5, to-check=0/5)

Number of files: 5
Number of files transferred: 5
Total file size: 4.31G bytes
Total transferred file size: 4.31G bytes
Literal data: 4.31G bytes
Matched data: 0 bytes
File list size: 240
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 4.31G
Total bytes received: 107

sent 4.31G bytes  received 107 bytes  2.10M bytes/sec
total size is 4.31G  speedup is 1.00

First Kernel Patch

posted Oct 25, 2009 4:21 AM by Bruno Braga   [ updated Oct 25, 2009 6:22 AM ]

This is interesting topic I would like to share.

Recently, I participated on the 1st Japan Linux Symposium, held in Tokyo last week, and I could meet many amazing people from the Linux movement. One of them was Greg Kroah, who is currently one of the top maintainers of the Linux Kernel, working closely to Linus Torvalds.

His presentation was about "Write & Submit Your First Linux Kernel Patch", and gave me insights that it is easier to participate than anyone thinks. Of course, you gotta clean out a lot of toilets until you get somewhat noticed, but at least I could grasp that it is not so difficult as it seems.

Of course, Greg helped a lot (thanks) to incentive everyone to participate. Not all open source communities are like that, a kind of pride thing and inner-circle closed (which, IMO, goes against the OS principle, but I agree that some level of control is required to avoid chaos). Well, the differences happen a lot in the Kernel as well (you can see some here), some getting even too intense, but in general the community is pretty opened to newcomers.

Well, anyway, during the 45min lesson with him, plus a little (hours) of search on the web, I could finally place my first contribution (yet to be accepted to the main tree though): http://patchwork.kernel.org/patch/55147/

So I will document here my steps (you can also see similar stuff from here and here).

The official presentation and guidelines from Greg are available at: http://github.com/gregkh/kernel-tutorial

1) Kernel Source code

Follow the steps below for starting to work on the kernel source code:

# get the kernel source code
$ git clone git://github.com/github/linux-2.6.git
$ cd linux-2.6

# create a git branch "tutorial" for you to work on (it is not recommended to work on the master)
$ git checkout -f -b tutorial

2) Updating the source code

The example below was based on the "homework" given to me to work with. I suppose that anyone can start from the staging drivers, which is a part of the Kernel source code that is not officially accepted, so it should cause no big trouble to other modules or the core code.

# check the file consistency (Greg gave us a particular driver on staging drivers to fix coding style) - you can search yourself for other files with issues
$ ./scripts/checkpatch.pl --file drivers/staging/comedi/drivers/adl_pci8164.c

# -----------------------
# the result should be something like:

WARNING: printk() should include KERN_ facility level
#131: FILE: staging/comedi/drivers/adl_pci8164.c:131:
+    printk("comedi: attempt to attach...\n");
[...]
total: 0 errors, 10 warnings, 391 lines checked

The checkpatch.pl script is amazing, as it shows which consistency errors and warnings should be treated in the code. This also serves as guidelines for starters such as myself to get used to the coding style of the Kernel, and what you should pay attention to when developing things from scratch.

Once you do your changes, you do the check again:

# check the file consistency again (until there are no errors//warnings)
$ ./scripts/checkpatch.pl --file drivers/staging/comedi/drivers/adl_pci8164.c

# -----------------------
# the result should be something like:

total: 0 errors, 0 warnings, 392 lines checked
drivers/staging/comedi/drivers/adl_pci8164.c has no obvious style problems and is ready for submission.


Don't forget to compile the kernel or module where you changed, otherwise you will make a lot of people angry later... This part was a little bit tricky for me (although others did not face the same problem).

# re-compile the kernel to make sure everything is fine (but only partial, that will be more than enough for this exercise)
$ make M=drivers/staging/comedi/drivers/

If you have problems here, try to look for examples on "make menuconfig" and MakeFiles.

3) Patching

When your changes are ready to be patched, you can then commit them to your git branch:

# commit your changes to your git branch
$ git commit -a
# This action will open the VI editor (in Ubuntu is set by default - search the web if you need to manually define one), so that you can place the commit message (what is your change about).

# prepare the patch for the changes you've made
$ git format-patch -s -n master..tutorial

# this action will create a patch file, with name based on the commit message you wrote before.
# -----------------------
# the result should be something like:

0001-Corrected-coding-style-excessive-curly-braces-and-p.patch

# check the file consistency for the patch
$ ./scripts/checkpatch.pl --file 0001-Corrected-coding-style-excessive-curly-braces-and-p.patch


# -----------------------
# the result should be something like:

total: 0 errors, 0 warnings, 126 lines checked
0001-Corrected-coding-style-excessive-curly-braces-and-p.patch has no obvious style problems and is ready for submission.

I found some articles informing ways to add the "Sign-off" automatically, by updating the mains bash script (~/.bashrc). This did not work for me very good. So what I did was to update the git configuration myself, for the current clone:

# commit your changes to your git branch
$ git config -a

# define special configuration
# see details below on msmtp
$ git config --add sendemail.smtpserver /usr/bin/msmtp
$ git config --add user.name "BRAGA, Bruno"
$ git config --add user.email bruno.braga@gmail.com

# my git configuration is:
$ git config --list
# ---------------
sendemail.smtpserver=/usr/bin/msmtp
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.url=git://github.com/github/linux-2.6.git
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.master.remote=origin
branch.master.merge=refs/heads/master
user.name=BRAGA, Bruno
user.email=bruno.braga@gmail.com
# ---------------

# use --global if you want those settings to be default for any git clone.


Lastly, you need to send the patch to the maintainers and the mailing list:

# send the email using git functionality
$ git send-email --from "BRAGA, Bruno <bruno.braga@gmail.com>" --to "LKML <linux-kernel@vger.kernel.org>" --cc "Greg Kroah <greg@kroah.com>" 0001-Corrected-coding-style-excessive-curly-braces-and-p.patch

Here there is another trick. To use the "git send-email" tool, actually you need to define a mailing tool and the SMTP settings for it. I've seen that people is mainly using msmtp tool for that, so I decided to go the same way. So here is what I did:

# install msmtp
$ sudo apt-get install msmtp

# configure msmtp
$ vim ~/.msmtprc

# this will open the VI editor (you can use another one if you like), then you add the settings:
# --------------
account default
host smtp.gmail.com
port 587
auth on
user {your account name here}@gmail.com
password {your password here}
tls on
tls_starttls on
tls_trust_file ~/.certs/Thawte_Premium_Server_CA.pem
from {your account name here}@gmail.com
# --------------

# since you NEED to leave your password as text on this file, it is recommended to change the permissions (default permissions allow everyone to see this file):
$ chmod 600 ~/.msmtprc

# the job here is not finished, as you need to leave the Thawte certificate file on the defined address. So here is how you do it:

# download the certificates (Verisign offers that)
$ wget https://www.verisign.com/support/thawte-roots.zip

# unpack the files (I recommend doing that on /tmp/ folder)
$ unzip thawte-roots.zip

# go to the folder where you need to get the certificate (this may vary a little bit on time, but you can just look for it within the contents downloaded)
$ cd Thawte SSLWeb Server Roots/thawte Premium Server CA/

# create the certificate directory if you don't have one
$ mkdir ~/.certs

# copy and properly rename it (remove spaces is good one)
$ cp Thawte Premium Server CA.pem ~/.certs/Thawte_Premium_Server_CA.pem

If you have issues with the msmtp, see other posts, such as in here.

Done!

Not that difficult, huh? Of course, first patch experience is mere fixing code formatting and style, but it is a starting point, and you should be proud to participate. Greg himself told that his first patches were similar stuff, and even the almighty Googlers' (Google employees) contributions are partially english spelling fixes (you can see statistics at Greg Kroah Hartman on the Linux Kernel 28:00 - I am serious!)... So, what are you waiting for?

Merging PDF files

posted Oct 24, 2009 12:38 AM by Bruno Braga   [ updated Oct 24, 2009 12:54 AM ]

I was looking at the Linux Kernel in a Nutshell, written by Greg Kroah, and found that the PDF version of the book was all splitted in small files by chapter. I just wanted to wrap them all up into a single file (obvious reasons), so I started looking for solutions on PDF merging. This one here (by lobner) was pretty suitable for me (from the extracted directory containing all files):

gs -q -sPAPERSIZE=letter -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -sOutputFile=LinuxInNutshell.pdf title.pdf ch00.pdf part1.pdf ch01.pdf ch02.pdf ch03.pdf ch04.pdf ch05.pdf ch06.pdf part2.pdf ch07.pdf ch08.pdf part3.pdf ch09.pdf ch10.pdf ch11.pdf part4.pdf appa.pdf appb.pdf LKNSIX.fm.pdf

This command (uses GhostScript) gives me some warnings, but since the output file is OK, I didn't even run to the trouble of torubleshooting it...

   **** Warning: File has a corrupted %%EOF marker, or garbage after %%EOF.

   **** This file had errors that were repaired or ignored.
   **** The file was produced by:
   **** >>>> Acrobat Distiller 4.05 for Sparc Solaris <<<<
   **** Please notify the author of the software that produced this
   **** file that it does not conform to Adobe's published PDF
   **** specification.

[...] (this goes multiples times, once per file)

By the way, thanks Greg by the book! I promise I'll read it before annoying you with compiling issues and stuff!

Lan-HD (automating samba shares)

posted Oct 17, 2009 8:13 AM by Bruno Braga   [ updated Oct 24, 2009 12:37 AM ]

I just purchased a NAS Bufallo LS-XHL 1TB, (also called Lan hard drive) to ease the pain of saving all my files inside my laptop (and of course, considering that I have multiple computers at home). It was my first time setting up this, and it is always gratifying to see the advertisement and manuals saying: "for Windows Vista or Mac OS". Well, of course that means nothing as I already know that there aren't many Linux only customers (this reality will change, we already got 1% of the cake! ).

Anyway, just to keep here my experience with it.

Plugging the HD in to my network

Note: Image extracted from manual.

Locating the HD

I figured that it would connect to my network (with offers DHCP) by default. The issue was just to locate it. Thanks to nmap, this is very easy:

nmap 192.168.X.1/24

This command will do a scan/portscan on every IP from segment X (router's default segment, usually 1 or 11) from 1 to 254.

And, bingo! I found my hard-drive (just reminding that nmap gets me much more than just IPs, and I could verify that default settings came with no sharing options, maybe they were expecting a direct USB connection or an installation application that would guide the user). As we don't have any of that on Linux, we do it the hard way...

Starting Nmap 4.76 ( http://nmap.org ) at 2009-10-18 00:59 JST

Interesting ports on 192.168.X.Y:
Not shown: 989 closed ports
PORT      STATE SERVICE
80/tcp    open  http
139/tcp   open  netbios-ssn
445/tcp   open  microsoft-ds
515/tcp   open  printer
873/tcp   open  rsync
3689/tcp  open  rendezvous
8873/tcp  open  unknown
9050/tcp  open  tor-socks
22939/tcp open  unknown


Detecting that the port 80 (HTTP) was opened, I figured that everything could be done with a web interface (only if I have looked th e complete manual online, that does not come with the product itself...)

So I just opened the browser and entered http://192.168.X.Y/ and voila!



I will skip the boring configuration for this (I am sure no one will have problems here).

Opening the HD

Once I properly configured (opened access as FTP, Windows and Mac, restricted to users: admin for read/write with proper password) the HD, I noticed (thru nmap) that now ports 21 and 22 became opened. My first action then was to open from Nautilus GUI, by typing:

ftp://admin@192.168.X.Y/disk1/sharedfolder/

This works fine, but I got (from the manual for Mac connection) that Windows support is interpreted in Unix environment OS's with Samba, so I gave it a try to:

smb://192.168.X.Y/sharedfolder/

It works much better/faster. I will keep this one. Note the difference for connecting with FTP: you need to add the root name of the disk to make it work.

Automating Samba Shares


As I got everything running, I thought that it would be nice to have the HD always available to me, as if it was there (being plugged in to my network, unlikely to be shutdown). So I searched a bit here and here, and everything works great for me now.



But, for personal references, I will leave reported what I did:

# check if you need to install samba dependencies
sudo apt-get -y install smbfs

# create media folder to be mounted (NW-HD-1TB is the name I chose to give to my HD)

sudo mkdir /media/NW-HD-1TB

# create credentials for fstab
sudo vim /root/.credentials.NW-HD-1TB

# and write the following to this file:
username=admin
password=yourpassword

# Remove default privileges for this file (644 - everyone can see it)
sudo chmod 600 /root/.credentials.NW-HD-1TB

# edit the fstab file
sudo vim /etc/fstab

# and add the following at the end of the file:
//192.168.X.Y/share/ /media/NW-HD-1TB   cifs  auto,credentials=/root/.credentials.NW-HD-1TB,iocharset=utf8,uid=bruno,gid=users,file_mode=0775,dir_mode=0775 0 0

# you are done!

# next time you reboot, it will be there, but you can also check if everything went fine by typing (to re-mount everything on fstab)
sudo mount -a


Final notes (about references):
  • The references where I learned this from get confused on the Ubuntu users and the HD server users. They are not related at all. For instance, the "admin" user I am using does not even exist in my ubuntu.
  • The credentials file is a MUST, but place a more specific name (something like: credentials-hd1) unless you plan to use a single one for every similar operation you need in fstab.

Search in Files from Command Line

posted Oct 15, 2009 12:28 AM by Bruno Braga   [ updated Oct 15, 2009 1:29 AM ]


This becomes handy once in a while, if you are connecting to some server via SSH, or simply don't want to open a GUI text editor for a simple search.

for f in `find ./ -name '*' 2>/dev/null`; do grep -nsiHI keyword $f; done

This would search recursively from the current directory. If you do not wish that, just use a simpler "for f in *", it should do the trick. The arguments from grep are also pretty cool, and it is worth to take a look if you have time.

‹ Prev    1-10 of 38    Next ›

 BRAGA, Bruno

 



Brazilian currently based in Japan, working on Information Technology.