Tag Archives: Linux

bash

Use column to display tsv files in columns

If you have a tab separated file and view it in a pager like less or more, the columns never line up. Here is a simple way to make those columns appear correct.

column -t file.tsv

For example, here is a file with three columns of words, displayed with cat

If we pass that to column with the -t option to detect the columns, we get nicely organised columns:

However, note that this is not exactly correct, notice that “Paradigm shift” has been split into two columns because the -t option uses whitespace by default, so to display the columns using tabs, we need to add a -s option:

column -t -s$'\t'

Reading and writing to the same file

If you try to modify a file (removing all empty lines for example) using a command like:

 cat file.txt | sed '/^$/d' > file.txt

you will end up with and empty file.txt. The reason is that bash parses the command line looking for “metacharacters” (  “|” , “>” and “space”  in this case) that separate words, then groups and executes those words according to their precedence. This means that “> file.txt”  get executed FIRST. This creates an empty file.txt (overwriting any existing file) and a “process”  to redirect standard output to that file. Then “cat file.txt” get executed, but by now file.txt is empty. So “cat file.txt” outputs 0 lines,  “sed ‘/^$/d’ ” deletes all 0 empty lines, and 0 lines get written to file.txt . This works as “intended” and bash outputs no error.

You can get around this using a temporal file.

 cat file.txt | sed '/^$/d' > tmp_file.txt
mv tmp_file.txt file.txt

But, as file.txt is technically a new file you might lose some information, in particular permissions and whether file.txt was originally a symbolic link or not.

Other options is to use sponge, which is part of moreutils and sadly not standard in many systems.

 cat file.txt | sed '/^$/d' | sponge file.txt

 

Linux commands for Mac OS X

We all know that Mac OS X is UNIX certified allowing command line users to use Terminal to execute quick Bash or Perl script pipelines. However, for us that are more familiar with the GNU Linux environment, there are some slight variations in the commands we often use, such as the –complement flag in cut, or the -h flag in sort. These options are non-existent in the OS X versions of the commands, and there are probably more instances of these that I haven’t run into yet.
There is a very simple way to install these GNU commands for your Mac using Homebrew:
brew install coreutils
This will allow you to use the GNU versions of the commands, for example, gsort is used in place of sort, and gcut in place of cut.
If you don’t have Homebrew installed (it’s almost like using apt-get to install programs and commands) you can install it by first installing Xcode and making sure command line tools is installed for Xcode. Then run the following command:
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Some online blogs recommend installing XQuartz afterwards too.
Note: this was performed on OS X Yosemite version 10.10.3

Changing the volume name on an LVM machine

Changing the volume group name on an LVM machine is easy, but doing it so that it doesn’t break everything is hard. Here are some pointers on how to change the volume group name on an Ubuntu LVM machine.

First, use

vgdisplay

to show your current volume groups, choose the one you want to rename and rename it using vgrename.

vgrename oldname newname

Now you need to edit /etc/fstab to replace the old name with the new one everywhere in that file.

vi /etc/fstab

Edit /boot/grub/grub.cfg and do the same thing

perl -i.old -npe 's/oldname/newname/g' /boot/grub/grub.cfg

Update the image ram disk:

update-initramfs -u   (this updates the current version)

And reboot.

It may well crash here, especially if the volume name has changed. If you end up in busybox, look in /dev/mapper and see what the volume is called. Next, reboot the machine (you probably need to power cycle it for this), and then when the grub menu comes up listing your machines, edit the line that says linux /dev/mapper/…. so that the name listed there is the same as the one you saw.

You should be able to reboot now, and then run

update-grub2

Finally, check that the name you provided in /etc/fstab was correct (mine was not, so I re-edited that file).

 

 

 

Become Even More Productive – 2 New Cool Things!

Over the past couple of days I’ve been toying with a couple of neat ways to access / visualize data. Rob showed me Cooliris, which was pretty mind blowing as far as just looking like it came straight out of a sci-fi movie. More seriously, I’ve found that it is actually a pretty efficient way to do image searches. It displays images much more quickly than a simple google image search result.

The other awesome thing that I’ve found is the Compiz-Fusion suite of window management plugins for Linux. (You can download a version for your distro here). It allows you to manage a large number of extended desktops; this is particularly useful on the EEEPC netbooks which have small monitors! Other cool features worth checking out are the Ring-switch plugin ( + tab for application switches), Desktop Cube & Rotate Cube (Alt-Ctrl-Mouse button 1 click and drag). Turning transparency on the cube is actually pretty nice because it allows you to view a number of desktops all at once.

Some cool things to do just for looks are: 
Enable Skydome on the cube (use custom backgrounds while in cube view).
Try out the water plugin (ctrl +  to initiate, Shift+f9 and shift+f8 for rain control).
Turn on wobble plugin for your application windows.
Check out the wiki to learn even more awesome stuff you can do!

Now that you’re all informed, install this stuff on your linux machines, go out, and be impressive!