Monthly Archives: June 2015

2015 SDSU Metagenomics Workshop

The 2015 SDSU Metagenomics Workshop is designed to be a combination of lectures, discussions, and practical hands on experience to bring people up to date on data analysis for metagenomics.

The workshop is being held in Adams Humanities Room 2108 from 10 am – 6 pm every day from June 22nd – 26th, 2015.

Registration is closed.

The agenda is online here, and will be updated as we progress.

We will use a VirtualBox virtual machine during the class. More information about the image and how to download is here. (Please note, the image is still subject to change, and so don’t download it yet!)

Linux commands for Mac OS X

We all know that Mac OS X is UNIX certified allowing command line users to use Terminal to execute quick Bash or Perl script pipelines. However, for us that are more familiar with the GNU Linux environment, there are some slight variations in the commands we often use, such as the –complement flag in cut, or the -h flag in sort. These options are non-existent in the OS X versions of the commands, and there are probably more instances of these that I haven’t run into yet.
There is a very simple way to install these GNU commands for your Mac using Homebrew:
brew install coreutils
This will allow you to use the GNU versions of the commands, for example, gsort is used in place of sort, and gcut in place of cut.
If you don’t have Homebrew installed (it’s almost like using apt-get to install programs and commands) you can install it by first installing Xcode and making sure command line tools is installed for Xcode. Then run the following command:
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Some online blogs recommend installing XQuartz afterwards too.
Note: this was performed on OS X Yosemite version 10.10.3

Virtual Machine Images

We have several virtual machine images that we use for courses and other reasons.

You can use most of these with Virtual Box, a free emulator.

  • 2015 SDSU Metagenomics Workshop Image [image no longer available]

Here are instructions for setting up a Virtual Box Virtual Machine.

If the disk is limited for space, you can share a folder with your host machine. Here are instructions for setting up a shared folder on your machine

Resizing a kvm image from ubuntu

Because the ubuntu install uses LVM it is not as simple to resize a KVM image.

First, shutdown the original image, and make a back up of that image. This is important!

Next, increase the size of the image. For example, this will add 5GB to your image.

qemu-img resize image.img +5G

start the image running, and log into it.

virsh start image

Use parted to edit the drive. Note: don’t use fdisk for this, as the sizes never work out. Parted allows you to just increase the size of a partition, whereas in fdisk you delete the partition and create a new one (if you do this it is critical the new ones start at the same point as the old ones, and I couldn’t get fdisk to do that).

parted /dev/vda

Set the units to sectors

(parted) unit s

Display the current partition table

(parted) print

Increase the size of the extended partition . Set the end to 1 less than the disk size listed in print. Mine looked like this: Disk /dev/vda: 52428800s and so I set it to 52428799s

(parted) resizepart 2

Increase the size of the logical (lvm) partition

(parted) resizepart 5

use the same end as before.

Now we need to expand the size of the physical and logical volumes. First, lets see their current size:

pvdisplay

and now resize the the physical volume to occupy the whole disk:

pvresize /dev/vda5

Similarly, for the logical volume:

lvdisplay

extend the size of the disk. We do this in two steps. The first one causes an error and then we use that for the second command. Here the key is to use a size bigger than you added to the disk (I only increased my disk by 5G).

lvextend -L+10G /dev/vname/root

This will cause an error:

 Extending logical volume root to 28.74 GiB
 Insufficient free space: 2560 extents needed, but only 1224 available

So we use lvextend again to add the number of extents that are available:

lvextend -l+1224 /dev/vname/root

and finally extend the disk:

resize2fs /dev/vname/root

You have now extended the size of the disk!

 

Changing the volume name on an LVM machine

Changing the volume group name on an LVM machine is easy, but doing it so that it doesn’t break everything is hard. Here are some pointers on how to change the volume group name on an Ubuntu LVM machine.

First, use

vgdisplay

to show your current volume groups, choose the one you want to rename and rename it using vgrename.

vgrename oldname newname

Now you need to edit /etc/fstab to replace the old name with the new one everywhere in that file.

vi /etc/fstab

Edit /boot/grub/grub.cfg and do the same thing

perl -i.old -npe 's/oldname/newname/g' /boot/grub/grub.cfg

Update the image ram disk:

update-initramfs -u   (this updates the current version)

And reboot.

It may well crash here, especially if the volume name has changed. If you end up in busybox, look in /dev/mapper and see what the volume is called. Next, reboot the machine (you probably need to power cycle it for this), and then when the grub menu comes up listing your machines, edit the line that says linux /dev/mapper/…. so that the name listed there is the same as the one you saw.

You should be able to reboot now, and then run

update-grub2

Finally, check that the name you provided in /etc/fstab was correct (mine was not, so I re-edited that file).

 

 

 

fastq to fasta

We often have people ask us how to convert fastq files to fasta format. We have a variety of code on this website, but sometimes that is not easy enough.

Here are a couple of ways to do it on the command line: using a PERL script written by Bas, using the command line, or using prinseq-lite. Here is a C++ version that you can compile (e.g. with c++ -o fastq2fasta fastq2fasta.cpp) and run on your machine.

We also have a simple form that converts fastq files to fasta files (DNA only … it does not give you the quality scores).