Thursday, August 30, 2007
Tuesday, August 28, 2007
Search and Replace Inline many files
One line way if you have perl to search and replace instances of a certain text with something else. Don't forget to escape special characters w/ a backslash (\).
perl -pi -e 's/find/replace/g' *.txt
Or, to change matching files in a hierarchy:
find . -name '*.txt' |xargs perl -pi -e 's/find/replace/g'
perl -pi -e 's/find/replace/g' *.txt
Or, to change matching files in a hierarchy:
find . -name '*.txt' |xargs perl -pi -e 's/find/replace/g'
Thursday, January 11, 2007
Difference between Unix mtime, ctime, atime
Unix keeps track of 3 types of timestamps with a confusion happening between ctime and mtime:
atime: stands for access time which is when the file was last read.
ctime: is the inode change time. When does the inode change, when you of course update a file, but also when you do things like changing the permissions of the file but not necessarily its contents.
mtime: is the modification time, so if you change the contents of the file, this time is updated. Keep in mind when you change the contents of a file and and save it, the inode gets updated as well for that file, so ctime always changes too.
Therefore, atime is when you last read a file, ctime and mtime can be different depending of if you just modified the inode or the contents of the file (which updates ctime as well).
Let's say you use backup software that only backs up files that have changes in the ctime. This means that if you had a file and you changed the permissions, but not its contents, the backup software would see this as an updated file and still back it up.
Let's say you had a file you're restoring from 2 years ago, and some backup programs are able to restore the mtime as well, so if today is 2007, the file would show up with a date of 2005 in the timestamp when you do an "ls -l". However, ctime would be today in 2007 because that's under system control, and as soon as you restored the file back, inodes changed, and ctime is current.
If you have a backup program that does not restore mtime, then that's okay since you can change the modification time with the "touch" command. For example if you need to change the date to Jan 11, 2005 12AM to file.txt, then you can go:
# touch -t 200501111200 file.txt
So mtime is the timestamp of a file, and ctime is kept by the system, and when mtime changes ctime always changes.
atime: stands for access time which is when the file was last read.
ctime: is the inode change time. When does the inode change, when you of course update a file, but also when you do things like changing the permissions of the file but not necessarily its contents.
mtime: is the modification time, so if you change the contents of the file, this time is updated. Keep in mind when you change the contents of a file and and save it, the inode gets updated as well for that file, so ctime always changes too.
Therefore, atime is when you last read a file, ctime and mtime can be different depending of if you just modified the inode or the contents of the file (which updates ctime as well).
Let's say you use backup software that only backs up files that have changes in the ctime. This means that if you had a file and you changed the permissions, but not its contents, the backup software would see this as an updated file and still back it up.
Let's say you had a file you're restoring from 2 years ago, and some backup programs are able to restore the mtime as well, so if today is 2007, the file would show up with a date of 2005 in the timestamp when you do an "ls -l". However, ctime would be today in 2007 because that's under system control, and as soon as you restored the file back, inodes changed, and ctime is current.
If you have a backup program that does not restore mtime, then that's okay since you can change the modification time with the "touch" command. For example if you need to change the date to Jan 11, 2005 12AM to file.txt, then you can go:
# touch -t 200501111200 file.txt
So mtime is the timestamp of a file, and ctime is kept by the system, and when mtime changes ctime always changes.
CentOS4 / RHEL4 Megaraid Drivers
I inherited an old Dell Poweredge 2400 as a test server from my boss, and I decided to put CentOS 4.4 on it. Apparently, Red Hat removed out-of-the-box support for old school Megaraid drivers. Dell would call these the PERC2/DC based systems. I Google'ed around and found a lot of forum posts regarding this, but there was always a discrepancy with the instructions that didn't fully get everything to work. Basically, since CentOS/RHEL 4.x does not autodetect these drivers any longer, you'd have to manually load the drivers before installing.
All forum posts point to Tom Sightler's great blog
that covered about 95% of what I needed. Below is a recap of the steps that I took which was adapted from Tom's great article. The other 5%-- however, since my Dell was a dual processor system, I needed to follow his steps and keep in mind that I needed to recompile drivers for a SMP system (I got the non-SMP drivers by compiling on my single processor workstation next to me.) If I had a dual processor system laying around, this would have made things easier. Still, even after successfully following Tom's steps and making it able to boot into the non-SMP kernel it wouldn't boot even after using the correct SMP drivers now with a new initrd.
So if you have a SMP system, I was able to find some drivers provided by Jackson Tech Support in New Jersey and actually load these drivers directly during the install and got everything up and running really fast.
Therefore, if you have an SMP system that's 32 bit Intel, this is the easy part (no drivers need to be compiled). Courtesy of Jackson Tech Support: his post or if that link is dead by now you can get it at my site.
Here's the simple steps for SMP now:
1) Copy those drivers onto a floppy on another machine, for example:
# dd if=megaraid.img of=/dev/fd0
2) Boot off the Linux install CD on the computer that you wish to install the drivers on, and type "linux dd" at the boot prompt.
3) Now when the system asks if you have any drivers, put in the floppy that you just copied the drivers to and let it load them.
NOTE: if the below steps don't work, you may first want to unload the incorrectly detected megaraid drivers by pressing ALT+F2 and going to the shell to do the following:
lsmod (see what drivers were detected)
rmmod megaraid_mbox
rmmod megaraid_mm
lsmod (verify they are removed)
insmod /mnt/floppy/megaraid.ko
Hit ALT+F2 to get back to the menu
4) There should be an option that allows you to choose drivers from a menu. These are the ones you want:
LSI MegaRAID for Legacy Dell PowerEdge Servers (megaraid)
Old Adaptec 28xx, 29xx, 39xx (aic7xxx_old)
5) Now when you're finished, go ahead and let anaconda start and your drives should now be detected!
Now if you have a non SMP system, I don't have packaged drivers for you, but you can follow these steps carefully to compile the single processor drivers on another Linux workstation, unload the incorrectly detected drivers, load the correct drivers, install Linux, then make a new initrd to ensure the boot loader chooses the right version of the Kernel that has your drivers loaded into it.
1) Follow Tom's steps to create the driver disk for single processor, and copy them to a floppy.
1) Boot off the Linux CD and choose "linux dd" at the prompt. When it asks for drivers, ALT+F2
2) Mount floppy w/ your driver disk
mkdir /mnt/floppy
mount /dev/fd0/ /mnt/floppy
3) lsmod and remove your megaraid drivers that it detects:
lsmod (see what drivers were detected)
rmmod megaraid_mbox
rmmod megaraid_mm
lsmod (verify they are removed)
insmod /mnt/floppy/megaraid.ko
(pause a little bit maybe and lsmod again to see if driver loaded)
4) ALT+F1 to go back and resume install via anaconda
5) When done, DO NOT REBOOT, but hit ALT+F2 again to goto the shell.
6) copy the megaraid driver to the drivers
cp /mnt/floppy/megaraid.ko /mnt/sysimage/lib/modules/2.6.9-42.EL/kernel/drivers/scsi
7) set /mnt/sysimage to the root
chroot /mnt/sysimage
8) edit modprobe, and make sure alias scsi_host adapater is alias scsi_hostadapter megaraid instead of something like megaraid_mbox
vi /etc/modprobe
9) switch to /boot and create a new initrd file
/sbin/mkinitrd initrd-2.6.11.EL.img.megaraid 2.6.9-11.EL
10) Overwrite old initrd with new:
cp initrd-2.6.9-11.EL.img.megaraid initrd-2.6.9-EL.img
11) ALT+F7 to switch back to installer and reboot
12) Make sure you choose the UP kernel instead of the default SMP kernel that Grub chooses upon bootup, or your system will not start.
Viola! Your RHEL4 based system can now boot up with the correct RAID drivers!
All forum posts point to Tom Sightler's great blog
that covered about 95% of what I needed. Below is a recap of the steps that I took which was adapted from Tom's great article. The other 5%-- however, since my Dell was a dual processor system, I needed to follow his steps and keep in mind that I needed to recompile drivers for a SMP system (I got the non-SMP drivers by compiling on my single processor workstation next to me.) If I had a dual processor system laying around, this would have made things easier. Still, even after successfully following Tom's steps and making it able to boot into the non-SMP kernel it wouldn't boot even after using the correct SMP drivers now with a new initrd.
So if you have a SMP system, I was able to find some drivers provided by Jackson Tech Support in New Jersey and actually load these drivers directly during the install and got everything up and running really fast.
Therefore, if you have an SMP system that's 32 bit Intel, this is the easy part (no drivers need to be compiled). Courtesy of Jackson Tech Support: his post or if that link is dead by now you can get it at my site.
Here's the simple steps for SMP now:
1) Copy those drivers onto a floppy on another machine, for example:
# dd if=megaraid.img of=/dev/fd0
2) Boot off the Linux install CD on the computer that you wish to install the drivers on, and type "linux dd" at the boot prompt.
3) Now when the system asks if you have any drivers, put in the floppy that you just copied the drivers to and let it load them.
NOTE: if the below steps don't work, you may first want to unload the incorrectly detected megaraid drivers by pressing ALT+F2 and going to the shell to do the following:
lsmod (see what drivers were detected)
rmmod megaraid_mbox
rmmod megaraid_mm
lsmod (verify they are removed)
insmod /mnt/floppy/megaraid.ko
Hit ALT+F2 to get back to the menu
4) There should be an option that allows you to choose drivers from a menu. These are the ones you want:
LSI MegaRAID for Legacy Dell PowerEdge Servers (megaraid)
Old Adaptec 28xx, 29xx, 39xx (aic7xxx_old)
5) Now when you're finished, go ahead and let anaconda start and your drives should now be detected!
Now if you have a non SMP system, I don't have packaged drivers for you, but you can follow these steps carefully to compile the single processor drivers on another Linux workstation, unload the incorrectly detected drivers, load the correct drivers, install Linux, then make a new initrd to ensure the boot loader chooses the right version of the Kernel that has your drivers loaded into it.
1) Follow Tom's steps to create the driver disk for single processor, and copy them to a floppy.
1) Boot off the Linux CD and choose "linux dd" at the prompt. When it asks for drivers, ALT+F2
2) Mount floppy w/ your driver disk
mkdir /mnt/floppy
mount /dev/fd0/ /mnt/floppy
3) lsmod and remove your megaraid drivers that it detects:
lsmod (see what drivers were detected)
rmmod megaraid_mbox
rmmod megaraid_mm
lsmod (verify they are removed)
insmod /mnt/floppy/megaraid.ko
(pause a little bit maybe and lsmod again to see if driver loaded)
4) ALT+F1 to go back and resume install via anaconda
5) When done, DO NOT REBOOT, but hit ALT+F2 again to goto the shell.
6) copy the megaraid driver to the drivers
cp /mnt/floppy/megaraid.ko /mnt/sysimage/lib/modules/2.6.9-42.EL/kernel/drivers/scsi
7) set /mnt/sysimage to the root
chroot /mnt/sysimage
8) edit modprobe, and make sure alias scsi_host adapater is alias scsi_hostadapter megaraid instead of something like megaraid_mbox
vi /etc/modprobe
9) switch to /boot and create a new initrd file
/sbin/mkinitrd initrd-2.6.11.EL.img.megaraid 2.6.9-11.EL
10) Overwrite old initrd with new:
cp initrd-2.6.9-11.EL.img.megaraid initrd-2.6.9-EL.img
11) ALT+F7 to switch back to installer and reboot
12) Make sure you choose the UP kernel instead of the default SMP kernel that Grub chooses upon bootup, or your system will not start.
Viola! Your RHEL4 based system can now boot up with the correct RAID drivers!
Wednesday, January 10, 2007
RPM Rollbacks
Ever afraid of updating packages or installing software on Linux? Those of us used to Windows would just go to "ADD/REMOVE" to remove a piece of software that you don't like. In OS X, you'd just simply drag things to trash.
However, what if you wanted to update your MySQL or PHP packages because there were new vulnerabilities or bugs discovered in your current version? Even if you had a development/test environment, it'd be nice to be able to go back to the previous version if the newly installed one breaks things or acts weird. Most Linux distros use Red Hat's RPM, and if you use RHEL or derivatives such as CentOS, you work with RPMs all the time. We can solve this issue with RPM rollbacks and transactions.
Let's say you're rolling out a new update for PHP, php-4.3.9, and you currently have php-4.3.8. You can install the new version and save the old package for rollback with the following command:
# rpm -Uvh --repackage php4.3.9
Preparing... ############################# [100%]
Repackaging...1:php ############################# [100%]
Upgrading...1:php ############################# [100%]
If you decide to erase an RPM, you can add the --repackage to your command andit'll save it:
# rpm -e --repackage
OK, now let's say this install of PHP broke things on your website, and for the timebeing you want to go back to the way things were. You can use the --rollback switchand specify a date or time. You can actually rollback to a target date, e.g.
# rpm -Uvh --rollback '5 minutes ago'
Rollback packages (+1/-1) to
Thu Jan 10 13:26:12 2007 (0x2f24d6f1):
Preparing... ########################### 100%]
1:php ########################### [ 33%]
BAM! Now you're back to where you were before. Now I know it's a hassle to rememberthe --rollback command every time, and if you forget, then you're kinda screwedbecause you don't have anything to go back to if needed. Therefore, you can edityour RPM macro file to use this option by default. Go ahead and find that file,usually located at /etc/rpm/macros. Add the line if it doesn't already exist:
%_repackage_all_erasures 1
You might also want to add the following in the macros file as well to put yourrollback repository of old RPMs into a specific directory or location:
%_repackage_dir /PATH/TO/LOCATION/
If you use up2date, you can add support for rollbacks by running up2date-config andthen clicking the "Retrieval/Installation" tab. There should be a checkbox nearthe bottom of the window that says "enable RPM rollbacks ...", and you can choosethe "Package storage directory" below that as well.
up2date might be deprecated now because a lot of people now use Yum as a convenientway to resolve dependencies, similar to Apt-get (but maybe not as good). You canautomatically support RPM rollbacks on all Yum updates if you add the following line to your Yumconfig file (usually /etc/yum.conf):
tsflags=repackage
Ok, that's all I have to say about that. RPM rollbacks can for the most part helpyou keep software up to date, and gives you an option to return to a previous statebefore installing these new packages if things go wrong.
However, what if you wanted to update your MySQL or PHP packages because there were new vulnerabilities or bugs discovered in your current version? Even if you had a development/test environment, it'd be nice to be able to go back to the previous version if the newly installed one breaks things or acts weird. Most Linux distros use Red Hat's RPM, and if you use RHEL or derivatives such as CentOS, you work with RPMs all the time. We can solve this issue with RPM rollbacks and transactions.
Let's say you're rolling out a new update for PHP, php-4.3.9, and you currently have php-4.3.8. You can install the new version and save the old package for rollback with the following command:
# rpm -Uvh --repackage php4.3.9
Preparing... ############################# [100%]
Repackaging...1:php ############################# [100%]
Upgrading...1:php ############################# [100%]
If you decide to erase an RPM, you can add the --repackage to your command andit'll save it:
# rpm -e --repackage
OK, now let's say this install of PHP broke things on your website, and for the timebeing you want to go back to the way things were. You can use the --rollback switchand specify a date or time. You can actually rollback to a target date, e.g.
# rpm -Uvh --rollback '5 minutes ago'
Rollback packages (+1/-1) to
Thu Jan 10 13:26:12 2007 (0x2f24d6f1):
Preparing... ########################### 100%]
1:php ########################### [ 33%]
BAM! Now you're back to where you were before. Now I know it's a hassle to rememberthe --rollback command every time, and if you forget, then you're kinda screwedbecause you don't have anything to go back to if needed. Therefore, you can edityour RPM macro file to use this option by default. Go ahead and find that file,usually located at /etc/rpm/macros. Add the line if it doesn't already exist:
%_repackage_all_erasures 1
You might also want to add the following in the macros file as well to put yourrollback repository of old RPMs into a specific directory or location:
%_repackage_dir /PATH/TO/LOCATION/
If you use up2date, you can add support for rollbacks by running up2date-config andthen clicking the "Retrieval/Installation" tab. There should be a checkbox nearthe bottom of the window that says "enable RPM rollbacks ...", and you can choosethe "Package storage directory" below that as well.
up2date might be deprecated now because a lot of people now use Yum as a convenientway to resolve dependencies, similar to Apt-get (but maybe not as good). You canautomatically support RPM rollbacks on all Yum updates if you add the following line to your Yumconfig file (usually /etc/yum.conf):
tsflags=repackage
Ok, that's all I have to say about that. RPM rollbacks can for the most part helpyou keep software up to date, and gives you an option to return to a previous statebefore installing these new packages if things go wrong.
Subscribe to:
Posts (Atom)