Linux

Dr. Tom V. Mathew

February 26, 2014


Contents

1 Basic commands

1.1 cd

To change the directory
Command Function
cd go to home directory
cd .. go to one directory down
cd MainDirA go to directory named MainDirA
cd MainDirA/SubDirA go to directory named SubDirA in the directory MainDirA
cd ../../MainDirB/SubDirB go to directory named SubDirB in the directory MainDirB from MainDirA/SubDirA

1.2 rm

To remove or delete a file/directory
Command Function
rm fIle deletes the file named File
rm -i fIle (interative mode) ask for confirmation before deleting the file named File
rm -r Dir deletes the directory named Dir
rm -ri Dir (interative mode) deletes the directory named Dir
rm -f File (force mode) deletes the file File if exists, otherwse no error is shown
rm -v File (verbose mode) deletes the file File and prints what the system did.

1.3 lsList Files

Command Function
ls list file in the current directory
ls -l list file with all fields
ls -o list file with some fields

1.4 date

Date gives the current date and time as follows.
date
Mon Aug  9 12:58:02 IST 2004
To get in dd/mm/yyyy format
date +\%d/\%m/\%y
09/08/2004
To get in dd/mm/yy format
date +\%d/\%m/\%y
09/08/04
To get in dd/mm/yy format (Alternate)
date +\%D
09/08/04

1.5 sort

sort command sorts a file, say file.tex.
sort file.tex
to sort a numeric file
sort -n file.tex
to sort a file in reverse order
sort -r file.tex
to sort a numeric file in everse order
sort -nr file.tex
to sort list of directories in the order of the group (i.e. sorting different fields) The fourth field is the group +3 indicate the fourth field starting with zero. -b indiacte ignore leading blanks
ls -l | sort +3 -b
To sort by the size of files
ls -l | sort +4 -b -n

1.6 tail

show the tail portion of a file, file.tex
tail file.tex
Show that last three lines of file.tex
tail +3 file.tex

1.7 cmp

cmp command compares two files and says there is difference, but not what is the difference/
cmp file1.tex file2.tex

1.8 diff

diff command compares two files and says what are the differences
diff file1.tex file2.tex

1.9 touch

touch creates an empty file.
touch file.tex

1.10 wc

wc command counts characters, words and lines of a file
wc file.tex

1.11 wildcharacters

Suppose you have files as below:
file1.tex
file2.tex
file3.tex
file.tex
file.c
prog.cpp
file.doc
testfile
Then * indiacte all and ? single character, for example, the following command display all files
ls *
This command will display all file, except the test file
ls *.*
To display all tex file
ls *.tex
To display all .tex files with number 1,2,3 at the end,
ls file?.tex
To display all c files,
ls *.c*
to display all files with file names start with small case alphabets
ls [a-z]*.*
to display all files with file names start with alphabets
ls [A-z]*.*
to display all files with file names start 1,2,3,4 or 5.
ls [1-5]*.*
to display all files with file names start with alphabet or digit
ls [0-9,A-z]*.*
to display all files with files that does not start with digit
ls [!0-9]*.*
to display all files with files that does not start with digit, but second character is digit
ls [!0-9][0-9]*.*

1.12 grep

To check whether is word or character is in a file. Eg. to see whaere all the word 'display' finds in file 'linux.tex'
grep display linux.tex
Eg. to see whaere all the word 'display all fil' finds in file 'linux.tex'
grep 'display all fil' linux.tex

1.13 File redirection

The output of many commands can be rdirected to files. To direct the out put of ls -l to a file called file.dat. If file.dat is not existing, it will create, otherwise it will overwrite.
ls -l > file.dat
To direct the out put of ls -l to a file called file.dat by appending the file.
ls -l >> file.dat

1.14 Piping

The output of a command can be fed to another command. To find how many files are there in this direcotriy.
ls | wc -l
ls will generate the list of files and feed to wc -l command which will count the number of lines. This is equivalent of
ls > t
wc -l t
rm t
Another example. To see whether user 'ce753' is logged in
who | grep ce753
The piping can be nested. If you want to see the latest 10 files and sort them
ls -t | tail +10 | sort

1.15 Mislaneous

pid Process ID's PATH variables

Command Function
who Shows current users
pwd Shows the present working directory
mkdirDirA Creates the directoryDirA
cat file show (concatenates) the file named file
more file show the file page by pagefile
less file show the file page by pagefile
cp dirA/file1.tex . copy using relative path with respect to current directory
cp /dirA/file1.tex . copy using relative path with respect to user home area
cp /dirA/file1.tex . copy using absolute path (true for other commands like mv, rm, etc.

2 Network commands

2.1 telnet

telnet 10.104.2.22
ce753
****
10.104.2.22 is the ip address of the machine. ce753 is the userid and **** is the password.

2.2 ssh

ssh ce753@10.104.2.22
****
10.104.2.22 is the ip address of the machine. ce753 is the userid and **** is the password.

2.3 ftp

File transfer protocol (ftp) to transfer files between two machins. You are in machines A and you want to transfer file from/to machine B. Machine B has IP (10.104.2.1). You have a file called aaa.tex which you want to transfer to B and from B you want to transfer a file called bbb.pdf to A.
ftp 10.104.2.1
ce753
****
bin
hash
put aaa.txt
get bbb.pdf
Note that bin will set binary moded and hash will set hash mode. Suppose you have many files, say a1.tex, a2.tex, ... a9.tex to transfer from A to B. Also you have to transfer bbb.tex, bbb.pdf, bbb.html and bbb.fig from B to A. Login as above. Set to binay and hash mode as above.
mput *.tex
mget bbb.*
Before each file transfer, ftp will ask whether you want to tranfer that file or not. Please press 'y' for yes and 'n' for no. If you dont want to be asked every time set the prompt off as below.
prom
mput *.tex
mget bbb.*
If you want to see which the default direcotries use the command below:
pwd
lcd
pwd will give the present working direcory of B (10.104.2.1) and lcd will give the local current directory. Suppose your files are in direcotry BBB in machine B and AAA direcotry of F drive in machine A. you can do this by:
pwd
cd BBB
pwd
lcd
lcd F:\AAA
lcd
To see some other commands and brief details of each command
help
help bin
Note that by, ftp you cannot transfer a directory. For that you have convert the directory to file and them transfer and then convert back to the directory. The two common way of doing is by converring to a zip file or tgz file.

2.4 Display to other Linux Machine

2.5 Display to other Windows Machine

How to use putty and xMing

3 Security commands

3.1 chmode

Command Function
d file status d = direcotry, l = link, - = normal file
r read permission for owner - = no permission
w write permission for owner
x execute permission for owner
r read permission for group
w write permission for group
x execute permission for group
r read permission for public
w write permission for public
x execute permission for public
chmode 777File read write execute permission for owner, group and public

There is a alternate way of doing this.

chmode u+x file.tex permits user to have execute permission
chmode g+x file.tex permits group to have execute permission
chmode o+x file.tex permits others to have execute permission
chmode a+x file.tex permits all to have execute permission
chmode u+r file.tex permits user to have read permission
chmode g+r file.tex permits group to have read permission
chmode o+r file.tex permits others to have read permission
chmode a+r file.tex permits all to have read permission
chmode u+w file.tex permits user to have write permission
chmode g+w file.tex permits group to have write permission
chmode o+w file.tex permits others to have write permission
chmode a+w file.tex permits all to have write permission
chmode ugo+rwx file.tex permits all user to have read-execute and write permissions
chmode a+rwx file.tex same as above (permits all user to have read, execute and write permissions)
chmode a-rwx file.tex deny all user to read, execute and write permissions
chmode a+rwx direcotory permits all user to have read, execute and write permissions of the directory
chmode -R a+rwx direcotory permits all user to have read, execute and write permissions of the directory as well as all the files inside that directory

4 Scripts

4.1 isimple

What ever commands you give in the linux command prompt can be given in a file and execute the script. for example, if you execute a command like clean, then ls -l, and date one by one. The whole can be put into a file say 1eg.bash as below
clear
ls -l
date
This can be executed by any of follwing way:

Note: for the last case 1eg.bash should have executable permission.

4.2 Variable definition

2eg.bash
#
# Defines variables
#
X=One_Word
Y=Words with spaces
Z="Words with space in quotes"
var="Varible with maby characters"

echo -e "variable X\t" $X
echo -e "variable Y\t" $Y
echo -e "variable Z\t" $Z
echo -e "variable var\t" ${var}

4.3 File existance testing

#
# Checks whether the file defined by vaiable F exists or not
#

F=file.tex

if [ -e "$Y" ]; then 
	echo "File $F exists"
else
	echo "File $F DOES NOT exists"
fi

4.4 test

#test
X=Prog.C

if [ -n "$X" ]; then 
	echo "X-file : $X"
fi

if [ -n "$Y" ]; then 
	echo "Y-file : $Y"
else
	echo "Y-file : not defined"
fi

M=makefile
if [ -e $M ]; then 
	echo "file exist : $M"
fi

for X in A B C
do
	echo $X
done

for X in p*.tex
do
	cat $X
done


#!/bin/bash
X=0
while [ $X -le 10 ]
do
	echo $X
	X=$((X+1))
done


#!/bin/bash
echo "Current "
X=0
while [ $X -le 20 ]
do
	Y=p$X.tex
	if [ -e $Y ]; then
	cat $Y
	fi
	X=$((X+1))
done


#!/bin/bash
# Another Script to rename all
# c file to cpp
#
for X in *.c
do
	mv $X ${X/"c"/cpp}
done

4.5 nptel

#!/bin/bash
#
#	script file to list all tex files

echo "List all the files"

cmd="latex2html -nonavigation -noinfo -noaddres -split 0 -dir"
dire="scripts/tmp1/"
XM=0
PWD1="../"
PWD="/faculty/tse/public_html/07Nptel04/"
MOD="mtexf/"
LEC="ltexf/"
HEA="h"
PRE="p"
TAI="t"
EXT=".tex"
OUT="ceTEI/"
while [ ${XM} -le 1 ]
do
	dinpMod=${PWD}${XM}${MOD}
	#-------------------
	if [ -d ${dinpMod} ]; then
		echo ${dinpMod}
		doutMod=${PWD}${OUT}${XM}${MOD}
		if [ -d ${doutMod} ]; then
			echo "Direcotry exists -> ${doutMod}"
		else
			echo "Creating directory -> ${doutMod}"
			mkdir ${doutMod}
		fi
		XL=0
		while [ ${XL} -le 1 ]
		do
			dinpModLec=${dinpMod}${XL}${LEC}
			if [ -d ${dinpModLec} ]; then
				echo ${dinpModLec}
				doutModLec=${PWD}${OUT}${XM}${MOD}${XL}${LEC}
				if [ -d ${doutModLec} ]; then
					echo "Directory exists ->-> ${doutModLec}"
				else
					echo "Creating directory ->-> ${doutModLec}"
					mkdir ${doutModLec}
				fi
				XP=0
				while [ ${XP} -le 2 ]
				do
					fModLecH=${dinpModLec}${HEA}${XP}${EXT}
					fModLecP=${dinpModLec}${PRE}${XP}${EXT}
					fModLecT=${dinpModLec}${TAI}${XP}${EXT}
					if [ -e ${fModLecP} ]; then
						doutModLecP=${doutModLec}${PRE}${XP}
						if [ -d ${doutModLecP} ]; then
							echo "Directory exists ->->-> ${doutModLecP}"
						else
							echo "Creating directory ->->-> ${doutModLecP}"
							mkdir ${doutModLecP}
						fi
						ls ${fModLecH}
						ls ${fModLecP}
						ls ${fModLecT}
						cat ${fModLecH} ${fModLecP} ${fModLecT} >  p.tex
						echo "${cmd} ${doutModLecP} p.tex"
						${cmd} ${doutModLecP} p.tex
						#echo "${cmd} ${doutModLecP} ${fModLecP}"
						#${cmd} ${doutModLecP} ${fModLecP}
					fi
				XP=$((XP+1))
				done
			fi
			#------------------------------------
		XL=$((XL+1))
		done
	fi
	#--------------------
	XM=$((XM+1))
done

4.6 awk program

The source tex file contains patterns like "begx" and "endx" and this script will convert the portion between the pattern into small tex files.
#
#       bash script file for extracting
#       tex files, with head and tails
#       for NPTEL Web course
#
#       Dr. Tom V. Mathew (vmtom@iitb.ac.in)
#
#       created : 10-sep-2004
#
finp="11intro_pav_design.tex"
#
beg="begx"
end="endx"
fout="p"
ext="tex"
#
#       get how many  pateern exist in the file
#
#       get the number of patterns
#
N=$(grep begx ${finp} | wc -l)
echo "No of p files $N"
#
#       use awk to extract the portion
#
i=1
while [ ${i} -le ${N} ]
do
        awk "/${beg}${i}/,/${end}/" ${finp} > ${fout}${i}.${ext}
#       awk "/${beg}${i}/,/${end}/" ${finp}
        i=$((i+1))
done
#
#       we have now p?.tex
#
X=1
while [ ${X} -le $N ]
do
        ln -s head.tex h${X}.tex
        ln -s tail.tex t${X}.tex
        X=$((X+1))
done

5 awk

5.1 File reading

To read a file and write two fields to anoter file use the follwing awk script

awk '{print $1" "$3}' inFile > outFile
The inFile is
one two three
abc def ghi
x y z
and the out file is
one three
abc ghi
x z

5.2 awk and convert

To convert all jpg images to into thumpnails. First create a script file and then execute the script file.
ls *.JPG | awk '{print "convert "$1" -resize 5% ../" $1}'  > convert.bash
source convert.bash

5.3 To convert a text database to html

This database will extract the first word, second word, from each line and write to the file. Each work can contain space, and the words are separated by tab.

I=faculty.txt
F=faculty.html
echo "<!-- Dr. Tom V. Mathew -->" > $F 
echo "<html><p>" >> $F 
echo "<BODY bgcolor=\"#ACD2C7\" background=\"../database/01image.gif\" text=\"#000000\" link=\"#0000ff\" vlink=\"#ff0000\" alink=\"#00ff00\">" >> $F
echo "<font face=\"Comic Sans MS\" color=\"552277\">" >> $F
echo "<h1>Transportation Faculty</h1>" >> $F
echo "<font face=\"Verdana\" size=\"4\">" >> $F
awk 'BEGIN {
	RS="\n"
	FS="\t"
}
{
	print "<font color=\"6600CC\" size=\"5\"><strong>"
	print $1
	print $2 
	print "</strong><br></font><font color=\"0000ff\">"
	print " <a href=\"mailto:" 
	print $3 
	print "@civil\.iitb\.ac\.in\">"
	print $3"@civil\.iitb\.ac\.in</a>" 
	print "</font><font color=\"000000\"><strong>"
	print $4 
	print "</strong></font><font color=\"006633\">"
	print $5"</font>"
	print " <a href=\"http://www\.civil\.iitb\.ac\.in/~"$3
	print "\" target=\"new\">Home</a></p>" 

}' $I >> $F
echo "</p></html>" >> $F

For example: this imput is converted to an html:

Prof.	S L Dhingra	dhingra	7329	Transportaion Eonomics
Prof.	P K Sikdar	pksikdar	7314	Transportaion Systems Planning
Prof.	K V K Rao	kvkrao	7305	Transportaion Planning
Prof.	Tom V Mathew	vmtom	7349	Transportaion Networks

The output is:

Transportation Faculty

Prof. S L Dhingra
dhingra@civil.iitb.ac.in 7329 Transportaion Eonomics Home

Prof. P K Sikdar
pksikdar@civil.iitb.ac.in 7314 Transportaion Systems Planning Home

Prof. K V K Rao
kvkrao@civil.iitb.ac.in 7305 Transportaion Planning Home

Prof. Tom V Mathew
vmtom@civil.iitb.ac.in 7349 Transportaion Networks Home

6 Misc Scripts

6.1 perl script to search and replace

If you want to seach and replace a word in a number of files (say all tex files) simultaneous, you can use this script.
perl -p -i -e 's/search-word/replace-word/g' *.tex

6.2 Automatic file backups

#	This script will make a copy of the given file
#	by appending the file name with a date stamp.
#	For example
#
#	Input	:	file.tex
#	Action	:	source update
#	Output	:	file.tex.2005-09-20-12-47-02
#
#	The date format contains the year, month, day
#	hour, minute, and second. This is usefule for
#	taking backpup
#
#	vmtom@civil.iitb.ac.in
#
F=file.tex
#
D=$(date +%Y-%m-%d-%H-%M-%S)
#
cp file.tex file.tex.$D
#
Save the above code in file say update. You can use this by replacing file.tex with your file.

6.3 Merging two PDF files

gs -q -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -sOutputFile=merged.pdf a.pdf b.pdf c.pdf

6.4 TECP enquiries

To see all the tcp inquiries

[root@tula 4303links]# /usr/sbin/tcpdump -n arp

6.5 Pipe commands

Pipe command to find all cpp file and copy to some folder

find . -name '*.cpp' -print | xargs cp -v -t ./source/
My pipe command to kill an httpd process (not very good)
ps -ae | grep http
kill -9 $(ps -ae | grep http | cut -f3 -d" ")
to use in gemini. Need refinement for cut to get exact PID.

My pipe command to kill an httpd process (better)

/opt/pkgs/apache/bin/apachectl stop
ps -ae | grep http
ps -ae | grep http | awk '{print "kill -9", $1}'> t
source t
rm t
/opt/pkgs/apache/bin/apachectl start
to use in gemini. Need refinement for to pipe kill.

6.6 Linux File System Quotas

Step 1: Edit /etc/fstab to add "usrquota" to the partition

	/dev/hda2     /phd   ext3    defaults,usrquota              1    2

Step 2: Add the file aquota.user

	touch /phd/aquota.user
	chmod 600 /phd/quota.user

Step 3: reboot the system

	shutdown -r now

Step 4: Construct the quota data base

	quotacheck -vgudinf /phd

Step 5: Set soft and hard limit for user p5test

	edquota -u p5test
 
Step 6: Set the same limit for all other users

	edquota -p p5test -u ***

	where *** indicate list of users (one by one)

Step 7a: Check the quota of all users

	repquota /phd

Step 7b: Check the quota for a user

	quota -u p5test

Note 1: Assumes that the quota is on in the partition

	quotaon -v /phd

	(This should be part of system start up)

Note 2: if quotacheck command given sime error for Read only file
	try this:
	mount -o remount,rw /phd
	then delete the (hard or soft link, or locked file)

6.7 C/C++/Java Code cleaner

Dowload Astyle application from the follwing site.

http://sourceforge.net/projects/astyle/

Unzip the folder and if you are in Windows, then an exe file under the AStyle/bin folder will work for you. In linux you have to compile the src files. In windows execute the following command.

AStyle.exe --options=style.txt < yourFile.cpp > newFile.cpp

where style.txt contains your options. My options file is

--style=allman
--indent=tab
--indent-classes
--indent-switches
--indent-cases
--indent-labels
--indent-preprocessor
--indent-col1-comments
--min-conditional-indent=0
--max-instatement-indent=60
--break-blocks=all
--unpad-paren
--delete-empty-lines
--break-elseifs
--add-brackets
--convert-tabs
--align-pointer=type
--align-reference=type
--preserve-date
--verbose
--lineend=linux

6.8 Video converter

To convert (or transcode) an array of MPG files to mp4 files in Linux/Ubundu using vlc

#
#	Script to convert/transcode MPG files to mp4 and mp3 files
#	command to execute:
#
#		source vlcscript.bash
#
#	Report bugs to: tfrdn7@gmail.com
#
#	created : Wed Mar 20 20:42:42 IST 2013
#
#	Step 1: Give the files
#
#
array=(
file_1.MPG
file_2.MPG
file_3.MPG
)
#	Script begins here
#
count=${#array[@]}
index=0
Cnt=0
F="logVlc.txt"
echo "Log file" > $F
#
while [ "$index" -lt "$count" ];
do
	X=${array[$index]}
	# Step 2: file extension for video and audio
	#
	O=${X%.MPG}".mp4"
	#
	A=${X%.MPG}".mp3" # Step 2: give the file extension
	#
	let "index++"
	Cnt=$((Cnt+1))
	if [ -e ${X} ]; then

		#	VIDEO conversion
		#
		if ! [ -e ${O} ]; then
			echo -e "Video compreesion of file" ${O} " " $index "/" $count
			vlc -I dummy -vvv ${X} --sout '#transcode{vcodec=h264,scale=0.67,acodec=mpga,ab=128,channels=2,samplerate=44100}:std{access=file,mux=mp4,dst='${O}'}' vlc://quit
			echo -e "Processed video file" ${X} " " $index "/" $count >> $F
		fi
		#
		#	AUDIO conversion
		#
		if ! [ -e ${A} ]; then
			echo -e "Audio extraction of file" ${A} " " $index "/" $count
			vlc -I dummy -vvv ${X} --sout '#transcode{vcodec=dummy,acodec=mp3,ab=128,channels=2,samplerate=44100}:std{access=file,mux=raw,dst='${A}'}' vlc://quit
			echo -e "Processed audio file" ${X} " " $index "/" $count >> $F
		fi
	fi
done
echo -e "\nCompleted\n"

To convert (or transcode) all MPG files in the current folder to mp4 using vlc player in Windows. Note: if the file exist, it will skip.

@echo off
for %%a in (*.MPG) do (
   if not exist %%~na.mp4 (	
      "C:\Program Files (x86)\VideoLAN\VLC\vlc" -I dummy -vvv %%a --sout=#transcode{vcodec=h264,scale=0.67,acodec=mpga,ab=128,channels=2,samplerate=44100}:standard{access=file,mux=mp4,dst=%%~na.mp4} vlc://quit
   ) else (
      echo The file %%~na.mp4 exist
   )
)
To convert an MPG file to flv file using VLC command line for uploading in youtube.
vlc input.MPG --sout '#transcode{vcodec=h264,acodec=mp3,ab=128,channels=2,samplerate=44100}:std{access=file,mux=mp4,dst=output.flv}' vlc://quit

To convert a raw MPG file to compressed mp4 file using avconv command line for uploading in youtube.

sudo apt-get update
sudo apt-get install avconv
sudo apt-get install libav-tools
sudo apt-get install x264
sudo apt-get install libavcodec-extra-53
avconv -i M2U00009.MPG -vcodec libx264 -b:v 600k -maxrate 600k -bufsize 1000k -deinterlace -threads 0 -acodec libvo_aacenc -b:a 96k out.mp4

#       Script to convert raw MPG files to mp4 files
#       command to execute:
#               source avconc_script.bash
#       Report bugs to: tfrdn7@gmail.com
#       created : Wed Mar 20 20:42:42 IST 2013
#       updated : Sat Oct 26 10:24:05 IST 2013
#       Step 1: Give the files
#
array=(
file1.MPG
file2.MPG
)
count=${#array[@]}
index=0
Cnt=0
while [ "$index" -lt "$count" ];
do
        X=${array[$index]}
        # Step 2: give the file extension
        #
        O=${X%.MPG}".mp4"
        echo -e "Processing file" ${X} " " $index "/" $count
        let "index++"
        Cnt=$((Cnt+1))
        if [ -e ${I} ]; then
                avconv -i $X -vcodec libx264 -b:v 600k -maxrate 600k -bufsize 1000k -deinterlace -threads 0 -acodec libvo_aacenc -b:a 96k ${O}
        fi
done
echo -e "\nCompleted\n"

6.9 mysql

# Mysql Command to take the full backup

mysqldump -u root -prootxyz --all-databases > allmySqlDataBaseBkp_20131017.sql

# Use command prompt commands

# Log on to the sql

mysqldump -u root -prootxyz

# Select the databases

show databases;

# select on databse e_emech

use e_mech;

# to see tables

show tables;

# show the keys in a table 

describe bookingdetails1;

# show all the contents of a table

select * from bookingdetails1;

# show matching some key

select * from lai where tid=20131017115557;

#copy table from one database to another

rename table d_meet.bookingdetails1 to d_ce208.bookingdetails1;

# delete one entry from the table

delete from bookingdetails1 where bid=253;

# Remove all the contents of the table 

truncate bookingdetails1;

# Delete databse

drop database d_meet;

7 Links

Before we start: Why Linux is better Why Linux is Great Unix Tutorial for beginners The right place for the beginner. Getting Started with Linux - Course Material Little more detailed list. Norman Matloff's Unix and Linux Tutorial Center By a Comp. Science Prof. All pages in PDF format. Also contains some c tutorials. Summary list of commands (one page) Linux tutorials This is an in-depth tutorial. Linux tutorials Include how to install, configure, and administer the systems. NC State University Some basic definitions and commands of Unix/Linux. Has a small pdf file that contains a summary of commands Advanced Bash-Scripting Guide Bash Guide for Beginners



Prof. Tom V. Mathew 2014-02-26