unix intermediate - Day 3

• man sed
• sed is a stream editor. It works on one line of input at at time. Upside/downside.
• be prepared for version and vendor differences
• two modes
  • command line: sed 'command' file[s]
  • script driven: sed -f scriptfile file[s]
• The original file is unaffected
• syntax sed -e 'CriteriaAction'
• if no address is indicated, the processing happens on every line
• address are like:
  1,50 lines 1-50
  25 line 25
  30,$ lines 30-EOF
  /cat/, /dog/ lines between the match for cat and dog
• common uses
  • substitution
    

    	#sub xx with yy on all lines containing TARGET
    	/TARGET/s/xx/yy/gi
    	#or just sub xx with yy
    	s/xx/yy/	#sub first occurrence on the line
    	s/xx/yy/g	#sub all occurrences on teh line
    	s/xx/yy/4	#sub 4th occurrence
    	s/xx/yy/gi
    	

  • printing a range
    

    	#quit on line 11; suppress printing, then print lines specified
    	sed -n -e '11q;5,10p'
    	

  • translation
    

    	#translate a to 1, b to 2, etc. on all lines containing TARGET
    	/TARGET/y/abc/123/g
    	

  • deletion
    

    	# delete all lines containing TARGET
    	/TARGET/d
    	

  • insertion
    

    	#insert text before *line* containing target
    	/TARGET/i\	
    some text\	
    notice the backslash\	
    before the lines\	
    except the last one
    	

• appending
  

  	#append text after line containing target
  	/TARGET/a\	
  some text	
  goes here
  	

• replacement ("change")
  

  	#replace TARGET with lines
  	/TARGET/c\	
  some text\	
  goes here
  	

• list mode
  

  	#show non-printable chars
  	#l as in llama
  	#usually seen as sed -e 'l'
  	l
  	

• multiple commands in the same scriptfile
  

  	/somelocation/i	inserted text
  
  	/someotherlocation/i/
  	more inserted text
  	

• combining multiple commands at the same address
  

  	/TARGET/{
  	someCommand
  
  	anotherCommand
  	}
  	

• reading in from an unrelated file
  

  	/TARGET/{
  	r sourcefile
  	/TARGET/d
  	}
  	

• writing out to an unrelated file
  

  	/TARGET/w outfile
  	

• famous one-liners
  • sed G - doublespace a file
  • sed "G;G" - triplespace a file
  • sed = filename | sed 'N;s/\n/\t/' - insert line numbers; note \t is tab.
  • sed 's/^[ \t]*//' - remove leading whitespace
  • sed 's/[ \t]*$//' - remove trailing whitespace
• sed FAQ

awk

• man awk
• be prepared for version and vendor differences
• command line: awk 'command' file[s]
• script driven: awk -f scriptfile file[s]
  notice similarity to sed syntax
• Fields
  • delimited: FS=":" or -F:. Default seperator is a space, :, | and \t are common.
  • $0 is the entire line, $1 is first field, $2 is second field, etc.
  • NR is the current line number
  • NF number of fields/words
  • {print NR, $1, $3}
  • advanced: fixed-length and multiline records
• Criteria
  • /foo/ work on those lines that contain foo, printing by default
  • && and
  • || or
  • ! negation
  • common uses
    • grep-like searching

      	#print out all lines in infile that contain bash
      	#$0 == the current line
      	'/bash/ {print $0}'  #printing the line is the default
      	

    • printing out a given field
      	awk -f myAwkScript /etc/passwd
      
      	#setting the field seperator
      	FS=":"
      	#print the fourth field from each line that contains TARGET
      	/TARGET/ {print $1		$4}
      	

  • finding text in a field

    	awk -f myAwkScript /etc/passwd
    	#setting the field seperator
    	FS=":"
    	#print the fourth field if it contains bash
    	/bash/ {print $7}
    	

  • NF = number of fields in current record
  • NR = number of the current record (usually line number)

  
  #variables declared in begin statement
  BEGIN { print FILENAME; p = 0 }
  /From / && / 199/           { p = 0 }
  /From / && /cc.adfa.oz.au/  { p = 1 }
  { if (p > 0) print "Here it is: ", $0 }
  END   { print "End ..." }
  
  
  

teaser: expect

teaser: perl

teaser: gcc/make

using cpio for incremental backups

• use find or a file list to ID what gets backed up
• make an archive: cpio -o > archive.cpio
• unpack an archive: cpio -imd 'pattern' < archive.cpio

Conclusion

Where to go from here

• other coursework
• outside work

Remember, your login is based on your machine's hostname, not on any other number.
~/[initials] refers to the subdirectory under your homedir, named after your initials. Everything except for .dotfiles will be stored in your ~/[initials] directory.

© 1995-2001 jason carr
Distributed under the terms of the GNU Free Documentation License.