Archive

Archive for October, 2011

sed – Print lines that do not match a pattern.

October 25, 2011 1 comment

Problem:
I am checking that a file containing questions have question marks. So I need to print out all the lines that do not have question marks. (NB. One question per line.)

Solution:

sed -n '/?/!p' questions.txt | less

From http://www.grymoire.com/Unix/Sed.html#uh-32

Reversing the restriction with !
Sometimes you need to perform an action on every line except those that match a regular expression, or those outside of a range of addresses. The “!” character, which often means not in Unix utilities, inverts the address restriction. You remember that

sed -n ‘/match/ p’

acts like the grep command. The “-v” option to grep prints all lines that don’t contain the pattern. Sed can do this with

sed -n ‘/match/ !p’ </tmp/b

Source:
http://www.grymoire.com/Unix/Sed.html

Advertisements
Categories: bash Tags:

python – TypeError: coercing to Unicode:

October 25, 2011 Leave a comment

Problem:
Was getting the following error. Did not understand why it kept on complaining about receiving a list when I clearly passed on a csv file.
The name of my file is update_records and I would run it as follows on the command line.

shell> ./update_records.py records.csv

Traceback (most recent call last):
File “./update_records.py”, line 156, in
main()
File “./update_records.py”, line 130, in main
subscriber_data = _get_data(input_file)
File “./update_records.py”, line 44, in _get_data
csvdata = csv.reader(open(csvfile, “r”))
TypeError: coercing to Unicode: need string or buffer, list found

I finally realised that at some other point in my code I do

122         opts, args = getopt.getopt(sys.argv[1:], '', ['help'])
123 
124         for opt, arg in opts:
125             if opt in ('--help'):
126                 _help()
127 
128         input_file = args

input_file now contains the list [‘records.csv’] and that is what is passed on to csv.reader(open(csvfile, “r”)) in csvfile and that is what generates the error.

Solution:
Changed input_file = args to input_file = args[0] to pick the first item from args.

122         opts, args = getopt.getopt(sys.argv[1:], '', ['help'])
123 
124         for opt, arg in opts:
125             if opt in ('--help'):
126                 _help()
127 
128         input_file = args[0] # Fixed!

Categories: python

sed – Substituting text on certain lines

October 14, 2011 Leave a comment

Problem:
Needed to remove sales totals for some days in a month. Had to change from

shell> less Oct2011_totals.csv
2011-10-01,586
2011-10-02,399
2011-10-03,761
2011-10-04,517
2011-10-05,773
2011-10-06,1882
2011-10-07,1759
2011-10-08,1948
2011-10-09,111258
2011-10-10,105451
2011-10-11,129304
2011-10-12,131212
2011-10-13,140890
2011-10-14,132396
2011-10-15,0
2011-10-16,0
2011-10-17,0
2011-10-18,0
2011-10-19,0
2011-10-20,0
2011-10-21,0
2011-10-22,0
2011-10-23,0
2011-10-24,0
2011-10-25,0
2011-10-26,0
2011-10-27,0
2011-10-28,0
2011-10-29,0
2011-10-30,0
2011-10-31,0

To

2011-10-01,0
2011-10-02,0
2011-10-03,0
2011-10-04,0
2011-10-05,0
2011-10-06,0
2011-10-07,0
2011-10-08,0
2011-10-09,111258
2011-10-10,105451
2011-10-11,129304
2011-10-12,131212
2011-10-13,140890
2011-10-14,132396
2011-10-15,0
2011-10-16,0
2011-10-17,0
2011-10-18,0
2011-10-19,0
2011-10-20,0
2011-10-21,0
2011-10-22,0
2011-10-23,0
2011-10-24,0
2011-10-25,0
2011-10-26,0
2011-10-27,0
2011-10-28,0
2011-10-29,0
2011-10-30,0
2011-10-31,0

Solution:
sed looked like a simple enough solution.

shell> sed  '1,8 s/\(2011-10-[0-9]\+\),[0-9]\+$/\1,0/' Oct2011_totals.csv > temp_file && mv temp_file Oct2011_totals.csv
Categories: bash Tags:

Confoo 2011 presentation

October 7, 2011 Leave a comment

2011-03-12 – Confoo 2011 presentation
Slides are available from the commandlinefu presentation at Confoo 2011: http://presentations.codeinthehole.com/confoo2011/

Categories: bash Tags: