Posts Tagged ‘python’

Install black on VIM

April 26, 2020 Leave a comment

What is black?

Black is the uncompromising Python code formatter. By using it, you agree to cede control over minutiae of hand-formatting. In return, Black gives you speed, determinism, and freedom from pycodestyle nagging about formatting. You will save time and mental energy for more important matters.

Blackened code looks the same regardless of the project you’re reading. Formatting becomes transparent after a while and you can focus on the content instead.

Black makes code review faster by producing the smallest diffs possible.


Installation steps
1. Deactivate any active virtual environments you may have running.

$ deactivate

Failure to do may lead to the following errors after completing all the installation steps on the github page.

When running :Black

Traceback (most recent call last):
  File "", line 1, in 
NameError: name 'Black' is not defined

opening a *.py file

Error detected while processing ~/.vim/plugged/black/plugin/black.vim:
line  124:
Traceback (most recent call last):
  File "", line 56, in 
ModuleNotFoundError: No module named 'black'

2. Install a plugin manager. Eg Vundle

$ git clone ~/.vim/bundle/Vundle.vim

3. Edit .vimrc to install plugin.
With Vundle

$ less .vimrc
set nocompatible              " be iMproved, required
filetype off                  " required

" set the runtime path to include Vundle and initialize
set rtp+=~/.vim/bundle/Vundle.vim
call vundle#begin()

" let Vundle manage Vundle, required
Plugin 'VundleVim/Vundle.vim'
Plugin 'psf/black'

" All of your Plugins must be added before the following line
call vundle#end()            " required
filetype plugin indent on    " required

" Specify the virtual environment path for black
let g:black_virtualenv='~/.vim/black'
" Run black on save
autocmd BufWritePre *.py execute ':Black'

4. Install the black plugin

$ vim

You may get the following error

Please wait, one time setup for Black.
Creating a virtualenv in /home/vagrant/.vim/black...
(this path can be customized in .vimrc by setting g:black_virtualenv)
The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.
    apt-get install python3-venv
You may need to use sudo with that command.  After installing the python3-venv
package, recreate your virtual environment.
Failing command: ['/home/vagrant/.vim/black/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']
Error detected while processing /home/vagrant/.vim/bundle/black/plugin/black.vim:
line  194:
E880: Can't handle SystemExit of python exception in vim
Press ENTER or type command to continue

5. Install python3-venv

$ sudo apt-get install python3-venv

6. Recreate the virtual environment

$ rm -rf .vim/black
$ vim
Please wait, one time setup for Black.
Creating a virtualenv in /home/vagrant/.vim/black...
(this path can be customized in .vimrc by setting g:black_virtualenv)
Installing Black with pip...
DONE! You are all set, thanks for waiting ✨ 🍰 ✨
Pro-tip: to upgrade Black in the future, use the :BlackUpgrade command and restart Vim.
Press ENTER or type command to continue

7. Last step of installation

vagrant@vagrant:~$ cd .vim/bundle/black/
vagrant@vagrant:~/.vim/bundle/black$ git checkout origin/stable -b stable
Branch 'stable' set up to track remote branch 'stable' from 'origin'.
Switched to a new branch 'stable'
vagrant@vagrant:~/.vim/bundle/black$ cd ~

8. Test black
Add the following to and write (:w)

j = [1,

Black should reformat the file in place and give you

j = [1, 2, 3]

You may get the following error

Error detected while processing BufWrite Auto commands for "*.py":
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 92, in Black
  File "", line 128, in get_configs
"" [New] 5L, 26C written
Press ENTER or type command to continue

Try running :Black inside to see the actual error

Traceback (most recent call last):
  File "", line 1, in 
  File "", line 92, in Black
  File "", line 128, in get_configs
AttributeError: module 'black' has no attribute 'find_pyproject_toml'

The above error may occur if you skipped step 7. You could also run

System details
– Ubuntu 18.04.3 LTS
– VIM 8.0 with python3 support
:echo has(‘python3’)


Categories: vim Tags: , ,

The Art of Subclassing talk by Raymond Hettinger

February 9, 2016 Leave a comment

All problems have simple, easy-to-understand, logical wrong answers. Subclassing in Python is no exception. Avoid the common pitfalls and learn everything you need to know about making effective use of inheritance in Python.

Categories: Interesting Tags: ,

Appengine – Appstats (python)

September 15, 2011 Leave a comment

Setting up Appstats for python and djangoappengine.

Assuming you have your djangoappengine setup as explained here.
Now following the instructions from

1. Add this to


    # ...

2. Add this to app.yaml

- appstats: on

That’s it. Now start your server is running

python2.5 runserver

Point your browser to
You should see something like

Now add some data by visiting some pages eg (Do this in a different tab then refresh.) You should have some stats now.

That’s it. Very simple and straight forward instructions from the docs.

Categories: appengine, django Tags: ,

Automatically update spreadsheets and graphs – Part 2

May 20, 2011 Leave a comment

Finally got some time to work on this. So I modified the initial code I posted here. This “newer” version is able to work with the spreadsheet and worksheet name that you pass on to it.

The data I have in the csv file (mydata.csv) looks like this

Date, Toy Cars sold
2011-01-01, 91
2011-01-02, 92
2011-01-03, 93
2011-01-04, 94
2011-01-05, 95


shell>python mydata.csv
shell>python --user [username] --pw [password] --sname [spreadsheet name] --wname [worksheet name] mydata.csv
shell>python --user [username] --pw [password] --cname [category] --sname [spreadsheet name] --wname [worksheet name] mydata.csv

If your email address is not in this format “” then use the full email address for your google apps account “”.


import csv
import getpass
import gdata.spreadsheet.service
import getopt
import sys
import urllib

def get_data(csvfile):
    """Read data from csv file.

    Read data from the given csv file and return it as a list.

    Keyword arguments:
    csvfile -- The csv file that has the data to be written to the worksheet file.

    reader = csv.reader(open(csvfile, "r"))

    data = []
    for row in reader:

    return data
def get_items(feed):
    """ Get the items in the feed.
    Either a list of documents that the user has or a list of worksheets within
    a given spreadsheet.

    Keyword arguments:
    feed -- The feed (xml file).
    items = {}
    for entry in feed.entry:
        id_parts = urllib.unquote(':','/').split('/')
        key = id_parts[len(id_parts) - 1]
        items[entry.title.text.lower()] = key
    return items

def usage():
    """ How this works """ 
    usage_text = """python --user [username] --pw [password] 
                --cname [category] --sname [spreadsheet name] 
                --wname [worksheet name] csv"""
    print usage_text

# Parse command line options
    opts, args = getopt.getopt(sys.argv[1:], '', ['user=', 'pw=', 'cname=', 'sname=', 'wname='])
except getopt.error, msg:

user = ''
pw = ''
cname = ''
sname = ''
wname = ''

# Process options
for option, arg in opts:
    if option == '--user':
      user = arg
    elif option == '--pw':
      pw = arg
    elif option == '--cname':
      cname = arg
    elif option == '--sname':
      sname = arg
    elif option == '--wname':
      wname = arg

# Get the csv file
    data, = args
    sheet_data = get_data(data)
except ValueError:
    print "You must specify a csv file"
except IOError:
    print "You must specify a csv file"

while not user:
    print 'NOTE: Please run these tests only with a test account.'
    user = raw_input('Please enter your username: ')

while not pw:
    pw = getpass.getpass()
    if not pw:
        print 'Password cannot be blank.'

while not sname:
    sname = raw_input('Please enter a spreadsheet name: ')
    if not sname:
        print 'Spreadsheet name cannot be blank.'
while not wname:
    wname = raw_input('Please enter a worksheet name: ')
    if not wname:
        print 'Worksheet name cannot be blank.'

    category = cname
    email = user
    password = pw
    source = category 
    spreadsheet_name = sname.lower()
    worksheet_name = wname.lower()
except NameError:
    cname = ''

gd_client =
gd_client.ClientLogin(email, password, source=source)

gs_client = gdata.spreadsheet.service.SpreadsheetsService() = email
gs_client.password = password

    # Get the SpreadSheet.
    if not category:
        feed = gd_client.GetDocumentListFeed()
        query =[category])
        feed = gd_client.Query(query.ToUri())

    docs = get_items(feed)

    #print docs # Comment this out to see the spreadsheets a user has.

    curr_key = docs[spreadsheet_name]

    # Get the WorkSheet within the SpreadSheet.
    feed = gs_client.GetWorksheetsFeed(curr_key)

    sheets = get_items(feed)
    #print sheets # Comment this out to see the worksheets a given spreadsheet has.

    curr_wksht_id = sheets[worksheet_name]

    # Write the data in the csv file to the given worksheet.
    rowx = 1
    for row in sheet_data:
        rowx += 1
        for colx, value in enumerate(row):
            colx += 1
            gs_client.UpdateCell(rowx, colx, value, curr_key, curr_wksht_id)

except KeyError:
    print "Sorry, the spreadsheet or worksheet name you specified does not exist."

All the code is from the sample files that come with the API. Mostly from these two files /gdata-2.0.14/samples/docs/ and /gdata-2.0.14/samples/spreadsheets/

1. If you give the wrong spreadsheet key.

  • gdata.service.RequestError: {‘status’: 400, ‘body’: ‘The spreadsheet at this URL could not be found. Make sure that you have the right URL and that the owner of the spreadsheet hasn't deleted it.’, ‘reason’: ‘Bad Request’}

2. Attempting to write data to the wrong location.
In my case I was attempting to write to a cell at position row 1, col 0 instead of row 1, col 1. The error clearly stats R1C0.

  • gdata.service.RequestError: {‘status’: 400, ‘body’: ‘Invalid query parameter value for R1C0.’, ‘reason’: ‘Bad Request’}

3. Using wrong email address.
I was using the wrong email address. So I specified the full email address otherwise it assumes that your address is

  • gdata.service.Error: Server responded with a 403 code
Categories: python Tags: , ,

Automatically update spreadsheets and graphs – Part 1

May 9, 2011 5 comments

Problem: Automatically update a spreadsheet and its corresponding graphs.

I had a number of solutions or so I thought.
1. Use python and xlwt. xlwt does not work with graphs. Atleast not at the moment.
2. Use python uno. The server running the script would need to have a local version of open office.
3. Use googledocs and python. (Final code here)

So far the 3rd solution is looking promising. I have a sample script and a sample the google doc that I was to edit.


import gdata.spreadsheet.service

def _CellsUpdateAction(row,col,inputValue,key,wksht_id):
    entry = spr_client.UpdateCell(row=row, col=col, inputValue=inputValue,
            key=curr_key, wksht_id=curr_wksht_id)
    if isinstance(entry, gdata.spreadsheet.SpreadsheetsCell):
        print 'Updated!'

def _PrintFeed(feed):
    for i, entry in enumerate(feed.entry):
        if isinstance(feed, gdata.spreadsheet.SpreadsheetsCellsFeed):
            print '%s %s\n' % (entry.title.text, entry.content.text)
        elif isinstance(feed, gdata.spreadsheet.SpreadsheetsListFeed):
            print '%s %s %s' % (i, entry.title.text, entry.content.text)
            # Print this row's value for each column (the custom dictionary is
            # built using the gsx: elements in the entry.)
            print 'Contents:'
            for key in entry.custom:  
              print '  %s: %s' % (key, entry.custom[key].text) 
            print '\n',
            print '%s %s\n' % (i, entry.title.text)

email = ''
password = 'password'

spr_client = gdata.spreadsheet.service.SpreadsheetsService() = email
spr_client.password = password

feed = spr_client.GetSpreadsheetsFeed()
# List the spreadsheets that available.
# Totally unneccessary but just here so that you can see what index to give to input variable below.
# Get the first one. (Otherwise pick the desired index)
input = 0 
id_parts = feed.entry[0].id.text.split('/')
curr_key = id_parts[len(id_parts) - 1]
print curr_key

feed = spr_client.GetWorksheetsFeed(curr_key)
id_parts = feed.entry[0].id.text.split('/')
curr_wksht_id = id_parts[len(id_parts) - 1]
print curr_wksht_id

# I just did it this way to test to see if it works.
date1 = "2011-01-01"  # Overwrites the existing dates.
date2 = "2011-01-02"  # Overwrites the existing dates.
date3 = "2011-01-03"  # Overwrites the existing dates.
weight1 = "30"
weight2 = "60"
weight3 = "90"
weight4 = "80"

_CellsUpdateAction(2, 2, "Date", curr_key, curr_wksht_id)
_CellsUpdateAction(3, 2, date1, curr_key, curr_wksht_id)
_CellsUpdateAction(4, 2, date2, curr_key, curr_wksht_id)
_CellsUpdateAction(5, 2, date3, curr_key, curr_wksht_id)
_CellsUpdateAction(2, 3, "Weight", curr_key, curr_wksht_id)
_CellsUpdateAction(3, 3, weight1, curr_key, curr_wksht_id)
_CellsUpdateAction(4, 3, weight2, curr_key, curr_wksht_id)
_CellsUpdateAction(5, 3, weight3, curr_key, curr_wksht_id)
_CellsUpdateAction(6, 3, weight4, curr_key, curr_wksht_id)


Some of the errors I got trying to get the above script working include

gdata.service.RequestError: {‘status’: 400, ‘body’: ‘We're sorry, a server error occurred. Please wait a bit and try reloading your spreadsheet.’, ‘reason’: ‘Bad Request’}

gdata.service.RequestError: {‘status’: 400, ‘body’: ‘Invalid query parameter value for R0C0.’, ‘reason’: ‘Bad Request’}

Before running code.
Graph before running the code.

And this is the graph after running the code.
Graph after running code.

My main objective was updating a spreadsheet and having the graph change as well. So far python + googledocs works very nicely for me. I will upload more code once I get a chance to work on this some more.

Most of the code came from the example in the Google Spreadsheets Data API.

Instructions on how to install Google Spreadsheets Data API for python is available here. The instructions worked for me except that I used sudo python install to install the package.

Part 2

Categories: python Tags: , ,

Printing days of the month.

March 24, 2011 1 comment

I needed to have the dates of the month for reporting purposes. As time was not on my side, I had my collegue do this for me in python. Later on I managed to do this in bash. (NB: The sample code below does not give the exact same results.)

Here is the bash version.

# DESCRIPTION: Lists the days of the given month for the given year in the
#              following format yyyy-mm-dd. If either value is not specified
#              then the current MONTH and/or YEAR is used.


#=== FUNCTION ================================================================
#           NAME: usage
#    DESCRIPTION: Display usage information for this script.
#    PARAMETER 1: ---
function usage()
cat <<- EOT
Lists the days of a given month in yyyy-mm-dd format.

usage: $0 [MONTH] [YEAR] [--help]

Optional values
    MONTH -- the month to use
    YEAR -- the year to use
    help -- display this message

# Display usage
if [ "$1" == "--help" ]

# Assign default values if needed.
if [ -z "$month" ]
    month=$(date "+%m")

if [ -z "$year" ]
    year=$(date "+%Y")

# main
for day in $(cal $month $year);
do  echo $day  | \
    sed -n -e "s/^[0-9]$/${year}-${month}-0&/" \
           -e "s/^[0-9][0-9]$/${year}-${month}-&/" \
           -e "/^${year}-/p" | sed "s/.*/&,0/"

And here is the verbose python version.

"""Create a file containing the days in a given month, for a given year.

For a given month, create a two column csv file, the first column holds the
days in the month. The date format is YYYY-MM-DD. The second column holds data
values, they are all set to zero.

The following should be specified.
 year            The desired year.
 month           The desired month.
 output_file     The file to write the dates to.

 --help          Print this message and exit.

import calendar
import sys
import getopt

USAGE_TEXT = """Usage: year month output_file
For more verbose descriptions, use --help """

def help():
 print >> sys.stderr, __doc__

def usage(code, msg=''):
 """Print usage or error.

 Keyword arguments:
 code -- The exit status code.
 msg -- The message to print out.

 print >> sys.stderr, msg

def create_date_file(file, month, year):
 """Create date file.

 Keyword arguments:
 file -- The name of the file to write to.
 month -- The desired month.
 year -- The desired year.

 # Get the last day of the month
 last_day = calendar.monthrange(year, month)[1]

 temp_file = open(file, 'wb')
 for day in range(1, last_day + 1):
 temp_file.write(str(year) + "-" + str(month).zfill(2) + "-" +
 str(day).zfill(2) +",0 \n")

def main():
 opts, args = getopt.getopt(sys.argv[1:], '',

 for opt, arg in opts:
 if opt in ('--help'):

 year = int(args[0])
 month = int(args[1])
 file = args[2]
 create_date_file(file, month, year)

 except getopt.error, msg:
 usage(1, msg)
 except IndexError:
 usage(1, USAGE_TEXT)
 except ValueError:
 usage(1, USAGE_TEXT)

if __name__ == '__main__':



Categories: bash Tags: ,