Archive

Archive for the ‘python’ Category

Deploy Any Python Project to Kubernetes

April 16, 2020 Leave a comment

As your project grows, it might get to the point that it becomes too hard to handle with just single VM or some simple SaaS solution. You can solve that by switching to more robust solution like Kubernetes. That might however, be little too complex if you are not familiar with it’s concepts or if just never used it before. So, to help you out – in this article – we will go over all you need to get yourself started and have your Python project deployed on cluster – including cluster setup, all the Kubernetes manifests and some extra automation to make your life easier down the road!

This is a follow up to previous article(s) about Automating Every Aspect of Your Python Project, so you might want check that out before reading this one.

TL;DR: Here is my repository with full source code and docs: https://github.com/MartinHeinz/python-project-blueprint

https://martinheinz.dev/blog/20

Source:
Python Weekly

Categories: Interesting, kubernetes, python Tags:

Printing the generated query including parameters

January 16, 2020 Leave a comment
Categories: Interesting, python Tags: ,

Getting Started with Heroku – notes

January 6, 2020 Leave a comment

Notes:

1. Install the heroku cli.
2. If you have a preexisting git repository with the desired files to be pushed to heroku.
You need to have at minimum a Procfile and requirements.txt file

$ less Procfile
web: gunicorn wsgi:app
$ less requirements.txt
Flask==1.0.3
alembic==1.0.10
...
gunicorn==20.0.4

3. Merge changes into master branch if working off a different branch.
4. Login into heroku.
$ heroku login
5. Create a heroku app. You can specify a name for the app. Below command generate a random name.
$ heroku create
To create an app outside the default region of the us.
$ heroku create midnight-cask-76349 --region eu
6. Create a heroku postgres database.
$ heroku addons:create heroku-postgresql:hobby-dev
7. Push changes up to heroku app. Alembic changes get applied automatically on successful deploy.
$ git push heroku master
8. Set application `Config vars` under settings on the dashboard
9. Get database application details from dashboard. Follow link from database details under installed addons section. THe database url details will be available under the linked settings. Also have instructions on how to connect to database
$ heroku pg:psql postgresql-sparkle-97339 --app midnight-cask-76349
10. To manually run alembic
$ heroku run alembic upgrade head
11. To check logs
$ heroku logs --tail
12. To get database credentials
$ heroku pg:credentials:url
13. You can clone the heroku repo for the app locally
$ heroku git:clone -a midnight-cask-76349

Sources:
https://stackabuse.com/deploying-a-flask-application-to-heroku/
https://stackoverflow.com/questions/34379130/pushing-to-heroku-fails-no-such-app-fatal
https://elements.heroku.com/addons/heroku-postgresql
https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-gunicorn-and-nginx-on-ubuntu-14-04
https://blog.dbi-services.com/a-free-postgresql-cloud-database/
https://devcenter.heroku.com/articles/heroku-postgresql#designating-a-primary-database

Categories: postgresql, python Tags:

SSLError(“Can’t connect to HTTPS URL because the SSL module is not available.”)

November 30, 2019 Leave a comment

Problem:
I was not able to run $ python3.7 -m pip install --upgrade pip on my Ubuntu 14.0 (Trusty) box. I was getting an error similar to

Could not fetch URL https://pypi.org/simple/slackclient/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host=’pypi.org’, port=443): Max retries exceeded with url: /simple/slackclient/ (Caused by SSLError(“Can’t connect to HTTPS URL because the SSL module is not available.”)) – skipping

Solution:
I had to manually install openssl and python3.7 for it to work on Ubuntu Trusty.

Sources:

https://help.dreamhost.com/hc/en-us/articles/115000702772-Installing-a-custom-version-of-Python-3

https://help.dreamhost.com/hc/en-us/articles/360001435926-Installing-OpenSSL-locally-under-your-username

Categories: python Tags:

Improve your tests in Django with fakes and factories

August 12, 2019 Leave a comment

Introduction
In our Django apps, we have the M(odel)T(emplate)V(iew) structure (mostly known as MVC). When we want to test its functionality we usually have to create some model instances and work with them and the database.

A nice and easy approach for doing so is to create fake data in our tests in the form of factories and fakers.

In the following article, we are going to look at some practical examples and techniques. This will improve the readability and behavior of your tests.

https://hacksoft.blog/improve-your-tests-django-fakes-and-factories/

Source:
https://www.pythonweekly.com/

Categories: python Tags: ,

OSError: [Errno 8] Exec format error

June 5, 2019 Leave a comment

Problem:
$ python3 run.py
* Serving Flask app "petsrus.petsrus" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: on
* Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
* Restarting with stat
Traceback (most recent call last):
File "run.py", line 4, in
app.run(host="0.0.0.0", port=5000)
File "/home/vagrant/env/lib/python3.4/site-packages/flask/app.py", line 944, in run
run_simple(host, port, self, **options)
File "/home/vagrant/env/lib/python3.4/site-packages/werkzeug/serving.py", line 1007, in run_simple
run_with_reloader(inner, extra_files, reloader_interval, reloader_type)
File "/home/vagrant/env/lib/python3.4/site-packages/werkzeug/_reloader.py", line 332, in run_with_reloader
sys.exit(reloader.restart_with_reloader())
File "/home/vagrant/env/lib/python3.4/site-packages/werkzeug/_reloader.py", line 176, in restart_with_reloader
exit_code = subprocess.call(args, env=new_environ, close_fds=False)
File "/usr/lib/python3.4/subprocess.py", line 537, in call
with Popen(*popenargs, **kwargs) as p:
File "/usr/lib/python3.4/subprocess.py", line 859, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.4/subprocess.py", line 1457, in _execute_child
raise child_exception_type(errno_num, err_msg)
OSError: [Errno 8] Exec format error


Solution:

Added #!/usr/bin/env python to the file

#!/usr/bin/env python
from myapp import app

if __name__ == "__main__":
app.run(host="0.0.0.0", port=5000)


Source:

https://stackoverflow.com/questions/27606653/oserror-errno-8-exec-format-error

Categories: python

Vim as a Python IDE

February 1, 2019 Leave a comment

VIM as a Python IDE

This project aims to use Vim as a powerful and complete Python IDE. In order to do that, we curated a list of awesome plugins available in the community and provided an automatic installation procedure for this set.

https://rapphil.github.io/vim-python-ide/

Source:
https://www.pythonweekly.com/

Categories: Interesting, python, vim Tags:

Designing Well-Structured REST APIs with Flask-RestPlus: Part 1

December 11, 2018 Leave a comment

This is the first part of a two-part series. In this post, I will introduce Flask-RestPlus and demonstrate how to start organising APIs around its simple REST-based conventions. Next time, I will address the topic of request/response marshalling (serialisation) and validation.

https://preslav.me/2018/12/02/designing-well-structured-rest-apis-with-flask-restplus-part-1/

Categories: python Tags:

pip – ImportError: cannot import name main

December 7, 2018 Leave a comment

Problem:
Getting above error when running pip

ubuntu@ubuntu-xenial:~$ pip install --upgrade pip
Collecting pip
  Downloading https://files.pythonhosted.org/packages/c2/d7/90f34cb0d83a6c5631cf71dfe64cc1054598c843a92b400e55675cc2ac37/pip-18.1-py2.py3-none-any.whl (1.3MB)
    100% |████████████████████████████████| 1.3MB 646kB/s 
Installing collected packages: pip
Successfully installed pip-8.1.1
You are using pip version 8.1.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.


ubuntu@ubuntu-xenial:~$ pip install --upgrade pip
Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    from pip import main
ImportError: cannot import name main

Solution:

ubuntu@ubuntu-xenial:~$ python -m pip install -U pip
Requirement already up-to-date: pip in ./.local/lib/python2.7/site-packages (18.1)

Source:
https://stackoverflow.com/questions/28210269/importerror-cannot-import-name-main-when-running-pip-version-command-in-windo

Categories: python

Brett Slatkin – How to Be More Effective with Functions – PyCon 2015

November 8, 2018 Leave a comment

Functions improve readability, encourage reuse, and facilitate refactoring. Python has many unique features that make functions significantly more powerful. This talk will show you the best ways to use functions in Python: when *args is helpful and when it’ll crash your programs; how to use generators for arguments and return values; the value of keyword vs. keyword-only arguments; and more!

Categories: Interesting, python