Home > python, vagrant > Running a pypi mirror on your laptop with devpi-server

Running a pypi mirror on your laptop with devpi-server

Problem:
I got tired of having to repeatedly run devpi-server every now and again. Figured it was about time I just had a dedicated box running on my laptop.

Solution:

$ mkdir /home/user/devpi_mirror
$ cd devpi_mirror
$ vagrant init "ubuntu/trusty64"

Edit your Vagrantfile and make the following changes:

  1. Uncomment the forwarded port mapping part. Note the new ports.
  2. Uncomment the private_network section. Note the IP address.
$ less Vagrantfile 
 22   # Create a forwarded port mapping which allows access to a specific port
 23   # within the machine from a port on the host machine. In the example below,
 24   # accessing "localhost:8080" will access port 80 on the guest machine.
 25    config.vm.network "forwarded_port", guest: 3141, host: 3141

 27   # Create a private network, which allows host-only access to the machine
 28   # using a specific IP.
 29    config.vm.network "private_network", ip: "192.168.38.18"

Start up virtual box and devpi-server

$ vagrant up --provider virtualbox 
$ ssh vagrant@192.168.38.18
$ sudo apt-get update
$ sudo apt-get install python-pip
$ sudo pip install devpi-server==2.5.0
$ devpi-server --version
2.5.0
$ devpi-server --port 3141 --host 192.168.38.18
2016-03-02 20:46:39,633 INFO  NOCTX Loading node info from /home/vagrant/.devpi/server/.nodeinfo
2016-03-02 20:46:39,636 INFO  NOCTX generated uuid: e494e5c5156f4c9299339c4298cfd07c
2016-03-02 20:46:39,640 INFO  NOCTX wrote nodeinfo to: /home/vagrant/.devpi/server/.nodeinfo
2016-03-02 20:46:39,649 INFO  NOCTX DB: Creating schema
2016-03-02 20:46:39,656 INFO  [Wtx-1] setting password for user u'root'
2016-03-02 20:46:39,656 INFO  [Wtx-1] created user u'root' with email None
2016-03-02 20:46:39,657 INFO  [Wtx-1] created root user
2016-03-02 20:46:39,657 INFO  [Wtx-1] created root/pypi index
2016-03-02 20:46:39,664 INFO  [Wtx-1] fswriter0: committed: keys: u'.config',u'root/.config'
2016-03-02 20:46:39,665 INFO  NOCTX retrieving initial name/serial list
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:315: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:120: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
2016-03-02 20:46:50,499 INFO  [Wtx0] fswriter1: committed: keys: u'root/pypi/initiallinks'
2016-03-02 20:46:50,739 INFO  NOCTX Found plugin devpi-server-2.5.0 (/usr/local/lib/python2.7/dist-packages).
2016-03-02 20:46:50,740 INFO  NOCTX Found plugin devpi-server-2.5.0 (/usr/local/lib/python2.7/dist-packages).
2016-03-02 20:46:50,792 INFO  NOCTX devpi-server version: 2.5.0
2016-03-02 20:46:50,793 INFO  NOCTX serverdir: /home/vagrant/.devpi/server
2016-03-02 20:46:50,793 INFO  NOCTX uuid: e494e5c5156f4c9299339c4298cfd07c
2016-03-02 20:46:50,793 INFO  NOCTX serving at url: http://192.168.38.18:3141
2016-03-02 20:46:50,794 INFO  NOCTX bug tracker: https://bitbucket.org/hpk42/devpi/issues
2016-03-02 20:46:50,794 INFO  NOCTX IRC: #devpi on irc.freenode.net
2016-03-02 20:46:50,794 INFO  NOCTX Hit Ctrl-C to quit.
serving on http://192.168.38.18:3141

On a separate box

$ telnet 192.168.38.18 3141
Trying 192.168.38.18...
Connected to 192.168.38.18.
Escape character is '^]'.
^]

telnet>
$ sudo pip install -i http://192.168.38.18:3141/root/pypi/ markdown
Downloading/unpacking markdown
  http://192.168.38.18:3141/root/pypi/markdown/ uses an insecure transport scheme (http). Consider using https if 192.168.38.18:3141 has it available
  Downloading Markdown-2.6.5.zip (412kB): 412kB downloaded
  Running setup.py (path:/tmp/pip_build_root/markdown/setup.py) egg_info for package markdown

Installing collected packages: markdown
  Running setup.py install for markdown
    changing mode of build/scripts-2.7/markdown_py from 644 to 755
    Converting docs/install.txt -> build/docs/install.html
...
    changing mode of /usr/local/bin/markdown_py to 755
Successfully installed markdown
Cleaning up..

Change you pip.conf to include a link to your local devpi-server. This means that you can now just run “sudo pip install markdown”

$ vim $HOME/.pip/pip.conf
[global]
index-url = http://192.168.38.18:3141/root/pypi/+simple/

Note:
I tried this with devpi-server version 2.1.5 and it did not work for me. So if you have a similar issue just move to a higher version before you give up.

If you get plugin errors like the ones below when checking the version of devpi-server then install devpi-web. Hopefully, it will fix your issue.

pluggy.PluginValidationError: Plugin 'devpi-web' hook 'devpiserver_on_changed_versiondata' argument 'projectname' not available plugin definition: devpiserver_on_changed_versiondata(stage, projectname, version, metadata) available hookargs: __multicall__, stage, project, version, metadata

Update 2016-10-28
Use supervisor to run devpi-server
1. Install supervisor.

$ sudo apt-get install supervisor

2. Update supervisor config with devpi server settings.

$ less /etc/supervisor/conf.d/devpi-server-local.conf
[program:devpi-server-local]                              ; Change the program name
command=/usr/local/bin/devpi-server --port 3141 --host 192.168.38.18  ; Command to start application
user=devpi_server_local                           ; User to run as
stdout_logfile=/var/log/devpi-server/devpi-server.log       ; Where to write log messages
redirect_stderr=true
autostart=true
stopasgroup=true
startsecs=10
startretries=5
stdout_logfile_maxbytes=50MB

3. Add a user

$ sudo adduser --system --group devpi_server_local
Adding system user `devpi_server_local' (UID 109) ...
Adding new group `devpi_server_local' (GID 115) ...
Adding new user `devpi_server_local' (UID 109) with group `devpi_server_local' ...
Creating home directory `/home/devpi_server_local' ...

4. Create log folder

sudo mkdir /var/log/devpi-server

5. Restart supervisor

$ sudo service supervisor restart
Restarting supervisor: supervisord.

6. Check that devpi-server is running.

$ sudo supervisorctl 
devpi-server-local               RUNNING    pid 5793, uptime 0:00:21
supervisor> exit
$ 

Source:
http://doc.devpi.net/latest/quickstart-pypimirror.html
http://blog.wearefarm.com/2015/07/14/devpi-server/

Advertisements
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: