Generating JSON from SQL

May 3, 2017 Leave a comment

Generating JSON in PostgreSQL can be several times faster than copying the relational data to the application and then generating the JSON.


Categories: Interesting, postgresql Tags: , ,

How To Build a Prediction API in 10 Minutes with Flask, Swagger, and SciPy

April 29, 2017 Leave a comment


I’ve seen a lot of hype around Prediction APIs, recently. This is obviously a byproduct of the current data science fad.

As a public service, I’m going to show you how you can build your own prediction API … and I’ll do it by creating a very basic version in 10 minutes.

We will build an API that will determine if we should provide credit to someone based on certain demographic information.

We will use Kaggle’s “Give Me Some Credit” dataset as the basis for this example.

Go to the “Give Me Some Credit” page, and download the files.

You will have 4 files:

  • cs-training.csv
  • cs-test.csv
  • sampleEntry.csv
  • DataDictionary.xls

We will only need the cs-training.csv and DataDictionary.xls files for this project.

Create application folder

Use the following commands to create a directory and move into it.

In the directory create a file called We will use incrementally…

View original post 545 more words

Categories: Interesting

You don’t need a Linkedin account

April 26, 2017 Leave a comment

In recent years, Linkedin has perceivably become a rather important part of the modern business world. People use this social network to search for jobs, advertise jobs, and get their own work-related resume out there into the spotlight. Which is why I always get a funny look when people ask me to add them on Linkedin, and I tell them, I don’t have one.

The same why I told you why you should not be using Facebook back in 2010, and the arguments still hold valid, I would like to tell you why you might want to entertain the idea of not having a business profile on a social media site, and why this could actually be good for your career. To wit, let us philosophize.

Categories: Interesting

Installing Elasticsearch 2.4

April 25, 2017 Leave a comment

Problem: Elasticsearch 2.4 on vm.

Pretty straight foward if you following the instructions on the Elasticsearch site.

1. Setup a new box.

$ vagrant init ubuntu/trusty64
$ vagrant up
$ vagrant ssh

2. Download java.

3. Install java.
As per instructions below.

Once you have java running on your vm.

$ java -version
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

4. Install Elasticsearch.
As per guide below.

$ cd elasticsearch-2.4.4/bin
$ ./elasticsearch
[2017-04-25 09:52:24,646][INFO ][node                     ] [Thundra] version[2.4.4], pid[7102], build[fcbb46d/2017-01-03T11:33:16Z]
[2017-04-25 09:52:24,652][INFO ][node                     ] [Thundra] initializing ...
[2017-04-25 09:52:25,905][INFO ][plugins                  ] [Thundra] modules [reindex, lang-expression, lang-groovy], plugins [], sites []
[2017-04-25 09:52:26,009][INFO ][env                      ] [Thundra] using [1] data paths, mounts [[/ (/dev/sda1)]], net usable_space [34.6gb], net total_space [39.3gb], spins? [possibly], types [ext4]
[2017-04-25 09:52:26,011][INFO ][env                      ] [Thundra] heap size [1015.6mb], compressed ordinary object pointers [true]
[2017-04-25 09:52:26,012][WARN ][env                      ] [Thundra] max file descriptors [4096] for elasticsearch process likely too low, consider increasing to at least [65536]
[2017-04-25 09:52:30,427][INFO ][node                     ] [Thundra] initialized
[2017-04-25 09:52:30,429][INFO ][node                     ] [Thundra] starting ...
[2017-04-25 09:52:30,657][INFO ][transport                ] [Thundra] publish_address {}, bound_addresses {[::1]:9300}, {}
[2017-04-25 09:52:30,669][INFO ][discovery                ] [Thundra] elasticsearch/I1jPodmCSGG4YanF0bceyQ
[2017-04-25 09:52:33,738][INFO ][cluster.service          ] [Thundra] new_master {Thundra}{I1jPodmCSGG4YanF0bceyQ}{}{}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2017-04-25 09:52:33,786][INFO ][http                     ] [Thundra] publish_address {}, bound_addresses {[::1]:9200}, {}
[2017-04-25 09:52:33,788][INFO ][node                     ] [Thundra] started
[2017-04-25 09:52:33,844][INFO ][gateway                  ] [Thundra] recovered [0] indices into cluster_state
$ curl 'localhost:9200/_cat/health?v'
epoch      timestamp cluster       status shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1493114161 09:56:01  elasticsearch green           1         1      0   0    0    0        0             0                  -                100.0%
Categories: elasticsearch Tags: , ,

Enable ssh X11 forwarding on Vagrant boxes

April 22, 2017 Leave a comment

I got the following error when trying to run tkinter on a vagrant box. (Ubuntu/trusty32).

TclError: no display name and no $DISPLAY environment variable

Forward X11 Display to the host machine.

Following the tutorial shown in the source section below, the minimum I needed to do was to add

config.ssh.forward_x11 = true

to the Vagrantfile.

$ less Vagrantfile
  # Every Vagrant development environment requires a box. You can search for
  # boxes at = "ubuntu/trusty32"

  # Forward X11
   config.ssh.forward_x11 = true

Then, vagrant up, vagrant ssh, then tried to run the python code again. Code works as expected.


Categories: vagrant

Redis as a JSON store

April 21, 2017 Leave a comment

tl;dr a Redis module that provides native JSON capabilities – get it from the GitHub repository or read the docs online. Both JSON and Redis need no introduction; the former is the standard data interchange format between modern applications, whereas the latter is ubiquitous wherever performant data management is needed by them. That being the […]

via Redis as a JSON store — thoughts…

Categories: Interesting Tags: ,

The real prerequisite for machine learning isn’t math. It’s data analysis.

April 21, 2017 Leave a comment

When beginners get started with machine learning, the inevitable question is “what are the prerequisites? What do I need to know to get started?” And once they start researching, beginners frequently find well-intentioned but disheartening advice, like the following: You need to master math. You need all of the following: – Calculus – Differential equations […]

via The real prerequisite for machine learning isn’t math, it’s data analysis — thoughts…