Right-joining a table that uses soft delete in Python SQLAlchemy

Saturday September 02, 2017

Typically when implementing a new SQL table, I use soft deletes over hard deletes. Soft deletes enables one to make audit operations without having a separate analytics system, and in many cases (say, purchases which reference old inventory), soft deletes are a requirement.

The only issue is when you try to join a table with soft deletes, funny things can happen. Take the following query, where we have accounts, and active credit cards for people to perform payment, which can …

Python Serialization Performance

Saturday January 30, 2016

As projects reach scale of moderate complexity, they must represent objects in many forms. As a json record from an API endpoint, cache, or database, as a thrift object for RPC, or as an object in-memory.

At Uber, we've begun to use the schematics library for our in-memory representations. It provides a canonical form for an object to take, which can then be serialized into thrift, sql, or various json forms. The native serialization works great for the base case …

Puffin, the Python stream manipulation tool

Thursday December 18, 2014

This is a horrifically abbreviated post, but I was recently grepping through lots of sys files, summing up columns, and doing those sorts of file manipulations. After struggling with awk and sed, I decided there must be a better way. So I created puffin. It's a slick tool to use, check it out.

Submitting My First Patch to Python

Thursday February 20, 2014

I recently discovered my first bona fide bug in Python. I tracked down the issue, submitted a patch, which was thereafter accepted. I'm writing this blog post to show how easy it is to contribute to open source, how approachable the Python community is, and because I'm downright proud that I've now contributed to Python!

Monkey-patching the Python logging module

Saturday February 08, 2014

I took an interesting foray into the Python standard library the other day. It was the first time that I had dug into CPython's source code myself, so I figured my experience might be interesting to others considering doing the same thing.

Logging is a great thing - I'll just state that flat out. And Python's logging module is pretty darn good as well. An easy-to-use API, easy switching between different levels, output to stdout or to a file, and so …

Install Python's PIL on OS X Mavericks

Friday February 07, 2014

Got this solution from Stack Overflow.

Basically, previously had the following error:

_imagingft.c:73:10: fatal error: 'freetype/fterrors.h' file not found

#include <freetype/fterrors.h>


1 error generated.

You need freetype installed (use brew). But that creates the freetype2 library. So just do a

ln -s /usr/local/include/freetype2/ /usr/local/include/freetype

And then you'll be able to do the following successfully!

pip install -U PIL --allow-external PIL --allow-unverified PIL

Installing OpenCV 2.4 on Mac OS X Lion

Tuesday April 16, 2013

Some quick help on installing OpenCV on Mac Lion. I had a lot of trouble for a long time, and the default installation instructions had me keep on getting Segmentation Fault: 11 when I would try to import cv2. I discovered the default installation instructions create a reference to the incorrect version of Python. Here's how to fix that, playing off the regular build instructions.

mkdir build
cd build
cmake -D PYTHON_EXECUTABLE=/Library/Frameworks/Python.framework/Versions/2.7/bin …