Kenneth Reitz

Be Cordial or Be on Your Way

2012-05-14T10:00:00Z

Contributing to Open Source can be stressful.

By opening a Pull Request to a popular project, you're inviting the maintainers and all of their users to analyze and scrutinize your code. Often, they will.

Most of the time, the maintainers of the projects do their best to be kind. If they have feedback or criticism, it's generally positive. Unfortunately, this isn't always the case.

Every once in while, maintainers and community members will shun and insult a contributor's work. This is a perfect example of what you should never do.

What if the Pull Request in question was the person's first time ever contributing to a project? They could be scared away from the world of open source forever.

Instead of simply yelling at a potential contributor and causing a mess of negativity and stress; the maintainer should take the opportunity to educate the user. They should point the user to a guide that outlines what the proper protocol for his project is.

As a project maintainer, it's extremely important to be respectful to your contributors. They often take what you say very personally.

Publishing Open Source

I don't handle stress and negativity well at all. I almost instantly get a terrible headache, have difficulty concentrating, lose my appetite, and my general quality of life degrades almost immediately.

There's a difference between being stressed and being in a stressful situation. One can be caught in a tremendously stressful situation and not be stressed at all with the right mixture of perspective and empathy.

By publishing your code (or anything, really) on the open internet, you're inviting the whole world to analyze and scrutinize it, line by line. That takes guts. People may love it, hate it, disagree with your general approach, or be convinced that you're doing their community a huge disservice. That can be stressful.

Don't worry about non-constructive feedback. It serves no purpose. Realize that some people just take things way too seriously. Tell them to be on their way.

Listen to everyone, then disregard it.

—311

Reflect

Take some time to reflect. Remember why you're doing this in the first place. I write open source software because I love software and want to help make my world a better place.

Why are you? If you don't have answer, maybe you shouldn't be doing it. If you find yourself maintaining a project purely out of obligation, maybe you should be doing less.

Don't be an asshole.
Make your contributors feel welcome and appreciated.
Always say thank you.

Before you tweet, comment on a pull request, or comment publically about a project, remember that it's often not a peice of code you're critiquing — it's a human being.

I don't have the energy or patience for non-constructive feedback. Be cordial and positive or be on your way.

Repository Structure and Python

2012-04-16T21:00:00Z

This opinionated guide was written for my best-practices guide, The Hitchhiker's Guide to Python.

It is based on the recommendations of the Distribute project, personal experience, and good intentions.

It's Important.

Just as Code Style, API Design, and Automation are essential for a healthy development cycle, Repository structure is a crucial part of your project's architecture.

When a potential user or contributor lands on your repository's page, they see a few things:

Project Name
Project Description
Bunch O' Files

Only when they scroll below the fold will the user see your project's README.

If your repo is a massive dump of files or a nested mess of directories, they might look elsewhere before even reading your beautiful documentation.

Dress for the job you want, not the job you have.

Of course, first impressions aren't everything. You and your colleagues will spend countless hours working with this repository, eventually becoming intimately familiar with every nook and cranny. The layout of it is important.

Sample Repository

tl;dr: This is what I recommend.

This repository is available on GitHub.

README.rst
LICENSE
setup.py
requirements.txt
sample/__init__.py
sample/core.py
sample/helpers.py
docs/conf.py
docs/index.rst
tests/test_basic.py
tests/test_advanced.py

Let's get into some specifics.

The Actual Module

Location		`./sample/` or `./sample.py`
Purpose		The code of interest.

Your module package is the core focus of the repository. It should not be tucked away:

./sample/

If your module consists of only a single file, you can place it directly in the root of your repository:

./sample.py

Your library does not belong in an ambiguous src or python subdirectory.

License

Location		`./LICENSE`
Purpose		Lawyering up.

This is arguably the most important part of your repository, aside from the source code itself. The full license text and copyright claims should exist in this file.

No excuses.

Setup.py

Location		`./setup.py`
Purpose		Package and distribution management.

If your module package is at the root of your repository, this should obviously be at the root as well.

Requirements File

Location		`./requirements.txt`
Purpose		Development dependencies.

A Pip requirements file should be placed at the root of the repository. It should specify the dependencies required to contribute to the project: testing, building, and generating documentation.

If your project has no development dependencies, or you prefer development environment setup via setup.py, this file may be unnecessary.

Documentation

Location		`./docs/`
Purpose		Package reference documentation.

There is little reason for this to exist elsewhere.

Test Suite

Location		`./test_sample.py` or `./tests`
Purpose		Package integration and unit tests.

Starting out, a small test suite will often exist in a single file:

./test_sample.py

Once a test suite grows, you can move your tests to a directory, like so:

tests/test_basic.py
tests/test_advanced.py

Obviously, these test modules must import your packaged module to test it. You can do this a few ways:

Expect the package to be installed in site-packages.
Use a simple (but explicit) path modification to resolve the package properly.

I highly recommend the latter. Requiring a developer to run setup.py develop to test an actively changing codebase also requires them to have an isolated environment setup for each instance of the codebase.

To give the individual tests import context, create a tests/context.py file:

import os
import sys
sys.path.insert(0, os.path.abspath('..'))

import sample

Then, within the individual test modules, import the module like so:

from .context import sample

This will always work as expected, regardless of installation method.

Some people will assert that you should distribute your tests within your module itself — I disagree. It often increases complexity for your users; many test suites often require additional dependencies and runtime contexts.

Makefile

Location		`./Makefile`
Purpose		Generic management tasks.

If you look at most of my projects or any Pocoo project, you'll notice a Makefile laying around. Why? These projects aren't written in C... In short, make is a incredibly useful tool for defining generic and platform agnostic tasks for your project.

Sample Makefile:

init:
    pip install -r requirements.txt

test:
    nosetests tests

Other generic management scripts (e.g. manage.py or fabfile.py) belong at the root of the repository as well.

Regarding Django Applications

I've noticed a new trend in Django applications since the release of Django 1.4. Many developers are structuring their repositories poorly due to the new bundled application templates.

How? Well, they go to their bare and fresh repository and run the following, as they always have:

$ django-admin.py start-project samplesite

The resulting repository structure looks like this:

README.rst
samplesite/manage.py
samplesite/samplesite/settings.py
samplesite/samplesite/wsgi.py
samplesite/samplesite/sampleapp/models.py

Don't do this.

Repetitive paths are confusing for both your tools and your developers. Unnecessary nesting doesn't help anybody (unless they're nostalgic for monolithic SVN repos).

Let's do it properly:

$ django-admin.py start-project samplesite .

Note the ".".

The resulting structure:

README.rst
manage.py
samplesite/settings.py
samplesite/wsgi.py
samplesite/sampleapp/models.py

Much better.

The Future of Python HTTP

2012-04-02T12:00:00Z

I like to think Requests is mostly analogous to Werkzeug in terms of purpose, functionality, and goals. One is for servers while the other is for clients.

Werkzeug and Flask were huge inspirations for Requests' design. As a matter of fact, Requests contains a decent bit of Werkzeug's internal data structures.

So, why are they separate projects?

Brainstorming

At PyCon 2012 a few weeks ago, Andrey Petrov, Armin Ronacher, Paul McMillan, and myself got into a room for a brainstorming session around the possibility of formally combining our efforts.

My expectations going in weren't that high, but that quickly changed once we were all in the same room. We discussed the general state of Python HTTP, security concerns, distributed services, and web application testing.

Today, making real HTTP Requests to an in-process WSGI app with a real HTTP client is not simple. Can you imagine writing real OAuth tests for your application with the same HTTP consumer your clients will use?

The root of the problem is that WSGI doesn't map 1:1 to HTTP.

So, instead of taking the WebOb approach of using WSGI as the common protocol between services, why not use HTTP itself? The rest of the world uses HTTP as the most-common denominator after all.

After a few hours, we drafted up a solid plan:

Consolidate shared code between Requests and Werkzeug into a new httpcore module.
Move WSGI-specific Werkzeug code into a new wsgicore module.
Make HTTP (vs WSGI) the common protocol between services.
Provide a transport adapter mechanism for mocking and emulating HTTP services.

The Architecture

Requests, Flask, and Werkzeug will remain the same to the end user.

Behind the scenes, the same functions used to generate a request will be used to consume it. For example, stream handling, header parsing, and form-encoding will all be synchronous functions from httpcore.

Adapters

Transport Adapters will provide a mechanism to define interaction methods for an "HTTP" service. They will allow you to fully mock a web service to fit your needs.

Gloriously simplified example (implementation subject to change):

class DistributedAdapter(BaseAdapter):
    def __init__(self):
        self.connect_pool = …

    def send(self, request):
        """Takes a Request object, returns a Response object."""
        # Whatever needs to happen here.
        …

HTTPCore

HTTPCore will be comprised of the code currently shared by Requests and Werkzeug, general HTTP utilities, and base objects / data structures.

Specifically, it will provide:

Request and Response objects
General HTTP Utilities
Common Data Structures (MultiDict, CaseInsensitiveDict)
Common Data Structure Utilities (merge_kwargs)
Stream Handling (make_line_iter, make_chunk_iter)
HTTP Parsing (http-parser)
URI/IRI Parsing and Handling Functions (uricore)
SSL Utilities
Cookie Handling
Base TransportAdapter

WSGICore

WSGICore will extend the framework that HTTPCore provides to be used by WSGI applications. It will replace the WSGI-specific parts of Werkzeug:

Request and Response objects
WSGI Transport Adapter (HTTP <-> WSGI)
WSGI Utility Functions

Distributed Services

In addition to testing web applications, this new Adapter system will provide a fantastic mechanism for distributed services.

In Requests, you'll be able to mount external services to the routing mechanism by mocking HTTP. To Requests, it'll be an HTTP Service, but in reality the service could be anything: a random number generator, ZeroMQ socket, proxy, WSGI application, &c.

Here's some theoretical example code:

import requests

from webscale import DevNullAdpater
from wsgicore.adapters import WsgiAdapter
from haystackapp.core import app as haystack

s = requests.session()
s.mount('null:', DevNullAdapter())
s.mount('http://haystack', WsgiAdapter(app=haystack))

# Make a request via DevNullAdapter
r = s.get('null://someurl')

# Make a request via Haystack WSGI App
r = s.get('http://haystack/index')

# Make a request via standard HTTPS
r = s.get('https://github.com/')

Long-term Advantages

There's a number of advantages to this design and approach in the future:

Requests will be able to use the same cache backends for HTTP Requests that Flask/Werkzeug does for views. They will be moved to cachecore.
Security enhancements (e.g. DNSSEC) can live in httpcore rather than waiting for a PEP or standard library implementation.
Django could potentially utilize the security features provided by httpcore.
Django/Flask could potentially use Requests as their respective official test clients.

Development

If you have thoughts to share, feel free to discuss this with us on Freenode at #cores.

There's little code to show at the moment, but you can track the development over on GitHub:

https://github.com/core

Xcode, GCC, and Homebrew

2012-02-16T10:00:00Z

Several months ago, I got fed up with having to download Xcode to build my software. I took the Xcode installer, ripped out all of the parts I didn't need, and made a nice installer for GCC. It ended up being ~200MB in size. It took 2 minutes to download.

OSX-GCC-Installer was born. Perfect.

Unfortunately, I couldn't include the 10.6/7 SDKs due to licensing restrictions. These SDKs include CoreAudio, CoreData, OpenGL, and more. Most software that isn't build specifically for OSX would build perfectly. Unfortunately, some software added some needless system dependencies, though (I'm looking at you, Node).

I stuck it up on GitHub, and much to my delight, it became a pretty big hit. It solved a lot of headaches for a lot of people.

Today, the project has 1649 watchers on GitHub and has been downloaded 53,400 times. That's 13.7 Terabytes of transfer. Thanks, GitHub, for the generous hosting.

Homebrew did their best to support the project, but the official stance was "if you buy a Mac, you buy the whole package", pointing everyone to install full Xcode if they had any problems. Far from ideal, but I was content.

Apple's Interest

Meanwhile, Apple reached out to me to discuss some details about OSX-GCC-Installer. They took an interest in shipping something official, but they weren't sure of the proper approach to take to support projects like Homebrew.

I hopped on the phone, explained the specific needs of Homebrew, the restrictions with the proprietary headers and licensing, and what the ideal situation would be.

Months passed and I didn't hear anything. Until today.

Command Line Tools for Xcode

Today, Apple added a beautiful new package to their official developer tools suite: Command Line Tools for Xcode. It's a 171 MB download that includes all of the tools a Homebrew should ever need. Best of all, it contains the proprietary headers that I couldn't ship myself.

You can download and try it out today. All you need is a free Apple ID.

You'll want to uninstall Xcode first:

$ sudo /Developer/Library/uninstall-devtools --mode=all

Major props to Apple for making this happen.

The Future

This is an incredible day for the Homebrew community. You can now setup a complete OS X develop environment with a single 171.7 MB package download. It's official. It's legal. It'll be maintained.

Homebrew is going to officially support the package too.

Max Howell:

To stave off further questions, yes, you can use the "Command Line Tools for Xcode" package with Homebrew. And we will support it.

Mike McQuaid:

We will recommend you don't use Xcode from the App Store but instead the command-line-tools package (which can also be installed from inside of Xcode and lives happily alongside it).

Open source is incredible.

How I Develop Things and Why

2012-01-27T10:00:00Z

I've always considered myself a bit of a software junkie. Nothing excites me more than a great piece of new software.

Some of my best childhood memories are our trips to Grandma's house, where I'd have access to a computer with a dial-up connection that I'd use to obtain freeware and shareware. I'd bring 4 or 5 floppies with me and try to cram all the games, waveform editors, and utilities that I could sneaker-net home.

Luckily today, excellent software written with passion oozes out of the app ecosystem. OS X and the App Store really fuel an economy of software built for humans by people that care.

Unfortunately, this doesn't always hold true in developer software — text editors, modules, libraries, toolchains, &c. We are forced to deal with APIs on a daily basis that were not built with the user in mind. Over-engineering surrounds us as developers. Things that should be simple are often needlessly complex for the sake of being complex and "proper".

Why should consumer apps and developer APIs be treated differently?

Have an Issue

The first step to developing something great is to have a real problem. You can't solve a problem properly if you don't experience it firsthand.

On the consumer app side of things, a great example of this is Microsoft OneNote. Have you used OneNote? It's incredible.

Essentially, OneNote is a hierarchical freeform note-taking software that assumes nothing: you can type, use handwriting, embed files, cross-link notes, sync them online, &c.

Unfortunately, OneNote is only available on Windows. This kills me. I would love to think that Microsoft's would port this lovely peice of software to OS X, but I doubt it will ever happen.

If I ever decide to actually ship a consumer product, it will be something akin to OneNote for OS X. It would be incredible. It may not be for many, but for people that resonate with my problem, it will work wonderfully. It would be a reaction to a real problem, not an engineered app an entrepreneur thinks will fill a gap so he can make some fast cash.

GitHub wasn't built for the developer community at large; the founders built GitHub for themselves. The problem they solved simply happened to resonate with millions of developers because they themselves happen to be developers.

37Signals didn't build Basecamp for a world full of project managers and consultants; they built it for themselves. They also developed Ruby on Rails for themselves, as Ruby developers that were repeating themselves too often.

How pragmatic.

These companies didn't need to commission lengthy case studies and perform market analysis. They didn't setup faux AdWords to measure the effectiveness of various marketing copy. Yet, they are astronomically successful. How is this possible? They know exactly what they want to build, how it should function, and how it should look because they were building it for themselves and not for others.

Let's go back to the developer's side of things.

A great example is my Requests module. I was a heavy user of Convore at the time, and I wanted to interface with it programmatically. So, I set out to build a Python module that wrapped the Convore HTTP API. Unfortunately, this was easier said than done. Dealing with Python's standard library for HTTP was a complete and total nightmare. It was over-engineered.

I love Python because it's a language designed for Humans. Why should modern HTTP be so difficult? So, I sat out to discover what it was that I wanted, and built exactly what I needed. It resonated well with others.

Nothing is more satisfying than using your own tools to Get Things Done.

Respond with a README

Before I start writing a single line of code, I write the README and fill it with usage examples. I pretend that the module I want to build is already written and available, and I write some code with it.

This has an incredible effect: instead of engineering something that will only get the job done, you start to interact with the problem itself and build an interface that reacts to it.

You discover it. You respond to it.

Great sculptures aren't manufactured — they're discovered. The sculptor studies and listens to the slab of marble. He identifies with the stone. Then, he responds. He enables the marble to speak for itself, setting free something beautiful that hidden was inside all along.

He responds.

This is what responsive design is all about. It's not merely a method to engineer a web design that will function on a phone, tablet, and desktop.

Beware lest you lose the substance by grasping at the shadow.

Responsive design is about making a design that identifies and understands itself enough to respond to the environment it's placed in. It is about setting your design free from arbitrary constraints. It is setting free something beautiful that was inside all along.

This is known as Readme-Driven Development. I call it Responsive API Design.

Build

Now that you know what your API is: Build it. Make it happen. If there's a significant amount of complexity behind a simple call, make a layered API: a porcelain interface that sits on top of a verbose API that sits on top of an low level integration interface.

The user API is all that matters. Everything else is secondary.

Once your software is released, improve it! Add new features, better security, optimal performance, and rigidity. But never compromise the API.

Manifesto

Build things that you want. Build things that you need. Build things for you.

The Golden Rule™:

Do unto others as you would have them do to you.

Adapted to:

Build tools for others that you want to be built for you.

Major Progress for Requests

2012-01-16T10:00:00Z

Requests has changes a lot over the past few months. An update is long overdue.

In case you aren't familiar, Requests is Python HTTP for Humans.

A Slow and Painful Death for Urllib2

The most notable change has been the complete removal of urllib2 as a dependency. Requests started out as a clean wrapper around urllib2, abstracting away the complex hacks required to accomplish basic tasks.

But, as the feature set grew (as well as the bug list), the project became impossible and frustrating to maintain. Supporting the simple use case of "send basic auth on a 404" took several hours.

Things shouldn't be this way. Not in Python.

It quickly became obvious that urllib2 was the problem.

The first step was decoupling all depending functionality on urllib2 handlers. Redirections, Basic and Digest Authentication, File Uploads, Cookies, &c. Easier said than done.

I started to work on the requirements for the new backend I planned to build. It would sit directly on top of httplib. I knew I wanted to add keep-alive in the near future, so it would need to support that, along with adding some file upload mechanisms.

That's when I found urllib3.

Keep-alive and Connection Pooling

Despite its name, urllib3 is actually a thin wrapper around httplib with a few niceties added: connection reuse, keep-alive, and file uploads. It was exactly what I needed.

So, I reached out to Andrey Petrov on Twitter. Much to my surprise, he loved the idea of embedding urllib3 within Requests. We worked together for a few weeks to make some required changes and watched the work slowly came to fruition.

By random chance, I had the opportunity to get to know Andrey at PyCodeConf, while all of this was in the making. We've since become good friends. Open source is awesome.

Thanks to the urllib3 collaboration, Requests has first class support for HTTP/1.1 Keep-alive and Connection Pooling:

s = requests.session()

# New connection to Httpbin.org.
s.get('http://httpbin.org/get')

# New connection to Github.com.
s.get('http://github.com/kennethreitz')

# Reuse old connection to Httpbin.org.
s.get('http://httpbin.com/ip')

Elegant. Thread-safe. Automatic. Awesome.

It took a few months to bring to a full release, but the move went resoundingly well. Porting Requests to Python 3 will now be significantly simpler, and the codebase is a joy to maintain again.

SSL Verification

The biggest milestone yet has also been met: automatic SSL Certificate validation with hostname verification. This is huge.

Secure HTTP? In Python? Say it isn't so!

Any request to an https resource that does not have a valid certificate will raise an exception, unless otherwise specified. Just like a web browser.

Requests ships with the exact Certificate Authority Bundle that Mozilla's Firefox ships with, and it is updated on a regular basis.

For example, this website doesn't have a valid certificate:

>>> requests.get('https://kennethreitz.com')
Traceback (most recent call last):
...
requests.exceptions.SSLError: hostname 'kennethreitz.com' doesn't match either of '*.herokuapp.com', 'herokuapp.com'

And Github does:

>>> requests.get('https://github.com')
<Response [200]>

Of course, you can disable this functionality explicitly or specify your own private CA Bundle in your codebase. Requests will also honor the REQUESTS_CA_BUNDLE and CURL_CA_BUNDLE environment variables, just like curl.

Usage

In terms of numbers, Requests has been doing very well recently.

72,000+ PyPi Installations
1,700+ GitHub Watchers

That's a lot of GitHub watchers. In fact, if you sort all of the Python projects on GitHub by watchers, the only projects ahead of Requests are Flask, Tornado, and Django.

This blows my mind. Of course, this is a silly vanity metric. Open source isn't a contest.

It's encouraging, nonetheless.

Joining Heroku!

2011-12-01T20:20:00Z

I am extremely happy to announce that I am joining the Heroku team!

If you aren't familiar, Heroku is a Polyglot Platform as a Service for hosting web applications. The product is executed so well that they are responsible for an entire micro-industry of PaaS platforms for different languages (Python included). Company after company strives to become the "Heroku for x".

Until recently, there were only a few good options for modern Python deployment. However, a few months ago, Heroku added official Python support to the Cedar stack — it is beautiful.

The Role

I will be responsible for the technical design of Heroku's Python offering, as well as Heroku’s voice in the Python community.

Other responsibilities will include:

Contributing to open source projects.
Driving best practices adoption.
Be passionate about the general success of Python.

Best. Job. Ever.

Arc90 + Readability?

Arc90 + Readability and myself have parted ways on excellent terms. Leaving the team was a very difficult decision to make. Readability is a hard-working team of incredible individuals, working passionately on one of my favorite online tools.

Readability is very much in a "move as fast as possible" stage of growth. Unfortunately, my passions for workflow automation, best-practices, and autonomy proved to mix harshly with the style of Readability's operation.

We're very proud of what we built over the past six months, and I look forward to seeing the platform grow.

Heroku's Values

I've always found myself attracted to beautiful and elegant architectures and APIs. Heroku is a natural perfect fit. They're inspiring as both a Product, a Platform, and a Company. Here are a few selections of team marketing:

On craft:

"Heroku is a do-ocracy. Makers of every stripe are welcomed here. Amazing ideas and execution are rewarded with independence and trust."

On productivity:

"From customer facing roles to our developers, we believe in building, not meeting."

Lastly, my personal favorite:

"We spend as long as it takes to make things right."

Needless to say, I'm tremendously excited about becoming an official Herokai™.

Static Sites on Heroku Cedar

2011-11-26T05:20:00Z

Heroku's excellent Cedar Stack has first-class support for Python, Ruby, Node.js, Java, Clojure, and Scala applications. Unfortunately, there's no obvious way to serve static sites without first fronting them with a Rack or WSGI application.

The Cedar stack has unofficial support for Custom Build Packs, which allow you to compile your own language runtime on top of Cedar. My plan was to build an custom nginx build pack for Cedar, but that turned out to be completely unnecessary.

Did you you know that Cedar has full (unofficial) support for PHP applications, fronted with Apache? When you push up a repository with an index.php file at the root, Apache and PHP are bundled into your application at runtime.

Elegant Static Sites on Cedar

First, lets turn your site into a PHP "application":

$ touch index.php

Next, we can fully disable Apache's PHP engine:

$ echo 'php_flag engine off' > .htaccess

When you push this up, you'll have a bare Apache instance serving up the contents of your site to the world. Best yet, you can do all of the stupid .htaccess tricks that you could on on any traditional shared hosting platform.

It works great! This very website is served this way.

Requests v0.6.0 Released!

2011-08-17T05:20:00Z

Requests v0.6.0 was just released! This is big.

If you're unfamiliar with Requests, the Python HTTP Module for Humans, it strives to keep things as simple and elegant as possible. Learn more.

Sessions

I'm extremely happy to announce that Requests now has support for sessions!

A session is created with a simple context manager. The resulting object has a method for each type of request, just like Requests itself.

Any optional parameters can be passed to requests.session, and will be included in each Request method call. Cookies are also persisted throughout all requests within a session.

with requests.session(auth=('username', 'password')) as s:

    r = s.get('https://api.github.com')

    r.status_code
    # 200

Learn more about sessions.

Status Code Reference

When reading HTTP code, remembering the exact meaning for every status_code can be difficult. Especially if you're working with a robust API. So, I've included a flexible lookup object to reference a status_code within your code.

>>> requests.codes.ok
200

>>> requests.codes.im_a_teapot
418

>>> requests.codes['precondition_failed']
412

Dict Cookies

Requests has had CookieJar support for a while, but that's far from ideal. CookieJars require an enormous amount of manual code. So, now you can give Requests a simple dictionary wherever it expects a CookieJar, and it'll automatically generate a CookieJar for you!

Best yet, if a server responds with Cookies in any request, you'll have a simple dictionary waiting for you in Response.cookies. If you want the full CookieJar, you can access it at Response.request.cookiejar.

Hook System

I've added a new hooks system for both signaling and modifying the method arguments, requests, and responses during the request lifecycle.

This will be extremely useful for using Requests to wrap web service APIs. Adding custom authentication mechanisms (OAuth anyone?) should be as simple as a few callbacks within a session.

Learn more about Hooks.

&c

I saw a large number of users relying on the order of the request function parmaters, instead of naming the arguments explicitly. So, now everything but url (and sometimes data) is passed via **kwargs, so you have to do things properly. This also gives me some flexiblity in making API decisions in the future.

Community Update

This weekend, Requests had its biggest explosion ever. The documentation has been accessed 43,000+ times by 25,000+ unique visitors in the past 30 days. The repo is approaching 700 watchers on GitHub (trailing just behind Celery), and the latest release has been downloaded over 2000 times from PyPi in the last month alone.

Another big milestone: we now have 25 contributors! Many thanks to everyone involved :)

Changelog

Here's a full list of what's new on v0.6.0:

New callback hook system
New persistent sessions object and context manager
Transparent Dict-cookie handling
Status code reference object
Removed Response.cached
Added Response.request
All args are kwargs
Relative redirect support
HTTPError handling improvements
Improved https testing
Bugfixes

Source Code

Requests is open source (ISC Licensed), and will make your world a better place. I promise :)

I Use This

2011-07-30T15:10:00Z

A while back, I published a post about my setup, inspired by the awesome articles on the UsesThis site.

As times change, so does my toolset. So, here's a list of my latest favorite things. I love them, and think you should use them too.

ep.io

Perfectly Pythonic WSGI Hosting.

There are a number of new Heroku-like Python hosts out there, but most of them leave a bad taste in my mouth. This is mostly because they only support Django, and not generic WSGI (e.g. Flask apps).

Ep.io supports any WSGI application. It charges based on usage and offers a generous amount of free usage per application. Redis support. Cron jobs. Optional git/mercurial integration. The architecture and configurations are very elegant. I love it.

I host a most of my open source applications, including this very website, on ep.io. I couldn't be happier.

[ ep.io ]

gaug.es

Google Analytics is an extremely power analytics platform that can be used to drive important business decisions and analyze deep trends.

I don't need any of that.

I need realtime traffic information, visitor browsers and devices, and a simple list of referral URLs.

Check out Gauges. Realtime, simple, and elegant analytics. No domain limits. It's provided by a group of great guys that are passionate about making the service as awesome as possible. It really shows.

[ gaug.es ]

DNSimple

Stop using GoDaddy. Now. They have extremely misleading billing practices, terrible ads, and — worst of all — one of the worst DNS management interfaces on earth.

DNSimple has the simplest and most elegant DNS management system I've ever used. Everything just works — instantly. One-click record templates for GitHub Pages, Google Apps, etc.

There's a small subscription fee, tiered by the number of hosted domains. If you find yourself thinking, "Why would I pay for DNS hosting?": You get what you pay for.

It doesn't matter where you register your domains, but I register all of mine through DNSimple. The registration cost is a little high compared to a few providers, but — again — you really get what you pay for.

I got a signed postcard from them in the mail last year, wishing me a Happy New Year. These guys care.

[ DNSimple ]

Sublime Text 2

It's time to face the facts: TextMate is dead. The author has been lying to his loyal community for a long time now. It's time for us to move on.

I looked for a worthy replacement for a long time. This wasn't an easy task. I believe that I have tried literally every editor ever released for every major operating system within the past 10 years. I'm really picky about my development environment; It's part of who I am.

Sublime Text 2 is incredible.

It offers TextMate theme and snippets support, extremely optimized workflows, and a full internal Python API for writing plugins. Coupled with SublimeLint and Soda Theme Dark, it's perfect.

It's available for OS X, Linux, and Windows too.

[ Sublime Text 2 ]

Readability

Full disclosure: I work for Readability. However, that's completely irrelevant. Readability is easily one of my favorite tools ever.

If I land on an article that's either ugly or too long to read at the moment, I hit ` in Chrome. I'm presented with a highly polished, content-only version of the article with elegant typography.

Top notch browser plugins. First-class Typography. Offline HTML5 apps for every device I own. A daily digest of my articles, sent to my Kindle automatically? Yes, please.

[ Readability ]

Canon 35mm f/1.4L

One of the few things that gets me away from my machine is photography. Check out my Flickr stream. It's very much a hobby, but I quite enjoy it.

My lens kit has gone through quite a few iterations. It took me a while, but I'm finally got rid of my last zoom lens, and have moved on to all primes. Zooms seem to suck creativity out of me. Constraints are good.

My new favorite lens is the Canon EF 35mm f/1.4L. I'm blown away by this thing. It's exactly what I've been looking for. I can't believe it hasn't been updated since 1992.

I've noticed a bit of a fanastic subculture behind this lens since I got it. If you ever take a look at the White House's Flickr Stream, you'll see the excellent work of Pete Souza. He uses the 35L in almost every indoor photo. Incredible.

Before, I typically shot with my moderately telephoto 85mm f/1.2L (my photon vaccuum). I still use it on certain occasions, but the 35L is really bringing my photography to a new level.

The 35L & 85L are an unstoppable duo.

WebFaction

I've never been a fan of shared hosting at all. These guys proved me wrong.

They really know their stuff. Python is a true first-class citizen on their boxes: 2.4, 2.5, 2.6, 2.7, 3.1, 3.2 are all available, with most of the popular supporting libraries (PIL, etc), out of the box. One click installers for WordPress and more with 100% perfect permissions set. Nginx-fronted Apache, giving you the best of both worlds. A standard account gets you user-level SSH access to the box. Awesome.

If I need to host a PHP or static app, they're my first stop.

[ WebFaction ]

Read the Docs

A product of last year's DjangoDash, Read the Docs is the best Sphinx documentation site around. It's open source too.

python-requests.org, python-guide.org, and tablib.org are all hosted on RTD. You should use it too.

[ Read the Docs ]

Bonus Things

Requests v0.5.1 Released

2011-07-24T02:00:00Z

Requests v0.5.1 was just released.

If you're unfamiliar with Requests, the HTTP Module for Humans, it strives to keep things as simple and elegant as possible. Learn more.

Community Update

I'm really happy with the direction that Requests is going. The repo is approaching 400 watchers on GitHub (trailing just behind the Pyramid framework), and has been downloaded over 1100 times from PyPi in the last month alone.

Another big milestone: we now have 20 contributors! Many thanks to everyone involved :)

If you head over to python-requests.org, you'll be greeted by the new Requests mascot, Rez the sea turtle.

This October, I'll be presenting at PyCodeConf in Miami, FL. Topics will include Requests' API design and how libraries like it help to make Python more accessible.

API Design and Pragmatic Python

Unfortunately, solving simple problems with Python isn't always import antigravity. This talk will analyze the high barriers of entry that clutter the Python landscape. We'll discuss ways to make Python more accessible for newcomers and less of a headache for seasoned veterans.

New Features

The v0.5.x series has brought along some awesome features so far:

PATCH Support!
International Domain Name Support!
Forced Basic Authentication (for 404'd responses; default)
Querystrings available for all methods
python-requests.org default User-Agent header
Sexy HTTPBin Test Suite
settings.verbose configurable stream for debugging and logging.
Access response headers without fetching entire body (read())
Use lists as dicts for parameters w/ multiple values
Dict-configurable Proxies
Accept-Encoding: gzip default on.

Moving Forward

In the short-term, I plan to remove Request's dependency on Poster, and replace the multi-part file upload functionality with internal code.

Once Poster is removed, Python 3.1+ support will be added.

Once the port to Python3 is complete, elegant HTTPS certificate verification will be added.

While those development tasks are being carried out, I'm planning on adding some more exhaustive documentation to the website: tutorials, FAQ, use cases recipes, scenarios, &c.

Source Code

Requests is open source (ISC Licensed), and will make your world a better place.

GitHub Reflog v1.7.19

2011-07-23T19:03:00Z

Original Post on GitHub.

Welcome to the The GitHub Reflog — the chronicle of remarkable GitHub repos and community activity. For previous editions, check out The Reflog Archive.

This week, we're extending a warm welcome to The League of Moveable Type's excellent open source fonts to GitHub!

Featured Repo of the Week

ryanmcgrath/wii-js

Nintendo's ever-popular Wii console has a web browser, but little development is done with it. This JavaScript project offers a stable, robust, and documented API for wii/html interaction.

While not entirely practical for actual application development, this project aims to be a great avenue for teaching kids about programming.

Wii-js supports up to four wiimotes interacting with a page at a time. Check out the demo page (on your wii) to see it in action.

Awesome Repo of the Week

danlucraft/git.js

This is nothing short of awesome. A git implementation written in pure JavaScript. Similar to the Dulwich project, for Python.

Git.js is MIT licensed. It features both a command-line git client for node.js and an in-browser repo API for accessing repos available via HTTP.

Remarkable Repos

j2labs/brubeck: A flexible web framework for Python, powered by Mongrel2, Eventlet, ZeroMQ, and DictShield. To get a feel for the framework, check out the example project.
arthuredelstein/clooj: a lightweight IDE for Clojure, written in Clojure. Powered by Swing, Clooj is cross platform and can be used as both a standalone application or embedded editor.
jceb/vim-orgmode: This awesome plugin brings emacs' org-mode note-taking system to the vim community. Access your notes on your iPhone with mobileorg over WebDAV or Dropbox.
Khan/khan-mobile: The source code of the Khan Academy's new mobile app.

Promising Repos

urbancoding/jenx: A Jenkins build server monitor for Mac OS X, powered by MacRuby. This app sits in your status bar and reports the status of all your Jenkins builds. Click on a job, and it opens it up in your browser. It even supports Growl!
jesusabdullah/pyee: A port of node.js' popular EventEmitter system to Python. Simple and strait forward.
memo/iSteveJobs: This useless (but fun) project replaces all the faces detected in a webcam feed with Steve Jobs. It is powered by the openFrameworks C++ Framework.
kennethkufluk/Twitter-Matrix: This HTML5 project takes a Twitter list's public feed, and displays it. Matrix-style. Check out the demo page to see it in action.

Feedback is appreciated! Send any questions, suggestions, and anonymous tips to reflog@kennethreitz.com.

For more open source news, check out The Changelog and github/explore.

Legit: The Sexy Git CLI

2011-06-27T02:00:00Z

GitHub for Mac is not just a Git client. This comment on Hacker News says it best:

They haven't re-created the git CLI tool in a GUI, they've created something different. They've created a tool that makes Git more accessible. Little things like auto-stashing when you switch branches will confuse git veterans, but it will make Git much easier to grok for newcomers because of the assumptions it makes about your Git workflow.

Why not bring this innovation back to the command line Git interface?

Enter Legit

Legit is a new tool that allows you to interface with a Git respository much like you would with GitHub for Mac—from the command line.

Switching branches becomes seamless. Any pending changes are automatically stashed and restored. Same goes with syncing your repository. And merging (grafting). And branching (sprouting).

Managing remote branches becomes as simple as publish branch and unpublish branch. Simple.

Installation

Installing Legit is easy with pip (Python 2.6 or 2.7 required):

$ pip install legit

This makes the legit command available. Run it within a repository.

Available commands:

sync [<branch>]: Syncronizes the given branch. Defaults to current branch. Stash, Fetch, Auto-Merge/Rebase, Push, and Unstash.
branches: Get a nice pretty list of available branches.
switch <branch>: Switches to specified branch. Automatically stashes and unstashes any changes.
sprout [<branch>] <new-branch>: Creates a new branch off of the specified branch. Defaults to current branch. Swiches to it immediately.
graft <branch> <into-branch>: Merges specified branch into the second branch, and removes it. You can only graft unpublished branches.
publish <branch>: Publishes specified branch to the remote.
unpublish <branch>: Removes specified branch from the remote.

Moving Forward

Legit is very much in beta. I'm using it for real work already, so don't hesitate to try it out. As feedback and contributions come in, I expect the feature-set to evolve. We'll see. That's up to you.

Static binaries will be available soon for all major platforms.

Source Code

Legit is open source (BSD Licensed), powered by Clint, GitPython, and good intentions.

Code on GitHub

Announcing HttpBin.org

2011-06-13T01:00:00Z

The development of Requests, the Python HTTP Module for Humans, led to some annoying testing practices. Relying on random websites and services in order to test different capabilities of the HTTP client became annoying quickly.

PostBin.org was perfect for testing POST request behavior, but is usless for other situations. I was hoping to extend its functionality to other request types, but it turns out that PostBin runs on the Google App Engine platform. No.

Thus, httpbin.org was born.

Example Endpoints

To get a feel for what HttpBin does, here are a few endpoint examples:

$ curl http://httpbin.org/ip

{"origin": "::ffff:24.127.96.129"}

$ curl http://httpbin.org/user-agent

{"user-agent": "curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3"}

$ curl http://httpbin.org/get

{
   "args": {},
   "headers": {
      "Accept": "*/*",
      "Connection": "close",
      "Content-Length": "",
      "Content-Type": "",
      "Host": "httpbin.org",
      "User-Agent": "curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3",
      "X-Forwarded-For": "::ffff:24.127.96.129",
      "X-Forwarded-Protocol": ""
   },
   "origin": "::ffff:24.127.96.129",
   "url": "http://httpbin.org/get"
}

$ curl -I http://httpbin.org/status/418

HTTP/1.1 418 I'M A TEAPOT
Server: nginx/0.7.67
Date: Mon, 13 Jun 2011 04:25:38 GMT
Connection: close
x-more-info: http://tools.ietf.org/html/rfc2324
Content-Length: 135

Moving Forward

HttpBin will be packaged and released on PyPi soon, for local development use and requests-tests runs on ci.kennethreitz.com. I need to determine a portable pattern for this.

In the coming weeks, I'd like to add a few new new endpoints: /deflate, /basic-auth, &c. Contributions are welcome.

I'm considering adding optional request logging / history to the service, powered by Redis. A new /post request, for example, would be redirected to a new URL (e.g. /post/c1548ed) that can be POSTed to repetitively. This will give HttpBin feature parity with postbin.

Source Code

HttpBin is open source (ISC Licensed), powered by Flask, Werkzeug, and good intentions.

Joining Arc90 + Readability

2011-05-27T22:35:00Z

I'm extremely happy to announce that I am joining the Arc90 team to work on Readabilty!

Arc90 is a NYC-based software consultancy firm, r&d lab, and product incubator. The company is a design-driven group, built around skills rather than hierarchy. They strive to create the best software possible; passion and design pours out of everything they ship. I love that.

As a self-professed software aficionado, I couldn't have picked a more inspiring group of people to work with. I immediately accepted their offer without any hesitation.

Unfortunately, my previous employer and I have parted ways for various reasons, stemming from the standard major-corp intellectual property agreement (IPA). I couldn't agree to the new terms because of the restrictions it would impose on my passion — open source.

For those interested, I highly recommend Intellectual Property and Open Source by Van Lindberg. If you do any open source software development of any kind, you absolutely must read this book.

Readability

Readability is easily one of the highest quality web applications around. I use it every day.

Simply put, it is a beautiful application that allows you to curate a personal reading list of articles you find online.

Let's say, you encounter a thought-provoking article hosted on the typical ad-ridden website with terrible typography. Activate the Readability browser addon (or bookmarklet) and the page is transformed into a typographically elegant version of the page, containing only the article content. You can then happily read the content of the article or save for later enjoyment.

Users pay a self-set monthly subscription fee ($5+). 70% of your subscription is distributed amongst the creators of the content you have saved. The remaining 30% goes back to Readability. Users receive an incredible reading experience, and the content creators get generously reimbursed for their hard work.

Beautiful.

Marco Arment, creator of Instapaper, loves what Readability is accomplishing:

Trust me, these guys really know their stuff, and their heads are in the right place: there are no sinister motives or shady practices. It works exactly the way you’d expect, and is one of the most positive, constructive efforts I’ve seen in the online publishing world in a long time.

I couldn't be more excited.

GitHub Reflog v1.5.16

2011-05-16T22:35:00Z

Original Post on GitHub.

Welcome to the The GitHub Reflog — the weekly chronicle of remarkable GitHub repos and community activity. For previous editions, check out The Reflog Archive.

Thanks to the awesome work of Alexander Marshalov, the Reflog archive is now being translated into Russian!

Featured Repo of the Week

yostudios/Spritemapper

Spritemapper is a command-line application that generates CSS sprites from your existing CSS files. It utilizes advanced techniques, like simulated annealing, to merge the referenced PNG images into one and generate a new CSS file utilizing the new sprite map.

It also contains its own PNG and CSS parser. All you need is a Python interpreter. If you've ever had to debug a PIL installation, you know how important this is.

Awesome Repo of the Week

jstrait/beats

This project is nothing short of amazing.

BEATS is a command-line audio sequencer, written in Ruby.

Songs are programmed in YAML. The input files define instruments (samples), bar sequences, and the full song structure. The whole song can then be exported to LPCM WAV for later enjoyment / mixing.

Remarkable Repos

tweetdeck/TDOAuth: A small and elegant OAuth v1.0 implementation for iOS applications. Written by Max Howell, of homebrew fame, this BSD-licensed implementation strives to be a major improvement over the other OAuth solutions available to Objective-C developers.
sporst/SWFREtools: A toolset for everse engineering and performing security audits on Adobe Flash Player and compiled SWF files. It features an SFW Parser, SWF Minimizer, Flash Debugger, and Metrics Generator. This GPLv2 licensed project is powered by both Java and Python.
leftnode/get-shit-done: A small script for toggling the blocking of a list of configured domains on your system, so you can Get Shit Done™.
microsoft-dpe/watoolkitios-lib: This set of libraries, written by Microsoft, allows Objective-C developers to utilize the Azure Cloud Storage platform from an iOS application. Check out the example application repo to see it in action.
enonic/cms-ce: New community edition of the Enonic CMS, bsaed on Java's Spring Framework and JEE. Unlike the commercial version, the AGPLv3 licensed Community Edition lacks a plugin engine and LDAP/AD support.
wbsun/kgpu: This extremely ambitious project allows the Linux kernel to offload data-parallel computations to a CUDA-enabled GPU. It aims to "make the Linux kernel truly parallelized: not only processing multiple requests concurrently, but can also partition a single large requested computation into tiles and do them on GPU cores."
jiminoc/goose: Instapaper-like HTML content and article extractor, from Gravity Labs. Written in Java, this Apache v2 licensed project will take any url you throw at it, and return the content.

Promising Repos

bobthecow/coda-cli: Command-line tool for launching and Panic's Coda editor (similar to TextMate's mate command), making it a full *nix citizen. Check out the project page to see usage examples.
skammer/vim-css-color: This epic plugin bring color to your Vim installation's CSS colors. It currently supports RGB, RGBA, and HTML Color Names. (Note: 256-color terminal required).
bradfitz/campher: You can now run Perl on top of the Go stack. Seriously. This esoteric project has quite a bit of ambition and an awesome sense of humor.
wavded/humane-js: An unobtrusive browser notification system, aimed to be as simple as possible. This fully themeable system is compatibly with all JS frameworks and browsers. Check out the demo page to see it in action.
brixen/poetics: A native CoffeeScript interpreter, running directly on the Rubinius VM. Need I say more?

Feedback is appreciated! Send any questions, suggestions, and anonymous tips to reflog@kennethreitz.com.

For more open source news, check out The Changelog and github.com/explore.

Requests v0.4.0 Released

2011-05-15T22:35:00Z

Requests v0.4.0 was released late last night!

If you're unfamiliar with Requests, it is an HTTP Module for Python that strives to keep things as simple as possible. Learn more.

It is being used internally at Twitter, Inc and The Library of Congress. If your company (or project) is using Requests internally, send me an email!

Since the initial release, 3 months ago, Requests quickly became my most popular Python project. It's approaching 200 Watchers on GitHub, and has a number of happy repetitive users:

Daniel Greenfeld: Nuked a 1200 LOC spaghetti code library with 10 lines of code thanks to @kennethreitz’s request library. Today has been AWESOME!
Kenny Meyers: Python HTTP: When in doubt, or when not in doubt, use Requests. Beautiful, simple, Pythonic.
Rich Leland: Requests is awesome. That is all.
Steve Pike: I can never remember how to do it the regular way. import requests; requests.get() is just so easy!

Community Update

Requests now has it's own documentation website: python-requests.org, hosted by the excellent Read the Docs project. The User and API docs are still a work in progress.

I decided to change the project tagline from "Most Python HTTP Modules suck. This one doesn't." to "Python HTTP Requests for Humans.". While I did enjoy the bit of personality that the previous tagline brought to the project, there's no need for the passive agression. Only Positivity going forward.

New Features

Version 0.4.0 features the new Response.history attribute, which contains a list of requests leading up to the response. If you ask for a URL, and it bounces off of 3x 301 Redirects, those 3 requests will reside in the Response.history atrribute.

Requests now features a wonderful Case-insensitive Header Dictionary, so you can handle unpredictable response headers easily. One server will respond with content-encoding, while another will respond with Content-Encoding. r.header['content-encoding'] will return the proper value in both scenarios.

Requests now transparently supports Unicode URLs (e.g. 'http://ko.wikipedia.org/wiki/위키백과:대문').

We also automatically decompresses GZip'd Encoded responses, without the need to send up an Accept-encoding header.

Urllib2 has a major recursion issue when HTTP BASIC Authentication is required, but bad credentials were provided. Requests now handles this scenario properly.

New Blog: Finally Static!

2011-04-21T02:45:00Z

I've been wanting to convert my website from a WordPress-powered application to a static one for a long time now. I tried a few different systems, but nothing was flexible enough for my needs.

I almost sat down and wrote something myself, but luckly I found Alex Metaireau's incredible Pelican project. So, now my website is statically generated with a combination of all my favorite tools: ReStructured Text, Jinja2, Python, Git, and Jenkins.

The move is far from perfect, but I'm glad I finally did it. All existing comments are gone. There's no longer a mobile version. Many of the old blog posts are malformed (and a big chunk should just be removed entirely).

But, these things will happen in time. It feels good to start fresh.

GitHub Reflog v1.4.20

2011-04-20T19:03:00Z

Welcome to the The GitHub Reflog — the weekly chronicle of remarkable GitHub repos and community activity. For previous editions, check out The Reflog Archive.

This week, we welcome Python's simplejson module and The Joy of Clojure code examples to GitHub!

Featured Repo of the Week

maccman/spine

Spine is a micro MVC framework for building JavaScript applications. While similar in concept to Backbone.js, Spine contains some key differences. It offers class inheritance, controllers, a model system with a full ORM, and HTML5 local storage support.

Spine comes with demo todo system and contact manager applications to give you a feel of its capabilities. It currently powers the JavaScript for the open source campfire-esque Holla project.

Check out the elegant documentation to learn more.

Awesome Repo of the Week

aarongough/flea

Flea is a mini Lisp interpreter written in Ruby, inspired by portions of Scheme. Obviously, this isn't designed to be used in production for any reason. The purpose of the project is to simply showcase the versatility of dynamic languages like Ruby.

Run $ gem install flea to give it a spin.

Remarkable Repos

anchepiece/statuspanic: A clone of Panic's infamous Status Board, powered by PHP. It features bar charts, weather widgets, and a countdown timer. Head over to the example installation to see it in action.
hanwen/go-fuse: Native FUSE bindings for the Go programming language. Go-fuse strives to be as fast as possible. Support for individual filesystems isn't included (at this time), but example implementations of zipfs and unionfs are.
gmarik/vundle: A true plugin manager for VIM. Inspired by the ever popular pathogen for VIM and bundler for Ruby, Vundle gives you the ability to search, install, update, and configure your scripts, all from interactive mode.

Promising Repos

boxedice/python-daemon: A simple Python daemonizer class. This project allows your Python application to continue running in the background as a daemon. Features include scheduling, a standard PID file interface, and support for Unix, Linux and OS X.
88mph/wpadmin: Command line tool for WordPress administration, written in PHP. Currently, it can manage user accounts and configuration values. In the near future, you'll also be able to publish posts and pages right from your terminal. There's a small todo list, so pull requests are welcome!
disqus/mule: A full distributed test runner framework for Python, powered by Celery. Beware, it's undergoing heavy development and is far from stable. When it's ready, it will include out of the box support for xunit, multiprocessing, and unittest2.
seppo0010/redislite: SQLite-style local database for Redis instead of SQL. This BSD licensed project is is still in its early stages, but will be extremely useful once it is complete. Definitely keep your eye on it.
dbyrne/4clojure: An interactive tutorial website for beginners learning Clojure (similar to Try Redis). There's a bit of work to be done still, so check out the issues page for ways to help.

Feedback is appreciated! Send any questions, suggestions, and anonymous tips to reflog@kennethreitz.com.

For more open source news, check out The Changelog and github.com/explore.

GitHub Reflog v1.4.12

2011-04-12T19:03:00Z

Welcome to the fourth edition of The GitHub Reflog — the weekly chronicle of remarkable GitHub repos and community activity. For previous editions, check out The Reflog Archive.

Featured Repo of the Week

apenwarr/sshuttle

This nifty project got quite a bit of press this week, and has already become indispensable part of my toolkit. Sshuttle calls itself a "Poor Man's VPN". It allows you to connect to a remote machine as a non-privileged user and forward every port on its network to your local machine. Once enabled, all traffic is being routed through the server. If the server has network access to machines that your machine cannot, you can now access those boxes as if they were on your local network.

Port forwarding with OpenSSH is hardly new, but sshuttle forwards all ports instead of only ones you specify. The only dependency on the server is OpenSSH itself.

Liftoff!

Awesome Repo of the Week

asarazan/Narwhalingus-EP

@asarazan found a great new use for GitHub this week. His band, The Bristol 7's, decided to publish their EP under the Creative Commons license. In the spirit of open source, he decided to put it up on GitHub!

The repo includes final mixes as well as the individual audio tracks for remixing. Pull requests welcome!

Remarkable Repos

jonromero/music-as-data: This Clojure project offers a live programming environment for generating music sequences. It allows you to treat the sequences as data and apply transformations to them in live environments. Check out the project page to learn more.
37signals/pow: This new project from 37 Signals is a zero-config development server for Rack applications. Once you run its super simple installer, go to ~/.pow and make a symlink to your app's directory. And, that's it! The app is now available locally at http://appname.dev.
DanielWaterworth/raphters: This new framework is for developing web applications in C. Yes, you read correctly. If you're a C aficionado that has always dreamed of building a modern webapp in your favorite language, Raphters is a great place to start. It provides a simple API for handling cookies, sessions, and templates. The resulting applications can be deployed with FastCGI.

Promising Repos

cldwalker/tux: Tux is a Ruby shell interface for Sinatra. It gives you a simple interface to interact with all parts of your application from the commandline. It can directly call your helpers and views, or generate your own request/response data.
ded/Ender.js: This new JavaScript module is actually 8 minimal utilities smashed into one: klass, qwery, bonzo, bean, script.js, reqwest, emile, and underscore.js. Running in at only 8KB, Ender.js combines the of all of these powerful modules into one elegant API. Features include: a class system, custom selector engine, DOM manipulation, events system, asynchronous dependency loading, simple animations, and a robust extension API. The codebase is powered by git-submodules and can be rebuilt with Node.js.

Feedback is appreciated! Send any questions, suggestions, and anonymous tips to reflog@kennethreitz.com.

For more open source news, check out The Changelog and github.com/explore.

GitHub Reflog v1.4.04

2011-04-04T19:03:00Z

Welcome to the third edition of The GitHub Reflog — the weekly chronicle of remarkable GitHub repos and community activity. For previous editions, check out The Reflog Archive.

Community Update

Last week, Apple's MacRuby project began to officially migrate to GitHub for its development. This week, the Eclipse Foundation and the Glasgow Haskell Compiler both created official GitHub mirrors. GHC even accepts pull requests!

Welcome, guys!

Featured Repo of the Week

divegeek/uscode

This small project from Shawn Willden is simple, but ambitious. It is a repository that contains a complete, up-to-date version of the United States Code, in the most hacker-friendly format possible: version-controlled plaintext. The repo is updated regularly with official modifications from the US Federal Government websites. Curious to see what modifications were made? Run git diff.

The purpose of this repo is explore new ideas related to the legislation process in the digital age. Check out the readme to learn more.

Awesome Repo of the Week

scoates/owty

Sean Coates found a novel use for GitHub this week: comparing beer recipes.

He's using the new image compare views to showcase the difference a little yeast makes in his homebrewed cream ale.

|MERGE|

Cheers! :sparkles: :beer: :sparkles:

Remarkable Repos

dotcloud/cloudlets: This project is awesome. It's a tool for creating small, universal server images under version control. The resulting images can be exported to "any bootable format known to man" for deployment in virtually any environment. Formats include: Xen, KVM, EC2 AMI, Tarballs, and ol' fashioned bootable CDs. Soon, cloudlets will offer support for full VM generation (vmkdk, xen, and qemu) and multi-image stacks. If you're looking to create turn-key server images for your application or platform, this is a great place to start.
josevalim/elixir: This is a functional language runtime built on top of Erlang. It provides object orientation, a Ruby-esque syntax, and can interoperate seamlessly with standard Erlang code and datatypes. The project is still under development, but already has a large contributor and user base. If you'd like to assist in its development, checkout the project roadmap and send a pull request.
ilektrojohn/creepy: Ever wonder what personal data is floating around the web for the world to see? Checkout creepy. Point it to all of your social network accounts, and it displays a map filled with your past locations. Data is fetched from a number of social APIs and photo services. It even goes as far as to parse Geo-location tags from uploaded image EXIF data. Creepy!
jordansissel/fpm: This is an interesting project. FPM, "Effing Package Management", takes out all the headaches of creating .deb and .rpm packages for your own consumption. It allows you to distribute your application in any style and configuration you want, abandoning all distro-specific best practices. The author designed it to be as simple as possible: "Here's my install dir and here are some dependencies; please make a package". This may or may not be a good idea. Give it a try and decide for yourself :)

Promising Repos

javisantana/whitebrd.me: This brand new project looks very promising. It's a webapp that provides realtime collaborative sketching. It's essentially Etherpad but for simple drawings. To try it out, open up The Official GitHub Reflog Whiteboard™ and start sketching!
atomicobject/objection: This is an MIT licensed dependency injection framework for iOS and OSX ObjectiveC applications. It strives to be as lightweight, yet flexible, as possible. The project is still fairly young and has a small todo list, so forks are encouraged.
mikeal/spider: This node.js project is a simple programmatic web spider, powered by jQuery. It has a very small code base (~200 SLOC), yet offers custom user agent strings, connection pools, and memory cache.

Feedback is appreciated! Send any questions, suggestions, and anonymous tips to reflog@kennethreitz.com.

For more open source news, check out The Changelog and github.com/explore.

GitHub Reflog v1.3.28

2011-03-28T19:03:00Z

Welcome to the second edition of The GitHub Reflog. This is a new weekly chronicle of remarkable GitHub repos. For previous articles, check out The Reflog Archive.

Featured Repo of the Week

koggdal/ocanvas

oCanvas is a middle layer for generating Canvas regions. Instead of dealing directly with pixels, you manage a set of display objects. oCanvas provides an elegant API and full event system for manipulating these objects, performing all the hard work in the background. From there you can create, manipulate, and animate to your heart's content.

Check out the polished demo page to see it in action.

Remarkable Repos

evanphx/gx: Gx is a Git tool that assists in merge conflict resolution. If you're using Git properly these are hard to come by, but every once in a while, a merge conflict is unavoidable. Git's merge resolution interface is a little difficult to deal with sometimes, so Gx gives you a simple interface for handling those conflicts.
BigZaphod/Chameleon: This project's massive undertaking is to replicate Apple's UIKit, used extensively for writing iOS apps, for OSX. It's an cleanroom implementation that mirrors the API exactly. The project is currently 9 months in development has about 60% UIKit coverage. Chameleon's best feature is that it allows you to build apps that target both iOS and OSX with the same codebase. If you're developing OSX applications, this is definitely with taking a look at.
jugyo/earthquake: This is an awesome new Twitter client for your terminal. Written in Ruby, Earthquake features proper OAuth authentication, streaming support, terminal colors, and tab completion. It appears to have been implemented specifically for the recent earthquakes in Japan. It's always great to see the open source community continuously stepping up to assist in times of need.
twigkit/tempo: Tempo is zero-dependency JavaScript template engine built on top of old-fashioned HTML. It's designed to make rendering of JSON content as simple as possible. It currently features nested data support, boolean conditionals, and gracefully falls back if JavaScript isn't available.

Promising Repos

shinh/maloader: This project looks very promising. It's a x86-64 Mach-O executable loader for Linux. It can run most of the Xcode toolchain from Linux. It can currently run Xcode's gcc-4.2, otool, dyldinfo, dwarfdump, cpp-4.2, dsymutil, and others. Maloader plans to support commandline applications only at this time.
antimatter15/player: This unique MP3 player is an excellent showcase of HTML5's capabilities. Point it to a folder on your system, and it recursively finds MP3 files, parses the ID3 tag metadata, and plays them from the browser. Check out the demo to give it a try!

Feedback is appreciated! Send any questions, suggestions, and anonymous tips to reflog@kennethreitz.com.

For more open source news, check out The Changelog and github.com/explore.

GitHub Reflog v1.3.21

2011-03-21T19:03:00Z

Welcome to the first edition of The GitHub Reflog. This is a new weekly installment chronicling awesome GitHub repos, brought to you by Kenneth Reitz, one of the classy gentlemen behind The Changelog and a ton of open source projects.

Featured Repo of the Week

devstructure/blueprint

This is one of the coolest projects I've seen in a long time.

Simply put, Blueprint reverse engineers servers. It scans your system, records installed packages from various managers (aptitude, ruby's gem, python's pip, etc), archives software built from source, and records configuration file changes. It then packages your entire server configuration into a bootstrap.sh file with an attached archive. You can take that bootstrap package and run it anywhere to replicate your server setup.

Then, Blueprint takes it to the next level: it generates full recipes/manifests for both Chef and Puppet with the blueprint-create {-C | -P} command.

Of course, QA is required for the resulting recipes, but it's a definitely a great first step for streamlining your existing systems.

Bon appétit!

Awesome Repo of the Week

drbrain/meme

I CAN HAS MEME? Meme is an awesome Ruby gem for generating memegenerator.net images from the command line. It features SPARTA, PHILOSORAPTOR, KEANU, TROLLFACE, and many more. The resulting pull requests are definitely worth looking at.

$ meme US_POINT 'I want you' 'to merge this'

Remarkable Repos

taitems/Aristo-jQuery-UI-Theme:

This is not a new project, but it received a lot of attention this week. Aristo is a beautiful theme for jQuery UI that emulates the elegant style of the Cappuccino JavaScript framework. jQuery UI has a bad reputation for having less-than-desirable themes available for its widgets. Aristo strives to fix that.
brainsik/virtualenv-burrito:

This project simplifies the virtualenv/wrapper setup in your Python development environment. Sprinters at last week's PyCon 2011 had to spend a large amount of time setting up their environments before they could contribute. This project aims to remove that overhead.
maccman/ichabod:

This project lets you run headless WebKit JavaScript tests from the command line. It currently offers Jasmine and QUnit testing support. It also gives you an interactive JavaScript console that has access to the Ruby stack.

Promising Repos

twitter/commons:

Twitter's internal common libraries for the JVM. Readme and build instructions haven't been written yet, but this looks like an extensive library suite. Everything from a memcached interface to an argument parser is included. Check out the documentation to learn more.
extend/cowboy:

This "Small, fast, modular HTTP server" is implemented 100% in Erlang. It's an ambitious project that's still in the early development stage. Contributors are welcome!

Feedback is appreciated! Send any questions, suggestions, and anonymous tips to reflog@kennethreitz.com.

For more open source news, check out The Changelog and github/explore.

Requests: Python HTTP Module

2011-02-19T01:22:00Z

I’m happy to announce the release of my latest Python module: Requests.

One of the most frustrating experiences I have to face, as a Python developer, is the pain of making HTTP requests. The available APIs are insane. I have to look up everything that I want to do and constantly refer to the documentation. I skip writing fun tools from time to time, simply because I don’t want to go through the pain of making a simple PUT request with Basic Authentication.

Things shouldn’t be this way. Not in Python.

Enter Requests

import requests

Let’s check out the awesome new Convore API.

>>> r = requests.get('https://convore.com/api/account/verify.json')
>>> r.status_code
401

Oops, we need to be authenticated. Let’s add our credentials and try again.

>>> conv_auth = requests.AuthObject('requesttest', 'requesttest')
>>> r = requests.get('https://convore.com/api/account/verify.json', auth=conv_auth)
>>> r.status_code
200

It worked! Let’s poke around.

>>> r.headers['content-type']
'application/json'

>>> r.content
'{"username": "requeststest", "url": "/users/requeststest/", "id": "9408", "img": "censored-long-url"}'

Requests is capable of much more than this, but the API stays this simple throughout.

Features

Requests strives to be as simple yet powerful as possible. It’s designed to fit the 90% use case. You might not be able to add proxies, client caching, and raw socket access, but you will have everything you need most of the time:

Extremely simple GET, HEAD, POST, PUT, DELETE requests
Receive simple response header dictionary, content, and status code
Module-level HTTP Auth registry
Eventlet/Greenlet support

The API is quite simple:

5 main functions: get(), head(), post(), put(), and delete()
Attach params/data
Add HTTP Basic Authentication with a parameter
Add a dictionary of HTTP headers with a parameter
Add a CookieJar with a parameter
Attach File uploads with a parameter

Installation

To install Requests, simply:

$ pip install requests

Or, if you must:

$ easy_install requests

But, you really shouldn’t do that.

Roadmap

Requests currently supports CPython 2.5, 2.6, 2.7, and PyPy-c v1.4.

In the future, Python 3.x will be supported. There are no intentions to support Python 2.4 and older.

If you’d like to help or have a feature suggestion, fork me!

[Source on GitHub] [PyPi Listing]

Tablib v0.9.4 Released

2011-02-18T01:22:00Z

Tablib v0.9.4 was released today.

For those unfamiliar with Tablib, it’s my format-agnostic tabular data library. To learn more, see Tablib.org.

Finally, we have Python 2.5 support! This should open some new possibilities, particularly for those of you on Google App Engine.

This update brings:

Python 2.5 Support

Internal pre-2.7 OrderedDicts

AnyJSON System

Secret surprise for the PEP8-strict folks

Various bug fixes

Thanks to tox, the codebase is now continuously tested against CPython 2.5, 2.6, 2.7, and PyPy-c v1.4.

The next steps are to add:

Tablib.ext Namespace Package (for django-tablib, tablib-extras, &c.)

Hooks System (e.g. post-append, pre-export, &c. )

[Tablib.org] [Source on GitHub] [PyPi]

Tablib Community Update

2011-01-31T01:22:00Z

Just wanted to give everyone a quick update on the status of Tablib and its future. Updates will be more regular in the future.

For those unfamiliar with Tablib, it’s my format-agnostic tabular data library. To learn more, see Tablib.org.

Since its release, Tablib has been installed over 2,500 times (from PyPi alone), has 124 watchers on GitHub, and multiple contributors. It is now being used in production at major corporations and development shops throughout the world, including: The Library of Congress, National Geographic, and Discovery Channel.. If your company is using Tablib, feel free to let me know!

Typical case reports include: presenting customer data, statistical data export/manipulation, sampling systems, personal finance management/calculation, and much more.

The community reaction to the initial release was more than I had hoped for. My Changelog article was trending for days on HackerNews, Reddit, and GitHub. This brought quite a few emails, bug reports, and contributions. I was quite surprised, considering documentation hadn’t even been written yet!

Needless to say, the project has significantly matured since. New features and infrastructure include:

Extensive documentation (http://tablib.org)

Extensive Unittesting + Continuous Integration System

Transposition

Databook support (e.g. Excel workbooks w/ multiple sheets)

Row & Column appends

Validation hooks

Advanced unicode (Python 2.x)

Row objects w/ tags

Section separators

Dataset sorting and filtering

Pickling Support

Improved XLS formatting w/ header frames and newline handling

No dependencies, all modules needed are vendorized

Advanced format support framework

A handful of these features have been provided by the work of a few generous contributors.

Luke Lee

Luca Beltrame

Josh Ourisman

Also worth mentioning is the excellent django-tablib extension by Joshua Ourisman. It allows you to seamlessly generate Tablib Datasets from Django QuerySets.

The Future of Tablib

Right now, Tablib’s biggest weakness, ironically, is tabular data’s biggest strength: big data. If you have experience dealing with big data in Python (e.g. 8GiB+ CSV files), I’d love to get your insight on how to proceed.

I will doing a lighting presentation on Tablib at PyCon 2011 in Atlanta, GA this March. I hope to increase awareness, gather feedback/suggestions, and recruit big data developers to assist.

Also planned is the tablib-extras project. The plan is to support various backends, like Google Fusion Tables, and a SQLAlchemy Mapper.

[Tablib.org] [Source on GitHub] [PyPi]

ShowMe v1.0.0 Released

2010-09-16T05:41:00Z

This weekend, I released a new Python module to PyPi: ShowMe v1.0.0. ShowMe is a simple set of function decorators that give you easy diagnose common problems in your Python applications.

Basic Usage

@showme.trace
def complex\_function(a, b, c,\*\*kwargs):
    ....

>>> complex_function('alpha', 'beta', False, debug=True)
calling haystack.submodule.complex_function() with
   args: ({'a': 'alpha', 'b': 'beta', 'c': False},)
   kwargs: {'debug': True}

It currently supports:

Argument call tracing
Execution Time (in seconds)
CPU Execution Time
Docstrings

Roadmap

Add @showme.locals Support
Add @showme.globals Support

Installation

pip install showme

[Source on GitHub]

Tablib Dataset Library v0.6.1 Released!

2010-09-13T01:22:00Z

I'm pleased to announce a new Python module: Tablib. Tablib is a simple module for working with tabular datasets. It allows you create tables of data using standard Python datatypes, manipulate them, and easily export to Excel, JSON, YAML, and CSV. **Basic Usage**: import tablib

headers = ('first_name', 'last_name', 'gpa')
data = [('John', 'Adams', 90), ('George', 'Washington', 67)]

data = tablib.Dataset(*data, headers=headers)

You can maniuplate your data like a standard Python list: >>> data.append(('Henry', 'Ford', 83))

>>> print data['first_name']
['John', 'George', 'Henry']

>>> del data[1]

You can easily export your data to JSON, YAML, XLS, and CSV. >>> print data.json [{"first_name": "John", "last_name": "Adams", "gpa": 90}, {"first_name": "Henry", "last_name": "Ford", "gpa": 83}]

>>> print data.yaml
- {age: 90, first_name: John, last_name: Adams}
- {age: 83, first_name: Henry, last_name: Ford}

>>> print data.csv
first_name,last_name,age
John,Adams,90
Henry,Ford,83

>>> open('people.xls', 'w').write(data.xls)

Excel files with multiple sheets are also supported (via the `DataBook` object). [Source on GitHub] [PyPi Listing]

The Setup

2010-08-03T00:43:00Z

I'm Kenneth Reitz, one of the two co-founders of Züm Hosting. I'm a passionate Python developer for NetApp. I typically spend my nights developing a number of open source projects, and my weekends architecting the web.

What hardware are you using?

My sole machine is a 2010 MacBook Pro running a Core i7 processor at 2.66 GHz w/ 8GB of DDR3 RAM. It's 15" non-reflective screen runs at a beautiful 1680 x 1050. It features both a 250GB SSD and 500 GB HDD, for a total of 750GB of local storage, thanks to the OptiBay. I often have it hooked up to one of many 23" LCD monitors. For general audio, I sport a pair of Bowers & Wilkins P5 Headphones. If I'm in a high-fidelity mood, I listen through a pair of open-backed SR-60s from Grado Labs. For telephony, I have an HTC Supersonic (EVO) running running the Nighty Android 2.2 (Froyo) CyanogenMod v6. This also functions as a perfectly portable hotspot. I read documentation and play arcade games on a 64GB Apple iPad (wifi). I play a PSP Go solely for Castlevania: Symphony of the Night. I have a large number of very powerful Linode servers at my disposal for various projects and builds. My system is backed up regularly to a cheap 1TB Western Digital drive. My media archives are backed up to a 1.5TB Seagate drive. My media exists on a NAS attached to my Netgear WNR3500L router. My code is backed up to various Git repositories local and across the net. All web projects and databases of relevance are backed up to Amazon S3 on a nightly basis, along with all my tweets, Flickr photos, and Gmail messages.

And what software?

I develop and live on the latest build of Snow Leopard. My main Python environment is the latest 2.7 release, along with well managed virtualenvs for all other releases via Buildout. I also employ a very large number of virtual machines, via VMWare Fusion 3. I have a particularly efficient XP VM that is always up, providing me with OneNote 2010. I'd do anything for Mactopia to port OneNote to OS X. Anything. My development environment consists of TextMate + PeepOpen, [most recently] PyCharm, SequelPro, Titanium Mobile, Transmit, and GitX. I love using HTTPScoop for request sniffing. My favorite command-line utilities are: Terminal.app w/ Zsh + Git, JoelTheLion's autojump, Defunkt's Hub and Gist, Mcxl's homebrew, nicksieger's jsonpretty, and Rtomayko's bcat. On servers, I typically run the latest Ubuntu Server release, or CentOS (but only if I have to). I carry out heavy-duty writing with Ulysses Core. Soulver is the best number-crunching application around. I don't know how I lived without it. My audio production work is carried out in Adobe Audition CS3 and FL Studio 9. I use Adobe CS5 Master Collection for light design work and photo retouching. Photoshop Lightroom for kosher photography dark room work. I've been meaning to get into Fireworks for wireframing, but haven't had the time as of late. As for architecture work, I utilize OmniGraffle for diagraming, Mockingbird / Balsamiq for mockups, and MindNode Pro for mind-mapping. For note taking, as mentioned, I use OneNote 2010. Nothing compares, except my Soft Large Squared Moleskine Notebook. My tasks are managed with Things.app. I can't wait till they roll out proper syncing. If they don't do it fast enough, I'll be switching to Macman's Taskforce. I use Kindle for purchased books, and iBooks for PDF reading on the iPad. I consume Google Reader via Reeder, the most beautiful application I have ever used. For Remote Desktop I use CoRD.app. Backups are performed by arRsync.app, a beautiful interface for rsync. My desktop's windows are managed well with Cinch.app. My music library is provided by Spotify at offline, 320KBPS Ogg Vorbis goodness. In the rare occasion that Spotify is missing an artist, I use 320KBPS MP3s and FLAC recordings for the ultra-special tracks (e.g. Liquid Tension Experiment). I buy my MP3s from the (wonderful) Amazon MP3 Store. I keep my video library nice and organized in iTunes, regardless of format, with iFlicks. I stream 1080 videos, NetFlix, and BlueRay Disks with a Play Station 3 Slim. I play emulated retro video games Wii. Best $200 I've ever spent. For communication, I use iChat + Chax, Linkinus, Echofon, and Skype.

What would be your dream setup?

No more chords.

Unix Exit Status Code Reference

2010-07-13T19:51:00Z

I always find myself constantly Googling the list of unix status codes (typically defined in sysexits.h).

0 # successful termination 64 # base value for error messages
64 # command line usage error
65 # data format error
66 # cannot open input
67 # addressee unknown
68 # host name unknown
69 # service unavailable
70 # internal software error
71 # system error (e.g., can't fork)
72 # critical OS file missing
73 # can't create (user) output file
74 # input/output error
75 # temp failure; user is invited to retry
76 # remote error in protocol
77 # permission denied
78 # configuration error

Terminal Productivity App: AutoJump

2010-07-07T03:20:00Z

On average, I'd say I spend 65% of the workday in a terminal session. About 95% of that time is within the same same 4 directories. cd foo & cd bar can get old. AutoJump is a "cd command that learns". It tracks shell history to detect which directories you spend the most time in, and allows you to jump to them without any directory context. It even supports Zsh tab completion. :)

From this:

cd /Users/kreitz/repos/public/gistapi\

To this:

j gistapi

Life is good.

[Project Page] [Source on GitHub]

Ventures: Python Development at NetApp

2010-05-26T18:15:00Z

For the past year I've been employed at Clutch, Inc in Winchester, Virginia, as a Systems Engineer and Web Developer. I've spent most of my time there developing content-based websites and web applications in PHP, advanced JavaScript, Grails, and the like. I even had the opportunity to develop the occassional Python application for server-side data manipulation/migration. But alas, all good things must come to an end.

To that end, I have accepted a new position as a Senior Python Developer at Network Appliance (RTP Campus), starting next week.

One of the best parts of this opportunity will be my ability to work remotely from my hometown of Winchester, Virginia. I'll be setting up office in Bright Cowork on the walking mall.

While moving from service-oriented web development to internal tool/application development will be a change of pace, it's a very welcome one, and I look forward to every moment of it.

I plan to leave my web development efforts to my various open source projects and client work, but on an as-need basis. My next step is to to focus my personal efforts on a new market entirely: iPhone app development & the iTunes store.

GistAPI.py v0.1 Released

2010-05-16T21:41:00Z

Today I released GistAPI.py v0.1.2. It features a highly-optimized Gist object model and API wrapper which allows you to consume Gists in your next Python application. GitHub just rolled out a miniature pre-release of the Gist API last month, so API functionality is pretty limited at the moment. More features will be added as soon as the API is updated. ## Usage from gistapi import *

gist = Gist('d4507e882a07ac6f9f92')

gist.description          # 'Example Gist for gist.py'
gist.created_at           # '2010/05/16 10:51:15 -0700'
gist.public               # False
gist.filenames            # ['exampleEmptyFile', 'exampleFile']
gist.files['exampleFile'] # 'Example file content.'

Source on GitHub

Semantic Versioning

2010-05-09T08:46:00Z

While listening to this week's episode of The Changelog, I came across Tom Preston-Werner's Semantic Versioning Specification. I love what I found. For many years, the open source community has been plagued with version number dystonia. Numbers vary so greatly from project to project, they are practically meaningless. This specification seeks to put an end to this madness with a small set of practical guidelines for you and your colleagues to use in your next project. Included is a helpful guideline for correlating repository tagging. In good taste, Mojombo also opened up the website. If you'd like to contribute to the site, just fork the repository, make your changes, and send a pull request. The way it should be. [Semantic Versioning] [Source on GitHub]

Dev Tool: Ghost #manage /etc/hosts

2010-03-08T08:15:00Z

The Ruby community has really been blowing me away lately with their array of indispensable web development tools.

Ghost is no exception to this rule. It is a simple command line application for adding and removing 127.0.0.1 entries in your /etc/hosts file. I can't believe I hadn't thought of this sooner.

Example Usage

$ ghost add mydevsite.local [Adding]
mydevsite.local -> 127.0.0.1

$ ghost add staging-server.local
67.207.136.164 [Adding] staging-server.local -> 67.207.136.164

$ ghost list
Listing 2 host(s): mydevsite.local -> 127.0.0.1
staging-server.local -> 67.207.136.164

$ ghost delete mydevsite.local
[Deleting] mydevsite.local

Installation

$ gem install ghost

Yes, it's really that easy. Make sure to checkout the GitHub Repo to contribute!

Spotify in the US? Yes please.

2010-01-17T19:03:00Z

I spent about 8 hours last night obtaining a Premium Spotify account in the US, and I've never been happier. As you know, Spotify is only available in the UK, Spain, and France. So, the only way to signup for an account it to take a trip overseas... virtually.

Step 2: Install Ubuntu and Boot Your VPS

Install Ubuntu Server on your new british hackbox. You can SSH in to test it out. Have fun. Make sure openssh-server is installed. ##

Step 3: Edit Hosts File

Append the following lines to your /etc/hosts file:
127.0.0.1 spotify.com
127.0.0.1 www.spotify.com
[your vps ip] hackbox

Step 4: Open a Reverse SSH Forwarding Tunnel

sudo ssh -C
root@hackbox -L 443:spotify.com:443 sudo ssh -C root@hackbox -L
80:spotify.com:480

Congratulations! You're now a Brit.

Step 5: Create an Account

While creating an account, you are prompted for you postal code. I did some google-fu and used SO23 8TH as my postal code for Winchester, Hampshire UK. Download the client.

Step 6: Enjoy :)

You now have unlimited access to a library of ~8,000,000 tracks, as if they were on your own computer. Once every two weeks, you'll have to reopen your SSH tunnels and login at spotify.com to prove you'll be good to go.

This restriction is lifted if you signup for a Premium account. I highly recommend this, as it allows you to listen to your music at 320 kbps. I'd tell you how, but I'd rather enjoy the fruits of my hard labor.

Do You Develop Software or Experiences?

2009-12-20T07:05:00Z

I read an interesting article today on Apple's marketing strategy. A certain section stood out to me, regarding their hardware manufacturing: > Apple is an experience company. They're a high-end marque; if they > were in the automobile business, they'd be BMW, Mercedes, and > Porsche rolled into one. They own about 12% of the PC market in the > USA... but 91% of the high end of the PC market (laptops over $999, > desktops over $699).

— Charlie Stross of Antipope.org.

## The Point: Seth Godin and 37Signals both recommend marketing yourself to the early adopters and geeks. Apple, however, does the opposite. They market themselves to the middle market — the incredibly non-technical. The geeks come on their own, with no marketing needed. They love it. And this makes Apple *enormously* successful. **Develop your software/experience for the masses**. Make the geeks love it, but make them find it on their own. Just a thought.

Facebook Lite Open to Public!

2009-09-10T22:05:00Z

It's official, Facebook Lite was opened up to the US public about 10 minutes ago.

This slimmed down version of Facebook is wonderful for those of us on slow internet connections (or tethering from our iPhones), this is a dream come true.

Reminds me of the good ol' days, when I had to go to http://gmu.facebook.com to see my facebook friends.

Here you can see it in action:

|image|

Try it out now!

Your Degree Is Worthless; Collaborate.

2009-09-01T21:17:00Z

I’ve always been a self-motivated learner as well as a free thinker. I was never one to get involved in cliques or social ladders. Despite the fact that I was raised being constantly told that grades were the single most important thing in my life, I could never accept that. So I didn’t. I just did enough to get by. I didn’t pay attention much in class. I had no reason to. Class was beyond boring. So I’d spend all hours of the night hacking away on my computer, soaking in all I could, and most of the school day sleeping during class when I could get away with it. When there was a test, I’d try my hardest to stay awake, answer the questions as best I could – typically earning a low C in my Honors/AP Classes, and a low B in my “Ordinary” Classes. I found a good balance. Why would I study for 30 hours a week to get better grades when I could get by with 0 hours? So, I graduated. And I did the next logical thing: I went to college. Ahh, college. The most important decision you could ever make in your life. The time to “make it or break it”. Where every young man goes to be become a man. So I went. While high-school never engaged me, I assumed that courses about career-relevant subjects would interest me in a university setting. I was wrong. I was very, very wrong.

College Life

I studied Plato, Homer, and Socrates, Turing Machines, and Single-State-Automata. I analyzed the progression of American Popular Music from the 1920s to today. I learned how to draw, play the marimba properly, and splatter paint on walls. I attempted jujitsu. Wonderful life experience. But what does any of this have to do with setting the foundation of a career? Nothing. At all. Of course, the higher up you go, the more relevant the courses get to your chosen major. But I didn’t want to wait. Especially when I was paying $20,000 a year for this. It was mostly useless information. The average college education in America costs 9 cents a minute. Every minute. Every day. A complete waste. And I was working 35 hours a week as a graphics design intern, working at odd hours of the night, attempting to pay for all of this. It was impossible, and I was unengaged.

The Plunge

So, after totally losing interest in class or anything related to it, I gave up and dropped out. I didn’t want to get further in debt. So I moved back home, defeated, and tried my hardest to get my life back in order. I got my high-school job back at McDonald’s. I worked harder than I ever have in my life. I didn’t have anything else better to do, so I worked as much as I could all the time. I worked one 65 hour week - but that got old really fast. I was again, unengaged. Then, one day, I quit McDonald’s without notice. Not best practice of course, but I didn’t want to spend any more time there. It’s strange what a terrible work environment it is. After a few short weeks, you begin to think there isn’t anything outside those walls. It was clear that it wasn’t getting me anywhere, so I set out with my laptop to try to find something better.

How It All Turned Around

I spent a lot of time on some freelance websites, where you bid for odd jobs, usually settling for some ridiculously low amount of money here and there. That didn’t last long. I remembered coming across a guy on Twitter from my hometown, Winchester, Virginia, who was quite into the internet and technology. That’s pretty rare in these parts, so I looked him up. What I found was a local cowork center. I went and checked out the cowork, and what I found blew my mind. In this little building off the historic old-town walking mall was a room. Inside: the COO of a major internet company, tech consultants, graphic designers, writers, author, bloggers, freelancers, and so much more. I met everyone in and around town. I sat in on think tank lunches. People cared about what I had to say. We collaborated.

Collaboration is Everything.

For the first time in my life, I was meeting interesting people with awesome experience, willing to share and collaborate what they have learned with me. And I did the same. I soaked in endless amounts of information. One simple room full of a few people turned on some switch in me that the education system had failed to do year after year after year: teach me something. I was finally engaged. *Fully* engaged. As soon as I realized that, my entire life changed. I started thriving on my own, getting dozens of clients. Suddenly, I had a life with significantly less stress and worry. No tuition fees! I realized how valuable my skills were and how I didn’t have to be part of the institution if I didn’t want to be. I rose above. I am now a Web Applications Developer at a respectful technology firm. No degree. No debt. Only an open mind. I gain more knowledge and experience in a single workday than I did during my entire college career. I’m in the real world. And I’m loving every minute of it.

In Conclusion

Looking back, I’m comfortable saying that dropping out of college has been the best decision I’ve ever made. If I would have gone through the entire education program, what would I have to show for it? $150,000 in debt, a piece of paper, and four years less of your life. No real experience. No connections. Just a piece of paper. And nothing more. Please bear these things in mind before you decide to spend $150,000 on a bachelor’s degree in a field you’re not so certain about. Personally, I’d rather spend that money on something that will actually benefit me: like a house.

The Call for an Open Source Social Network

2009-08-16T20:16:00Z

Lately, I've been tossing some ideas around that I feel would benefit the Social Web as a whole. It’s been going through some rough times lately, and I think it’s time for a change. Or so I thought. My first idea was to create a site that was rather decentralized, allowing all of your content to exist on other sites, but still allowing for you to interact without locking a user in. As it turns out, this site already exists. It is called FriendFeed. `I love FriendFeed <http://kennethreitz.com/blog/friendfeed-is-awesome/>`_. About a week after I decided that I wanted to get into it, Facebook decided to purchase it. How sad. They claim that it will still be up and running, but we’ll see if that proves to be the case (I have my fingers crossed). Pownce (which was powered by Python / Django) was shut down when it was purchased. I pray this not be the case. Think about this now: Why is that even an option? Social networking is all about community and building tribes – So why do we need to have an organization in charge of our chosen communication platform? Here is my proposal: Create a community-driven, community-developed, and community-controlled social networking site that is truly open source. The community could take care of everything from feature development to content control. No random shutting down, buy-outs, or merges. No more random change of Privacy Policies or Content Ownership battles. No more worries. The only group of people who would be benefitted would be the community itself, not some company. An open minded network full of open minded people working for the better of the community. # The Open Web of Flow

Step One: Solid Platform Choice

The platform choice is the most important. You've seen what happens when the wrong tool is chosen for the job: look at Twitter: Bad Planning. We're all familiar with the Fail Whale but we shouldn't. At all. So what do we use? There's an array of options. .NET? HAHA! Did you know that there's even a social platform built on top of Microsoft's Sharepoint? I bet you didn't. I've installed it a number of times. It's fantastic (for people who need it). But we don't. At all. What type of open source project aside Mono is driven by NET developers anyway? They are in an entirely different mindset than us. Anyway, the answer is obvious: Django on a LAMP Stack (Linux + Apache + MySql + Python). We can all agree that Python is freaking amazing. And it's certianly not going anywhere any time soon. I think google has proven time-and time again that Python is the language for just about any job. And when Google's unlayden-swallow project is complete, all (typically negligible) performance issues will be eliminated. Done. ### Step Two: Basic Information Architecture

We need to decide how the whole system will work. FriendFeed has an excellent system in place. Lets use it. Users can tie everything in from all of their other websites and steam it on their profile, and display it all on one page. Everything's streamlined, commentable, hookable, and readily accessible. Google Profiles rock. But that is definitely an abandoned project. Lets mix that with a FriendFeed-style activity stream. Done. ### Step Three: Sustainability, Audience, and Accessibility

How will we pay for it? How will we get people to contribute? How will be get people to use it? Answer: Twitter is getting old and it's getting old fast. It's time for something new. Lets blow them away and they will ALL hop on board. Allow for easy external-account migration and account creation and we'll be golden. ### Step Four: Technical Planning and Engineering

This is all the stuff end users don't have to worry about. Performance. Design. System Administration. Database Engineering. The geek stuff. I mean come on guys, how awesome would it be to be able to make a commit to the Twitter live SVN Branch? Epically awesome. ### Step Five: Community + Collaboration

Once the community gets going, there will be no stopping it. In conclusion, think about what we have to lose? Are you with me?

What's In a Design?

2009-08-09T05:16:00Z

Disclaimer: I am not an Apple fanboy. Apple makes a fortune off of speeple. An Apple FanBoy blindly follows everything the cult leader, Steve Jobs has to say. They go out and purchase every iProduct that Apple realeases and does so with a smile on their face, not realizing that over the course of four years, they have spent about $20,000 on Apple products alone. They leak photos of an upcoming product with incredibly low resolution so the masses can pick and decipher the image, spending countless hours just trying to imagine what that mysterious new button does. And people do it! They completely take advantage, but rightfully so. Who could blame them? They are, after all, a business. It would be foolish for them not to, because that FanBoy pays thousands of dollars to watch a 3 hour long commercial, and they love it! They love OSX for no apparent reason other than they are told to. They go out and buy every single new iPhone the day that it comes out, when there's clearly nothing wrong with the one they had. Year after year, they pour more and more money into their investment in nothing more than a brand that they thing will change who they are as a person. They want to be Steve Jobs. They write endless posts about how much money they spend on Apple products every month, and just can't wait to get their hands on the next one. This is not that post and I am not that guy. I do, however, believe that my recently purchased 13" Unibody MacBook Pro is the greatest laptop ever manufactured on Earth to date. But not because Apple made it. That's irrelevant. Now please, don't get me wrong: I love Apple. And that's because, they have a superior operating system and they put an extraordinary amount of design and thought into every nook and cranny of every system that they release. This system is designed well. Incredibly well. On a side note, I firmly believe that OSX is the greatest desktop operating system of all time, and I see no chance of this changing any time soon. From many different standpoints (End User, Prosumer, Developer, Newbie) this holds true. I am a fan and follower of one thing and one thing only: Design. And, for now, Apple hands-down has the greatest sense of design, both software and hardware in my opinion. But if something else comes along made by someone else, I'll be sure to give that a try with an open mind. :)

I Finally Got a MacBook

2009-08-07T05:56:00Z

Yesterday I stopped by the Apple store in Fair Oaks Shopping Center in Fairfax, VA and finally purchased a MacBook. I've been wanting one for quite a while – ever since I fell deeply in love with OS X after adopting an old 12" G4 PowerBook I've been using for the past 5 months.

I decided to get the 2.5 GHz 13" MacBook Pro. This will be my full-time machine for the next 4+ years hopefully. With specs this good and a case this sturdy, I see no reason why it shouldn't last me for years to come.

I am thoroughly pleased with the purchase so far. If you've been planning on getting a MacBook, now is the time to get one. The base line 13" Aluminum Pro model is only $1100 right now – cheaper than most of the refurbished polycarbonates with lower specs. Anyway, I'll post lists of my software choices (including TextMate Plugins and Bundles) soon for those interested. Stay tuned!

Back to What I Really Love

2009-07-21T11:46:00Z

A couple of months ago, I took a position at a company that tailors Microsoft SharePoint solutions. The business model was very strong (and successful), the work was challenging, and there was lots of opportunity. At the end of the day though, I just couldn't get past one thing: Microsoft and.NET. This is not stuff that I wanted to spend the rest of my life. The company offered very generous compensation for Microsoft Certification – but what good does that do me? I could do it, but it would only be for the money. I love working with software of all kinds, but the development I was doing did not sharpen my skills in any way. So, I decided that the longer I stayed, the more I would simply be delaying my leaving. So another opportunity came up, and I decided to take it. I am now a developer working with PHP, Symfony, Java, Groovy, Grails, Apache, and more. The list grows every day. Now life is good. It's never been better in fact.

Wasted Talent

2009-07-15T18:30:00Z

I'd say that 95% percent of the students that I met at George Mason University my Freshman year had no idea why they were even there. This is so sad.

For me, going to college was more about getting out of the house than anything else. And it worked. I did get out of the house. And I learned a lot. I found my passion.

Other students worked their butts off trying to take as many credits as possible, and when asked what they want to do for a living they don't have the slightest idea. Why would you go to college if you don't even know what you want to major in? I would think that spending all that money would be in order to achieve a goal, but apparently not.

They are just along for the ride.

Fallibilism

2009-07-15T05:11:00Z

Everyone seems to think that they are always right. It's pretty funny when you think about it. Because they aren't. At all. That's why I'm a falliblist. I believe that others are often right, and since I'm human, and prone to error, I'll always consider anything that anyone else has to say. But hey, I could be wrong. :)

|image|

Revolution vs. Innovation

2009-07-15T01:08:00Z

I've been thinking alot about this cloud-computing "movement" that has been a buzz word for the past year and half or so. Being able to access anything from anywhere? Awesome, but I can do that now.

I don't really get it why everyone's like "oh this will totally change computing as we know it". I beg to differ. Perhaps it will change the way we develop, or organize. But not how Desktops sell. Or eliminate the need for desktop software. The desktop is not going to die. Not from Azure, at least.

Amazon has EC2, Microsoft has Azure, and Google has AppEngine. These are fantastic tools, but they are nothing new – Just something someone else thought up, executed properly – followed through and improved upon.

Bill Gates came up with the idea of a true Software Company. This was revolutionary. Apple took Bill's model, and innovated. They improved upon it. And look at what's happening.

I really doubt that Google's new OS is going to bring us anything we don't have already. I do think, however, that it will build on things we already have.

When Chrome came out, it didn't offer anything that we didn't already have. Sure it's a fantastic browser, and I don't want to discount that. But the ability to run "web apps" as applications is nothing new. I had been using Mozilla's Prism for at least a year before Chrome was announced. And on OSX I had been using FluidApp, which is like Prism on Crack.

It seems like the smart thing to do in the tech world nowadays is to follow through with great ideas – even if they aren't yours.

Mozilla's team came up with the idea of running web apps at application with SSB's (single site browsers), but implementation of Prism was slow, and incredibly buggy. They didn't follow through. Google did. and Google won.

Maybe I don't need to come up with a revolutionary idea. Maybe I just need to be innovative.

What Seperates?

2009-07-15T00:28:00Z

I want my thoughts to be heard. I want to share the things I find on the web every day. I want to be a part of a community. I want to lead the community.

That's why I'm here, and that's why I'm writing this.

Instapaper: Best Web App Ever Created

2009-07-13T22:54:00Z

Out of all the startup applications that I have ever used, Instapaper (from the creators of micro-blogging site Tumblr) is by far the most innovative and useful. I use it on a daily basis. It not only saves me hours upon hours of time, but it allows me to focus more on the task at hand and boosts my productivity levels through the roof.

Whenever I'm surfing the web, I have a bad habit of clicking open new tabs almost constantly. It's not uncommon for me to easily have over 60 tabs open, and in the process of sorting through all these wonderful links, the task at hand drowns in a sea of information.

Well, I don't have this problem anymore, thanks to Instapaper. When I find something interesting online now, and it's not related to the task at hand, I hit the nifty "read later" bookmarklet. This ads it to my list of unread marertial on Instapaper.com. I can go there at any time and read any of the pages that I didnt' have time to read before!

Seriously guys, this is amazing.

What's better yet, is that they have an iPhone app. I can read my synched list of hundreds of unread pages at any time in Kindle-quality formated pages. Even without an internet connection.

Surely life doesn't get better than this.

`Instapaper <http://instapaper.com>`_

The Universal Flaw in Commercial-Based OS's

2009-07-10T21:24:00Z

Designers and Developers around the world, I present to you the flaw prevalent in all of today's commercial operating systems. This is not a security hole, nor is it a CPU-capping bug. It's more of a world-view. We're simply looking at things the wrong way. What ever happened to the days when a computer was a *tool*, rather than an *experience*? Nowadays, computers are viewed by the public as a way of expressing oneself. Don't get me wrong, a computer can be all of that and more. Certainly, I find a level of solace and self-identity in my software/hardware setup. But, a computer, more primarily, is so much more than that. A computer, in today's modern consumerist mindset, is a box that runs applications that are made by other people. You will use the applications either because you think it will enhance your quality of life, make you more productive, cure you from boredom, or just be plain neat. No other options exist, just the software that big-name companies produce. Sure, this method does sell well and, in the scheme of things, makes a company a large amount of money in a short amount of time (which is the point of a business, is it not?), but perhaps those big-name companies should think a little more long-term for the sake of us all. What does this method truly accomplish? It makes people buy lots of computers that have power way beyond practicality. This, in term, makes software development freeze. In case you haven't noticed, we can't do much with computers nowadays that we couldn't do 10 years ago, except perhaps check our bank accounts online, and that's hardly a major breakthrough rather than an new-found application to keep everyone's interest. It self-destructs in the end when you think about it. And this, my friends, is the central heart of our problem. In our "ever-changing" world of computer software, there is little to be found that is truly new or exciting. We have a bad habit of putting a new face on an old concept and calling it by a different name, when, in reality, we've run out of good ideas.

Django Remote Development Server

2009-05-08T04:03:00Z

If you've worked with Django much at all, I'm sure you've had this problem: wanting to access the built-in development webserver remotely. Typically, this integrated mini-server ignores all requests from any IP Address other than 127.0.0.1 . If you run the following command, however, it will be accessible remotely. VERY useful for remote dev work.

./manage.py runserver 0.0.0.0:8000

Enjoy!

Development, Django, Python

Remote TextMate Development via SSH and Rsync

2009-05-08T00:39:00Z

I am a huge fan of TextMate. In my opinion, it is by far the greatest text editor ever conceived by mankind. It has a couple of shortcomings, however. One of which is that it has no built-in FTP or SFTP support. Remote file editing is a bit of a bear here if you like to view folders in the project drawer on the side. Options for remote editing with TextMate: - Cyberduck FTP client - MacFUSE + SSHFS - Rsync + SSH

Cyberduck

The `Cyberduck <http://david.olrik.dk/files/Synchronize_remote_directory_rsync_ssh.zip>`_ option is very very useful. While in the FTP client, you simply click "Edit in TextMate" and the client will download the file for you, open it in your editor, and – here's the awesome part – it automatically uploads the file every time you save it. This works great when working with one file at a time. The drawback, however, is when working with large projects. Toggling between many files can be an albatross without the project drawer (cyberduck understandably doesn't allow you to edit an entier folder), so MacFUSE is the next logical choice. ## MacFUSE + SSHFS

MacFUSE + SSHFS works great, and allows you to mount an SSH folder as a mountpoint on your local system. VERY USEFUL. You can open this folder with TextMate. This is perfect for smaller projects. However, with larger projects, this makes opening the folder in TextMate almost unbearable as it checks the status of every single file. Too slow :P ## Rsync + SSH

So here's the final solution: Rsync + SSH. This allows me to automatically sync my working copy with my server and allow for snappy file interactions without having insane latencies for starting up and bandwidth hogging! To remotely sync over SSH, run the following code: rsync -avz -e ssh remoteuser@remotehost:/remote/dir /target/dir/

Hint: If your remote working copy is a subverson checkout, you can add --cvs-exclude

System Message: WARNING/2 (<string>, line 2); backlink

Duplicate explicit target name: "textmate".

into the rsync parameters, and it will exclude the ".svn" folders! You can then open this directory in TextMate and make all the changes you want, and then sync after ! There is also a wonderful TextMate Bundle for Remote Rsync + SSH within TextMate. Development, OSX, rsync, TextMate

Crossing Over to the Dark Side

2009-05-06T13:49:00Z

Well, I've done it. I've crossed over. To .NET.

I will continue to be a complete open-source junkie of course, but during the work hours, I will no longer be working with PHP and Python. I will now be working with .NET and SharePoint.

I was presented with a rather good reason for this actually:

"Using SharePoint gives our clients a tremendous amount of security – they can trust that if, for whatever reason, something were to happen to us, their software could still be serviced by someone else." That's certainly justifiable and understandable. That's a pretty good idea actually. I like it.

After looking into ASP.NET, I was actually quite surprised. The code is a lot cleaner than PHP. It looks rather powerful actually.

Anyway, that's why I've crossed over. No hard feelings :)

Software Development vs. Computer Science

2009-05-05T20:52:00Z

Most developer job applications that I see have a "BS in Computer Science or equivalent experience" requirement.

During my studies in Computer Science at George Mason University, though short, I learned a number of things. One of them was what a waste it was to learn such higher math in my field. I want to develop software, not develop the most cunning-edge earth-shattering algorithms. I don't want to reinvent ssh or find a better way to implement pgp keys. Of course, those things are necessary in certain fields, but not in mine.

I was taught that Programming is to Computer Science as a Telescope is to Astronomy. Its a tool to get to a means. If this is true, than why wasn't I thought how to make software?

Many Software Developers also have degrees in Electrical Engineering. Why is this? A CS degree doesn't seem to quite fit either. Perhaps we should make a separate degree for Software Development?

Computer Science should be separated from Software Development. They should be two different Degrees.

When I realized this, I started spending my time focusing on design rather than math. I learned a great deal of things from color theory to relational spacing and I found myself a new home: web design. It's a beautiful field. I started to spend my own time learning software development, rather than spending hours studying Calc 2.

The Ultimate RSS Feed Reader

2009-04-23T01:35:00Z

What would the ultimate RSS Reader have? Hmmm.... Features: - Easy-to-manage import/export - Available anywhere and in a variety of formats - Easy access to both urls and inline-viewing abilities - Customizable fonts - Takes up less than 200 MB of RAM (this rules out all Adobe Air Applications) - Auto feed sniffing from urls - The ability to play podcasts and other media that is thrown at it (unlike the PSP) - Syncing for offline viewing - Expendable

Closest contender: Google Reader! It offers all of this functionality and more. It makes my world a better place to live. `Google Reader <http://reader.google.com>`_

Contact Syncing for Massive Productivity Booster

2009-03-25T06:12:00Z

Google significantly increased my quality of life recently. How? Let me explain. While I am a firm believer that cloud computing should never be viewed as a replacement for the current desktop/ model, I must say that I am now a huge fan of storing my data on the internet. Not all of my data, keep in mind, but information that needs to be accessed by multiple computers, of course – but that goes without saying.

I have always had the problem of not being able to keep track of all my data. I switch computers and operating systems so often, I can't keep track of my contacts at all. And when I update one, I have to go through and update many different databases – not the most efficient method.

So, the ultimate solution is – obviously – to consolidate all of my contacts into one database. I used to keep all of my contacts on my Samsung Blackjack cellphone. This worked well, since I could easily sync it with a computer. This caused a problem, however: I could only sync it with one system. If I was at work or on another system, I had no way to get to my friend's email addresses.

Last month, Google Contacts started to support Exchange Syncing, which happens to work flawlessly with my Windows Mobile 6.1 install.

My life will never be the same. Thank you, Google. I am eternally grateful.

More details soon!

Python + Regular Expressions

2009-03-17T05:30:00Z

Have you ever needed to parse through large amounts of text looking for a specific pattern? Patterns like “one capital letter followed by three numbers” or “dd/mm/yyyy”? This is known as Pattern Matching. Regular Expressions allow easy syntax for pattern matching, and is an invaluable skill to add to one’s toolkit, no matter what your area of expertise/practice is. Whether you’re writing a Compiler, Form Validator, Text Editor, Django Project, or Language Translator, Regular Expressions will always prove to be invaluable. Here is a very basic overview of some syntax: ‘\d’ represents a digit. ‘\s’ represents whitespace. ‘.’ represents any character. If you have worked with Python for very long, you are probably already familiar with the concept. Take a look at the following code:

print("Rounded = %05d" % (42))

This makes sure that the digit printed has 5 digits, and will automatically add 0’s to compensate. If you understand this concept, then you shouldn’t have a problem. Perl-style Regular Expressions are a very widely-accepted implementation, and Python has built in support for this mini-language! It’s easily accessible, so let’s get started. The included ‘re’ module will give us everything we need to get started: import re

Lets give our new module a try! It will enable you to do anything you could ever want with regular expressions. Here’s a quick example of some basic use:

import re

string0 = 'Kenneth Reitz is a cool guy!'
regExp = r’kenneth[- ]?reitz’

if re.match(regExp, string0, re.IGNORECASE):
   print “True”
else:
   print “False”

This script takes the string 'Kenneth Reitz is a cool guy', and searches for 'kenneth reitz' inside of it. If ‘kenneth reitz’ is found within string0 (re.match compares the expression with the string), the script will print True, if not, it will print False. Additional parameters can be passed to the re.match function when needed. Note the re.IGNORECASE flag used here –- This tells the function be case-insensitive. Once you master the regular expression syntax, you’ll realize how truly powerful they can be. The options become limitless and the usefulness becomes undeniable. Here’s another example:

import re

string0 = '10.03.1988'
regExp = r'^\d\d[./]\d\d[./]\d\d\d\d?$'

if re.match(regExp, string0):
   print 'True'
else:
   print 'False/

When run, this script prints out True. If we were to change string0 to ‘10.03.88’, it would print False. Simple, isn’t it? Now, while a True/False return could be useful in certain applications (i.e. form validation), most of the time, we’re going to want to have a bit more information in order for our checks to be useful. We can tell Python to show us the data that matches our query. To do this, we’re going to have to break our expression up into different groups. In the date we have defined, there are three obvious groups we could separate this into: the day, month, and year. While defining a Regular Expression, you can use parentheses to define groups: regExp = r’^()././$’

This separates our expression into 3 separate groups. Python also supports turning a Regular Expression string into an heavily-supported object with the re.compile() function. Once you define a string as a Regular Expression object, you can use the built in methods to preform powerful parsing. Now we can ask python what is in those groups:

import re

string0 = ‘10.03.1988’

regExp = re.compile(‘^()././$’)
regExpMatches = regExp.match(string0)

if re.match(regExp, string0):
   print(“Day: %s\nMonth: %s\nYear: %s” % (regExpMatches.group(1), \
     regExpMatches.group(2), regExpMatches.group(3)))
else:
   print(“Invalid Date.”)

When executed, this script parses through our validated date, breaks it down into groups, and prints the following:

Day: 10
Month: 03
Year: 1988

The possibilities are limitless! Here’s a quick run-down of the re module’s functions, strait from the Python documentation for reference: match: Match a regular expression pattern to the beginning of a string. search: Search a string for the presence of a pattern. sub: Substitute occurrences of a pattern found in a string subn: Same as sub, but also return the number of substitutions made. split: Split a string by the occurrences of a pattern. findall: Find all occurrences of a pattern in a string. compile: Compile a pattern into a RegexObject. purge: Clear the regular expression cache. escape: Backslash all non-alphanumerics in a string.

Remember, you can always type help(re) (after importing the re module) into the Python interpret to take a quick look at the module’s built-in documentation. Good luck and happy coding!

A New Spin to Software Platform Design

2008-11-11T19:31:00Z

*I wrote this article two years ago, before I found OS X. I wish I would have known then.* As I've said before, I find many reasons to believe that modern commercialized software platforms are severely lacking in many, many areas. This should not come as a surprise to anyone. Perhaps basic utility-inclusion is not the only solution though. Perhaps the basic priority structure and ethics that development and marketing teams utilize should be forced into question.

Essentially, most major computer software corporations, are, all in all, trying to make money. No matter how hard you try to find a way around this, or justify why these companies try to do the things that they do, the only answer is money. These companies are simply trying to make a quick buck. This concept has worked incredibly well for years, but we seem to have a bit of a problem with foresight. After a while, people get rather bored with the same old concepts being presented to them in new and exciting ways. This is why Microsoft needs to release a new operating system every once in a while. Microsoft's current problem is that the masses are beginning to realize their other options.

So here's my proposal for the long-term design and marketing strategy 2.0:

An operating system should first be a place of power, consistency, stability, scalability, and flexibility. Included would be a robust and fully scriptable toolset which can be manipulated and presented both graphically and statistically.

Second, the user interface should be very well thought out and planned, with ample room for improvement down the road. Its purpose should first be a place of usability, workflow, and creativity. Task-related workflow and presentation customization, accessible to all types of users, is crucial to the success of the UI. Second, the User Interface should be a mode of personal expression and aesthetic preference. This should never take precedence over the overall stability, usability, or general usefulness of a desktop system, for any given reason.

Lastly, the user application platform system needs to be designed. A centralized repository of applications is an incredibly efficient method for application distribution. This repository would be a dynamic, centralized database of application software and packages that are intended for different groups of people. Most major Linux distributions use this heavily, as well as Apple for it's iPod and iPhone applications, and it has been proven to work well.

Any of these rules should have the ability to be broken easily by advanced power users for technical reasons/needs. This should be in no way advertised or demonstrated.

Anyone up for the challenge?