Best Practices

Here are some best practices that I have learnt so far. If you find something to be inappropriate / inaccurate, please comment below so that I may update this post as necessary. Again, these are evolving thoughts and may change as I learn something new. I have put these things on GitHub too.

General

• Always make sure that you name the variables appropriately.
• Follow a general pattern. It should be clear what a function does and returns. stick to the standards.
• Experiment as much as possible before you use some thing in production.
• Open source almost everything as much as possible.
• Composability - Always keep this in your mind.
• Never change a function signature that some one else has written without discussing with your team first. This is the most stupidest thing to do.
• Write kick ass test cases. This will drag you in the beginning, but trust me, this will hold your back in the long run.
• Write as much extensive documentation as possible.
• Build a CI server. Have it test your code each time you commit your code. That way, everyone will immediately know if something is broken.
• Always stick to the standards and general patterns as much as possible. By that way, any new member in the team can figure things on himself without much help from the others.
• If the logic / idea behind the code is not obvious, either write a crystal clear documentation or rewrite the code so it is obvious.
• Remove redundant pieces of code, UI or anything.
• Never update a production server. If there are any compelling reasons to update your server, have an identical staging server, update/ upgrade it, run your kick-ass test cases and if everything seems to be right, then you may continue updating your production server.

Source Control

• Prefer Git unless you have any oher strong reason to use somthing else.
• Keep your runnables on root of your repo.
• Have to main branches - prod (for production) and dev (development) branches.
• And other branches as required - hotfix-* and feature-*
• You NEVER commit directly on prod.
• If there are any severe bugs in prod, then you may checkout hotfix-* from prod, fix the issue and merge it back into prod.
• The only activity on prod branch is to pull from a stable dev or a hotfix-*.
• If you want to add a new feature, then you should checkout feature-* from dev, Implement the feature and merge it back to dev.
• The activities on dev should only pulling from feature-* and pulling from prod which has a hotfix-* merge.
• Avoid usage of GUI tools. GUIs are great for code review. But always use git command line for committing and stuff.
• Try to user --no-ff to preserve branching history.
• We almost always write our new code at the end of the file. This will lead to merge conflict if all of us work on same part of file. Instead, try to write new codes other than the end of file. some where it is more appropriate. This will prevent merge conflict to an extent.
• It is always a good idea to have release notes with a diff of codes. Great for reviews.
• Merge vs. Rebase: Use merge when you want to preserve the branching tree. Using merge will result in a additional merge commit. Use rebase if you dont need preserve the branching tree and avoid the additional merge commit.

Startup

• Get out as soon as possible.
• Build a very minimal viable product and go out as soon as possible. You never know what users want. This will give an opportunity to learn and build more.

UI-UX

• After all, this is the part which users will be seeing.
• DO NOT make the user remember anything.
• Everything should be intuitive.
• Users should accomplish their tasks with as much lesser efforts as possible.
• Keep the interface as much simple as possible.
• Everything should be obvious.

JavaScript

• Make Fewer HTTP Requests
• Use a Content Delivery Network
• Add an Expires Header
• Gzip Components
• Put Stylesheets at the Top
• Put Scripts at the Bottom
• Avoid CSS Expressions
• Make JavaScript and CSS External
• Reduce DNS Lookups
• Minify JavaScript
• Avoid Redirects
• Remove Duplicate Scripts
• Configure ETags
• Make AJAX Cacheable

Misc

• If you want to succeed as bad as you want to breath, you will.
• If you use any open source projects, give an attribution to them.
• Contribute to their repo or donate something, no matter how small it is. It is these small things that move an open source projects forward.
• Open source your code as much as possible.
• If you are a startup, hire some freshers who are passionate about technology, and guide and mentor them. Turn them into some valuable stuff that a society could use. And make sure he will do the same thing to some other fresher when he is settled.
• Blog about problems you have faced and the approach that you took to solve it. It might help some one else.

Python Libraries: Django, Twisted, Tornado, Flask and Cyclone.

This post will soon be moved here
This blog has moved to http://dhilipsiva.com/blog/

This post is outdated.

NOTE: This post may be modified soon. I suspect that this will start a flamewar. I have written this post as far as I understood these libraries.

I have been writing production applications in python for almost 2 years now. And I have come across quiet a few libraries. I was confused and didn't know what to use at first. So I experimented with as many libraries as possible. So here is what I found until now.


"Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design". If you are building something that is similar to a e-commerce site, then you should probably go with Django. It will get your work done quick. You dont have to worry about too many technology choices. It provides everything thing you need from template engine to ORM. It will be slightly opinionated about the way you structure your app, which is good If you ask me. And it has the strongest community of all the other libraries, which means easy help is available.


"Flask is a microframework for Python based on Werkzeug, Jinja 2 and good intentions". Beware - "microframework" may be misleading. This does not mean that Flask is a half-baked library. This mean the core of flask is very, very simple. Unlike Django, It will not make any Technology decisions for you. You are free to choose any template engine or ORM that pleases you. Even though it comes with Jinja template engine by default, you are always free to choose our own. As far as I know Flask comes in handy for writing APIs endpoints (RESTful rervices).


"Twisted is an event-driven networking engine written in python". This is a high-performance engine. The main reason for its speed is something called as deferred. Twisted is built on top of deferreds. For those of you who dont know about defereds, it is the mechanism through with asynchronous architecture is achieved. Twisted is very fast. But is not suitable for writing conventional webapps. If you want to do something low-level networking stuff, twisted is your friend.

"Tornado is a Python web framework and asynchronous networking library, originally developed at FriendFeed. By using non-blocking network I/O, Tornado can scale to tens of thousands of open connections, making it ideal for long polling, WebSockets, and other applications that require a long-lived connection to each user". Tornado stands some where between Django and Flask. If you want to write something with Django or Flask, but if you need a better performance, you can opt for Tornado. it can handle C10k problem very well if it is architected right.

"Cyclone is a web server framework for Python that implements the Tornado API as a Twisted protocol." Now, what if you want somehting that is nearly as performant as Twisted but easy to write conventional webapps? Say hello to cyclone. I would prefer Cyclone over Tornado. It has an API that is very similar to Tornado. As a matter of fact, this is a fork of Tornado. But the problem is it has relativly small community. Alexandre Fiori is the only main commiter to the repo.

I am working on the tabulation. I ll be updating it as soon as I find time to complete it.

Want me to review any other library? Please leave a comment below.

Comments / Suggestions / Edits welcomed.

My 2012 python meme #2012pythonmeme

I have been writing production application in python for the past one year.

1. What's the coolest Python application, framework, or library you have discovered in 2012?

    MRJob - A python package that helps you write and run Hadoop Streaming jobs.

2. What new programming technique did you learn in 2012?

    I have learnt how to write highly modular and reliable code. How to do automated deployments. And how to auto-scale.

3. Which open source project did you contribute to the most in 2012? What did you do?

    Well most of my contributions were to non-python projects. But I did a very tiny contribution to cyclone.

4. Which Python blog or website did you read the most in 2012?
    Planet Python and couple of other blogs.


5. What are the top things you want to learn in 2013?
    Natural Language processing, Machine learning, Big Data in python.


6. What is the top software, application, or library you wish someone would write in 2013?
    A full stack framework that supports Hadoop and related technologies.

Want to do your own list?

here's how:copy-paste the questions and answser them in your blog.

tweet it with #2012pythonmeme hashtag

My first pull request

“A journey of a thousand miles begins with a single step.”

That single step for me, in my carrier, is learning python. A fun loving, magnificent beauty. For me, programming is like raising a baby. Initially, the baby girl doesn’t know anything. As a dad, its my responsibility to teach her what to expect, how to react & what to respond. For me, it is more than art. Its parenting. I take care of her so that, she never crashes. To ensure that she  can be fixed when someone breaks her, I should write great test cases. She is just a kid right? She is not mature enough to take care of her all by herself. So I have to make sure that she is always happy by employing some care taker who always look after her changes. Buildbot is her caretaker. When some one breaks her, buildbot will notify me where I can fix her right away.

So I was trying to get buildbot up and running. For some unknown reasons, it did not start. It just crashed. When I looked at the logs, it said that “twisted.web.errors.NoResource” was missing. Well that was weird. So I decided to go through the source-code of twisted. And “No Resource” was no where to be found. I got confused. Only then I found out that Twisted was updated to 12.2 just a few minutes ago.

There was a breaking change in it. “NoResource” has been removed. It was not there. The only thing that appeared to be similar to it was “Errors” So quickly without thinking anything else, I patched it and send a pull request.

I was happy, because it was my first contribution for a big project. Then next day when I checked my mail, I was sad. Looked like “NoResource” has been moved to “Twisted.web.Resource” package. I was dumb enough that I failed to see it. My pull request has not been accepted. It was fixed by some one else. Then I remembered something.

”I haven’t failed. I’ve just found 10,000 ways that won’t work.”

Yes, I just found one way in which I cannot submit a pull request.

“If You’ve Never Failed – You Haven’t Tried Hard Enough to Succeed”

“Success does not consist in never making blunders, but in never making the same one a second time.”

I am happy that my pull request got rejected. If a person always agrees to what you say, then you sort of never learn anything. It is only while someone disagrees you, you learn something new.