Adrian Andreias

Goodbye, Wordpres! Hello, Pelican!

I've just switched by blog from Wordpress to Pelican. Because serving static files is the fastest and most secure way to run a website.

The coolness factor is also way up if you're keeping your blog content and settings in git and you run fabric to build and publish:

fab build && fab publish

Fresh git tips

1. Pull all branches without merge commit

git pull --ff-only --all

Pulls all branches from remote and merges only if there are fast-forward changes (that don't require a merge), otherwise fails. If any of the pulls fails you can handle it individually - you need to do that anyway.

Unfortunately git pull --all only does fetch --all and not the merging part on all branches, but only on current branch. There's git-up, but I don't like the automatic rebasing part. I'll keep searching for a solution that fits my needs. A little bash script in .gitconfig alises would be probably easiest.

2. Colored log tree view

git log --oneline --all --graph --decorate --color

3. And alias'em: `git pa` and `git lg`

Edit your user's git config \~/.gitconfig to create shortcuts for the two git commands:

[alias] lg = log --oneline --all --graph --decorate --color pa = pull --ff-only --all

Git cancel pull merge and overwrite local files

I'm usually writing code on two machines: the Intel NUC running Arch Linux at the office and the Windows 8.1 HP Ultrabook at home (the Microsoft Office package keeps me on Windows. I tried Evolution with the Exchange plugin a few days ago on my Arch Linux and it was stalling the machine with my 5GB of mail and I had to cold reboot a few times a day). Good thing PyCharm allows me to use my license on multiple machines.

I have a personal repository on our Gitlab server and I use it to copy code from one machine to another:

1. I'm amending the last commit and push it to server from the machine I've last used:

git add . git commit --amend --no-edit git pus

1. On the second machine I usually just pull:

git pull

This might cause a merge conflict if I happen to have some small and unimportant changes on the second machine. This will always cause a conflict when you're actually overwriting the local history. If I am sure that I can dump the changes on the local machine I can force the pull to overwrote local changes:

# cancel the current merge that was initiated by git pull:
git reset --hard HEAD
# not normally needed since git pull already fetched:
git fetch --all
# use remote code and overwrite local code:
git reset --hard origin

And this little article should save me the googling I do each time I do a git pull. :)

g()()('al')

Interesting programming problem and Python solutions.

Jenkins build triggered from Gitlab

You'll want to trigger automatic builds whenever a commit is made in Gitlab. Here's a quick way to do it:

1. You need to have the Jenkins git plugin installed
2. Enable polling: Jenkins > project > "Configure" > check ">Poll SCM"
3. In Gitlab on the project's "Settings" page go to "Web hooks" and add a new hook by filling this address:

http://<<jenkins server>>/git/notifyCommit?url=git@<<git server>>:<<user or group>>/<<project>>.git

I've selected "Push events", "Tag push events" and "Merge Request events"

You can debug any issues by the "Git Polling Log" link in the left menu from the Jenkins project page.

Merge git personal/master to upstream/master

So you forked the project's repo to your own personal remote repo (in Gitlab/Github/etc.) and you committed your changes to personal/master and now you're trying to merge your commits to upstream/master. (Good idea to work in a branch, and not on master, by the way :)

You'll have a problem because both branches, from the personal and the upstream repos, are called master.

I gave up doing this when working with a single local git directory that has the origin/master branch tracking the personal remote repo and upstream/master branch tracking the main repo because would involve some non-straight forward configs that look hard to debug/maintain.

Another option is to do the merge in another directory. You'd have a separate local repo (besides the personal development repo) that is cloned from the upstream:

git clone git@gitserver:<<theproject>>/<<theproject>>.git .

and:

# add your personal (development) remote repository
git remote add <<adrian>> git@gitserver:<<adrian>>/<<theproject>>.git .
# create a local branch named <<adrian>>-master (avoiding the master name collision!) tracking <<adrian>>/master
git fetch <<adrian>> git checkout -b <<adrian>>-master <<adrian>>/master
# make sure you're on origin/master (main/upstream project)
git checkout master
# and merge your commits into main project
git merge <<adrian>>-master --ff-only
# send the commits to main project remote server
git push

Notes:

• I'm using Gitlab 7.0, but that should matter (it uses the git ssh user for key based authentication). I'm doing the merge from command line because I want to have fast-forward merges (without an extra commit for the merge - this is how the pull request merge works in the Gitlab web interface). Also, my pull request web page is crashing right now, but that's another story.
• replace <<adrian>> with your personal repo name
• same with <<theproject>>
• gitserver is obviously your own git server hostname

More git trickery

Difference between working directory/stage/HEAD

git diff

Shows the changes between the working directory and the index. This shows what has been changed, but is not staged for a commit.

git diff --cached

Shows the changes between the index and the HEAD (which is the last commit on this branch). This shows what has been added to the index and staged for a commit.

git diff HEAD

Shows all the changes between the working directory and HEAD (which includes changes in the index). This shows all the changes since the last commit, whether or not they have been staged for commit or not.

Modify commit history: edit/split/merge commits

Do not modify commits that have already been pushed! Unimaginable nasty things can happen!

There is a guide to splitting commits in the rebase manual. The quick summary is:

• Perform an interactive rebase including the target commit (e.g. git rebase -i "<commit-to-split>^" branch) and mark it to be edited. Note that double quotes are required in Windows, otherwise the caret symbol will be interpreted by cmd.exe by escaping the following character. You can also specify the commit before the one you want to edit, this time without the caret: git rebase -i <the-commit-before> branch
• When the rebase reaches that commit, use git reset "HEAD^" to reset to before the commit, but keep your work tree intact.
• Incrementally add changes and commit them, making as many commits as desired. add -p can be useful to add only some of the changes in a given file. Use commit -c ORIG_HEAD if you want to re-use the original commit message for a certain commit.
• You can also git commit --amend in order to modify a commit, otherwise new commits will be created.
• If you want to test what you're committing (good idea!) use git stash to hide away the part you haven't committed (or stash --keep-index before you even commit it), test, then git stash pop to return the rest to the work tree. Keep making commits until you get all modifications committed, i.e. have a clean work tree.
• You can also abort your rebase if you messed things up completely: git rebase --abort
• Run git rebase --continue to proceed applying the commits after the now-split commit.

Based on: http://stackoverflow.com/a/4307158

Restoring lost commits

Assuming you ran git reset --hard HEAD^ or just messed with rebasing causing you to lose a commit or more. Don't panic. Git should still have your commits in the data store until the next garbage collection cleans it up, unless you ran git gc.

So your commit(s) doesn't show up when you run git log, you should see them when running git flog. Identify in the flog the commit you want to switch to and run:

git merge <commit-hash>

Show file at specified revision

git show 2c624a9:a.txt git show HEAD~3:a.txt git show "HEAD^":a.txt

Django CreateUpdateView

Django is missing a generic class based view that handles both creating and updating a model object. As I was arguing in Ticket #22768 this is a common pattern that would save some code duplication and deserves a place in Django core. Until then here's an implementation:

class BaseCreateUpdateView(ModelFormMixin, ProcessFormView):
    """
    Base view for creating and updating an existing object.

    Using this base class requires subclassing to provide a response mixin.
    """

    object_url_kwargs = ['slug', 'pk']

    def get_object(self, queryset=None):
        if self.is_update_request():
            return super(BaseCreateUpdateView, self).get_object(queryset)
        else:
            return None

    def is_update_request(self):
        """
        Returns True if current request is an object update request, False if it's an object create request.

        Checks if the URL contains a parameter identifying an object.
        Possible URL parameter names are defined in self.object_url_kwargs
        """
        for object_kwarg in self.object_url_kwargs:
            if object_kwarg in self.kwargs:
                return True
        return False

    def get(self, request, *args, **kwargs):
        self.object = self.get_object()
        return super(BaseCreateUpdateView, self).get(request, *args, **kwargs)

    def post(self, request, *args, **kwargs):
        self.object = self.get_object()
        return super(BaseCreateUpdateView, self).post(request, *args, **kwargs)

Django shell magically auto-importing and auto-completing

Getting tired of writing log import lines in Django shell (from myapp.path.models import SomeModel)?

Switching between your code, documentation and Django shell way too often?

Buy this toothbrush and all your worries... uh... no, I mean there's something way smarter than hard working (or typing), it's called django-extensions. Once your run python manage.py shell_plus it will auto-import all your models and some more useful stuff and you get auto-complete (requires pip install ipython). django-extensiosn can do a lot more. Here's a short intro video: http://vimeo.com/1720508 and the docs.

Sexy (bootstrap) forms with Django: comparing options

While working on a new software product I started using Bootstrap. It's a quick way for a programmer to add some looks to his web application. We might end up not using it in production, since designers will have an word on this (I had a chat with a friend that does front end and he says they're not using Bootstrap, since it's cluttered). However, it's still nice to have something else than Times New Roman in the browser while writing code. Django forms can only render as_p (HTML paragraphs), as_table and as_ul (unordered list). Hopefully we'll have a as_div in a future Django version, though if we look at the source code in BaseForm you'll notice that it wouldn't be hard at all to write an as_div method the same way the others are written:

    def as_table(self):
        "Returns this form rendered as HTML <tr>s -- excluding the <table></table>."
        return self._html_output(
            normal_row = '<tr%(html_class_attr)s><th>%(label)s</th><td>%(errors)s%(field)s%(help_text)s</td></tr>',
            error_row = '<tr><td colspan="2">%s</td></tr>',
            row_ender = '</td></tr>',
            help_text_html = '<br /><span class="helptext">%s</span>',
            errors_on_separate_row = False)

But it's best to first search for a better wheel, than to invent a new wheel every time you get some crazy idea about how things should work. So I started to look for options in the open source community.

It's important to note the difference between a software product (or shrinkwrap as Joel calls it) and a web application/site that is deployed just once and/or for a single customer (which is very close to what Joel calls "internal"). There are design decisions right from the start, for instance:

django-crispy-forms

https://github.com/maraujop/django-crispy-forms

Is probably designed for "internal" applications or to be used mostly by python programmers, because if you want your app to be easily customizable by a number of people (most of them not knowing anything about Python and some not even knowing how to program) you wouldn't want to define presentation in Python code like this:

Div('form_field_1', style="background: white;", title="Explication title", css_class="bigdivs")

or like this:

TabHolder(
    Tab('First Tab',
        'field_name_1',
        Div('field_name_2')
    ),
    Tab('Second Tab',
        Field('field_name_3', css_class="extra")
    )
)

Ionel argues that it's DRY. It's true, but I also think web designers will dry up when they need to modify layout by writing Python code  :). He also says that Python errors will be in your face, while writing Django template code will be harder to debug. This is a good point. Crispy forms also seems to have the largest community and is well documented.

Advanced form configuration are done thorough a self.helper = FormHelper() in YourForm class. I wish crispyforms would allow to do more from the Django template in order to touch the Python code as little as possible (or even at all).

django-floppyforms

https://github.com/brutasse/django-floppyforms

Similar to crispy-forms it renders HTML fields through Django templates, which obviously introduces a performance penalty, but that's something you could live with considering you're not rendering 100 forms on each HTTP request. Django templating cache would give a considerable performance boost, by the way. This project seems to have less traction that crispy-forms (if look at Github's Starts and Forks for instance). With floppyforms you derive YourForm class from floppyforms.forms, which is less than ideal. You will get HTML5 form fields out of the box with floppyforms.

formulation

https://github.com/funkybob/formulation

The #django IRC channel suggests this and it seems to be a nice and simple solution. 200 lines also means that you can easily maintain this case the project hangs, but you need to write and maintain your own bootstrap html code. It obviously has much less features out of the box when compared to crispy-forms.

django-bootstrap-form

https://github.com/tzangms/django-bootstrap-form

This is another small project that seems to do its job, just has a lot of unanswered issues and seems somehow hanging.

Conclusion

No conclusion yet; django-crispy-forms looks to be the mature project to choose, but I just don't agree with the design decision to describe presentation (in detail) from Python code and only from Python code.