Posts Tagged ‘Python’

Kiwi PyCon 2013

Kiwi PyCon 2013 was held last weekend at AUT in Auckland.  This was the first time I’ve been able to attend; although I’ve planned to in previous years, the timing never worked out (even this year was very tight).

I missed Michael’s keynote, unfortunately, and also the first session, because it’s also the end of the winter sports season, and so I had to attend prizegiving that morning.  If the conference had been somewhere else, I probably couldn’t have attended at all, but since it was only ~65km from where I live, I could still attend most of the two days.

The first talk I attended was Integrating Javascript with Python Web Frameworks.  This is an interesting topic, and was well presented, but seemed more novice than intermediate to me. Unfortunately, most of my interest in this topic is more on the side of light Javascript integration (i.e. where the bulk of the work is still done on the Python side), and the talk focused on the other area.  However, that wasn’t a flaw in the talk, just a limit in what I got out of it (it seems likely that by the time that I ever do something that’s Javascript-heavy, if I ever do, everything will have changed considerably by then).

Next up was Testing and OpenStack.  I’m only vaguely familiar with OpenStack, and the abstract is very poor, but I don’t like changing tracks mid-session, and testing is of interest, so this was the talk to attend.  Again the presentation was fine, but the talk seemed more novice and less intermediate (some familiarity with OpenStack was assumed, but I don’t consider that means that the talk itself was at an intermediate level).  I didn’t really take much away from this.

The final talk in the session was Python and Creative Data Analysis.  This was a novice talk, and for someone at novice level, probably well worth attending (it was presented quite well).  For someone not a novice, even someone with nearly no matplotlib experience (like me), there wasn’t much to be gained.

For the final session of the day, I started with Dissecting Memory Mysteries of Python.  I had a lot of hope for this presentation, because this is an area of interest for me, and practical advice would be really useful for work that I do.  It was labelled as an intermediate talk, and I would say that was accurate (but I was probably hoping for more of an experienced level talk).  The presentation was ok, but suffered from having two speakers, one of whom spoke too quietly.  The talk did cover some of how memory management works in Python, but concentrated too much on reference-cycles for my taste (which I find in practice is  never the problem when short on memory), and their practical suggestions for improving memory usage weren’t really that good.  For someone with no previous understanding of how Python manages memory, this would probably have been a good talk.

The final talk of the day, Using Cython for distributed-multiprocess steganographic md5sum-collision generation. For… reasons, was the best of the conference.  The speaker was excellent – far superior to most of those at the conference, and the talk was entertaining and somewhat informative (mostly entertaining, but that’s exactly what it claimed to be, and those are often the best talks at a conference: pure information is better found elsewhere).  The only negative was that the abstract was pretty poor – it doesn’t really reflect what the talk was about at all.  If the conference had 5 talks of this quality, I would have no hesitation towards attending again.

The second (or third, if you count the tutorials) day was opened with a keynote by Jessica McKellar.  Jessica is a very good speaker, and the topic was suitable for a keynote, especially from someone with her background.  Essentially, she was advocating increasing involvement with and usage of Python, and going over what she saw as the key barriers to entry at the moment, and therefore what should be targeted in order to help Python remain popular (her brief explanation of why popular is important was well done).  Although I certainly agree with her central tenet (everyone needs to do a couple of small things and that’s enough to make a significant difference, and that difference is important), I don’t agree with all of the examples she gave.  Her four points were Windows, games, mobile, and science (the latter being an example of something that is working well; i.e. where we can copy ideas from and remain strong).

I started with Python on Windows, although I almost never use it any more.  There are some difficulties, although I think that if you’re the sort of person for whom programming is a good match, you can work through the issues without much difficulty – the challenge would be more with ‘casual’ programmers – e.g. a scientist that could use Python to do something, but isn’t really a developer.  Most of the challenges here I think can be addressed by documentation – e.g. the Python Tools for Visual Studio are excellent, fairly simple to use, and familiar to a lot of Windows developers.  All that they need is more people pointed towards them.  I don’t know enough about the casual programmer to know how they get started, but it seems like better documentation would help them too.  However, the core of my disagreement here is that I’m convinced that Windows use is imploding – this is going to happen more rapidly than people realise, and so putting a lot of effort in here is not a good use of resources.  It would be far better to target mobile instead.

Although Python isn’t necessarily the first choice one would make for games, I think the examples that Jessica gave were really just a re-hash of the Windows issues, and those will also just go away as fewer games are developed on/for Windows, in favour of mobile devices and possibly consoles.  I would argue here that again the resources would be better spent on making sure that developing games on the web, iOS, and Android is as straightforward and well-supported as possible.

Her last two points, mobile, and science, I do generally agree with, although I think she undersold the mobile issue.  For example, there are many people that will only be using a mobile device – they need to be able to actually develop on iOS or Android (or maybe Chrome OS), not just build for it.  There are some tools for this – some that are really pretty good, but more would be better.

The first proper talk of the day was Exploring Science on Twitter with IPython Notebook and Python Pandas.  Again, this was labelled as intermediate, but really was novice.  It was an ok introduction to IPython Notebook in particular, but not a lot more than that.  For a novice, or someone that had never heard of IPython or Pandas, probably valuable.  For others, not so much.

The second talk of the session was From nothing to daily use: How Python changed my work life.  This was correctly labelled as novice, but was by another excellent speaker.  There wasn’t much to take away in terms of Python development, but it was a great insight into non-programmers (sort-of) coming into Python, and how Python is in use at a central New Zealand organisation.  Again, if there were five talks of this quality, then PyCon NZ would be an easy recommendation.

The final talk of the session was Advanced methods for creating decorators.  This was correctly labelled as experienced (one of only two at the conference), and was exactly what it claimed to be.  The speaker read the presentation, which is never great (I’m not sure what I got from it that I wouldn’t get from reading an essay), but otherwise it was informative and well done.  More talks like this, especially if they were better presented, would be excellent.

After lunch I attended Publishing Theology with Python, again labelled as intermediate but actually novice.  For a novice talk, it was fine.  I was hoping for more practicalities, and more focus on getting Word documents to ReST (when actually that step is done by a raw copy and paste and manual markup), but it was moderately interesting.

Prior to afternoon tea were the lightning talks (around 12, I think).  These were generally very low quality – there were a couple that were worth listening to, but many that I would rather have skipped, and some where the speaker was both not good at speaking, and didn’t really appear to have anything to say.  Too many people used it as a “share what you are doing” opportunity, rather than something that would be of interest to the group.

The final session of the day started with Computational Advertising, Billions of records, and AWS – Lessons Learned.  This was listed as experienced, but was really intermediate (at best).  A poor abstract (just throwing together a lot of words, really), but an ok talk.  Either more practicalities (exactly how were things done), or being much more general would have improved this, rather than the middle ground that was taken.

The last session of the conference was the third-best: Testing for Graceful Failure with Vaurien and Marteau (actually Loads).  This was accurately intermediate (although on the novice side rather than the experienced side), and very well presented.  The two tools sound interesting, and definitely something that I’ll be checking out (these are the only two tools that I learned about at the conference that sound worth investigating).  It would be great to see more talks like this.

There’s not much to say about the rest of the conference – it was fairly typical (venue, food, etc – although the swag-bag was very light).  There were a lot of giveaways (t-shirts, books, licenses, etc) that took a lot of time and were a bit odd.  I don’t really understand why people would applaud someone who was just randomly selected to get some free stuff, or why the giveaways needed to be drawn out so much.  In general the organisers did a good job, though – it’s not really their fault that the quality of talks (and speakers) wasn’t as high as it could be (unless they turned down the good talks, which is unlikely).  It’s possible that I missed some great talks – I’ll be watching the videos once they are available (most of them, if not all).

Two issues that should be addressed for next year are the “audience level”.  Of the talks I saw, there was only one that was actually experienced, four that were actually intermediate, and six that were actually novice.  Many of the novice talks (and one of the intermediates) claimed to be higher.  The first problem is that the level need to be accurate, so that people can go into a talk with a realistic expectation of what’s required.  The second problem is that this skews far too far towards novices – there needs to be more experienced talks in particular, and more truly intermediate ones as well.

Overall, I wasn’t particularly impressed.  Since the conference was close to home, it only cost me around $400 (travel, registration, food), which is very cheap as far as conferences go, but could instead pay for a new iPad Mini (which I’d use for much more than a weekend), or several books, or licenses, or various other things where I think I would get better value.  If I was looking for work (or workers), or for collaborators on an open-source project, then I can absolutely see value.  However, I’m not doing any of those right now.  If the conference had more experienced-level talks, more inspiring talks, and more great speakers, then I would absolutely attend again.  If I was a novice, or low-level ‘intermediate’, then I would certainly see value.  The venue for 2014 wasn’t announced (Dunedin was two years ago, and they tend to alternate, so I would guess that Christchurch and Wellington are the two most likely, although Hamilton might be a contender), but as things are I’m not really sold on attending again, unless the speaker line-up looks particularly good, or something else changes (e.g. if I am looking for workers).

An option that I would be interested in, but I doubt will be done, is a “green” option for attending.  I would pay a discounted registration (90%?) to get access to live streaming video of the presentations and a posted t-shirt (or even the whole conference bag).  I can still interact with other attendees in real time (via Twitter, ADN, etc – questions can also be handled this way), if not to quite the same level, get much the same out of the conference, and save a lot of the cost (e.g. even this year with it so close to where I live, I would still save 75% of the overall cost), and be much more environmentally friendly (i.e. reducing wasteful travel).

PyPad

PyPad is a Python interpreter for iOS.  This sounds incredibly exciting, right – finally I can do proper development and run Python programs on iOS!  Unfortunately, that’s not really the case (mostly due to Apple’s restrictions).

PyPad lets you create multiple modules and execute each of them.  However, only a subset of the standard library is available, and there’s nothing iOS-specific available (so you can’t access the camera, or touch information, and so on).  Getting code in and out of the app is done via copy and paste.  The standard keyboard is provided, with start/pause/stop buttons.

I keep the app installed so that I can (via AirPlay mirroring) demonstrate simple Python snippets.  However, if I have an Internet connection available, then I can do that in Prompt (ssh’d to a server that has Python installed) much more elegantly.

The app is clearly limited by Apple’s restrictions as to what’s acceptable for iOS.  However, it does seem like it could do much more (e.g. see Codea) if more of the standard library was available (this would mean rewriting chunks, I presume) and if there were special iOS-specific modules available for accessing things specific to the device (especially for accessing touch and graphical output).  It could accept .py and text files from other applications, making it easy to get code in (e.g. from Dropbox) and share files (as text) – although perhaps that crosses Apple’s boundary for what’s ok.  It would be nice to include the Python documentation, too (I have a separate app for this, but it makes sense to have it in once place).

The app is only $2, so if you’ve any interest in Python on iOS, then I’d recommend buying it to have a look and to encourage more development.  You probably won’t end up using it that much, however.

Trac to iCal

One project I work on uses Trac and has a custom “due date” field (it doesn’t really have milestones – updates are more granular).  While this is useful, one problem is that I don’t check this Trac instance every day, and so sometimes I’ve missed deadlines because I haven’t noticed that they are due.  However, I do check my calendar every day (multiple times a day).  It seems like exporting this “due date” value into my calendar will help with this.

This little script creates a calendar that can be subscribed to.  I run it once a day (due dates don’t change very often), and have iCal set to update once a day, so it should work fine.  If it’s of use to you, use it (no restrictions).  Let me know if there are things that can be improved!  It requires the Python iCalendar module, which I already use for parsing public holiday data.

#! /usr/bin/env python

import sqlite3
import datetime

import icalendar

cal = icalendar.Calendar()
cal.add("prodid", "-//PROJECT NAME Trac Deadlines//trac.tonyandrewmeyer.com//")
cal.add("version", "2.0")
cal.add("method", "publish")
cal.add("x-wr-calname", "PROJECT NAME Ticket Due Dates")
cal.add("x-wr-caldesc", "Due dates for PROJECT NAME tickets")
db = sqlite3.connect("/trac_location/db/trac.db")
c = db.cursor()
c.execute("select t.id, t.owner, t.summary, c.value from ticket t, "
           "ticket_custom c where t.id=c.ticket and t.status!='closed'")
for ticket_id, owner, summary, due_date in c.fetchall():
    if not due_date:
        continue
    due_date = datetime.datetime.strptime(due_date, '%d/%m/%Y')
    due_date = datetime.date(due_date.year, due_date.month, due_date.day)
    event = icalendar.Event()
    event.add("summary", "PROJECT NAME #%s (%s): %s" % (ticket_id, owner, summary))
    event.add("dtstart", due_date)
    event.add("dtend", due_date)
    event.add("dtstamp", datetime.datetime.now())
    event.add("url", "https://trac.tonyandrewmeyer.com/ticket/%s" % ticket_id)
    event["sequence"] = datetime.datetime.now().strftime("%Y%m%d%H%M")
    event["uid"] = "ticket_due_%s@trac.tonyandrewmeyer.com" % ticket_id
    cal.add_component(event)
c.close()
db.close()
open("/var/www/subfolder/project-name-trac-due.ics", "wb").write(cal.as_string())

D520 Week Three – 2010

Last year Chapter Four of IronPython in Action was covered over two weeks (the lab is also a two-part exercise), and I felt that worked fairly well, so kept the same plan for this year, although the exact parts that were covered each week changed.  As usual, the students received notes [PDF], and a lab exercise [PDF], and two recommended reading items, both by Brent Simmons: one on how improving quality is non-linear, and one on how your own code is always improving.  The notes again cover the textbook, key points, and example code (although most of the example code is MultiDoc, so just links to the online copy. Continue reading

D520 Week Two – 2010

No radical changes from either last year’s week two or last week.  In a way, this is the real first week – in the previous week we learn about the course and about what IronPython is (and remember how to program in Python), but we don’t do much more than that.  In the second week, we really get into doing some actual IronPython programming.  I gave the students notes [PDF], and the first assessed lab exercise [PDF], and the recommended reading was two Joel Spolsky posts: one on (IT) customer service and one on how hardware improvements impact software design.  The notes are again in three sections: textbook chapters (this week is Chapter Three, a fairly essential introduction to .NET and IronPython), key points, and example code (from Chapter Three).  The lab is essentially the same as in previous years. Continue reading

What happened to the 2009 IPy Notes?

Around this time last year, I started posting weekly reports on the IronPython classes I was teaching at Northtec, including copies of the material that I was giving the students.  These stopped abruptly around the middle of the course.  The course did complete (and was fairly successful – I think the change to IronPython was definitely for the better), and I did continue making notes as I taught.  However, I didn’t manage to get them online.

On the 10th of October 2009, my father passed away (very unexpectedly, of a heart attack the day before).  As a result, I didn’t have as much extra time as I had anticipated, and I didn’t really have the energy/motivation to post the notes.  (The classes continued, except the week of the funeral, where the hours were halved).

The course began again last week (21st July 2010), and I’m going to try this again.  As I go, I’ll complete the 2009 notes as well (some/all of the links in the 2009 ones are broken, which I’ll fix, and when I get to where the notes stopped, I’ll post the notes that I took last year, although these will be less exhaustive than if I had done them at the time).

No promises, but I expect that I’ll be able to post throughout the entire course this year, and I’ll again make the material that I use available (you may use it under a Creative Commons license if you wish, although much of the material is references to IronPython in Action).

I had planned to present a short summary of the experience of using IronPython in this way at PyCon NZ, but I’m again unable to make it this year (for happier reasons – a close friend is getting married that day); perhaps next year!  I’m still interested in hearing from anyone else that’s using IronPython in the classroom.

More notes coming soon

I am still putting together the weekly D520 notes. However, the last couple of weeks have been a bit busy and so they’re waiting to be cleaned up and have links added. The intra-semester break starts next week so I’ll catch up then.

D520 – Week Six

Chapter 6 of IronPython in Action covers “properties, dialogs, and Visual Studio”.  This seemed an obvious place to insert the material on user-interface design that is normally covered in the course, and to look a bit more deeply than the textbook does at Visual Studio itself (and the Windows Forms controls and their properties).  I only scheduled a single week to cover this, but I suspected that it might take more than one (I left an empty slot in the schedule to cover one such over-run), and that was, indeed, the case.  The students received notes [PDF], slightly longer this week (covering the UI design material not in the textbook, as well as the usual chapter summary, key points, and examples, and the steps required to install IronPython support in the ‘Experimental Hive’ Visual Studio SDK), and a fairly simple lab exercise [PDF]. Continue reading

Mailbox flattening messages

The mailbox.py code does this to dump a message (e.g. when adding to a Maildir file):

def _dump_message(self, message, target, mangle_from_=False):
    # Most files are opened in binary mode to allow predictable seeking.
    # To get native line endings on disk, the user-friendly \n line endings
    # used in strings and by email.Message are translated here.
    """Dump message contents to target file."""
    if isinstance(message, email.Message.Message):
        buffer = StringIO.StringIO()
        gen = email.Generator.Generator(buffer, mangle_from_, 0)
        gen.flatten(message)
        buffer.seek(0)
        target.write(buffer.read().replace('\n', os.linesep))
    elif isinstance(message, str):
        if mangle_from_:
            message = message.replace('\nFrom ', '\n>From ')
        message = message.replace('\n', os.linesep)
        target.write(message)

(There’s a bit more that deals with other types of message).  Unfortunately, with some messages containing Unicode characters, this breaks with a UnicodeDecodeError:

Traceback (most recent call last):
  File "", line 1, in
  File "/usr/lib/python2.5/mailbox.py", line 245, in add
    self._dump_message(message, tmp_file)
  File "/usr/lib/python2.5/mailbox.py", line 203, in _dump_message
    buffer.seek(0)
  File "/usr/lib/python2.5/StringIO.py", line 106, in seek
    self.buf += ''.join(self.buflist)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd6 in position 106: ordinal not in range(128)

The message does flatten with message.as_string() – or str(message) – without any problems.  I’m not really sure why mailbox.py doesn’t just use as_string(), rather than create a generator itself.  I’m also not totally sure where the error is coming from, especially since it happens inside of seek().

For now, I can just call mailbox.add() with msg.as_string(), rather than with msg directly, and it’ll work fine.  At some point, if I have time, I’ll revisit this and try and figure out if it’s a Python bug that should be reported.

D520 – Week Five

Chapter 5 of IronPython in Action deals with XML, although it starts out covering some of the more advanced things you can do with functions.  I considered skipping this chapter (the function material is perhaps a bit advanced, and covering XML isn’t a necessity), but decided that it was worth learning about XML in .NET (since it’s so common) and that it would make using the MultiDoc example tricky (since the file format is XML) and I really wanted to use MultiDoc.  I gave the students notes [PDF], again covering the textbook material that we could look at, the tools (unchanged), key points, and a link to the MultiDoc example code.  We had a new lab exercise [PDF] this week (implementing Conway’s Game of Life), as well as two new recommended reading articles: a Lukas Mathis post about preferences and another Spolsky post, this one about “architecture astronauts”. Continue reading