I should probably preface this with saying that I’m probably going to come off sounding like a total hater. That’s not entirely untrue, but don’t take it personally, I just hate a lot of stuff.

I started writing out a whole long thing talking about every talk I saw, but I grew tired of that by about saturday morning, so here are just some of the highlights.

Friday started with keynotes and whatnot. It’s nice to hear Guido talk and all, but whatever, nothing too interesting here. First talk was Max Ischenko’s Simple WSGI Composition. I thought the title here was a little misleading; the talk was very Pylons-focused. I’ve never used Pylons myself, and I still don’t particularly want to, but it’s good to see how the other half lives now and then.

Next was Daniel Greenfeld’s A Short Pinax Tutorial. I don’t think I “get” Pinax. I was hoping that this talk might show me how to do what I wanted with it, but now I’m feeling more like the usage I want is not part of their agenda. I understand that, I’m all for opinionated software, I just like it better when they’re compatible with my opinions. Brant’s question afterwards pretty much summed it up for me: “Why is Pinax a thing?” I also feel that all the stuff they provide above and beyond just a collection of reasonably isolated apps just gets in the way.

I wish I had gone to the VisTrails talk.

Grig’s Creating RESTful Web Services With Restish was quite a bit better than I expected. I’ve seen a couple REST talks, and they’re usually pretty awful, essentially boiling down to someone slowly repeating “nouns and verbs”. I thought Grig delved a little deeper into the right things, and gave some nice comparisons to xml-rpc. I don’t remember much more specific, I’ll probably watch the video again, though.

Saturday’s keynotes were a little better. I’m not entirely proud of it, but my brain pretty much shuts down whenever anyone starts talking about IronPython. I just can’t bring myself to care about it at all. PyPy, however, is some voodoo space shit from the future and it scares the hell out of me.

My first talk, Demystifying Non-Blocking and Asynchronous I/O was probably my favorite of the entire conference. Peter Portante had some solid slides, and did an excellent job of explaining something I knew nothing about in a way I could easily follow. I highly recommend this one.

David Beazley’s Understanding the Python GIL was probably the most hyped talk of the weekend, and it did not disappoint.

The middle of the day saturday was a little weak, I sat it out for a little while, and worked on a little project for later. I did go see Titus’ Continuous Integration and Ned Batchelder’s Testing talks, though. Both were very strong, and relevant to my interests. Titus is a great speaker, and gave probably this year’s most entertaining talk for me at least. Ned was also wearing one of my company’s nosetests shirts, which marks him as a person of quality.

Saturday night was the most important event of the weekend, the Testing In Python BoF meetup. What started last year as just a quiet little meetup for a quiet mailing list somehow grew into a monster. We were packed into the largest open space room, and people were even standing out in the hall. I feel a little bad asking this, but who are these people? The TiP BoF had probably the highest concentration of “big names” of any open space, I get the feeling that they attract a lot of “groupies” who figure that anything Michael Foord or Jesse Noller is going to is something they should be at too. I suppose they’re not necessarily wrong in thinking that, but it does make getting enough beer a problem.

Sunday morning, I went to the Eventlet talk. It sounded like some really cool stuff, and I hope I can think of an excuse to use it. After that was the two Mercurial talks. The first one by Dirkjan Ochtman wasn’t quite everything I was hoping for. It seemed a little too aimed at people that didn’t know how DVCS worked, where I was hoping for something that actually delved into the code, showing how Mercurial specifically made it work. Ironically the next talk, Hg vs. Git had a bit more of that, and was very good. I still don’t know how to actually do anything in Git, but at least I’m thinking it might be possible to learn.

So yeah, Pycon is awesome.

/Library/Python/2.6/site-packages/setuptools/command/install_scripts.py:3: UserWarning: Module pkg_resources was already imported from /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/pkg_resources.pyc, but /Library/Python/2.6/site-packages is being added to sys.path
      from pkg_resources import Distribution, PathMetadata, ensure_directory
/Library/Python/2.6/site-packages/setuptools/command/install_scripts.py:3: UserWarning: Module site was already imported from /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site.pyc, but /Library/Python/2.6/site-packages is being added to sys.path
      from pkg_resources import Distribution, PathMetadata, ensure_directory
warning: no previously-included files matching '*.*' found under directory 'docs/_templates'

I try to be good about using virtualenv and pip and whatnot, but I probably slipped up somewhere. Either way, I finally got tired of seeing those warnings all the time and actually read them. Throwing caution to the wind, I deleted /System/​Library/​Frameworks/​Python.framework/​Versions/​2.6/​Extras/​lib/​python/​pkg_resources.pyc and everything seems to be working. hth

This would not do at all. I use virtualenv (and virtualenvwrapper) pretty religiously, and am loathe to install much of anything in the system-wide site-packages. I needed to get things to a point where a straight-up easy_install lxml would work. I didn’t wanna be messing around with custom install flags or whatever every time I cut a new env.

This is not a neat step-by-step guide for getting lxml working in OSX. This is just a record of some stuff that I saw, and some things that I did. Some combination thereof was sufficient to get things working.

For the record, I’m using lxml 2.2.2, libxml2 2.7.5, and libxslt 1.1.26.

One day, I fired up python, typed from lxml import etree and got an error much like the following:

>>> from lxml import etree
Traceback (most recent call last):
  File "", line 1, in 
ImportError: dlopen(/Users/nwilliams/.virtualenvs/lxml1/lib/python2.6/site-packages/lxml-2.2.2-py2.6-macosx-10.5-universal.egg/lxml/etree.so, 2): Symbol not found: _xmlFree
  Referenced from: /Users/nwilliams/.virtualenvs/lxml1/lib/python2.6/site-packages/lxml-2.2.2-py2.6-macosx-10.5-universal.egg/lxml/etree.so
  Expected in: dynamic lookup

>>>

Needless to say, I was less than pleased.

Looking though the source, I gathered that _xmlFree was a symbol exported by libxml2. My first thought was that lxml was somehow compiling against the old system version of libxml2, or for some other reason couldn’t find the new version.

I noticed a few funny things about the output when installing lxml. First, there were two lines right before all the heavy compilation work:

Using build configuration of libxslt 1.1.26
Building against libxml2/libxslt in the following directory: /usr/local/lib

The second line suggested that lxml was, in fact, seeing the version of libxml2 that I wanted it to. The first was a problem though, because normally, it looks like this:

Using build configuration of libxml2 2.7.5 and libxslt 1.1.26

For some reason, lxml wasn’t finding xml2-config. Doing export XML2_CONFIG=/usr/local/bin/xml2-config seemed to fix things.

The other issue was probably the important one. While compiling lxml, I would see a bunch of warnings that looked like this:

ld: warning in /usr/local/lib/libxml2.dylib, file is not of required architecture
ld: warning in /usr/local/lib/libxslt.dylib, file is not of required architecture

Also for another file or two, and all repeated a few times.

Checking file /usr/local/lib/libxml2.dylib told me that the files in question were only compiled as i386 binaries. I eventually found this page, which showed me how to compile libxml2 and libxslt as universal binaries. I did modify things somewhat, though, most notably adding 64-bit architectures.
The configure command I used for libxml2 was:

env CFLAGS="-arch i386 -arch ppc -arch x86_64 -arch ppc64" ./configure --enable-static=no --without-python --disable-dependency-tracking

And for libxslt:

env CFLAGS="-arch i386 -arch ppc -arch x86_64 -arch ppc64" ./configure --disable-dependency-tracking

The CFLAGS bit is necessary, that’s what’s making things work. I don’t actually know exactly what the point of --enable-static=no is. I’m sure it’ll come back to bite me in the ass at some point. --without-python sounds scary, but since lxml is actually an alternative to the gross “real” bindings, we don’t care about them. I also have no idea what --disable-dependency-tracking does, but make will fail without it.

Once I had these universal libraries built and installed, easy_install lxml worked like a charm.

I like when people talk about how the things I do at work are so amazing, essential, and hard to find people to do.

I started this job about a month and a half ago without out really knowing much about Python or testing. I was familiar with BDD and using rspec in dinky little Rails apps, but the function test side of things, as well as all the concerns that come with automation across multiple sites were unknown to me.

We pretty much exclusively use twill and nose for our tests, and I really don’t want to think how painful things would be without them. I might actually have to use watir or something…

I know if I actually go on and talk about how my company actually seems to think testing is important and part of the process then Terry will laugh at me, but I suspect that even just my experience so far has spoiled me for life.