Planet Zope

        Zope related news
 

Jan-05

---

This one, to be precise, and it includes nginx, Varnish, load-balanced ZEO clients, log rotation and more goodness.


Jan-02

Bootstrap.py versus pkg_resources.py

I’ve been using zc.buildout quite a bit over the past month.  Although it has been working, it has been doing strange things like using the wrong version of zope.interface.  Yesterday I finally figured out why, and today I found a possible solution.

It turns out that Ubuntu (8.10) provides a package called python-pkg-resources.  At least one Ubuntu package (Snowballz, a strategy game written in Python) pulls in that package automatically.  It installs a pkg_resources module in Python’s site-packages directory, but it does not install the rest of setuptools.

I can understand why Ubuntu chose to split up setuptools, but that choice causes havoc for the bootstrap.py module people use to install zc.buildout.  Here is what bootstrap.py is supposed to do:

  1. Download ez_setup.py and run it.
  2. ez_setup tries to import the pkg_resources module, but fails.
  3. The setuptools package is not found, so ez_setup downloads setuptools in a temporary directory.
  4. ez_setup alters sys.path to include the new setuptools package.
  5. bootstrap.py imports the pkg_resources module from the version of setuptools just downloaded.
  6. Ask pkg_resources about the installed setuptools package.
  7. Use setuptools to install zc.buildout.

Here is what bootstrap.py actually does when pkg_resources.py exists in the site-packages directory (differences emphasized):

  1. Download ez_setup.py and run it.
  2. ez_setup successfully imports the pkg_resources module from site-packages.
  3. The setuptools package is not found, so ez_setup downloads setuptools in a temporary directory.
  4. ez_setup alters sys.path to include the new setuptools package.
  5. boostrap.py continues to use the previously imported pkg_resources module.
  6. Ask pkg_resources about the installed setuptools package.
  7. pkg_resources does not find setuptools because pkg_resources does not notice the change to sys.path.  bootstrap.py fails.

At first, following ideas I gleaned from various posts about zc.buildout, I worked around this by deleting the setuptools egg and the pkg_resources module from site-packages.  I didn’t know exactly why this helped until I studied the problem.  It turns out that bootstrap.py was just not written to cope with a system-wide installation of pkg_resources.

Now I think I recognize another bad choice that zc.buildout has been making.  zc.buildout generates a “bin” directory full of Python scripts.  Those scripts prepend egg directories and egg zip files to sys.path before doing their work.  I noticed that sometimes the list of paths to prepend includes “/usr/lib/python2.5/site-packages”, which is already on sys.path.  I now suspect that whenever zc.buildout includes paths like that, it’s wrong, and the cause is a mixup involving a system-wide installation of pkg_resources, setuptools, or some other foundational package.

Here is a possible way to fix bootstrap.py.  Just before the “import pkg_resources” line, add this:

del sys.modules['pkg_resources']

This solved the bootstrap.py problem for me.  Altering sys.modules is rarely a good idea, but this might be a good exception to the rule.  I don’t believe we need to catch KeyError because ez_setup should have imported pkg_resources already.

Beyond this, there is probably more work to do to make zc.buildout produce correct scripts.

Whoever said computers behave logically must have been joking or delusional.  The people who provide the software never fully agree with each other–nor even themselves!



Dec-31

plone.org web site upgraded

Feed: Plone News
The plone.org web site has been upgraded, and is now running the upcoming version 3.2 of Plone.


Dec-30

Choosing a distributed revision control system


What’s nice about Subversion is that it’s easy enough for most people to wrap their head around it and therefore it’s supported well. Admittedly it can make branching a pain. Actually, that isn’t quite true. It’s a bit like speed which itself doesn’t kill you, just the sudden impact does. Likewise he problem is not with branching, it’s the merging that can make you pull your hair out. But it’s gotten better at that over the past releases.

Another issue with Subversion is the central repository. I however think the central repository idea fits many projects or even individuals in need of revision control very well. And if you want to mirror a repository for bandwidth or high-availability reasons, well that’s possible too since version 1.4 or so. I know, these mirrors have to be read-only, otherwise it’ll easily get messy. If you want to be able to commit to some local mirror and push back your changes, you should consider SVK. It mirrors foreign repositories locally, lets you create local branches and merge them back. SVK has a few problems of its own, but I’m not going to get into them here. Point is, if your project works best so that there’s a central repository, Subversion is a sensible choice. Thanks to solutions like SVK, people will also be able to work offline (I’ve used it this way) or be able to follow a project and add their modifications without leaving their cave.

Distribution is not for everyone

Since lately people have been trying to convince us that central repositories are not the way of the future, and neither is Subversion because it’s not suitable at all in a distributed environment. But in all honesty, after many years of contributing to various Open Source projects I haven’t actually had the need for such an environment. I get Linus’ points about how the Linux Kernel is developed, how he receives patches from his lieutenants who in turn receive them from somebody else. Surely all that needs a distributed system. But I’m sorry, the projects I’ve worked on just don’t have the man power to have people do nothing else but review patches sent to them, merge them with their private branches, push them along to other people, etc. In our central repositories we’re happy with having a few knowledgeable people watch the commit list (we call them the “checkin police” in Zope) and make sure that code and patches committed as well as the log messages match our quality standards. For everything else there’s buildbot.

All that’s not to say that distributed revision control systems aren’t nice because they are. I do understand why their users are smug. But I just won’t have their arrogance. Subversion has served many of us well over the years and now all of a sudden we’re idiots if we still like it?

That said, all distributed systems can do what Subversion can do (except partial checkouts of the repository apparently) so they seem worth a look. After all you get more features and no drawbacks, right?

Sorting out the contenders

There seem to be three serious options when it comes to distributed version control that all have about the same feature set: Bazaar, Mercurial and Git. Actually, that’s not quite correct. If you’re like me and have to or want to work with several Subversion repositories, Mercurial isn’t an option. As nice as Mercurial may seem (though a bit weird in its understanding of branches), you’ll have to realize that only Bazaar and Git have decent Subversion plugins that allow you to pull and push to a Subversion repository.

Both Bazaar and Git are installed easily on OS X if you have MacPorts. Just beware that if you want to build Git with SVN support, you should install it as follows:

sudo port install git +svn

For the following tests I took one of my private Subversion repositories (the smallest one in which I keep all the files related to my work at the university) and tried interacting with it from Bazaar and Git.

Bazaar

The first thing you’ll hear about Bazaar is that while the documentation is pretty good, it’s slow. And boy they aren’t lying. I haven’t done any measurements, but it felt even slower than Subversion on operations like printing the status of the working copy.

Like with Mercurial, Bazaar’s command set is quite easy to grasp for people who’ve been brainwashed by Subversion. In other words, people like me. However, I can’t help but think that’s because Bazaar isn’t actually too far away from Subversion, conceptually speaking. Sure, it has local repositories and all that, but in essence it seems to be geared towards a central repository when it comes to sharing your work. Not that that’s a bad thing, as I tried to point out above. I just somehow expected more. For instance, let’s say you have a checkout of something. This checkout can only be bound to one branch in a remote repository at a time. That means you can’t push some work to several repositories at the same time.

Like SVK, Bazaar has the concept of a working copy directly associated with a remote branch and working copies that represent local branches. If you have one of the former, the unbind/bind feature is quite neat. It tells Bazaar to temporarily stop sending every commit to the remote repository (e.g. while you’re hacking away on the train). Once you’re back with network connectivity, you rebind to the remote branch and can push your changes. Unfortunately, Bazaar wants you to push all these changes as one revision (to Subversion) even if you made several commits when offline. I’d rather have it reflect the individual commits.

Another downside in the communication with a Subversion repository is that it leaves turds in the repository, that is special directory properties that it uses to track which revision it has synced. In this respect it’s similar to SVK.

Git

Having read tutorials and guides a la “Git for SVN refugees”, I must get the impression that Git is surrounded by a cloud of fanboyism. Fanboyism per se is tolerable, but as I said above, I don’t like when it’s mixed with arrogance. I know that Subversion isn’t the bee’s knees, that’s why I’m reading this tutorial. You don’t have to tell how stupid I’ve been using Subversion all along and not helping Linus come up with Git.

That said, once you look past the fanboyism, you’ll realize Git is actually quite well composed. The commands are a bit weird at first but so far each one has done exactly what I expected it to do. Its concept of remote and local branches is absolutely easy to understand and since Linus designed it to do kernel development, you can easily manage a gazillion local and remote branches, do merges between them, etc. I’ll admit that it feels a bit weird in the beginning, but you’ll soon appreciate the niftiness.

Something that definitely takes getting used to is the way it represents branches. A checkout and a repository are inseparably the same thing, therefore switching between branches happens within the same checkout. I’m not yet entirely sure yet what to think of that, all I know is that you might easily forget which branch you’re currently in and do something to a branch that you meant to do in another one. That’s not a big problem with Git, though, because you can easily roll back commits. What is annoying, however, is that you can’t switch branches or rebase your changes on top of the latest changes from the repository you’re tracking (e.g. SVN) while having local modifications. I tend to keep local modifications in my working copy almost forever, for instance when I have a canonical version of a configuration file in the repository and I change it locally for a test installation. Git has ways around that annoyance, too, for instance I could use git stash to hide the local modifications temporarily, or I could make a local branch in which I can check in the modifications but never push them back to the tracked repository, just pull the latest changes.

Git’s Subversion integration is superb. There’s an excellent tutorial for people who’ve deserted from Subversion/SVK to Git. It also mentions how to interact with a Subversion repository. In fact, generally you read that Git’s supposedly not as well documented as, say, Bazaar. I can’t come to that conclusion. I’ve rarely needed the online user guide, the man pages are quite well sorted out. You’ll actually see them by either typing man git-cmd or git cmd --help.

The winner

So which is it going to be? Well, despite Bazaar’s Python bonus and Git’s initial weirdness, I’ve gone with Git for now. On the server side I’m keeping my Subversion repositories, at least for now. Because at some university machines or on Windows I only have a Subversion client (I suppose I could compile Git myself, not that the uni sysadmins like seeing such a thing). Also, I’ve set up my Subversion repository access via HTTP/HTTPS. That means I can view my repositories with a simple web browser or download the HEAD with wget if I don’t happen to have a Subversion client at hand at all. Finally, keeping Subversion around gives me the possibility to change my mind again and go for something else.

      


Dec-28

RelStorage 1.1.1 Released

For those just tuning in: RelStorage is an adapter for ZODB (the database behind Zope) that stores data in a relational database such as MySQL, PostgreSQL, or Oracle.  It has advantages over FileStorage and ZEO, the default storage methods.  Read more in the ZODB wiki.

Download this release from PyPI.  As I mentioned in another post, this release works around MySQL performance bugs to make it possible to pack large databases in a reasonable amount of time.

If you are upgrading from previous versions, a simple schema migration is required.  See the migration instructions.



Dec-24

Practical Plone 3, a new Plone book for beginners: pre-order now!

Feed: Plone News
The long-awaited introductory-level book for Plone 3, "Practical Plone 3," is on its way to the printers and will be hitting bookstore shelves in January.


Plone 3 trainings for you!


Developing for Plone is a daunting task to start, there is a lot of things to know. If you are looking into using Plone to develop your company website, or you are already a web developer and have customers asking about Plone, or if you have any other reason to get into Plone, you will have a lot of use of starting by taking a course.

If you are in North America, Nate Aune of Jazkarta has one session in Montreal in the mid of January. And if you are in Europe, Jörgen Modin has one in London in the beginning of February.

Yes, I’m good friends and work with both, so I’m biased, but they have both gotten rave reviews, so I’m not worried about recommending them.

Posted in plone      


XML   Syndicate
» ZopeNews rss1.0
Feeds   recent Feeds
01-05 01:50 Martin Aspeli
01-02 19:33 Picklebytes Zope
12-31 10:37 Plone News
12-30 11:39 philiKON - a jour
12-22 21:56 Lennart Regebro: ...
12-19 18:23 Zope.org
Feeds   recent PyPI
01-06 08:42 Products.PlonePopoll 2.7.0-beta2
01-05 23:56 Products.ZMySQLDA 3.1
01-05 23:29 repoze.who.plugins.formcookie 0.1
01-05 23:15 ZODB3 3.9.0a10
01-05 14:52 z3c.sqlalchemy 1.3.9
01-05 13:35 plonetheme.level2 1.1.1
01-05 13:35 plonetheme.tidyblog 1.2.1
01-05 13:22 plonetheme.subordinate 1.2.1
01-05 13:11 zopyx.multieventcalendar 0.1.3
01-05 11:06 collective.dancing 0.8.5
Bookmarks   del.icio.us
» del.icio.us/tag/grok
» del.icio.us/tag/plone
» del.icio.us/tag/repoze
» del.icio.us/tag/zope
» del.icio.us/tag/zope3
- - - - - - - - - -
del.icio.us is a collection of personal, categorized bookmarks
About   About PlanetZope
» PlanetZope aggregates
the weblogs and product
announcements of zope
related websites.
- - - - - - - - - -
9447 newsitems since june 2004
- - - - - - - - - -
It is made using Zope 2.7,
CMF 1.4.2 and Mark Pilgrim's UniversalFeedParser 4.2
- - - - - - - - - -
Contact: d2m
- - - - - - - - - -
Nearby planets
» Advogato
» Apache
» Debian
» Lisp
» Plone
» Python
» RDF
» Twisted
» XMLhack