Sublime Text packages: working in 2 and 3

I maintain the Git package for Sublime Text. It’s popular, which is kind of fun and also occasionally stressful. I recently did a major refactor of it, and want to share a few tips.

I needed to refactor it because, back when the Sublime Text 3 beta came out, I had made a branch of the git package to work with ST3, and was thus essentially maintaining two versions of the package, one for each major Sublime version. This was problematic, because all new features needed to be implemented twice, and wound up hurting my motivation to work on things.

Why did I feel the need to branch the package? Well…

The Problem

Sublime Text is currently suffering from a version problem. There’s the official version, Sublime Text 2, and the easily available beta version, Sublime Text 3. They’re both in widespread use. This division has ground on for around three years now, and is a pain to deal with.

It’s annoying, as a plugin developer, because of a few crucial differences:

Sublime Text 2:

  • Uses Python 2.7.
  • Puts all package contents into a shared namespace.

Sublime Text 3:

  • Uses Python 3.3.
  • Puts all package contents into a module named for the package.
  • Has some new APIs, removes some old APIs.

…yes, the Sublime Text 2 / 3 situation is an annoyingly close parallel to the general Python 2 / 3 situation that is itself a subset of the Sublime problem. I prefer less irony in my life.

Python

What changed in Python 3 is a pretty well-covered topic, which I’m not going to go into here.

Suffice it to say that the changes are good, but introduce some incompatibilities which need code to be carefully written if it wants to run on both versions.

Imports

If your plugin is of any size at all, you probably have multiple files because separation of code into manageable modules is good. Unfortunately, the differing way that packages are treated in ST2 vs ST3 makes referring to these files difficult.

In Sublime Text 2, all files in packages are in a great big “sublime” namespace. Any package can import modules from any other package, perhaps accidentally.

For instance, in ST2…

…gets us the Default.comment module, which provides the built-in “toggle comment on a line” functionality. Unless some other package has a comment.py, in which case who what we’ll get becomes order-of-execution dependent.

Note the fun side-effect of this: if any package has a file which shares a name with anything in the standard library, it’ll “shadow” that and any other package which then tries to use that part of the standard library will break.

Because of these drawbacks, Sublime Text 3 made the very sensible decision to make every package its own module. That is, to get that comment module, we need to do:

This is better, and makes it harder to accidentally break other packages via your own naming conventions. However, it does cause compatibility problems in two situations:

  1. You want to access another package
  2. You want to use relative imports to access files in your own package

The latter case, this is something which behaves differently depending on whether you’re inside a module or not.

Editing text

In Sublime Text 2 you had to call edit = view.begin_edit(...) and view.end_edit(edit) to group changes you were making to text, so that undo/redo would bundle them together properly.

In Sublime Text 3, these were removed, and any change to text needs to be a sublime_plugin.TextCommand which will handle the edit-grouping itself without involving you.

The Solution (sort of)

If you want to write a plugin that works on both versions, you have to write Python that runs on 2 and 3, and has to play very carefully around relative imports.

Python 2 / 3

A good first step here is to stick this at the top of all your Python files:

This gets Python 2 and 3 mostly on the same page; you can largely just write for Python 3 and expect it to work in Python 2. There’s still some differences to be aware of, mostly in areas where the standard library was renamed, or when you’re dealing with points where the difference between bytes and str actually matters. But these are workable-around.

For standard library reshuffling, checking exceptions works:

If your package relies on something which changed more deeply, more extensive branching might be required.

Imports

If you want to access another module, as above, this is a sensible enough place to just check for exceptions.

You could check for the version of Sublime, of course, but the duck-typing approach here seems more Pythonic to me.

When accessing your own files, what made sense to me was to make it consistent by moving your files into a submodule, which means that the “importing a file in the same module” case is all you ever have to think about.

Thus: move everything into a subdirectory, and make sure there’s an __init__.py within it.

There’s one drawback here, which is that Sublime only notices commands that are in top-level package files. You can work around this with a my_package_commands.py file, or similar, which just imports your commands from the submodule:

There’s one last quirk to this, which only applies to you during package development: Sublime Text only reloads your plugin when you change a top-level file. Editing a file inside the submodule does nothing, and you have to restart Sublime to pick up the changes.

I noticed that Package Control has some code to get around this, so I copied its approach in my top-level command-importing file, making it so that saving that file will trigger a reload of all the submodule contents. It has one minor irritation, in that you have to manually list files in the right order to satisfy their dependencies. Although one could totally work around this, I agree with the Package Control author that it’s a lot simpler to just list the order and not lose oneself in metaprogramming.

Editing text

Fortunately, sublime_plugin.TextCommand exists in Sublime Text 2, with the same API signature as in Sublime Text 3, so all you have to do here is wrap all text-edits into a TextCommand that you execute when needed.

Conclusion

Getting a package working in Sublime Text 2 and 3 simultaneously is entirely doable, though there are some nuisances involved, which is appropriate given that “run in Python 2 and 3 simultaneously” is a subset of the problem. That said, if you do what I suggest here, it should largely work without you having to worry about it.

Sublime Text 2 git plugin

I wrote a git plugin for Sublime Text 2.

I’d decided to try Sublime out for work to see how it compared to TextMate… and thus some degree of git integration was required. Given that it’s been out since January, I was surprised that there wasn’t already a solid git plugin.

I did find this one, admittedly, but I decided that I didn’t like how it fit in with Sublime. It’s built around menus and keybinds, whereas I felt that setting everything up as commands in the palette and hooking as much stuff as I could into the fuzzy search was the way to go.

Working on the plugin was a good exercise in getting me used to Sublime. I’m fairly sold on it as a result. It’s philosophically somewhat similar to TextMate, but with some of TextMate’s rough edges smoothed out.

(Short rant: if the recently announced TextMate2 alpha doesn’t get rid of the single-character undo buffer… I don’t know what I’ll do. It’s certainly the biggest single complaint I have about TextMate nowadays.)

Creating an image map from SVG

I was asked how I made the map in my examples earlier.

I wrote a small script to do it. (The script is quite limited – I only made it complete enough to handle the SVG files I was using. Others might break it. Also, it requires pyparsing… and hoo-boy is that slow.)

Example!

Wikipedia is good for this, and has provided me with the example file I’ll use, a map of the USA. If you have some GIS data already, I believe that ArcGIS 9.2 has native SVG support, or it looks like you can convert ESRI shapefiles with shp2svg.

My example file is filled with all sort of crud that isn’t a definition of state boundaries, though, so I need to get just that. Perusing it (in a text editor or a SVG editor like Inkscape) reveals that all the state borders are in a group named “States”. Helpful!

So I run my script: svg2imagemap.py Map_of_USA_with_state_names.svg 960 593 States

(The “960 593” is the size of the image I’m creating from the SVG file.)

This creates an html file named [svg name].html, so Map_of_USA_with_state_names.html. It only contains the area tags, so I dump them into an image map in a page set up like the one in the other examples…

And we get: A map of the USA.

Just to disclaim again: That script is unlikely to be immediately useful for any particular SVG image. You would almost certainly need to tweak it significantly to make it work for your purposes. But it’s a good start, at least.

One last time: I make no guarantee of this script working on an arbitrary SVG file. At best it’s an example of an approach to take. If you use it, expect to have to debug how it interacts with your particular file.

Python whitespace doesn’t matter

If you have some programming experience then there is one particular feature of Python that is likely to turn you off.

Whitespace matters. Indentation is significant.

This comes as a shock to many people who are used to it being meaningless. Most languages in common use designate code blocks with braces ({ ... }), or in some cases special keywords (e.g. Lua’s do/then ... end). If you want to write your program without any indentation whatsoever then it won’t stop you.

So having to pay attention to whitespace worries people. Puts them off trying Python. Makes them chafe at the thought of using it. Their freedom is being abridged!

This is really weird, since everyone agrees that consistent indentation is a good idea. In fact, it’s about the easiest and most effective thing you can do to make your code legible.

This is why objecting to syntactically meaningful whitespace in Python is a straw-man. All it’s doing is requiring you to not write unreadable and misleading code. In fact, if you have good habits already then you will never notice it. Ever.

(This post is an example of low-hanging fruit for a programming blog. Something to post about early on, for the sake of getting your opinion out there.)