Manila and the OPML Editor

If you’ve been following me for the last couple of months, may have noticed that I’ve been spending some time looking at Manila again.

Recently, I completed a set of updates to bring Manila up to speed when running in the OPML Editor, and with Dave Winer’s help, that work is now released as a set of updates to the Manila.root Tool.

How to get the updates

If you’ve already installed Manila in the OPML Editor, and you have automatic updates enabled for installed Tools (via Prefs > Web Services > Automatic Updates), then you likely already have the changes.

If you have Manila installed but automatic updates are turned off, you can update manually as follows:

  1. Bring the OPML Editor to the front.
  2. Choose Window > manila.root.
  3. Choose Tools > Update Front Tool…
  4. Click OK on the dialog that asks, “Get the latest “manila.root” code?”

If you don’t yet have Manila installed, you can install via the Tools Catalog page:

  1. Bring the OPML Editor to the front.
  2. Choose Misc > Tool Catalog…
  3. Click the Install link for manila.

What was updated?

Here’s a summary of the updates:

  • The relevant parts of the Frontier Admin interface were ported to the Manila Tool’s admin website, which runs via the Website Framework responder, instead of mainResponder.
  • The Tool’s sub-menu (Tools > Manila) was updated to work in the context of the OPML Editor.
  • Bugs were fixed related to the way mainResponder (on which Manila depends) works in the OPML Editor.
  • Picture shortcuts which linked to the database-backed URLs are updated if pictures have static URLs.
  • Several verbs in were fixed to run in the OPML Editor, and the Website Framework responder.
  • A few minor optimizations were made to improve performance.

Problems? Questions? Bugs?

If you run into any problems or bugs, or have questions about Manila, please ask on frontier-user.

A river of Frontier news

We’re working on Frontier now more than ever, but the cursor has shifted from this WordPress site to sites using the new tools we’re developing.

At the center of everything is a river of Frontier news that’s subscribed to all the feeds. It’s the quickest way to find out what’s new.

And there’s, which is an introduction to Frontier development in 2011.

And of course much of the info posted here is current and useful and not going anywhere.

When a “root” crashes

On the ec2-for-poets list, Michael Helleson writes:

“I’ve been getting errors with config.root during the nightly backup operations. It hasn’t been able to sucessfully back it up in a week or more. The size of the file in Guest Databases is over 700 megabytes.

“I attempted to do the filemenu.savecopy(“c:\config.root”) command and got this error.

A picture named crash.gif

“I checked in c: and there is no copy of config.root so I’m assuming the backup didn’t happen.

“I’m willing to let it go like this since River2/Blork still works just fine but I think if config.root gets much bigger it will be bad.”

What to do

This is a serious situation, do not let it go on. It may appear that everything is fine, but eventually you will lose the database.

1. Take your server off the air. Quit the OPML Editor.

2. If you’re running a Windows server, in the Task Manager app, force flaunch.exe to quit. This app may have been automatically launched by opml.exe. It periodically checks if opml.exe is running, and if not, it launches it. If you’re slow at this step you may have to repeat step 1.

3. Make a physical copy of the file that’s reporting the problem, in case something goes wrong with the recovery. Put the copy in a safe place, somewhere outside the OPML folder. The Desktop is where I usually put these files.

4. Re-launch OPML.

5. Bring the database with the problem to the front. Assume it’s config.root, as in Michael’s example.

6. Choose the Quick Script command from the Misc menu. This opens a small window that you can enter a script into.

7. Enter this into the Quick Script window: filemenu.savecopy (“C:\\config.root”) and press Enter. Wait.

8. If the script finishes, great. It was able to recover the database. But it probably didn’t.

9. If you got an error, it should have given you an idea where the problem is. In Michael’s case it was a sub-table of config.river2.feeds. Carefully navigate to that location in the database, and put the cursor on the problem table.

A picture named badspot.gif

10. Choose Copy Address from the Table menu.

11. Choose Quick Script from the Misc menu.

12. Enter table.jettison (@ then Paste then enter ) — and count to 10 then press Enter.

A picture named jettison.gif

This is an undocumented verb, used only in drastic situations like this one. Do not write scripts using it, because it doesn’t reclaim the space used by the object that is jettisoned. It’s safe to call in this situation because of that.

13. Immediately the bad bit disappears. (This happens instantaneously.)

14. Still in the Quick Script window, enter filemenu.savecopy (“C:\\config.root”) and press Enter. This should take a minute or so to run, and if that was the only problem in the database, it will run to completion without error. You can tell that it did because there’s a value in the bottom of the Quick Script window (screen shot) and there’s a file at the top level of the C disk called config.root.

15. Repeat steps 1 and 2 to quit the OPML Editor and flaunch.exe if it’s running.

16. In the filesystem, navigate to the location of the database that was crashed, and change its name from config.root to configCrashed.root. (If you get an error that the file is busy this probably means that OPML has re-launched. Repeat steps 1 and 2 until this works.)

17. Copy the saved file from the top level of disk C into the location of the crashed file. It should be considerably smaller. This is because all the unused space in the file has been “squeezed” out.

18. Relaunch the OPML Editor.

When this happens

You might wonder what causes a database to go bad.

It doesn’t happen very often, because the way the underlying code is written, it does all its work with a database quickly and doesn’t leave the file on disk in an invalid state. So when the program crashes, or if you force-quit it, almost always the data in the files are safe.

But occasionally the program crashes while it’s in the middle of writing out a critical resource, like the “avail list” or the items in a table. The database software that’s built into the OPML Editor complains if it finds something out of place. A “free” node where there should only be used ones. Or vice versa. This is an indication that a serious problem is coming soon. The data isn’t yet lost, but if you continue, it will be.

That’s why the first thing you must do when you notice the problem is to take a snapshot of the database, so if the problem gets worse you can try to recover from something better.

But knock wood, it doesn’t happen very often. But when it does, it pays to be prepared.

Bookmark this page

In all the years that the OPML Editor and its ancestors have been around, I’ve always meant to write one of these howtos. Now I have. When someone says they have this problem, point them to this page.

myPhotos.root archives your Flickr photos

myPhotos.root makes it easy (we hope) to create a backup archive of your photos from Flickr on the local file system and on Amazon S3. It also provides a testbed where we can play with new flows for pictures, once they’ve found a home outside Flickr.

Eventually we hope this will get a friendly browswer-based user interface. For now, you configure it by editing with the OPML Editor’s object database.

How to set up

1. Install myPhotos.root from the Tools Catalog page.

2. Wait a minute. Choose myPhotosData.root from the Windows menu.

3. Navigate to myPhotosData.users. Create a new sub-table with your username. It doesn’t have to be your Flickr username, but it’s fine if that’s what you choose. Wait a minute and the table will be initialized automatically.

4. Expand it. Then go into prefs, and enter your Flickr ID. (You can get your Flickr ID from this page. It’s in the right column, under Useful Values. Screen shot.)

5. It has automatically chosen a folder where it will store downloaded pictures. If you’d like them stored somewhere else, change the folder value. The three S3 prefs values are very important and hopefully self-explanatory.

6. Still in the user-level prefs table, set the enabled boolean to true.

7. Navigate to myPhotosData.prefs and set its enabled boolean to true.

What will happen

Once you flip the switch in the last step, at the top of the next minute, myPhotos.root will start downloading your pictures from Flickr, storing copies locally and uploading copies to S3. If you have a lot of pictures in your Flickr account, this could take hours to run.

If you quit the OPML Editor, next time you launch it, it should resume downloading where it left off, until it has downloaded all the pictures.

Once they’re all downloaded, it will start checking every ten minutes for new pictures. When it discovers that you’ve uploaded some, it downloads them and archives them on S3.

So Flickr remains a good way to publish your pictures.

CSS in worldOutline.root

Yesterday I worked closely with Dave Jones on making the worldoutline themable. The result is fairly magnificent. The tables and blockquotes are gone. Yet it all looks great, and designers can have their way with the way outlines are displayed.

All this involves some fairly major changes to the code that implements html directories, and to the templates that are used to generate them. If you’re running worldOutline.root, you should follow these steps carefully to make this corner-turn.

How to update

1. Choose Update opml.root from the File menu.

2. Bring worldOutline.root to the front and choose Update front Tool from the Tools menu.

3. Bring config.root to the front and navigate to worldOutline.prefs. If you’ve made changes to your templates, add “Old” at the end of their names, otherwise delete them. template and directoryTemplate. In a minute or less they will be automatically replaced with “virgin” copies that are designed for the new code.

That’s it.

How to control appearance

1. You can do it the obvious way, by adding a <styles> section to the <head> of the main template or link to an external stylesheet.

2. You can also use the OPML Editor’s Add Styles command (in the World Outline sub-menu of the Outliner menu). It’s explained in this howto.

3. There are more ideas we are exploring.

Styles, the worldoutline and OPML Editor

You can add styles to your worldOutline. And when we make the templates more style-friendly, which should be a quick bootstrap, that will mean you will be able to completely redesign the look of a hierarchy from within the OPML Editor.

How to

Here’s how it works, revealed in layers (as you’d expect since all this stuff cascades)…

1. To see what styles you can set, do a view-source on any page in a worldOutline server you can author for. One where you are either the author of the root outline, or one of your outlines has been included within the root outline, either by the author or one of the people he or she delegates to.

2. In the OPML Editor, add a style to any headline, by choosing the Add Styles command in the World Outline sub-menu of the Outliner menu. The styles are anything that can appear in the <style> section of the <head> section of an HTML document.

The styles accumulate in a hierarchic manner, with the outer styles appearing before the inner styles. That is, the inner styles can override the outer ones. Exactly as you’d want it to work.

3. When you save the outline, the new version of the outline is displayed. When you visit a page that is subordinate to a style or a collection of styles, they govern the appearance of the page, as any other styles in HTML would.

We’ve started a group project with this blog post on Scripting News. Since you have detailed knowledge of how the worldOutline app works, and presumably are running it on your server, here’s an additional hint.

4. There are two templates that you can edit that control how themable your worldOutline is. They are at: config.worldOutline.prefs.directoryTemplate and config.worldOutline.prefs.template.

You can add classes to HTML elements in either of those templates and that will allow authors to add styles to control their appearance, using the OPML Editor, as described above.


body {color: #DC143C; font-size: 95%;} — this is the styles attribute of the demo headline.

body {color: #008000; font-size: 105%;} — this is the styles attrbiute of the New York headline.

To see how it works, you have to do a View Source on the New York City page.

You’ll see the <style> element in <head> has two items in it, and the latter one overrides the former one.

That’s why the text is green and a little larger underneath New York City.

Obviously this is just the beginning. Next thing to do is to re-do the templates so that everything has proper classes on them, and the tables should go away as the layout mechanism.

If you have questions, ask them here or on the EC2-for-Poets mail list.

dnsApi.root, interface to is a domain name service that has an API.

dnsApi.root connects to it, providing a simple XML-RPC interface for creating, updating and deleting names.

The OPML Editor uses dnsApi.root to help implement the domain-mapping feature of worldOutline.root (and other features in the future).

How to install

1. You must be running the Windows version of the OPML Editor on your server, and have an account on and be running a copy of worldOutline.root.

2. From the Tools Catalog page click the Install link next to dnsApi.root.

3. Relaunch OPML Editor. Wait a minute.

4. Jump to config.dnsApi.prefs. Enter your username and password. Set enabled true. Wait a minute.

5. You should see your dnsimple-hosted domains show up in, and any records you’ve already established under that in each domain’s records table.

6. Then you can go into your OPML Editor user.opmlEditor.prefs.dnsApiServer to point to your server.