Setting up rssToBlog.root

This is a faceless app that routes text from RSS feeds to WordPress blogs. In the future, I hope to get it working with other brands of blogs. There are differences in their APIs.

How to set up

1. Bring the OPML app to the front and choose Tool Catalog from the Misc menu. Click on the Install link next to rssToBlog.root.

2. Choose Add blog from the rssToBlog sub-menu of the Tools menu.

3. A dialog appears, asking for the URL of the feed. Enter it. Click OK.

4. A table window opens with info about the feed. Open the prefs table, then open the params table contained within it. Fill in each of the entries, describing the WordPress blog that you want to synch with the feed. A screen shot of the table that connects my linkblog with its WordPress version. (Password omitted.)

5. When you’re ready, set the enabled boolean to true.

6. There’s another enabled boolean you must set true. Bring config.root to the front, navigate to config.rssToBlog.prefs. There it is. Once you set this true, at the top of the minute, rssToBlog will read the feed, and start shoveling content over to your WordPress blog.

Other prefs

There’s a template in the blog prefs table, that determines what the text looks like as it flows to the blog. The default is appropriate for a linkblog. However, if you have a blog like Scripting News, with titles and essays, it makes more sense to just set the template to <%description%>. Here’s the test WP blog I’ve set up to test this kind of connection.

In the top-level prefs table there’s a value called ctMinutesBetwReads. It defaults to 10. If you want less latency, lower the number. If you want to minimize bandwidth and don’t care about a lag betw feed and blog, increase it.

Is it only RSS?

No, it should work with Atom 1.0 as well as all versions of RSS, going back to 1.0. Hey it might even work with 0.91 or 0.92. but you don’t see too many of those around these days.

Setting up a Scripting2 server

Scripting2 is a full-strength outliner-based blogging tool. It’s what I use to edit Scripting News.

A few people in the EC2-for-Poets community have hacked their way to getting it to run on their systems. It wasn’t fair what they had to go through. I wanted to make it so this software could be installed as easily as Radio2. Now it’s pretty close, so I wanted to get the instructions up so people could test them.

1. If you haven’t already done so, set up OPML to be able to store files in your S3 space.

2. Bring the OPML app to the front and choose Tool Catalog from the Misc menu. Click on the Install link next to scripting2.root.

3. The web browser will come to the front and ask for a username and password. Before doing that, we have to set up a test account for you to work with.

4. Bring the OPML app to the front. Choose scripting2.root from the Window menu. Navigate to scripting2suite.utilities.afterInstall. Open it. Click the Run button at the top of the script. A dialog appears asking for the S3 path for your blogs. Suppose you want to put them in a folder called blogs in static.myhome.com. The path you would enter is /static.myhome.com/blogs/ — it’s important that you get the slashes right.

5. When the script completes, bring the browser to the front and enter the test/password where it asks for the username and password. If it worked, OPML will come to the front, with a workspace window open saying it’s time to write your Hello World post.

6. Write. When you’re ready click Save. Then View. You should see your first post in the browser.

Nicer URLs

The URLs that Scripting2 will generate for your site are the long ones that Amazon returns to us. But if you set up your bucket so that you’re using your own domain name, you can have shorter addresses. You just have to configure Scripting2.

Choose scripting2data.root from the Window menu, and navigate to scripting2data.prefs.statictext.s3.prefs.url. Here’s a screen shot. You’ll see a long URL, edit it to replace it with the short equivalent.

Your blog’s name, your name, etc

These bits are used in rendering your site, and without them being set, the site will work, but it will look strange in places.

To set them, you can either navigate to your prefs table under scripting2data.server.users, or use the web form. (Note that URL will only work on the server machine.)

New users

The test account was just to get your started. Now you should start a new account with your real username, and a real password. To do so, choose the New user command from the Scripting2 sub-menu of the Tools menu. It will ask for a username and password. After you’re done, you should re-login with that info and start again with your “real” blog.

Scripting2 mail list

If you’re going to run this server for real you should probably sign up for the list.

How to use the new host-a-website feature on S3

You can find all the links about the new feature on Werner Vogel’s blog. I share his enthusiasm for the new host-a-website feature. I was asking for it for a long time. I guess, from reading the tea leaves, he was too.

Anyway, I had to do a bunch of trial and error to get my first S3-hosted static site configured. I wanted to leave a howto behind me so next time I won’t have to do the head-scratching again. Maybe it will help you too.

Suppose the domain you want to host on S3 is r2.reallysimple.org. And the index file for that domain will be index.html. This is what you do.

1. Go to the S3 panel on the AWS website and create a new bucket called r2.reallysimple.org in the US-Standard region.

2. Upload a file called index.html to the new bucket. Say something simple like Hello World. You can copy the HTML from my test file if you want.

3. Click on Properties at the top of the right panel, then click on Website in the lower-right panel. Click the Enabled checkbox, and enter index.html as the Index Document. Click Save.

4. Click Permissions in the lower-right panel. Then click on Edit bucket policy. Paste in the template text you grab from this file. Edit it to replace YOUR-BUCKET-NAME-GOES-HERE with the name of your bucket (in the example r2.reallysimple.org). Click Save.

5. Now go to your domain registrar, and map r2.reallysimple.org as a CNAME to r2.reallysimple.org.s3-website-us-east-1.amazonaws.com.

Now you should be able to go r2.reallysimple.org and see the text of your index file.

How to synch a folder with S3

You can use the OPML Editor to keep a folder on your desktop in synch with a folder on Amazon S3. The synchronization is one-way. Files and folders only go up to Amazon from your desktop.

1. If you haven’t already done so, configure the OPML Editor to access your S3 account.

2. On the computer the OPML Editor is running on, visit this page. Screen shot.

3. The folder will be scanned periodically for new, updated or deleted files and folders. It defaults to a folder on your desktop named Amazon S3. If you move the folder you should change this pref.

4. S3 path is the path to the folder in your Amazon S3 storage. The URL is the web address we use for these files in automatically-generated files and logs.

How to map a domain to an S3 bucket

To map a domain to an S3 bucket:

1. Choose a domain name, and on your registrar’s site, make it a CNAME for s3.amazonaws.com. For this example I’ve chosen static.reallysimple.org.

2. Using the AWS Management Console, create a bucket with the same name.

3. Store stuff in the new bucket. You can use the name defined in step 1 to refer to the contents of the bucket created in step 2. Here’s the url of a file in my bucket.

That’s all there is to it. It’s really simple. Amazon did this in a very straightforward and elegant way.

Why would you want to do this? It’s neater, and it gives you flexibility down the road if you want to move the content somewhere other than S3.

Setting up a Radio2 server

Important: There’s a new howto doc for setting up a Radio2 server:

http://radio2oninstall.blorkmark.com/

Dave

 

Setting up a “feedhose” server

A few days ago I did a quiet launch of feedHose.root to a very small number of users.

It’s ridiculously easy to set up a new feedHose client, at least for the Hacker News hose, which is our canonical demo. You just install the tool from the Tool Catalog page, and choose Open Hacker News from the feedHose sub-menu of the Tools menu. A window opens and as soon as there’s a new item in the Hacker News feed, it displays in the window. I leave mine open on my second monitor and can always see out of the corner of my eye if there’s something new.

How to set up a server

Now, to the more interesting task of setting up a feedHose server. It’s more interesting because it involves script writing, but it’s very simple stuff.

1. We’re going to set up a hose that distributes links to NY Times stories.

2. First, decide what name to give your hose server. I call mine hosedave.reallysimple.org. Whatever you decide, go to your domain registrar, and point that domain to the server that feedhose.root is running on.

3. On your server, bring feedHose.root to the front, and open feedHoseSuite. Open the utilities table and you’ll see a script there called setupTimesHose. Double-click on the wedge next to its name to open the script. Click on the Run button. It will ask for the name you’ve chosen. Enter it and press OK. At this point you have a feedhose server running. You can test it by entering this URL, replacing my domain name with yours. http://hosedave.reallysimple.org/?timeout=1

4. Bring config.root to the front. Navigate to feedHose.server.hoses.nyTimes. Expand it. You should see four items: calendar, condition, prefs and stats. condition is a script. We’re going to edit it, so open it.

5. Read the script, especially the comment. Since the script unconditionally returns false, the hose as it’s currentlly set will distribute no items. We’re going to change that.

6. Add the following code, just before the return (false) statement:

local (url = string.lower (nameof (adrfeed^)))

if (url contains “nytimes”) or (url contains “nyt/mostrecentheadlines”)

return (true)

Here’s a screen shot of what the script should look like.

7. Click the Compile button at the top of the script.

8. Very important. The hose is hooked into the flow of items coming into your River2 installation. It will look at every item that comes in. You must be subscribed to at least one feed from nytimes.com for anything to show up in your hose. Don’t worry about duplicates, the code takes care of that and won’t send an item across two or more times.

9. That’s it. Your hose server is running and will return stories from the Times.

How to set up a client that listens to your server

Okay. If a tree falls in the forest but no one is there to hear it, what’s the big deal?

We have to set up a listener to see if your hose is working.

1. Hopefully you have another machine to play with. Your desktop computer or laptop would be fine. Install OPML if it isn’t already running. Install feedhose.root. Bring feedhose.root to the front. Open the utilities table in feedHoseSuite.

2. You should see a script there named watchTimesNews. Open it and run it. It will ask for the domain name of your feed hose server. Enter it and click OK. A window should open.

3. Leave the window open for a while. It may take a bit of time before there’s a new story. That’s how it goes in the new business.

That’s it for this tutorial. This is a programmer’s feature, so from here, what you want to do with this is up for innovation. You can create communities in hoses, where everyone in the community can post items to the windows on everyone else’s desktop.

I’m pretty sure this technology, which is basically the same as the instant outliner, using long-polling, will hold up pretty well. I guess we’ll find out!

How to configure OPML for Amazon S3

As with all instructions on this site, we’re explaining how this feature works in the OPML Editor distribution. This is the software that’s running on the system that EC2 for Poets installs.

First, you have to have an account on Amazon’s web services site, and have S3 enabled. If you’ve ever bought anything on Amazon, you already have an account, if not you have to create one.

To create an account, go to the web services site, click the Sign Up Now button, follow the instructions.

To connect with S3, go to the S3 page, click the Sign Up button, and do what they say.

There are two bits of information you need to get from Amazon, that you will give to the OPML Editor, that will enable it to write files to your S3 storage space. They are called the Access Key ID and the Secret Access Key. As with everything in AWS, these are confusing names for well-known concepts: “username” and “password.”

How to configure…

1. Open a web browser on the machine your copy of the OPML Editor is running on.

2. Go to this page and scan down the page with your eye, looking for the Access Credentials section. You should see a list that says “Your Access Keys.” There is probably one row in the list. Leave the browser here, and open a new tab.

3. In the new tab, go to this page:

http://127.0.0.1:5337/opmlEditor/prefs?title=AmazonS3

Here’s a screen shot of what it looks like.

4. Copy the Access Key ID from the page on the Amazon site (step 2 above) and paste it into the Access Key ID box on the OPML Editor page.

5. Do the same for the Secret Access Key.

6. You can ignore the other two options on the page — the checkbox for synching a desktop folder and the default bucket.

7. Click Submit.

If all goes well you should get a message that says it was able to connect with Amazon with the credentials you entered.

Updates to JSON parser and serializer

A bunch of bug reports followed by fixes and a major conceptual problem in the naming of the entry-points.

Most of the reports came in two comments on the scripting.com post.

The first, from Andrew Tunnell Jones, resulted in a new example that tests the two problems he caught.

1. We weren’t catching the “null” constant. This is encoded in Frontier as the built-in value nil. The serializer needed to watch for it, and it wasn’t.

2. The second is much more problematic. He included an empty list. This caught a problem in the way we view lists. Say you have a list named concepts with three values: 1, 15 and true. We create three items named concepts in the table, with each of the respective values. This works great until you give me an empty list. We create 0 items. Oops, we didn’t leave anything behind to serialize. So now we do. If there’s an empty list in the JSON text, we create an empty Frontier list. When you’re processing this, you should do nothing with it.

Peter Miller caught two problems..

3. A list with one element, a number, returns as a list with a string. This is actually a more general problem. It was always returning strings. A boolean true would return as the string “true” for example. I simply wrote the missing code.

4. The value “foo” returns as true. This is because the code expected either a list or a struct. A scalar value completely screwed things up. My design was wrong, we’re not just passing around tables that are either lists or structs, we have to be able to use JSON to represent single scalar values like “foo” and -198.02 and false. So I made the changes, which were easy, but there’s now a serious conceptual problem — the names of the routines: xml.jsonToTable and xml.tableToJson. That’s not what’s going on any longer. Better names would be json.compile and json.decompile. It’s still early enough in the process to make this change, I believe.

To get the fixes, choose Update opml.root from the File menu in the OPML Editor.

Update: Jim Roepcke discovered that xml.jsonToTable wasn’t properly handling strings containing a backslashed double-quote. Here’s a simple example. The problem has been fixed.

JSON parser for the OPML Editor

We now have a full-blown JSON parser running in the OPML Editor.

It’s written as a script, runs pretty fast.

It works exactly as xml.compile does. It takes two parameters, a string of JSON text and the address of a table. The table returns in the same format as xml.compile produces, so all the techniques we’ve developed for walking XML structures work equally well for JSON structures. They are the same.

There is also a serializer that works exactly as xml.decompile does.

The two routines are:

xml.jsonToTable (s, adrtable)

xml.tableToJson (adrtable) returns string

There is an important difference — XML has attributes and JSON does not. This came up when producing a JSON serialization of an RSS structure. Following other people’s examples, when I see an element that has attributes, I include the attributes as sub-items and create a sub-element called #value, which contains the element’s value. There is an optional parameter on xml.jsonToTable that says whether or not it should watch for these, and automatically collapse them.

The code for the parser has been released. If you update opml.root you’ll get it. You can read the source here. And if you have some valid JSON around, you can test the parser using this web app that I put together to help me test it.

http://json.scripting.com/

Bottom-line: Now I can build web apps that work with JSON.