I Was Really Stupid, and Greedy Too

They always say that when it rains it pours, and this week certainly was a good example of that to me. While in and out of various doctor’s offices and hospitals, a particular form of advertising that I used on Syndic8 became a huge issue with well-intentioned folks all around the world, inspiring a flurry of posts on various forums and resulting in Syndic8′s eviction from Google, as a purported keyword or link spammer.

Here’s my side of the story…

I registered Syndic8.com in August of 2001 and the site was online by September of that year. At first it ran from a hand-built server in my home office, through my DSL connection, which, at the time, cost me $10/GB after the first 3 GB of transfer per month. Bit by bit, the site became more feature-rich and more well known, and traffic started to build.

After months of data transfer bills in excess of $200, I had to find a better way to host it. I found that I could rent a machine at a hosting company for less than my bandwidth bill. I also tried to make the site self-funding in the process, adding some Amazon Associates and Cafe Press links along the way.

The site’s popularity grew and grew, and a friend asked me if he could purchase a link on the site. We agreed on a price, and I put that first link up. Once that first link was in place, potential advertisers noticed it, and they also expressed interest in links. While trying to keep the site from looking too busy, I managed to squeeze in a number of links on the front page and in other places. I was always up-front about this, and never resorted to hidden text or anything of that nature.

The money was pretty good, paying my ever-increasing hosting costs and letting me repay the personal deficit that I had built up over the first year or so of operation, and even building up a reserve in anticipation of further expansion of my server subscription (currently at over $600/month).

Early this year an existing advertiser approached me with an interesting proposition. They asked me to create some sub-domains on the site, point the DNS records for these sub-domains to their server, and to link to them from the front page of Syndic8. I took a look at their proposed content and it seemed pretty decent, articles about mortgages, insurance, and so forth, with some links to affiliate programs and some AdSense blocks. On the surface, all seemed well, and I agreed to take this on. I didn’t look at every page, but the ones I looked at seemed fine. The pages were, in fact, designed to get the attention of search engines. Given that there’s a whole industry built around SEO (Search Engine Optimization) I didn’t know that I had crossed a line here, but in retrospect it is clear that I had.

All was fine until early this month when someone “found” this (hiding in plain sight at the bottom of each and every page on Syndic8) and decided to make a public issue over it. In an instant, at a time when I was least able to respond, my little site was the subject of a lot of attention and scrutiny, none of it good.

In the space of a day, things progressed from “hey, look at this” to “those dirty rats” to “let’s tell Google guy” to “ok, they are toast.” The trial, sentencing, and conviction took place in the blink of an eye.

I became aware of this somewhere after the “dirty rats” stage. and wasn’t sure how to proceed. Never having been the subject of a public attack before, I wasn’t quite sure of the proper protocol for responding. It was interesting (and a bit disappointing) to see bold, vituperative public posts from people that I knew, had worked with, and had even helped in the past. I was labeled as a spammer, a hacker with no business sense, and accused of all sorts of indiscretions. Exactly one person thought to email me and say “I’m not taking sides here, but I know it must suck to be you right now.”

In this link-happy world, I have found that people are more interesting in talking about you than talking to you. In the past, I thought that a private conversation was the first step toward resolution in a situation like this. That’s not true anymore. Do something wrong and a hundred bloggers will be happy to publicly point at it and to take some joy in your trouble. I’m not complaining here, just stating a fact. The Cluetrain tells us that “markets are conversations”, but I think there is actually a reluctance to engage in private 1:1 discussion about this. Apparently it is easier to create a link than to send an email. For the record, anyone who wants to email me can find me at jeff@vertexdev.com.

What have I learned so far? First, be careful about slippery slopes. Once you take on a particular form of advertising, the next one doesn’t seem all that bad, but before you know it you are doing things that you wouldn’t otherwise do. Second, consider alternatives to the ad-supported business model. Lots of people seemed to think that I could have raised funds in this way. Perhaps, perhaps not. It is clear that trying to create something that’s large and self-sustaining requires more attention to the business end than I was capable of giving it. Third, respond, and respond fast when you make a mistake.

What’s next? Our mission remains unchanged, to create a large, public directory of reviewed RSS and Atom feeds. The offending ads are gone from the site, and I will certainly not be accepting any more ads of that nature. The site will stay up and running, perhaps with an alternate funding source. I’ll talk to Google and see if they will let me back in.

So that’s the story of my week so far, and it is only Thursday morning.

Syndic8 Web Service Functi0ns for Tagging

Because Syndic8 is a platform for all kinds of interesting applications (most of which I don’t know nearly enough about), a lot of the site functionality is also available in the form of XML-RPC web services.

To go along with the tagging functionality that I rolled out last week, here are a bunch of new tagging functions:

  1. TagFeed to assign a tag to a feed.
  2. GetTaggedFeeds to fetch the Feed Ids of all feeds with a given tag, for all users or for a particular user.
  3. GetAllFeedTags to fetch the set of tags used on all feeds, either globally or for a particular user.

These are all documented on the Syndic8 Web Services Page.

If you build something cool with this, please drop me an email.

Syndic8 / KnowNow Partnership

I have been working with the folks at KnowNow since the middle of 2000, when Kragen Sitaker and Adam Rifkin invited me to visit their Seattle “headquarters” (a couple of rooms with cardboard walls in a dilapidated building in Seattle). They’ve come a long way since then (and I have too).

We’ve been working toward real-time news distribution ever since, and our new partnership is the latest step in that direction.

Here is the official press release: KNOWNOW AND SYNDIC8 PARTNER TO OFFER LIVE, CONTINUOUSLY UPDATED RSS FEEDS FOR SUBSCRIBERS.

I’ll have the first round of integration done within the next couple of days, more news here when it is ready.

Syndic8 Feed Tagging

Sites such as del.icio.us and FlickR pioneered the concept of tags, or user-assigned labeling and categorization information. This has come to be known as a folksonomy.

After a couple of days of hard work, I am pleased to announce that Syndic8 now has a feed tagging system. Any logged-in user can add their own tags to a feed using a form in the top-left corner of each and every feed info page:

Before Tagging

The tags for each feed are shown in the top-left corner of the feed’s info page. Here are the tags for feed 130801:

Clicking on any tag name will display a list of all feeds with that tag, for example ‘cricket‘:

There’s also a way to see all of the tags, on the new tag list page:

The existing XML-RPC API will be extended to provide full read-write tag functionality. Already, any function which returns feed info now includes the list of tags assigned to the feed (if the “Tags” pseudo-field is specified). A lot more is coming.,

A9.com Search Integration

Jeff Bezos rolled out a new feature for Amazon’s A9.com at the ETech conference today. Using a simple extension to RSS, any search tool can be plugged in to the A9 search.

I have already done this integration, and you can now see the results of a Syndic8 feed search as another column on the A9 home page!

To see this in action:

  1. Visit a9.com.
  2. Click on the link titled “Add Columns to your Search Results”:
    Add Column to A9
  3. Scroll down until you find “Syndic8 RSS & Atom Feed Search”; it looks like this:
  4. Press the Add button.
  5. Return to the main page and do a search. Here’s the result column after a search for chocolate:

Featured Feed System, PayPal IPN

I implemented the Syndic8 featured feed system almost a year ago in response to repeated request from feed owners for a way to get some extra attention for their feed. For a small fee, almost any feed can be featured on the front page of Syndic8 for a week or more. We don’t allow feeds which are about drugs, gambling, or porn to be featured. If the feed has an image, the image will be displayed.

Greg Knollenberg of Writers Write has been featuring his feeds on a regular basis, and he’s written about his experiences here. Here’s an excerpt:

Each time we received excellent results. Hundreds of new people looked at our feed each week on Syndic8.com with many of these translating into direct clicks over to our blogs and websites.

Writers Write is a rich resource for practicing or aspiring writers of all kinds, by the way. There’s also Readers Read, for, uh, readers.

Anyway, the featured feed system has worked out well for our advertisers, and it has been a learning experience for me. I used the PayPal IPN (Instant Payment Notification) API to integrate the payments into Syndic8. When a user makes a payment for a featured feed using PayPal, PayPal performs an HTTP GET on a URL that I have configured in to my PayPal account. Code on my site then calls PayPal to validate the payment, and it sends me an email with a “click to feature” URL inside. I check the feed to make sure that it is within the featured feed guidelines, and then click the URL. I then send a customized automated reply to the requester, and the feed is featured.

IPN works really well, and it was pretty easy to get going. The PayPal sandbox system could use some help, but I’m starting to ramble, so I’ll end it right here.

Seattle Resources, Geo-Tagging RSS

I just found out about the Seattle Networking Group, and signed up for their newsletter.

I also like the Lost in Seattle site. They are linking the virtual world to the real world, mapping out the city block by block and linking to the businesses on each block. Think of it as yellow pages, but indexed by street instead of by business type. This, of course, gets into the whole concept of geo-location, as done by organizations like MaxMind and GeoURL.

And from there, we should talk about geo-tagging RSS feeds, of course. We’ve done some work to exploit this on Syndic8, but we need to get more feeds tagged to make it really useful. You can do queries like this:

http://www.syndic8.com/feedlist.php?ShowGeoIPLocation=se;Sweden

To retrieve, for example, feeds that are geo-tagged as located in Sweden. The matching is exact; you must specify a legitimate value consisting of a country code, a semicolon, and the country name. Here are some other values:

  • us;United States
  • at;Austria
  • au;Australia
  • ca;Canada
  • ch;Switzerland
  • dk;Denmark
  • fr;France
  • is;Iceland
  • nl;Netherlands
  • mx;Mexico
  • de;Germany

Ok, so this post has rambled a bit. I have an associative brain, and I’ll have to write about that someday too.

Centos – Free Version of Red Hat Enterprise

I am in the process of procuring a second machine to handle growth and some new features at Syndic8. One of the operating systems available from my hosting company is CentOS Enterprise Linux. I had not heard of this distribution before, so I spent some time checking it out. Basically, this is a complete and faithful rebuild of the Red Hat Enterprise Linux system.

Here is how they get the question “What are the differences between CentOS and Red Hat ES/AS?.”

A quick tour through the forums seems to indicate that the users are very happy with this product, and I expect I will be as well. I’ll know more within a few days, and I will be sure to post an update at that time.