Freshnews -- The Technology Preview

A Wrapper for Leafnode's Fetchnews

All You Need To Know

The freshnews Technology Preview is a program that will manage the news connection on a permanently-connected RedHat Linux system that is running a default installation of leafnode. Freshnews will give you the fastest possible posting of articles, make new articles available to you quickly and reduce the bandwidth used by fetchnews when you are not reading news.

You don't need to know more. Click over to the download page. Get the source code. Compile it. Edit your inittab. Be happy.

If you're the more curious type, read on!

What Is Freshnews?

Freshnews is a program that runs fetchnews for you. If you run leafnode, and have a permanent or autodial connection to your upstream server, then you probably have a line in a cron table that runs fetchnews. I've written freshnews to replace that line. Pretty simple, huh? But wait, there's more.

Running fetchnews from cron is ok, but it does not provide the best performance for your local news server. Using cron, you must choose either a very fast cycle to get a good quality feed or a very slow cycle to reduce the bandwidth you use. Most likely you have chosen something in between and do a refresh every hour. For most people, that will be an excellent solution. But maybe you're a news addict? Maybe your upstream supplier is unreliable? Maybe you participate in high-speed discussions that can turn around in an hour or two? If any of those problems irk you, then freshnews may be just what you need.

What's a "Technology Preview"?

I was rather impressed with Opera's idea of showing their code before it's time. They explained at www.opera.com that their code was not ready to release, not ready for beta, not even ready for alpha testing. But they wanted people to see the product anyway. That document may be gone now, but you can look around their site. As of this writing, I'm anxious to see the rumered TP-3 release of their Linux browser, any day now.

I've worked out most of the technical details of how freshnews should run. The code works for me, but needs a little fine tuning still. Before I fine tune it, I need to know if I'm doing this just for me or if other people are interested. Depending on the amount of interest in this Technology Preview, I will either hard-code my own setup into the code and remove this web page, or I will make an alpha version that includes one or more methods of customizing without recompiling.

What problems does freshnews solve?

When I started running leafnode, I quickly ran into a number of problems. Some of these were related to my need (professionally) for a good news feed, others were the usual problems everyone encounters before and during an upgrade of their upstream supplier's news server. I expect to run into these problems again, and I expect it will happen at the worst possible time. This program is partly my attempt to be pro-active and kill the monsters before they rise again. It's partly so I won't be tempted to open another window on my desktop so I can force fetchnews to run when I want news posted or a fresh set of headers to pick from.

Your upstream server doesn't accept a posting. Freshnews Makes repeated attempts to post the article. Each failed attempt is made after a longer delay than the previous attempt, until eventually no further attempt is made to post the article.
Your upstream server fails to accept the connection. Oops. Currently freshnews doesn't detect this. I'll add it to a future version. There are a couple of simple things I can do to detect this, but first I need to know how leafnode behaves in certain legitimate situations.
You have posted a message and are expecting a fast answer. Freshnews got your posting out fast. While you are reading news, it will poll your upstream supplier every 10 minutes, but just remember that you are at the mercy of your upstream supplier, your coorespondents supplier, and your coorespondents newsreading/posting habits. I've done what I can to speed things up at your (my) end, beyond that, usenet rules apply.
You've been away from the internet for days and are anxious to get a news fix. I've let your news queue get stale while you were away. Within a minute of you logging in, perhaps even before you start your newsreader, freshnews will start fetchnews running to get the latest posts into the queue. If you read news through a net connection, you will want to refresh your header list about two minutes after starting your newsreader.

What kind of fetch schedule does freshnews provide?

I've always thought that a picture is worth a thousand words. Maybe that's why I type so much, I'm trying to paint pictures. Here's a picture from my freshnews log file that shows the schedule since my last compile:


 91 since last, login time.   Sun Mar 26 00:28:18 2000
 12 since last,  read time.   Sun Mar 26 00:40:57 2000
 10 since last,  read time.   Sun Mar 26 00:51:23 2000
 10 since last,  read time.   Sun Mar 26 01:01:57 2000
 51 since last,  idle time.   Sun Mar 26 01:53:00 2000
 81 since last,  idle time.   Sun Mar 26 03:14:35 2000
162 since last,  idle time.   Sun Mar 26 05:57:01 2000
326 since last,  idle time.   Sun Mar 26 11:22:53 2000
 15 since last,   net time.   Sun Mar 26 11:38:39 2000
 15 since last,   net time.   Sun Mar 26 11:54:14 2000
 16 since last,   net time.   Sun Mar 26 12:09:57 2000
 40 since last, login time.   Sun Mar 26 12:50:38 2000

Here's a real picture of the intervals:

These lines are printed for debugging purposes each time freshnews decides to run fetchnews. Let's have a look at what this means:

91 since last -- It was 91 minutes since the last time that fetchnews ran. Everything within freshnews runs in seconds, but this line displays the interval rounded to the nearest minute.
login time -- shows which of the possible timing cycles is in effect. The possible values are:
- post -- a message was posted and freshnews is trying to have it passed to your upstream supplier.
- read -- freshnews has noticed that someone on the local system is using one of the newsreaders it knows about. For now, I've hardcoded the list as slrn, rn and tin. Don't try to change the list unless you are sure you know what you are doing. For now, if your reader is not recognized, just reduce CYCLE_NET to get better service. If you're wondering why "read" and "net" both exist, it's because it took me a while to figure out how to do "net".
- net -- freshnews has detected someone accessing the news queue. This detection works whether it is a local program that triggered read instead, or a local program that read does not recognize, or even some other kind of program on another computer entirely.
- login -- Someone is logged in on the local computer. Since leafnode is typically used for desktop installations, this is a pretty good way of getting the queue freshened before you start to read news. Of course, you might do something else entirly on your computer, so I don't get too eager about keeping the queue fresh.
- idle -- There is no indication of anyone using the news queue, but someone did use it in the last couple of days. This is the "cool down" cycle when the interval between running fetchnews is doubling each time.
- max -- The maximum allowed interval between running fetchnews. I've never actually tested this. Perhaps I'm a newsaholic? No, it's just that I've only had this program running for a week.
Sat Mar 25 22:15:02 2000 -- of course, each line is timestamped. If you look carefully at the timestamps, you will see that the intervals are not the clean, precise minutes indicated at the beginning of the line. In fact, freshnews takes some pains to see that fetchnews is not locked onto a particular part of the clock. A random element is added to the interval between tests so that freshnews usually sleeps in a little before testing if fetchnews needs to run.

Problems?

Since I'm just putting this page up, there are no known user problems!

If you have a problem, do not contact me until you have (a) tried hard to fix it yourself and (b) prepared a clear, concise description of the problem. DO NOT SEND SPAM OR UNSOLICITED COMMERCIAL EMAIL TO THIS ADDRESS. I can be contacted at ocurtin@usa.net.

Bugs

Bugs? In my code? You must be kidding.

Ok, seriously...

# Description Status

1 There is no mechanism for skipping idle and going directly to max cycle This would only be usefull for people with expensive diallup. Easy to add, but useless to me.

2 Nothing shows up in the log! Yeah. I send it under the "news" banner, apparently this is dissabled by default. I'm looking into it and will post instructions if there is an upgrade to the program.
Workaround: Redirect the output to a logfile as shown on the downloads page. A similar message is sent to stdout whenever the log message would appear.

#	Description	Status
1	There is no mechanism for skipping idle and going directly to max cycle	This would only be usefull for people with expensive diallup. Easy to add, but useless to me.
2	Nothing shows up in the log!	Yeah. I send it under the "news" banner, apparently this is dissabled by default. I'm looking into it and will post instructions if there is an upgrade to the program. Workaround: Redirect the output to a logfile as shown on the downloads page. A similar message is sent to stdout whenever the log message would appear.