NNTP Buffer version 1.10

Copyright (C) Angus J. C. Duggan 1997-1998

This is a simple *personal* news cache for Windows NT, designed for use on
portable machines. It acts as a buffer between the user and one or more NNTP
servers, downloading messages automatically while online and allowing them to
be read and postings to be made while offline.

This program runs as as a Windows NT service, it is entirely useless for any
other operating system. Also, don't even think about running a full or
anything like a full feed with this program; it is designed for smaller
volumes, of perhaps up to 100 groups or several tens of megabytes (no limits
are enforced, though). It has not been tuned for performance. It also hasn't
been tuned or tested with multiple simultaneous readers, though it should
work with them.

This program requires that feed sites support the NNTP NEWNEWS command. This
command is often disabled, check that your feed supports this command before
attempting to use NNTP Buffer.

Since this program's purpose is storing and expiring of news, you'll need
plenty of disc space to use it. I'm currently using about 5MB including the
config databases for 80 (mostly low-volume) groups.


The newsfeeds file contains patterns matching groups to NNTP servers, and
their expiry times and size limits (not yet fully implemented). Groups carried
are dynamic, depending on what the user selects. Groups are inactivated if
they are not read for a specified period (actually, if some groups are read
and others aren't, they are inactivated).


In key \\HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\NNTPBuffer:

PostingDir		($exedir\outgoing)
	Directory in which outgoing postings will be queued.
SpoolDir		($exedir\spool)
	Directory in which retrieved news will be stored.
ConfigDir		($exedir\config)
	Directory containing newsfeeds file, group and article databases.

ConnectTimeout		(5 seconds)
	Time in seconds allowed to find host when connecting.
ReconnectTimeout	(5 mins)
	Time in seconds before attempting to reconnect failed server
DisconnectTimeout	(10 mins)
	Time in seconds before disconnecting idle connections.
ExpireInterval		(24 hours)
	Time in seconds until next expire check after successful expire (no
	point much more than once a day).
ExpireWait		(5 minutes)
	Time in seconds to wait before trying first expire on startup.
ExpireRetry		(5 minutes)
	Time in seconds until next expire check after failed expire.
PostingInterval		(30 minutes)
	Time in seconds until next posting check after successful post.
PostingWait		(5 minutes)
	Time in seconds to wait before first posting check on startup.
PostingRetry		(5 minutes)
	Time in seconds until next posting check after failed post.
DownloadInterval	(3 hours)
	Time in seconds until next download check after successful download.
DownloadWait		(10 minutes)
	Time in seconds to wait before first download check on startup.
DownloadRetry		(15 minutes)
	Time in seconds until next download check after failed download.
InactivateDelay		(1 week)
	Time in seconds before inactivating group that has not been read.
RememberMax		(2 weeks)
	The maximum time to remember messages. The maximum of this number and
	the highest group expiry time is used to purge messages.
RememberMin		(1 week)
	The minimum time to remember message-ids. If any message is expired
	before this time, the message-id will be remembered for this time to
	prevent re-downloading.
ClientPort		(119/NNTP)
	Client port number to listen for newsreaders.
ServerPort		(119/NNTP)
	Server port number for sending new posts and downloading news.


The command line arguments supported by nntpbuff are:

	-d confdir	Sets the configuration directory
	-s spooldir	Sets the spool (incoming) directory
	-o postdir	Sets the posting (outgoing) directory
	-c portno	Client port to listen for newsreaders
	-p portno	Server port to get news feed from
	-i		Installs the service
	-r		Removes the service
	-q		View outgoing posting queue
	-x		Stop the service
	-Y group	Enable posting to group
	-N group	Disable posting to group
	-A group	Activate downloads for group
	-I group	Inactivate downloads for group
	-U group	Unsubscribe (delete) group
	-K msgid/num	Kill outgoing posting 
	-E start/stop/abort	Start/stop/abort expires
	-S start/stop/abort	Start/stop/abort download
	-P start/stop/abort	Start/stop/abort posting
	-D num		Run debug NNTP server. Flags are:
			0x0000004	Registry access
			0x0000008	Client list and newgroup
			0x0000020	Server connections
			0x0000040	Group access
			0x0000080	Article access
			0x0000100	Socket access
			0x0000200	File access
			0x0000400	Read filters
			0x0000800	Write filters
			0x0001000	Block downloading
			0x0002000	Download info
			0x0004000	Download stream
			0x0010000	Block posting
			0x0020000	Posting info
			0x0040000	Posting stream
			0x0100000	Block expire
			0x0200000	Expire info
			0x0400000	Expire removals
			0x1000000	Message ID access
	Only the last specified of -i -r -q -x -D -E -S -P will take effect.
	Default action is to start the service.


Nntpbuff uses three directories, by default called config, spool, and
outgoing, and by default located in the same directory as the nntpbuff
executable. The names of these directories can be changed with the command
line options, but it's not a good idea to do so. The config directory
contains the "newsfeeds" file, which describes where to get and put news
from, and the article, group, and message-id databases. The spool directory
is used for storing the articles that are downloaded. The group name
hierarchy is mapped onto directory names in it. The outgoing directory
contains postings queued for sending to an upstream server. On the whole,
postings made while on-line will go straight through, but off-line postings
will end here. On NTFS filing systems, nntpbuff supports hard links between
cross-posted files. It is a good idea to use an NTFS disc for the spool
directory anyway, because NTFS wastes less space when storing small files
(such as most postings) than FAT.


Compile the programs (or use the executables provided). Create a directory
where you want them installed and copy all of the executables and the example
config sub-directory over. Edit the "newsfeeds" file in the config
sub-directory to reflect which groups you will be reading and which servers
you will be getting them from. Create "spool" and "outgoing" sub-directories in
the installation directory. From within the installation directory, run
"nntpbuff -i". This will install the program as an NT service.

Next time you reboot, the service will start automatically. Alternately, you
can start it immediately by opening the Control Panel, starting the Services
applet, selecting "NNTP Buffer Service", and hitting the Start button. It is
a good idea to check the application log to see that it started OK, or what
problems there were if it didn't.

Now set your news reader to point at "localhost" and run it. The first time
you run, you won't get any news. You will have to try selecting the groups
you want explicitly to make them active, after which they will automatically
download news. If you do not read a group for a long while, it will revert to
inactive, and you will have to re-select it to make it active again.


Quite a lot of events are logged to the event log service. You can check when
the last successful download or expire ran, or what articles were posted, by
looking at the application log.


The artdump, groupdump, and msgdump utilities are for examining the contents
of the article, group, and message database, mainly to see if they are
corrupted. They cooperate with nntpbuff by using the same
publically-accessible mutexes to lock the databases. They can be provided
with globbed patterns to examine portions of the databases.

The rebuild utility can be used to re-create or repair the article, group and
message databases if they are damaged or deleted. After many operations,
these databases may grow rather large; they can be deleted and rebuilt if

The main nntpbuff executable may be used to activate, inactivate and remove
groups, deny or permit posting access to groups, start, stop or suspend
expiry, posting and news downloading, and to view and kill outgoing postings.


Comments line start with '#' as the first character. The format of the
newsfeeds file is:


Min is minimum number of days to keep article entries (if expiry is set, it
is forced to be at least as long as min). Default is the normal expiry
length, which is used if no expiry is set. Purge is the maximum length to
keep the article, regardless of what the expiry says. Default and purge can
be set to "*", to indicate that the article should never be expired or purged
respectively. (Expires headers are currently not honoured.)

The group list can include new expiry settings, introduced with a colon:


Group entries starting with "!" indicate that the pattern should not match.
So the following matches all alt groups execpt binaries ones:


There can only be one pattern for each host name. Patterns will be continued
onto the next line if the last character is ",". Each host name can only
appear once, but aliases which point to the same host can be used.

The first matching entry for a group will be used.

Expires headers are not currently honoured, the purge syntax is there for
future expansion. 

Other future additions will include adding "<size" syntax to prevent
downloading messages larger than that size. 

                                                     AJCD 27th September 1998

[Home] [Up]
Wrapped on 2nd October 1998 by angus@harlequin.com