The Unofficial Undernet Operators' Manual

 Draft Release 1 - Thursday May 9, 1996
 Originally by Tony Vencill, Aug 1, 1993
 Modifications by Chris Portman (_Chris_)

About this Manual
This manual exists as an attempt to provide a consistent set of guidelines for Undernet irc operators, so that they may all function as a team. It is stressed that this guide remains unapproved and probably is not reflective of the view of the entire Undernet oper base.
The foundation of this guide and most of its credit lies in the hands of Tony Vencill. After a recommendation by IRC's Rowan that the guide be updated after three years, this is an attempt by myself to modify it for any changes I've observed contradicting the old guide.
These changes are reflected as far as how Undernet administration and management has differed from the three year old guide through the period that I've been around. Comments and constructive criticisms are always welcome.

Thanks To:
Joe Diehl (joed) - For his feedback, comments and criticisms, as well explanations on personal etiquette and info on obscure operator commands.
Rowan Smith (rowan) - For his concise comments on killing
Teknix (teknix) - For a fantastic editing of the entire guide, comments, some rewording, and for running it through a much-needed spell check ;)

Index

About This Manual
Index
Introduction
Mailing List and FTP Information
Server Software Administration
ircd.conf
Y: Lines
I: Lines
C: and N: Lines
Autoconnects and AutoConnect Connection Order
Hostmasking
H: vs. L: lines
Q: lines
O: Lines
Sysadmin Approval
Test servers
Server Troubleshooting
Oper Responsibilities
Oper duties and powers
Killing
SQuitting and Connecting
Rerouting
Restarting and Dying
New Server Links
Granting Oper Status
Using UWorld and UWorld2
Clearing Channel Modes
OpCom KICK / OpCom MODE
UWorld AutoBans
Disabling/Enabling UWorld/UWorld2
Counting Clients from a Given user@host
Problem Solving
Handling Critical Problems

Introduction

The Undernet is a network of IRC servers linked together similar to EFNet, but operating in a vastly different atmosphere. The Undernet continues to be a very successful, friendly, and more usable environment in  comparison to EFnet, where conversation is not continually interrupted by netsplits, collides, channel takeovers, and other disruptions. Users are always welcome to seek help from a fantastic and always changing collection of volunteers.

The Undernet is successful today because it has been administered and operated by a team of hard-working, reliable people.

Although challenging at times, it is essential for Undernet opers to work together as a team. In order to provide a friendly, peaceful environment, it is important that all opers keep up to date on Undernet issues, be accessible and willing to help when problems arise, and monitor and participate in the various designated mailing lists and channel meetings. Furthermore, when the oper meetings are announced, it is expected that you make a strong effort to show up at these meetings. You are encouraged to interact with other opers and ask questions on things you're uncertain about.

If you run or assist in the administration of a server on the Undernet, it is expected that you spend time on the Undernet and be accessible if needed. Running a server on the Undernet is not just starting a process and leaving, but it is making a commitment to be a part of the net and to help it grow and flourish.

As part of this accessibility, your /admin information (ircd.conf's A line) should contain the nick and email address of at least one of your server's opers.

Mailing List and FTP Information

Several mailing lists for discussion on various aspects of the Undernet are currently available. These are as follows:
coder-com
Discussion of Undernet server (ircd) programming. Bug reports, bug fixes, enhancements, and other topics relating to server code.
cservice
[VERY BUSY] Discussion and copies of all channel applications. Many of the admins, reps, and helpers follow this list so assistance is often very speedy.
cservice-admins
[CLOSED-LIST] Administrative discussion of issues related to channel service.
dns
Discussion relating to the *.undernet.org domain tree.
doco-com
Documentation Committee -- Discussion of documentation on various topics relating to the Undernet and IRC.
oper-com
[CLOSED LIST] Discussion between Undernet irc operators.
opermeet
[CLOSED LIST] For posting of online operator meetings.
pr-com
Public Relations Committee.
routing-com
Discussion of topics related to the Undernet server tree. Routing issues, new link applications, hub and leaf status, etc. are all appropriate.
undernet-admins
[CLOSED LIST] Discussion between Undernet server administrators.
undernet-announce
Announcements of new servers and other important issues relating to the Undernet. No discussion, low feed.
wastelanders
Pretty much everything :). Please, constructive emails only ..No flaming, pointless arguing, or trying to get in the last word.

The mailing list manager is accessible by sending mail to majordomo@undernet.org and putting commands in the message body.
Three important to know commands follow:
subscribe To subscribe yourself to
unsubscribe To unsubscribe yourself from
help To request a mail server help message
To post to any of the lists, send mail to @undernet.org.
For example, doco-com@undernet.org.

The servers on the Undernet should operate as standard servers from any other server's point of view, using software and patches only found from ftp.undernet.org.
Important directories follow:
/users/ircd-dev/patches/approved [ Approved Server Patches ]
/users/ircd-dev/patches/rejected [ Rejected Server Patches ]
/users/ircd-dev/patches/pending [ Server Patches Not Yet Approved ]
/users/ircd-dev/servers [ Server Software ]
/pub/undernet/clients [ Client Software ]
/pub/undernet/docs [ Undernet/IRC Documentation ]
/pub/undernet/newlinks [ Info Files for New Server Links ]
/pub/undernet/scripts [ IRC Client Scripts ]
/pub/undernet/utils [ Various IRC-Related Utilities ]

Only patch your server with patches found at ftp.undernet.org or those posted to appropriate mailing lists from official sources. If in question, check with other opers or post concerns to the appropriate mailing list(s).

Server Software Administration

ircd.conf

If you're running a server on the Undernet, please be sure you understand the function of the ircd.conf lines. All of these lines are described in the INSTALL file in the doc directory, and in the example ircd.conf file.
The most important lines to know are the I, Y, C, N, K, H, and L lines.
All servers that can should make their ircd.conf files readable only to server operators or admins to further reduce oper hacking and to reduce the possibility of server juping by non-opers. The typical command for this is 'chmod 600 ircd.conf'.

Y: lines
Y lines allow you to adjust your server's behavior to suit different links. You can define separate classes for servers and clients, separate classes for servers with different link characteristics, and even classes for oper connections. Y line stats should be set and adjusted over time to get the best connection possible between servers and with clients. Because of the high number of clients on the Undernet, in order for your server to work efficiently, these lines should be optimized.
Please do not use excessively long ping times. Recommended client ping times are 120 seconds, and server ping times, 100 seconds. Please also ensure that the fields that define the number of client connections permit a reasonable number of clients to connect to your server.
The format of a Y line is as follows (taken from the INSTALL doc included with the server). A sample Y line with recommended values is shown below in the general format line. The class number (5 in the sample) is the number that identifies the specified set of stats and that will appear in your I, O, C, and N lines.
Y:::::
CLASS: Connection class number.
PING FREQUENCY: How long the server will let the connection remain silent before sending a PING message to ensure it is still alive.
CONNECT FREQUENCY: How often your server checks to see if it can connect to this server.
MAX LINKS: The maximum number of links this class will allow from automatic connections. /CONNECT overrides this feature.
SENDQ: The 'sendq' value for this class. If this field is not present, the default from config.h is used.
NOTE: leaving any of the fields out means their value is 0 (ZERO)!

example:
Y:23:120:300:5
define class 23 to allow 5 auto-connections, which are checked every 300 seconds. The connection is allowed to remain silent for 120 seconds before a PING is sent.
Another feature of connection class is the ability to do automatic routing by using the class as a 'priority'. If you are connected to a server which has a class lower than one of the servers that is 'behind' it, the server will disconnect the lower class one and schedule a 'new' connection for the higher class server.

I: lines
I lines allow clients to connect to your server. A sample I line and its format are shown below.

I:::::
I:*::*:1
# or if using Y line 5 for client connections...
I:*::*::5

C: and N: lines
C lines define to whom you can connect your server and N lines define who can connect to you. They almost always appear in pairs so that all the servers to whom you may connect may also connect to you. Also see the section on H and L lines.
A sample CN pair are shown below with their formats.

C:::::
N:::::
C:pv16.state.edu:old:pv16.state.edu:6667:5
N:pv16.state.edu:old:pv16.state.edu::5

The C and N line passwords need not be the same, but the password in your C line must match the password in the other server's N line, and vice versa.
If using link password encryption (ie: if CRYPT_LINK_PASSWORD is defined in your config.h file), then only the N line password must be run through the mkpasswd utility in ircd/crypt/. The C line password must not be encrypted.

Autoconnects and AutoConnect Connection Order
Filling in the target host port field in a C line allows your server to automatically attempt to connect to the named server under the right conditions. If your server gets disconnected, it may try to connect to any of these autoconnects, or if one of these autoconnects gets disconnected, your server may automatically try to reconnect the named server. On the other hand, a C line without a port number still allows an oper to force the connection but will never result in an automatic connection to the listed server. It is recommended that you limit your autoconnects to three servers and that these be Undernet hubs.
When arranging CN pairs in your ircd.conf file, keep in mind that the LAST CN autoconnect pair in the file is tried FIRST and the first autoconnection listed in the file is tried last. Obviously, order will not matter for connections that are not autoconnects.

Hostmasking
It is possible to run the net as a number of separate interconnected sectors using a technique called hostmasking. For example, if every server in canada presents itself to the US servers as *.ca (how a server presents itself is defined in the N lines), and the US servers present themselves to the canada servers as *.edu, then the servers can route normally within their sectors, but only one intersector (us--ca) link can be made. This is useful for minimizing intercontinental or overseas links while still maintaining the flexibility of allowing many servers to make this connection. However, if a sector is too small, routing within the sector may sometimes not be sufficient and a server may get isolated.

H: vs. L: lines
2.8+ servers require the use of an H or L line for each CN pair. If an H line is not provided, an L line will be assumed. It is recommended that you hub all your links unless they expressed disinterest in hubbing. It is imperative that you don't forget to H: line all hub servers you connect to.

Q: lines
Please never use Q lines on the Undernet. The use of Q lines can fracture the net.

O: Lines
For security reasons, every server should use oper password encryption. There is a define in the config.h file for CRYPT_OPER_PASSWORD which should be defined. This will require you to encrypt your O line passwords, for which there is a mkpasswd program in ircd/crypt/. Encrypting oper passwords makes it difficult for others to find out a password and become an oper illegitimately.
Cracking passwords is not impossible, and it is recommended mixing letters and numbers or lowercase and uppercase in oper passwords to make it more difficult to crack. A sample O line for me with password "rabbits" is shown below, using connection class 5.

O:<User@Host>::::
O:*cjp@net*.net.com:7RcBJo.:_Chris_::5
O:*man@*torfree.net:7RcBJo.:_Chris_::5
O:*jpl.gov:7RcBJo.:_Chris_::5

Sysadmin Approval
It is imperative that if you run a server on the Undernet, you seek approval or permission from whoever is in charge of the machine on which you plan to run it. The Undernet has lost a number of servers due to admins deciding the servers were not necessary, and we would like to avoid this situation whenever possible.

Test Servers
Many opers from time to time desire to set up a test server to test out a new version or patch. The lifespan of these servers should be short, spanning only the time required for the test, and the one uplink for the test server should be the testing oper's main server unless the test dictates otherwise.

Server Troubleshooting
A server might not start correctly if it cannot resolve all the host names in it's C/N lines into IP numbers. If you are having problems starting your server, you may wish to comment out all CN pairs but one, and for that pair use an IP address instead of a host name. If your server starts with this configuration, then you need to find which
hostname in the file your server cannot resolve. Alternatively, you could attempt to lookup each hostname using nslookup. If this doesn't solve the problem, consider enabling debugging in config.h and running your server with debug enabled (ircd -t -x ), or if you understand the output, use strace (strace ircd or strace -p pid>).

Oper Responsibilities

Oper duties and powers
As oper, you have several abilities that non-opers do not. Though some of these, such as rehash, only effect your server, most effect the entire Undernet. Thus you should use these powers responsibly and with care to provide a friendly, usable environment for other users.

Killing

USAGE: /kill
A user should be killed only if absolutely necessary or if requested by the user him or herself. Problems with users should be worked out through messaging if possible. Killing for revenge is frowned upon the Undernet, but examples of necessary kills are for users who are cloning, flooding, or attempting to flood (CTCP) users off IRC, and
other cases of abuse where they do not respond to requests to stop. These are only guidelines, of course; in the end the choice is yours, but keep in mind the philosophy of the Undernet when exercising a kill.
Mass killing can almost always be avoided. In many cases, killing the owner/source of the clones will cause the clones to be killed as well. Note that auto reconnecting clones use up more bandwidth with the kill and the connect than blocking them from channels and having them die by themselves. With the introduction of GLines (global
temp klines), you can gline them and have servers autokill them, where they are essentially klined from all servers for a given time period. At the time this manual was written, glines have yet to be approved; the guide will be updated once the command interface has been determined.

SQuitting and Connecting
USAGE:
/squit
EXAMPLE: /squit ann-arbor.* Rerouting
USAGE:
/connect
EXAMPLE: /connect chicago* 4400 hub.*
Causes hub.* to attempt to connect to chicago.* on port 4400
USAGE:
/connect
EXAMPLE: /connect hub.* 4400
Causes your local server to attempt to connect to hub.*, port 4400.

The commands squit and connect give you the ability to reroute the Undernet. The use of these commands, of course, is restricted to the connections that are listed in the servers' ircd.conf file. That is, you cannot connect two servers unless they have CN lines for one another. But even with this restriction there remains much flexibility
in the routing of the net. It is recommended that you wallop your planned squits and connects to see if other opers have any problems with your plans.
The squit command disconnects the named server from the server next closest to you. For example, if I am on server B in a net routed as A--B--C--D, and I squit D, the net will break into A--B--C and D. But if I squit C, the net breaks into A--B and C--D. Note this is not the same result that an oper on server D gets when squitting C, breaking the net into A--B--C and D. It is a very good idea to /trace the server you wish to squit before squitting it so you can judge what impact the squit will have on the net.

Please do not squit in an attempt to remove a server from the net, as this is pointless. Squit does not permanently disconnect a server; the disconnection is only temporary. The server will soon automatically reconnect. Squitting is not a reasonable means for dealing with problems on the net.

Rerouting
Rerouting can be beneficial at times, providing users with a less lagged connection, but can also be very disruptive, especially if many servers are rerouted at nearly the same time. Server outages, loss of connections, and other events can cause servers to switch to links besides their primaries. A netsplit can be much more annoying than a barely noticeable lag. There will be times, however, when rerouting is necessary. For example, if a hub is upgrading or for some other reason must shut down or restart, then connections may be rerouted to other servers. This rerouting would most likely provide a better final routing than the chaotic scramble for connections that occurs when a hub with many links shuts down. However, when rerouting, keep in mind the effect on users and attempt to warn using WALLOPS. To determine which servers are lagged, ping a large channel, or /trace different servers. If you are /dieing or /restarting your server, inform all local users by a /msg $.
If any rerouting is to be done, after the impact of the rerouting has been assessed with /trace, you should announce your intentions on WALLOPS to see if other opers have anything to say about your decision.

Restarting and Dying
Please note that the commands restart and die will cause a server to drop all links. Please use these commands responsibly. These commands also have the potential to effect many users of the net.

New Server Links
Links are only given out as approved by routing-com. The procedure for new links is as follows. Application sent to routing-com, routing-com informs the applicant, and should it be approved, the applicant contacts the hub admins to setup C/N pairs.

Granting Oper Status
Please be careful when granting oper status on your server. Keep in mind that opers can have a great effect on the net and that these opers will reflect directly on your server. Ask other opers their opinions of people you are considering adding to your oper list. Also, feel free to email any appropriate closed list.

Using UWorld and UWorld2
Uworld currently exists to provide opers with the ability to do the following: clear channel modes, block clients from channels, change channel modes, kick users on channels, and count the number of users matching a given mask.
The usage of these commands follows:

Clearing Channel Modes
USAGE:
/msg uworld clearchan #chan
AVAILABLE ON: uworld and uworld2
This command clears a channel of all modes: bans and any of the following: +nmlkistp.
UWorld does NOT deop chops (channel +o's); however, UWorld2 *DOES*.
Before using clearchan, try your best to determine if the person requesting a clearchan is a valid op on the channel. There's no good way to really do this aside from using your instinct. Try getting the person requesting the clearchan to get at least two other people to support the request. It's a tricky issue, but it's important that you don't just clearchan every unjoinable channel that you hear about. You're always welcome to wallop for feedback from other opers.

OpCom KICK / OpCom MODE
OpCom KICK and OpCom mode are methods available to operators for dealing with channel affairs without having operator status on the channel.
OpCom MODE usage is exactly the same as usage for the /MODE command. For example, to op _Chris_ and _Chris2_ on #narf, you would /msg uworld opcom mode #narf +oo _Chris_ _Chris2_.  OpCom KICK usage is also exactly the same as usage for the /KICK command. To kick _Chris_ off #narf for flooding, you would /msg uworld opcom kick #narf _Chris_ flooding.
I can't really see any reason why using opcom kick would be used, but that's how one does it.  Do not interfere in channels unless there has been a takeover situation and the channel is inaccessible or closed; for example, it is +i, +k, or +l. Furthermore, don't op in channels unless they are opless (or opless aside from X and W), or channel abuse such as flooding is occurring. In the latter case, it's suggested that an oper join, op himself, deal with the flooders, and leave the channel. 
Before opping in opless channels, join the channel and verify that the person you plan to op isn't a problem with the current channel members. Saying something along the lines of "hello, I'm an Undernet ircop; does anyone object to having _Chris_ opped?" is suggested.

UWorld AutoBans
USAGE:
/msg uworld autoban user@host
/msg uworld remautoban user@host
/msg uworld bans
AVAILABLE ON: UWorld only
Autoban causes UWorld to auto ban any client from the given user@host mask upon its joining of any channel. When a client matching the autoban list attempts to join a channel, ChanSvr will automatically ban them as per the mask specified when they were autobanned, proceeded by a '*!*'.
** AutoBans last ONE hour. **
If you wish to remove a ban early (or you made a typo), use the remautoban command.
Finally, a list of current auto bans is available with 'bans'.

How to disable and reenable UWorld/UWorld2
In the event of an emergency, and only an emergency, when UWorld or UWorld2 is acting up, the following enabling/disabling commands can be used. This is merely an FYI section, and the disabling of UWorld or UWorld2 will almost definitely never be necessary. Most definitely, you should use /wallops before doing any of the following. People will hurt you if you squit UWorld accidentally, don't question this :).

ENABLING UWORLD: Can not be done from irc, only WildThang can do it.
DISABLING UWORLD: /squit uworld.*
ENABLING UWORLD2: /msg {X/W} uworld on
DISABLING UWORLD2: /msg {X/W} uworld off

Counting Clients From a Given user@host
To have UWorld count the number of clients from a given host,
/msg uworld scan *mask@mask. Examples follow:
/msg uworld scan *cjp@*
/msg uworld scan *@*netcom.com
/msg uworld scan *portman@*torfree.net

Problem Solving
If you have or notice a problem on the Undernet that is not urgent, then the preferred way of dealing with the problem is to either contact the involved parties through IRC or email, or to post the relevant facts to appropriate mailing lists. Urgent in this case will mean not tearing the net apart or making it unusable. Whether you contact the involved parties yourself, or post, is up to you and will vary from situation to situation, depending on how accessible the parties are, what kind of problem it is, and how large a scope the problem involves. If you do post, you are encouraged to include all the facts and keep personal opinions and emotions to a minimum (or at least flag them as such) so that all opers may see the situation for what it is and make their own judgments. This also will keep us working together and not yelling across the mailing list at one another.

Handling Critical Problems
If an oper or server on the net is misbehaving, splitting the net or making the net otherwise unusable, then drastic measures may be in line. A server may be juped (replaced with a fake to prevent the real server connecting) to solve such a problem. Juping is certainly NOT a preferred method and should ONLY be used as a last resort, or when ABSOLUTELY necessary.
Be sure to consult with other opers online before taking such a drastic measure. Because of the robust linking process, the chances of this happening are very nil; but it could happen, be it an oper-gone-crazy, an oper password cracked, or a misbehaving server.