Customizing Backtalk Authentication

Version 1.3.29

© 2002-2003 Jan Wolter, Steve Weiss

Backtalk is an advanced web-based computer conferencing system written and distributed by Jan Wolter and Steve Weiss. For general information about Backtalk, see the Backtalk home page. This document describes methods for customizing Backtalk's authentication system.

It is common for administrators to want to integrate a conferencing system like Backtalk with web-based applications that might be running on their site. They may already have some sort of database with user names and passwords, and they may want those logins and passwords to work with Backtalk as well. This kind of integration is generally difficult, because there are lots of different kinds of authentication systems on the web, but Backtalk is far easier to adapt to some foreign authentication systems than most packages of similar complexity.

Backtalk can be configured to use a large variety of different databases and techniques for authentication simply by using different options on the configure command. These are described in the installation manual. These cover a wide range of options, and may suffice to integrate Backtalk with many pre-existing authentication databases. However, in many cases some customization the Backtalk will be required.

Starting with release 1.1.7, we have made a concerted effort to modularize the Backtalk authentication system to make it simple to add new authentication options or customize existing ones. In general this will still require at least a modest amount of programming. The beginings of a flexible SQL interface has also been added which should eventually make it possible to integrate Backtalk with just about any SQL database without doing much programming.

All of this is still at a somewhat immature stage. It will continue to improve with time.

1. User Databases

Conceptually, user information in Backtalk is divided over several databases. In some configurations they may not all really be separate, but it is useful to think of them as such. Backtalk has built-in support for multiple different implementations of most of these. We'll compare them to the Unix equivalents, just to give a clearer idea of what each database is. In most cases, actually using the Unix database is one possible (though unusual) option.

There are some databases that are not yet in modules, but which will probably be in future releases. These include users' settings (corresponding to the Unix .plan file and other "dot files" normally kept in the home directory), and user's participation files, which record which portions of the conferences they have read. It'll probably be a lot longer before the actual conference data is modularized.

Installers who wish to share user information between Backtalk and another program will need to decide which databases they actually want to share. If you are using basic authentication (--login=basic), then the authentication database is a must, because the user must be looked up in the authentication every time he or she tries to access Backtalk. If you are using form-based authentication (--login=form) then the session database is more important, because Backtalk must be able to look the session ID up in the session database on each hit. You don't necessarily need to share the authentication database in that case, since you can use the other program's login page to generate the session cookies. If you share the authentication database but not the session database, users would have to re-login when moving to Backtalk, though their password would always be the same.

The identity database is generally nice to share, so a person doesn't have to change their full name in two different places if they get married and take their wive's last names. Since you will not want to have multiple account creation tools, there would be little use in sharing the next uid database. The group database may be useful.

2. Sharing Stock Authentication and Session Databases

We'll start with a discussion of the most important of the shared databases: the authentication database. This is influenced significantly by the type of authentication system you are using. For lots of information on this, see Jan Wolter's Guide to Web Authentication.

2.1. Basic Authentication Systems

Basic authentication is a user login system that is built into virtually all browsers and http servers. Typically the user will see a pop up box appear with a login/password prompt when he tries to access a secure area. Actual authentication will be done by the HTTP server, not by CGI programs like Backtalk. With Apache, the authentication will be handled by some standard authentication module, like mod_auth, mod_auth_dbm or mod_auth_db, or by some similar third party module, like mod_auth_external, mod_auth_pam or any of zillions of similar things.

Generally, in such configurations, the format of the authentication database will be dictated by the HTTP daemon's authentication module, which must be able to read it to do authentication. Backtalk fully supports the formats used by mod_auth, mod_auth_dbm and mod_auth_db as well as a combination of mod_auth_external and pwauth that authenticates out of a Unix password file.

Note that with basic authentication, the user's browser remembers his login and password and resends them with every request. So every page must be able to check authentications, and thus must be able to read the authentication database. Thus if want user to be able to access two applications both running basic authentication then they need to share an authentication database.

Luckily, this is probably easy, because the actual authentication checking is done by the HTTP server. So you just have to use the same authentication module pointing to the same authentication database, and everything will be peachy. You'll need some administration and account creation tools that work with the database, but very likely either those from Backtalk or from the other application will do just fine.

Sharing session databases in basic authentication systems is even easier, since there aren't any.

2.2. Form-Based Authentication Systems

Basic authentication systems have an often shabby looking pop-up box, and do not allow a user to logout without exiting all browser windows, so sites seeking a professional look and feel often avoid them, prefering some form of form-based authentication.

In these systems, the users find a login and password box in a form on the main browser window. When he submits his login info it is passed to the CGI program, which checks that the password is correct, and gives access to the next page only if it was. To avoid forcing the user to login again for every new page thereafter, the CGI sends back to the browser not only the page, but some form of certificate that will be automatically resubmitted to the browser with each future request. Normally it will either be contained in a cookie, or will be passed from page to page as a CGI argument. In the simplest case the certificate could just be the user's login and password, but that has obvious security problems. Most commonly the certificate is simply a random string of characters called a session ID. The CGI maintains a database that tells which user each session ID was issued to, so that when a session ID comes back, it can easily identify the user. Session IDs usually expire after a short time of not being used. Logging out is handled by invalidating the session ID.

Note that such systems typically have only one page that actually checks logins and passwords and thus only one page needs to have access to the authentication database. This can be handled by either Backtalk or the other application, whichever is handy, so the authentication database does not have to be shared. Obviously, whichever application is handling logins should probably also be handling account creation and administration.

However, all secure pages will need to check the session database, so that must be shared between applications. This is going to be a bit more of a challenge to accomplish than sharing the authentication database under basic authentication was. In that case the HTTP server imposes some regularity on the database formats, while in this case an application could use any database format you'd care to imagine, and the odds that one program will be able to read the other's database are low. However, if it is in an SQL table, it may prove fairly easy to teach Backtalk to read it.

Another issue is the passing of the session IDs. Backtalk currently supports only the cookie option. If your other application passes it as a CGI variable, this is a problem that probably isn't very hard to solve but that we haven't addressed yet. The easiest solution may be to modify the login program to put the session ID in a cookie as well as passing it as a CGI variable.

You'll want to check SESSION_ID_SIZE in incl/sess.h to make sure that it is bigger than the session IDs generated by the other application. Otherwise Backtalk will truncate the session IDs passed to it, and fail to find them in the session database.

If you are going to use the foreign program to handle logins and authentication then you'll probably want to build Backtalk with the --auth=none option so it can't authentication. You'll want to set register_url in the script/config.bt so that Backtalk's "register" links will point to the foreign creation program. Also change script/public/login.bt to something like:

 /http_location (http://www.site.com/cgi-sys/otherprogram/login.cgi) store
With the URL pointing to the login page for the other site. This way if they hit Backtalk with a bad or expired session ID, they will be directed to the correct login page. This will probably break Backtalk's ability to get back to the page you originally requested after a re-login, unless the foreign login program has support for that and you can figure out how to tie into it.

3. Customizing The SQL Modules

If you are trying to share an SQL database with some application other than Backtalk, then you'll have problems with the fact that Backtalk's default SQL tables are likely to have different table names, field names and field sizes. They tables may even be structured differently. Thus, you may want to change the table definitions used by Backtalk to make them agree with your other applications. If you know SQL, this isn't difficult.

Backtalk does two things to achieve portability between different SQL systems. First, there is a collection of interface modules that give a common interface to different SQL servers. These include:

src/sql_pgsql.c
src/sql_mysql.c
src/sql_msql.c
src/sql_none.c
(At the time of this writing, only the src/sql_pgsql.c, src/sql_mysql.c and src/sql_none.c modules are fully implemented and tested.) These modules are designed to let Backtalk submit queries and get results from different SQL servers in a uniform way. Unless you want to support an SQL server not yet supported by Backtalk, you shouldn't have to fiddle with these.

However, different SQL servers also have small syntactical differences in the query languages that they understand. Thus, the queries submitted to different SQL servers might also have to be different. So Backtalk does not embed the SQL queries used for different operations in the C code. Instead, they are extracted at compile time from SQL query files stored in the sql subdirectory of the distribution. These will eventually include:

sql/pgsql/*
sql/mysql/*
sql/msql/*
Within each of these directories will be found the files containing the queries used to access each database. These include:
sql/pgsql/auth.sql
sql/pgsql/auth_ident.sql
sql/pgsql/group.sql
sql/pgsql/ident.sql
sql/pgsql/session.sql
sql/pgsql/nextuid.sql
The auth_ident.sql query set is used instead of the separate auth.sql and ident.sql if those are to be in the same table. Each of these files includes a set of definitions for the SQL queries to be used to execute different operations on the databases. By changing the queries, different database structures can be supported.

Note that if any changes are made to these files, then Backtalk must be recompiled. These are read only at compile-time, not at run-time.

Instead of formally documenting the format of these, we'll just look through some examples.

3.1. Macros

As an example, we'll start with the sql/pgsql/auth.sql file, which defines the default SQL commands to be used to create, access and update the SQL authentication database under PostgreSQL. The file starts with some macro definitions:
    &tab_auth = bt_user
    &col_auth_login = login
    &col_auth_pass = password
Any name starting with an ampersand is a macro. It is defined by assignment commands like the ones above. If the name appears anywhere later in the file, the value will be substituted. These three definitions define the name of the table and the two fields in it. If your table differs only in some field names (e.g., passwd instead of password), then it may be sufficient just to redefine these macros to make Backtalk work with it.

3.2. Create Query

Next we define the first of the queries. This is a query to create the authentication table:
    auth_create()
    {
        -DROP TABLE &tab_auth;
        CREATE TABLE &tab_auth (
               &col_auth_login     &type_login PRIMARY KEY,
               &col_auth_pass      &type_pass
        );
    }
Note that this is full of macros. The macros named &type_login and &type_pass are defined in the sql/pgsql/types.sql file. When they are all substituted in, this query look like:
    auth_create()
    {
        -DROP TABLE bt_user;
        CREATE TABLE bt_user (
               login        varchar(32) PRIMARY KEY,
               password     varchar(16)
        );
    }
This sequence of SQL statements is executed whenver we want to create the authentication databasse. This actually only happens during the installation process when you do 'make install-users', so if you aren't using the Backtalk install scripts to create your database, then there is no need to actually define this. You could just do
    auth_create()
    {
    }
The first SQL statement in the auth_create() query is prefixed with a dash. This means that it is OK if these query fails (normally this would be because the bt_user doesn't exist, so dropping it doesn't work, but that's fine).

3.3. Getpass Query

Next we define a more commonly used SQL query. This one is given a login name, and returns one row with one column containing the encrypted password for that account. It is used by Backtalk to authenticate users in form-based authentication systems.
    auth_getpass($login)
    {
          SELECT &col_auth_pass FROM &tab_auth WHERE &col_auth_login=$'login;
    }
Here we see another kind of variable. In addition to macros, which are expanded at compile time, we have parameters, which aren't expanded until the query is executed at run-time. These parameters have names that start with dollar signs. So after macro expansion, the auth_getpass() function would look like:
    auth_getpass($login)
    {
          SELECT password FROM bt_user WHERE login=$'login;
    }
If this is run with the login joe then any instance of the variable $login would be replaced with the value joe. In this case we have instead $'login which is replaced with the value quoted as an SQL string (typically enclosed in single quotes with any internal single quotes doubled, though this can vary on different SQL servers). Thus the actual query executed will be:
          SELECT password FROM bt_user WHERE login='joe';
After submitting this query to the SQL server, Backtalk will fetch results back, expecting to find one row of data containing just one column. Since the login field is defined as a PRIMARY KEY there will never be more than one row that satisfies this query.

It's perfectly possible to have more complex queries. If I had two tables, one with uid numbers and passwords and one with uid numbers and login names, then I could use a join:

    auth_getpass($login)
    {
          SELECT passtab.pass FROM passtab,nametab
             WHERE nametab.name=$'login AND passtab.uid=nametab.uid;
    }
It doesn't matter that the field name for the result is different (pass instead of password). Backtalk never pays attention to field names, only to column numbers.

If you wanted to give only some the accounts in the database access to Backtalk, the query could be restricted, for example:

    auth_getpass($login)
    {
          SELECT password FROM bt_user WHERE login=$'login
             WHERE enable_backtalk=1;
    }

3.4. Authentication and Identity Add Query

Further down, we find a query to add a new entry to the authentication table, which is used when creating new user accounts.
    auth_add($login,$pass,$uid,$gid,$fname,$dir)
    {
          INSERT INTO &tab_auth (&col_auth_login,&col_auth_pass)
            VALUES ($'login, $'pass);
    }
Note that this is passed several arguements that aren't stored in the table. In some installations the authentication and identity databases are shared, and thus stored in the same table. Then the auth_add() query actually adds all the data, while the ident_add() query, which normally creates new rows in the identity database, does nothing. So, if we look at the definitions in the sql/pgsql/auth_ident.sql file, we see:
    auth_add($login,$pass,$uid,$gid,$fname,$dir)
    {
          INSERT INTO &tab_auth
            (&col_ident_login,&col_ident_pass,&col_ident_uid,&col_ident_gid,
             &col_ident_fname)
            VALUES ($'login, $'pass, $uid, $gid, $'fname);
    }

    ident_add($login,$uid,$gid,$fname,$dir)
    {
    }
Thus, though the UID value is always passed to the query function, it isn't always used.

Note that in the shared auth_add() function, the insert statement refers to $uid instead of $'uid. We don't have any kind of typing here, so the query compiler doesn't actually know that the UID is a numeric value, not a string value. We need to be explicit about telling it which strings to quote and which not to.

3.5. Identity List Query

As a final example, the ident_users() query in sql/pgsql/ident.sql is used to list users in the identity database. Sometimes we don't want to list more than N users displayed per page, so this takes a optional argument:
    ident_users($n)
    {
          SELECT &col_ident_login,NULL,&col_ident_uid,
                 &col_ident_gid,&col_ident_fname,NULL
            FROM &tab_ident
            ORDER BY &col_ident_login
            $[n LIMIT $n];
    }
This query is expected to return six columns of data, the first with the login name, the second with the password (or NULL if the databases aren't shared), the third with the uid, the fourth with the gid, the fifth with person's full name, and the sixth with the directory name. Again, it is the column positions, not the field names that matter to Backtalk.

This also shows how optional clauses can be included. The LIMIT $n clause is only included in the query if the $n varible is defined.

Most likely, at least some of these fields will not exist in your identity database. Luckily, you don't really need them all.

login
Backtalk does need a login name. Some sites use an email address as the login name. This is fine for Backtalk. In any case, be sure that you have Backtalk configured with MAX_LOGIN_LEN at least as big as the biggest login that can stored in the database.
password
Return NULL if you don't have it or if in encrypted in some way Backtalk doesn't understand.
uid
Many systems don't assign unique ID numbers to users. With some SQL servers, you may be able to use the OID number of the database row. MySQL doesn't support this though, and they can change if the database is copied carelessly. Adding an auto-increment field to the database might work, if the other application is written so it won't be confused by added fields (a basic of good SQL programming, if you ask me). But if all else fails, just return a constant 1 for all users. Backtalk won't really mind much if users don't have unique uid numbers, so long as you aren't enabling direct execution, which you wouldn't do unless you are using Unix accounts anyway. The only loss is that if a user 'fred' gets his account deleted, and a new user takes out a new login with the name 'fred', then the new 'fred' will be able to edit the old one's postings.
gid
The foreign system may have no idea of groups. By default, Backtalk uses the group number 0 to indicate an admin account and 1 to indicate a user account. So if you don't need a Backtalk admin account, you could just return a constant 1 for this. Or you could do something like IF(username=='mike',0,1). This would make user 'mike' a cfadm, and everyone else a normal user.
fname
I've never seen an identity database that didn't have this, but often it is split into first and last names (a good practice, actually). Just return something like CONCAT(firstname,' ',lastname).
directory
Hardly any foreign systems will have home directories for their users. If you return NULL, Backtalk will concoct a suitable one based on the login name. This only solves half the problem though - the home directory probably was not created by the foreign registration program. So build Backtalk with the AUTO_CREATE_DIR option, and it will check for the existance of the directory every time a user logs in, and create it if it doesn't exist.

4. Writing Custom Database Modules

If you want to authenticate out of a database of a different type than those already supported by Backtalk, then you'll have to write some C code. In most cases this is relatively easy to do. Let's consider, for example, writing a new session database interface. If you look in the src subdirectory of the distribution, you'll find three existing session modules:
   src/sess_none.c
   src/sess_file.c
   src/sess_sql.c
Only one of these files will be linked in on any given build of Backtalk. The 'src/sess_none.c' file is used when we aren't using a session database. To create a new session module, you just need to write a new file, named something like 'src/sess_foo.c' and say '--session=foo' on the configure comand. The 'src/sess_foo.c' file would need to provide implementions of three functions: (Actually, it's generally smarter to ignore the IP addresses these days.)

Most other database interfaces are similar. The authentication and identity modules are a bit more complex, however, since they interact (sometimes being shared) and since not all interfaces implement all functions.

Each of these modules has both a source file and header file:

   src/auth_db.c         incl/auth_db.h
   src/auth_dbm.c        incl/auth_dbm.h
   src/auth_none.c       incl/auth_none.h
   src/auth_passwd.c     incl/auth_passwd.h
   src/auth_shadow.c     incl/auth_shadow.h
   src/auth_sql.c        incl/auth_sql.h
   src/auth_text.c       incl/auth_text.h

   src/ident_hash.c      incl/ident_hash.h
   src/ident_passwd.c    incl/ident_passwd.h
   src/ident_shadow.c    incl/ident_shadow.h
   src/ident_sql.c       incl/ident_sql.h
   src/ident_tagfile.c   incl/ident_tagfile.h
   src/ident_text.c      incl/ident_text.h    
   src/ident_yapp.c      incl/ident_yapp.h
For the most part, the headers just define which of the functions are actually implemented by any given module. If src/auth_text.c defines a auth_getpass(login) function, then src/auth_text.c will defined the symbol HAVEAUTH_GETPASS.

At compile time, Backtalk will figure out which of the available routines it actually will use in any given configuration, which depends on the combination of identity and authentication modules installed and on various other system configuration settings.

The complete set of functions and defines is summarized in the table below. (Sorry, This table is huge and not very Lynx-readable).

The modules included in the distribution are designed to be flexible and so implement as many different routines as possible, but custom modules can often be much simpler. If you have an existing database, you probably already have tools to create, edit and destroy settings, so you typically won't need any of the edit function. Mostly it will suffice to implement just one of the three "walk" functions.

functiondefinewhat it doesneeded? auth_textauth_db
auth_dbm
auth_sql auth_passwdauth_shadow 
auth_getpass(login) HAVEAUTH_GETPASS Returns the encrpyted password for the user, or NULL if the user is not found. Optional. Without it we can't check passwords or status, unless the auth file is shared with the ident file, in which case we'll use ident_getpwnam instead. Yes Yes Yes No. Use Default.No.
auth_checkpass(login,pass) HAVEAUTH_CHECKPASS Given a login and a plain text password, return true if they are valid. This is only used with --login=cookie. In that case we must have either auth_checkpass() or auth_getpass(). If auth_checkpass() is not implemented, we compare the result of auth_getpass() with the result of running crypt() on the password. No. Use Default. No. Use Default. No. Use Default. No.Yes.
auth_walk(flag) HAVEAUTH_WALK If flag is 0, return the login name for the "first" user in the authentication database. Otherwise, return the "next" user in the database. Required only if neither ident_walk() nor ident_walkpw() is defined. Yes Yes Yes No. Use Default.No. Use Default.
auth_seek(login) HAVEAUTH_SEEK Arrange things so that the next call to auth_walk(1) will return the user after the given user. Used only if neither ident_walk() nor ident_walkpw() is defined, and optional even then. If it is missing, we use auth_walk() to go sequentially through the database until the desired login is found. No. Use Default. Yes. Yes. No. Use Default.No. Use Default.
auth_newpass(login,encpw) HAVEAUTH_NEWPASS Change a user's password, given his login and encrypted password. Optional. If not implemented, Backtalk cannot change passwords. Yes. Yes. Yes. No.No.
auth_add(login,password, status,uid,gid,fname,dir) HAVEAUTH_ADD Add a user to the authentication database. The uid ,gid ,fname and dir arguments should be ignored if the authentication and identity databases are not shared. Otherwise they should be saved too. Optional. If not implemented, Backtalk cannot create users. Yes. Yes. Yes. No.No.
auth_del(login) HAVEAUTH_DEL Delete a user from the authentication database. Return non-zero if the user does not exist, 0 on success. Optional. If not implemented, Backtalk cannot delete users. Yes. Yes. Yes. No.No.
functiondefinewhat it doesneeded? ident_textident_hashident_sql ident_passwdident_shadow ident_tagfileident_yapp
  IDENT_STOREDIR Defined if the identity database stores a home directory for each user. If true, ident_getpwnam(), ident_getpwuid(), and ident_walkpw() should return the stored value. If not, they should return pw_dir=NULL. Optional. If directories are not stored, they are computed from the login name. Yes. Yes. No. Yes.Yes. No. No.
  IDENT_STOREEMAIL Defined if the identity database stores an email address for each user. This is normally not done - email addresses are normally kept in a different database, but the Yapp ident module does it for compatibility reasons. If true, ident_getpwnam(), ident_getpwuid(), and ident_walkpw() should return the stored value in the pw_shell field, which is otherwise not used. Optional. If email addresses are not stored in the ident database, then they are stored in the user's .backtalk file. No. No. No. No.No. No. Yes.
ident_getpwnam(login) HAVEIDENT_GETPWNAM Given a login name, return a 'struct passwd' (see `man getpwent') with at least pw_uid, pw_gid, and pw_gecos (fullname) fields set. pw_name, pw_passwd, pw_dir, and pw_shell may be either set to something, or NULL. Return NULL if no user with that name exists. Always required. Yes. Yes. Yes. Yes.Yes. Yes. Yes.
ident_getpwuid(uid) HAVEIDENT_GETPWUID Same as getidnam, but finds the user with the given UID number instead of the given login and the returned passwd structure must have both pw_name and pw_uid defined. Required only if built with --enable-exec option. (Which is only useful if using Unix accounts, with ident_passwd.c and ident_shadow.c). Yes. Yes. Yes. Yes.Yes. No. No.
ident_walkpw(flag) HAVEIDENT_WALKPW If flag is 0, return the passwd structure for the "first" user in the database. Otherwise, return the "next" user in the database. Returned password structure should have pw_name, pw_uid, pw_gid, and pw_gecos. Either ident_walkpw(), ident_walk() or auth_walk() is required. If this is not defined we do ident_getpwnam(ident_walk(flag)) or ident_getpwnam(auth_walk(flag)) so unless you can do better you don't need to implement this. Yes. No. Use Default. Yes. Yes.Yes. No. Use Default. No. Use Default.
ident_walk(flag) HAVEIDENT_WALK If flag is 0, return the login name for the "first" user in the database. Otherwise, return the "next" user in the database. Either ident_walkpw() or ident_walk() is required. If ident_walk() is not defined we do auth_walk() or ident_walkpw(flag)->pw_name, so this need only be provided if we can do something faster. No. Use Default. Yes. Yes. No. Use Default.No. Use Default. No. Use Default. No. Use Default.
ident_seek(login) HAVEIDENT_SEEK Arrange things so that the next call to ident_walk(1) will return the user after the given user. Optional. If not provided, we do ident_walk(0) followed by ident_walk(1) until we arrive at the given login. No. Use Default. Yes. Yes. No. Use Default.No. Use Default. No. Use Default. No. Use Default.
ident_newfname(pwd,fname) HAVEIDENT_NEWFNAME Change the user's full name. pwd will have the full set of old identity information including login name. Optional. If not provided, fullnames cannot be editted through Backtalk. Yes. Yes. Yes. No.No. Yes. Yes.
ident_newgid(pwd,gid) HAVEIDENT_NEWGID Change the user's primary group id. pwd will have the full set of old identity information including login name. Optional. If not provided, gids cannot be editted through Backtalk. Yes. Yes. Yes. No.No. Yes. Yes.
ident_newemail(pwd,email) HAVEIDENT_NEWEMAIL Change the user's stored email address. pwd will have the full set of old identity information including login name. Never called unless IDENT_STOREEMAIL is defined. Optional. If not provided, stored email addresses cannot be editted through Backtalk. No. No. No. No.No. No. Yes.
ident_add(login,uid,gid, fname,dir) HAVEIDENT_ADD Add a new entry into the identity database. Optional. If not provided, users cannot be created. Not required if identity database is shared with the authentication database. Yes. Yes. Yes. No.No. Yes. Yes.
ident_del(login) HAVEIDENT_DEL Delete the named user from identity database. Optional. If not provided, users cannot be deleted. Not required if when the identity database is shared with the authentication database. Yes. Yes. Yes. No.No. Yes. Yes.


contact page Wed Feb 26 16:33:42 EST 2003