AUTHOR: Matthias S. Benkmann <article at winterdrache dot de>

DATE: 2007-10-20

LICENSE: Creative Commons Attribution-Share Alike 3.0
         (http://creativecommons.org/licenses/by-sa/3.0/)

SYNOPSIS: More Control and Package Management using Package Users (v1.4)

DESCRIPTION:
-You want to know which packages your files belong to ?                 
-You want to deinstall software that doesn't have make uninstall ?      
-You are bothered by programs installed setuid root behind your back ?  
-You don't like packages quietly overwriting files from other packages ?
-You don't like package managers like RPM ?                             
-YOU WANT TOTAL CONTROL USING ONLY UNIX BUILTINS ?                      

ATTACHMENTS:
http://www.linuxfromscratch.org/hints/downloads/attachments/more_control_and_pkg_man/more_control_helpers.tar.bz2

PREREQUISITES:
For use with LFS book 6.2: Brain.
For use with LFS book later than 6.2: Brain (awake, good working condition).

HINT:

########################################################################### 
 Contents
###########################################################################

1. Preface
2. Overview

- PART 1: General Information -

3. Package Users
  3.1 Introduction
  3.2 User Name
  3.3 Groups
  3.4 Home Directory
4. Common Problems
  4.1 Introduction
  4.2 General Procedure
  4.3 Permission Changes
  4.4 Ownership Changes
  4.5 Write to Non-Install Directory
  4.6 Delete or Overwrite File
  4.7 /sbin/ldconfig
  4.8 What Commands to Run as a Package User
5. The more_control_helpers Archive
  5.1 Overview
  5.2 The Wrappers
  5.3 add_package_user/install_package
  5.4 forall_direntries_from
  5.5 uninstall_package
  5.6 list_suspicious_files/list_suspicious_files_from
  5.7 list_package
  5.8 grep_all_regular_files_for
  5.9 The etc Directory
  5.10 Temporary Files
  
- PART 2: LFS Specifics -

6. Pre-Chroot Phase (Chapter 5)
7. Chroot Phase (Chapter 6)
  7.1 Preparations
  7.2 Walkthrough: Installing linux-libc-headers
  7.3 Known Issues with LFS Packages
8. Sanity Checks
  8.1 Suspicious Files
  8.2 References to Temporary Files

- APPENDICES -

A. Security Issues
  A.1 NFS
  A.2 Daemons
B. Package Categories
C. Acknowledgements and Changelog
  

########################################################################### 
 1. Preface
###########################################################################

Let's say I have written a program that you would like to use. To make it
easier for you I come over to install it for you. Would you give me the root
account and then leave the room ? No ? Then why do you give it to complete
strangers who you have never seen in your life, to install software packages
pulled from some Internet server, that come with no warranty and don't even 
list their contents in the README, although they will happily spread them all 
over your system ?

It is a mystery why Unix admins who wouldn't even trust their employer with
more than a normal user account carelessly execute complex and incomprehensible
installation scripts with full root rights.

Users and groups are the basic security principle in a Unix system. They have
been used successfully for a long time to monitor who has created a file and 
to control who is allowed to delete or change it. But this control has only 
been imposed on the files of ordinary users. What a waste! I suggest to extend
this control to all system files. 

The general idea is to create package users, i.e. user accounts with restricted
rights, to build and install software packages, rather than doing these tasks 
as root. Not only does this give you more control over what build and install 
scripts may or may not do, it can also serve as a quite useful package 
management system.


#############################################################################
 2. Overview
#############################################################################

This hint is divided into 3 parts. The first part contains general information
about the package user method. This part is the most important part of the
hint. Read it thoroughly. The second part explains how to apply the package 
user method to the building of an LFS system. 
Finally, part 3 of this hint is the Appendix with information that would not
fit anywhere else or that is not of general interest.

It is inevitable that part 2 will become outdated with time as the LFS book 
changes and new versions of the software packages used with LFS are released. 
I make no attempt to track these changes. 
When someone reports an issue with a package I will incorporate
it into the hint, but larger changes that might be required due to changes in
the LFS build methodology could take a long time to get included. The reason 
for this (aside from lack of time) is that I consider part 2 as bonus material 
that helps people get started but is not essential. Part 1 describes the 
concepts, which are independent of package versions or the LFS book, and you 
will have to rely on this information whenever part 2 fails. Don't forget 
that part 2 only deals with the packages used by the LFS book. For all the 
other packages you install on your system after that even an up-to-date 
part 2 would offer no aid anyway.

The previous paragraph might sound discouraging, and as you read more from the
hint it is possible that you get the impression that the package user
method is complicated, causes lots of difficult problems and is overall too
much trouble for anyone but a real hardcore admin with programming experience.
But you would be mistaken. 
First of all, many things experienced as installation problems when working
with the package user system are in fact desirable features. 
If `make install' fails for some package, because it attempts to install a
file with the same name as a pre-existing file from another package, you
should not curse the fact that you have to spend additional time to resolve
this issue. Instead you should be happy that you have been alerted of this
collision that, had it gone unnoticed, could have messed up your system in
more or less subtle ways.
Secondly, the package user system is not an all-or-nothing approach. It
works on a per-package basis. If a package gives you too much trouble, you
can always decide to chicken out and finish the installation as root.
Finally, the more_control_helpers archive provided with this hint contains
several useful scripts that automate many aspects of software installation
as a package user and, together with the tips given in this hint, add a lot 
of value to the package user system. 
So do not pass judgement until you have read at least the complete part 1,
including the description of the more_control_helpers.


---------------------- PART 1: General Information --------------------------


#############################################################################
 3. Package Users
#############################################################################

 3.1 Introduction
 ----------------

The basic idea of this scheme is easily explained. Every package belongs to a 
certain "package user". When you install a package, you build and install
the package as this package user, causing all files that are installed to be 
owned by the package user. As a consequence all the usual package management 
tasks can be comfortably achieved through the use of standard command line 
utilities. A simple `ls -l <file>' will tell you, for instance, what package 
<file> belongs to and a `find -user ...' command allows you to perform an 
operation on all the files belonging to a certain package, e.g. delete them 
to uninstall the package.

But package management is not all that package users are good for. Because
package users do not have root-rights, the installation of a package is
limited in what it can do. One thing that a package user is not allowed to do,
for example, is to overwrite files from a different package user. Clashes
between different packages that want to install a binary, library or header 
file of the same name are more common than you might think. With package users
you never run the risk of package B's installation destroying files from 
package A silently without you noticing. Every attempt of doing this during
package B's installation will cause a "Permission denied" or
"Operation not permitted" error so that you have the chance of taking 
appropriate steps.
Another thing that package users are not allowed to do is install setuid root 
binaries. The decision to make a binary setuid root is also something that a 
prudent admin does not want to leave up to the creator of a software package.

Usually package user accounts have no valid password so that only root can su 
to a package user, which ensures that package users do not open an additional 
way into the system and undermine security. But you *may* set passwords 
anyway to allow a co-admin who you want to be able to install and maintain 
certain software packages to do so without having access to the actual root 
account. This co-admin could for instance install, delete, change additional 
libraries which might be necessary for his workgroup. He would be unable, 
though, to remove or modify libraries which don't belong to him/her, such as 
libc.


 3.2 User Name
 -------------

You don't need to drive yourself nuts trying to come up with 8 character
names for the package users. I always use the name of the package without 
the version number, including dashes and possibly exceeding 8 characters in 
length, e.g. "util-linux", and in the several years that I've been using this 
scheme I have not encountered any problems, nor has anyone else reported 
trouble. The 8-character limit on user names seems to be a thing of the past.

TIP:
  You can use bash's programmable completion feature to save yourself some 
  typing when entering commands that take a user name as an argument, such as
  su, finger or pinky. The command 
  
      complete -o default -o nospace -A user su finger pinky
      
  tells bash to tab-complete words as user names for the commands su,
  finger and pinky.
  With this in place you can simply type `su linux-li<TAB>' and bash
  will complete this to `su linux-libc-headers' (assuming that you have a
  package user named "linux-libc-headers").
  "-o default" tells bash that if a suitable user name does not exist, the 
  default completion shall be attempted. 
  "-o nospace" prevents the addition of a space after the completed word.
  
  This is a very useful command to put into root's .bashrc and .bash_profile.
  
  BTW, at http://freshmeat.net/projects/bashcompletion/
  you can find a project that offers sophisticated completions for many
  other commands.
  
  Or switch to zsh (http://freshmeat.net/projects/zsh/). It's more powerful
  and less buggy than bash.


 3.3 Groups
 ----------

Every package user belongs to at least 2 groups. One of these groups is
the "install" group, which all package users (and only package users) belong
to. All directories that packages are allowed to install stuff in belong to 
the install group. This includes directories such as /bin and /usr/bin but 
excludes directories like /root or /. 
The directories owned by the install group are always group-writable. 
This would be enough for the package management aspects, but without further 
preparation this would not give added security or control because every 
package could replace the files from a different package (the change would 
be visible in the output from `ls -l', though).
For this reason all install directories get the sticky attribute. This
allows users to create new files and delete or modify their own files in
the directory, but files from other users can not be modified or removed.
In the rest of this hint, whenever the term "install directory" is used, it
refers to a directory that belongs to group install, is group-writable and
sticky. IOW, to turn <dir> into an install directory you would do

    chgrp install <dir> && chmod g+w,o+t <dir>

Although the install group is crucial for the package user system, it is 
implemented as a supplementary group, rather than as the primary group for
package users. This has at least 2 advantages. 
One advantage is that this makes it easy to get a list of all packages 
installed on the system with the command

    grep install /etc/group

A more important point, however, is that the primary group is the
one that files created by the package user will belong to. So it will be 
printed in the output of `ls -l' and is subject to find's "-group" test. 
This makes it very useful for organizational purposes. 
Following are some suggestions for how to use the primary group. 

1. group name = user name

   Under this scheme the package user for the bash package would be
   bash:bash. `ls -l /bin/bash' would show something like this
   
   -rwxr-xr-x    1 bash     bash    1731859 Feb 30  2005 /bin/bash
   
   An important advantage of this scheme is that the user information is
   not lost when you make a file setuid root, which requires changing
   the file's owner. Because of this advantage, this scheme is the one
   recommended by this hint. However, the hint's instructions will work
   fine if you choose a different scheme.

2. group name = package category

   Under this scheme, you would have certain package categories, such as
   games, system, net,... and bash, being a system program, would possibly
   belong to the system group, so that `ls -l /bin/bash' would show something 
   like this
   
   -rwxr-xr-x    1 bash     system    1731859 Jul  4  1776 /bin/bash
   
   This system is nice, but probably not as useful as #1 above, unless you
   have a real use for this categorization.
   For a possible categorization see Appendix B at the end of this hint.

3. group name = identifier for a real group of people
   
   Under this scheme, the group would correspond to a real group of people in 
   meatspace, e.g. the group of admins responsible for the package. 
   If you need something like this you'll know best what it looks like and how
   to implement it, so no further discussion of this method will be given here. 
   

 3.4 Home Directory
 ------------------
 
Although it is well possible not to have a valid home directory for package
users or to have just one home directory shared by all package users, that
would be a wasted opportunity. Having individual home directories for the
package users offers a nice way to organize tarballs, patches, build scripts,
notes and all the other per-package information that you accumulate with time.

I suggest to use the home directory /usr/src/<package> for a package user
called <package> with the contents detailed below. The more_control_helpers 
archive contains scripts and skeleton files that implement this suggestion.

  .bash_profile: 
           You will usually want to have the same environment for all package
           users, so it is a good idea to make .bash_profile a symbolic link
           to a file in a central location. The more_control_helpers example
           uses /etc/pkgusr/bash_profile for this purpose.
           
  .bashrc: 
           As for .bash_profile a symlink is a good idea for .bashrc. The
           more_control_helpers example uses /etc/pkgusr/bashrc as link target.
           Under normal circumstances package users are not 
           (and even can not be) used for logging into the system, so there 
           is little reason to distinguish between login and non-login shells 
           for package users. Therefore, the example bashrc from 
           more_control_helpers simply sources .bash_profile.
           This makes sure that the same environment will be used, regardless
           of whether `su <package>' or `su - <package>' is used to become
           the package user.
           
  .project: 
           The contents of this file are printed by the commands
           `finger -l <user>' and 'pinky -l <user>' so .project is a
           good place for putting information about a package. You should 
           keep the contents of the .project files for your package users 
           up-to-date.
                       
  source code:
           The package user's home directory is the perfect place for storing
           a package's source code. This includes tarballs for different
           versions, CVS checkouts, unpacked source trees for building,...
           
  build script(s):
           Package user installations require more careful examination of build 
           and install messages than installations done as root, because of
           the package user-specific problems that can occur. Therefore it is
           unwise to simply copy'n'paste installation instructions from the
           LFS book. Build scripts allow you to use sophisticated output
           redirection for logging purposes that is impractical for direct
           entry on the command line. The build script skeleton included in
           the more_control_helpers archive demonstrates this.
                       

############################################################################
 4. Common Problems
############################################################################

 4.1 Introduction
 ----------------

Software installation is the crux of the package user system. Because 
installation scripts are often written under the assumption that they will be 
executed as root, they sometimes fail when executed by a package user.
Once this hurdle is passed and a package has been installed, there's usually no 
difference to a root-installation. A few programs insist that certain 
security-sensitive files be owned by root and will not execute otherwise,
but this is the rare exception. 
This chapter presents some more or less common problems that you will 
encounter when using package user accounts to install software, together with
guidelines on how to deal with these issues. 
Although I've said it before I will say it again: Many of the problems you
encounter during a package user installation are desirable features of the
package user system. You want installation to fail rather than have 
potentially dangerous actions performed behind your back with root rights.

 
 4.2 General Procedure
 ---------------------
 
When an installation fails it is almost always due to a "Permission denied"
or "Operation not permitted" error while executing a command during
`make install'. The first thing you have to do is identify the command that
is causing the problem. Usually you will find this in the make output right
before the error message. Once you have identified the culprit, you have to
decide whether the action that is attempted is illegitimate, partially
legitimate or completely legitimate. Illegitimate commands can simply be
removed from the Makefile. The other 2 possibilities are more difficult to 
deal with. You either have to change the condition that makes the command fail
or you have to change or sometimes remove the command and make a note if your
change suppresses a legitimate action.

After you've made changes to solve a certain problem, you reattempt the
installation and solve any remaining problems until the installation
succeeds. Once you've reached that point it is time to perform any remaining
legitimate actions that you've had to disable, such as make certain binaries
setuid root.

Note that often Makefiles are generated during the configure step, sometimes
even later in the build process. If you want to apply changes before the
configure step you will usually have to edit files called "Makefile.in".

 
 4.3 Permission Changes
 ----------------------

Some unsophisticated build systems that don't use the mkinstalldirs script to 
create installation target directories are very poorly written. Instead of 
testing whether a target directory exists, they simply attempt to create 
it with default permissions. This problem usually manifests as a line such
as "install -d $(prefix)/bin" in the Makefile. In the common case where 
prefix=/usr this would attempt to create the /usr/bin directory. If the target
directory already exists, as in this case, install will attempt to change its
permissions to the default permissions (or those passed on the command line).
Of course a package user is not allowed to change the permissions of
/usr/bin and so the command fails with a message like
"install: cannot change permissions of `/usr/bin': Operation not permitted"
This is an example of a completely illegitimate command. Just remove it from
the Makefile and everything's fine.

 
 4.4 Ownership Changes
 ---------------------

The most common situation when a package wants to change the ownership of
files during installation is when it wants to install setuid root binaries.
A common command to do this would be something like
"install -c -m 4755 -o root name /usr/bin/name" and the error message would
look like this:
"install: cannot change ownership of `name': Operation not permitted"
The change of ownership is hidden in the "-o root" switch to install, which
tells it to make the target file owned by root.
The command is at least partially legitimate, because you probably want the 
binary to be installed. Whether you actually want it to be setuid root is 
a different matter. The fact that a binary is commonly installed as setuid
root doesn't mean that you should make it so. You'll have to ask yourself if
normal users absolutely need to execute that binary. If you think they can
live without it you're better off not making it setuid root, because every
setuid root binary is a potential security hole. In any case you will
have to edit the Makefile and remove the offending switch, "-o root" in this 
case, so that the installation can succeed. Note that this will cause the
binary to be installed setuid <package>, which of course makes no sense at all.
If you don't intend to make the binary setuid root after the installation, it
is best to change the "-m 4755" to "-m 755", so that it won't be installed
setuid at all.

TIP:
  When you make a binary setuid root after the installation, use
  `chown root /usr/bin/name' and not `chown root:root /usr/bin/name'.
  This way you can keep original group of the file (i.e. the group of the 
  package user) intact. With the user name = group name scheme recommended for
  package users this makes sure that you can identify the source package of
  the binary even after making it setuid root.
  Note that as a security measure chown resets the setuid bit,
  so you will have to do `chmod u+s /usr/bin/name' after the chown.

 
 4.5 Write to Non-Install Directory
 ----------------------------------

Sometimes packages want to create files or directories in non-install 
directories. 3 situations have to be distinguished in this case. The 1st
possibility is that the target directory should be an install directory.
An example of this is /usr/share/aclocal. This directory is not among the
standard system directories created when building an LFS system. It will be
created by the first package that has files to install there and will be
owned by the corresponding package user. The next package that wants to write
in it will fail to install. The remedy is simple. Just make the directory an
install directory. You don't even need to be root to do it. The package user
that owns the directory has the rights to make that change.

The 2nd possible reason for a package wanting to write to a non-install
directory is that the failing command is only partially legitimate, i.e. you
do want to have installed whatever it is meant to install, but you want it in
a different location. For example some packages install binaries that are not
meant to be called directly. The default location for these binaries is
sometimes called libexec and with prefix=/usr the package will attempt to 
create /usr/libexec. In cases such as this you often don't have to change 
any Makefiles. There is either a configure switch to change the directory in
question or it is just a matter of overriding a Makefile variable as in
`make libexecdir=/usr/lib install'.

The 3rd possible reason for an attempt to write to a non-install directory is
that the command in question is illegitimate, i.e. you don't want to have
installed whatever the package wants to install. As usual with illegitimate
commands you can edit the Makefile and just remove them. In the case of
a whole directory whose installation you want to suppress it could be too
much effort to remove all of the offending commands that want to install
files there. In this case an approach similar to that from the previous
paragraph can be more effective. Either through configure switches or 
overriding of variables you change the directory in question to something
like <builddir>/foobar, where <builddir> is the directory in which build 
commands are run (i.e. usually the top of the unpackaged source
tree). This will cause the package to create the unwanted directory inside
the build tree, which doesn't cause any permission problems and has the nice
side effect that it'll be deleted together with the build directory when you
clean up after the build.


 4.6 Delete or Overwrite File
 ----------------------------

In a perfect world one package should not mess with another package's files,
but in the real world conflicts do happen occasionally. While a normal
sysadmin installing as root won't notice this until it's too late, an admin
employing the package user system will have to deal with conflicts right away.
When a package tries to overwrite or delete a file or directory that is owned
by another package the attempt will fail. It will fail even inside install
directories because of the sticky bit.
Although sometimes difficult to implement, the solution to such a conflict is
easy to describe. You need to either remove (or rename) the old file or 
directory before installing, or suppress the installation of the new file or 
directory. The installation of individual binaries is sometimes easy to
prevent. If you find a line such as "PROGRAMS=foo bar fubar barfu" in the
Makefile and "foo" is the name of the conflicting binary, just try removing
it from that list. That may be sufficient to prevent it from being installed.

 
 4.7 /sbin/ldconfig
 ------------------
 
Packages that install libraries sometimes run /sbin/ldconfig as part of their
installation so that the dynamic libraries are properly registered on the 
system. Because a package user is not allowed to overwrite /etc/ld.so.cache
ldconfig fails. This failure is commonly ignored in Makefiles, but you should
take note of it anyway, because you need to run ldconfig as root after
the installation. Alternatively, the more_control_helpers contain a wrapper 
program that calls /sbin/ldconfig and can be made setuid root.


 4.8 What Commands to Run as a Package User  
 ------------------------------------------

A common problem that new users of this hint have is to decide which commands
to run as a package user and which commands to run as root. The general rule
is that the only commands to run as a package user are those for building,
installing, removing and modifying the files that belong to *that* package 
user's package. Everything else should be run as root as usual. 
Some things you CAN/SHOULD NOT DO as a package user include

    - starting daemons
    - running udevstart
    - stripping /lib/*

 
############################################################################
 5. The more_control_helpers Archive
############################################################################

 5.1 Overview
 ------------

The more_control_helpers archive contains files to help you with building and
maintaining a system that uses the package user method. One thing that the
more_control_helpers archive contains are some LFS-specific temporary files 
that are only needed for the building of your LFS system and will not remain 
installed in a permanent location. Then there are the previously mentioned 
example files that demonstrate the suggested use of the package user home 
directories discussed earlier. Another group of files contained in the archive 
is a set of scripts that help with package management aspects, such as
creating new package users and checking which files a particular package has 
installed. Finally the more_control_helpers archive contains wrapper scripts
for some commands that handle many of the common problems discussed in the
previous chapter and make package user installations a lot easier.
 
 
 5.2 The Wrappers
 ----------------
 
The previous chapter discussed some common problems encountered during
package user builds and how to solve them. The solution to an installation
failure usually requires editing of one or more Makefiles. Making such changes
manually is annoying, even if it happens only occasionally, and whenever you
reinstall a package you have to make the changes again. Sed scripts and patches
can help with the latter problem, but they still have to be custom fitted to 
every package that needs them. There is a better solution, though. While there
exist countless ways to install files, only very few are commonly used by
packages. The 5 commands mkdir, chgrp, chown, chmod and install are responsible 
for most of the problems that arise during an LFS installation. This
prompted me to write wrapper scripts for these 5 commands that recognize
certain problematic patterns and deal with them automatically.

The instructions given in this hint in the LFS-specific part will instruct you
to install these wrappers in /usr/lib/pkgusr. If you do that and make sure 
that this directory is the first entry in the PATH of every package user, then
they will save you a lot of time and effort in dealing with recurring issues.
Note that if you want to choose a directory other than /usr/lib/pkgusr for
the wrappers, you need to be careful. Some configure scripts ignore certain
locations. A subdirectory of /etc would not work, for instance, because /etc
is one of these locations.  

It is important that you understand the limitations of the wrapper scripts.
They can fix some problems without user intervention, such as turning
newly created directories in /usr/share/locale into install directories.
But other problems by their very nature require manual intervention. When a 
program tries to install a setuid root binary, for instance, the wrapper 
scripts will suppress the attempt to change ownership of an installed file to 
root. While that allows `make install' to complete without error, it is only 
a partial solution. The wrapper scripts can not (and should not) take away
your responsibility for deciding whether the program in question should be
setuid root and to make it so, if necessary. To account for this, the
wrapper scripts will output warning lines to standard error that start with
"***" whenever they encounter a situation that needs to be reviewed.
Following the "***" in the message will be the original command that the
installation attempted to perform. 
You *must* check these "***" lines, examine the affected files or directories
and take appropriate action. Because of this it is imperative that you log
the messages output during a package installation and check these logs
religiously. The `build' script contained in the more_control_helpers archive
demonstrates some useful output redirection tricks to be used for this purpose.
The following 3 examples shall illustrate what kind of things you will have to
look for:

Example 1: "*** install -c rsh -o root -m 4775 /usr/bin/rsh"
   This message is output by the install wrapper during the installation of
   inetutils. The package wants to install the rsh binary setuid root. The
   install wrapper removes the "-o root" and changes the "-m 4775" to
   "-m 755" before passing the command on to the real install program.
   The important thing here is the "-m 4xxx", because this wants to set the
   setuid bit. Some install scripts throw in a "-o root" for good measure
   when installing an otherwise normal binary. In that case it's enough that
   the install wrapper strips out the "-o root" and you don't have to take
   further action. But when, as in the case of inetutils, the permissions
   indicate an attempt to make a binary setuid or setgid, then you will have to
   investigate. You need to decide if you want rsh to be setuid root and
   if you decide you do, you need to become root and issue commands like this:
       
       chown root /usr/bin/rsh
       chmod u+s /usr/bin/rsh
  
TIP:
   Be conservative with making binaries setuid. If you're unsure whether you
   will ever use a program (as non-root), you probably don't want it to be 
   setuid root. Keep in mind that you can always make the change later when
   you need it. When you apply this reasoning to rsh, for instance, you'll 
   probably end up not making it setuid root.


Example 2: "*** chgrp tty /usr/bin/write" 
   This is output by the chgrp wrapper during the util-linux installation.
   The util-linux package wants to install the write program as setgid tty,
   so that it is allowed to access other users' terminals. The chgrp wrapper
   prevents the changing of the group and the chmod wrapper prevents the
   setting of the setgid bit. You need to decide if you want the
   program to be setgid and if you decide in favor of this, do as root
   
       chgrp tty /usr/bin/write
       chmod g+s /usr/bin/write
   
    
Example 3: "*** install -d -m 755 /sbin"
   This is also from the util-linux installation. Util-linux, for no good
   reason, tries to recreate the /sbin directory. The install wrapper
   prevents this and you don't have to take any further action.

 
 5.3 add_package_user/install_package
 ------------------------------------
  
Whenever you install a new package on your system, you first have to create
a new user account, possibly create a new group and if you follow the advice
from this hint about making productive use of a package user's home directory,
you will have to set up that one, too. If you were to do all of this manually,
it would be a lot of work. The add_package_user and install_package scripts
in the more_control_helpers archive were written to automate this.

The install_package script is the one you will normally use to prepare for
installing a new package. It takes 3 parameters: the description of the 
package, the name of the package user account to create and the name of the
package user's primary group. So if you use the user=group scheme recommended 
by this hint and are as creative with your package descriptions as I am, then 
the command you'll use to prepare for installing package "foo" will be

    install_package foo foo foo

This command does 2 things. First it calls the add_package_user script with
the provided name, group and description plus sensible default values for 
add_package_user's other parameters. Then, after add_package_user has created 
the package user account, install_package automatically uses the su-command
to switch to the newly created account. If the default .bashrc and
.bash_profile scripts you use for package users contain the command "cd" as do
the examples in the more_control_helpers archive, you will be put right into
your package user's home directory so that you can start installing right away.

TIP:
  The install_package script can be called with a single argument that will
  be used as user name, group name and description. So instead of the above
  command a simple `install_package foo' would have sufficed.

The add_package_user script is responsible for the actual work of creating
a new package user account. Given a name, a group name and a description, it
will create a new user account with the provided primary group and the install
group as supplementary group. The groups will be created if necessary. 
add_package_user takes several arguments that determine the numeric ranges from
which it will pick the new user's UID and the GIDs for groups it needs to
create. add_package_user does not only create the package user account. It
will set up a home directory for it, too. You can either specify the directory
or go with the default, which is /usr/src/<name>, where <name> is the name
provided for the new account. If the home directory already exists, its
ownership and that of any existing contents will be changed to the new user.
If it doesn't exist, it will be created. 

The contents of /etc/pkgusr/skel-package will be copied into the new package
user's home directory (without overwriting pre-existing files). 
The more_control_helpers archive contains an example of a useful skel-package
directory. Note that symlinks are copied as symlinks, so skel-package is the
perfect place to put .bashrc and .bash_profile symlinks to a central location
that will ensure that all package user accounts have the same environment.
This is especially useful to make sure that all package users have the
wrappers directory in their PATH.

  
 5.4 forall_direntries_from
 --------------------------

The forall_direntries_from script is a very useful tool for common package
management tasks. It can roughly be described as a shortcut for 
"find / -user <name> -or -group <name>  <commands>", where <name> is the
first parameter to forall_direntries_from and <commands> are the remaining
parameters. However, forall_direntries_from takes care of making sure that
only relevant filesystems are scanned and shields you from certain unpleasant
surprises such as "Oops, I forgot that -depth negates -prune and have 
accidentally wiped out my home directory." or "Oops, I forgot to -prune /proc
and now I'm getting parity errors on my SCSI bus.". 

IMPORTANT NOTE: By default the forall_direntries_from script will only scan
the / filesystem and will not traverse other filesystems. If you have
relevant directories that need to be scanned on other filesystems, you will 
need to edit the script and add the respective mount point(s) to the
fs_to_scan list at the beginning of the script. The most likely candidate for
addition is "/usr".

Application examples:

Example 1: Create a tar.gz archive of all files that belong to <package>, e.g.
           for installing <package> on another machine without having to 
           recompile it there.

  forall_direntries_from <package> -fprint0 /tmp/files.lst
  tar --null -P -czf /tmp/archive.tar.gz --files-from=/tmp/files.lst


Example 2: Print out all setuid root binaries installed by <package>.
           (This only works if you use the user=group scheme.)
  
  forall_direntries_from <package> -perm +u+s -print


Example 3: List all binaries in /bin and /usr/bin belonging to "me" (i.e. the 
           package user executing the command) in alphabetical order.

  forall_direntries_from $(whoami) -path "*/bin/*" -printf "%f\n" | sort


Example 4: Uninstall <package>.
          
  See following section about the uninstall_package script.
 
                                   
 5.5 uninstall_package
 ---------------------

The uninstall_package script is basically a forall_direntries_from
application example in script form. The command `uninstall_package foo'
prints out the forall_direntries_from call that you have to use to delete
all the files of package "foo" (except for those in directories that 
forall_direntries_from is instructed not to scan) together with some
explanations. So in order to delete the files from package foo, you would 
execute `uninstall_package foo' and then copy'n'paste the command it prints
to the command line. As a safeguard the forall_direntries_from call has an 
"echo" in front of the "rm" and "rmdir" calls, so if you execute it, the files 
will not actually be deleted unless you remove both instances of "echo". 
It is recommended that you execute the command once with the echos and check 
the output to make sure that only the files you intend to be deleted are in 
the list. After you've confirmed that, you can use the shell's history to 
recall the command, edit out the instances of "echo" and remove the files 
for real.


 5.6 list_suspicious_files/list_suspicious_files_from
 ----------------------------------------------------

list_suspicious_files looks for filesystem entries that are out of the ordinary
in some way and prints a categorized list of them. Things that qualify as
suspicious include setuid and setgid binaries, world-writable files, symlinks 
that are possibly broken, hard links, install directories with unusual 
permissions and other stuff. You should run this script after you've finished 
your new LFS system and in regular intervals after that. Investigate the
listing closely.

TIP: 
  When you check the list of setuid and setgid files, don't forget to
  look at the actual user or group ownership of the file. It's easy to forget
  that, especially in the setuid case, because we often equate setuid with
  setuid root since setuid is seldom used with other user accounts.

list_suspicious_files_from takes a user or group name or a UID/GID as an 
argument and reports suspicious entries only when they are owned by the given 
user or group. Usually you would not call this script directly but instead
use list_package (described in the next section), whose output includes that 
from list_suspicious_files_from.

IMPORTANT NOTE: By default the list_suspicious_files script will only scan
the / filesystem and will not traverse other filesystems. If you have
relevant directories that need to be scanned on other filesystems, you will 
need to edit the script and add the respective mount point(s) to the
fs_to_scan list at the beginning of the script. The most likely candidate for
addition is "/usr".


 5.7 list_package
 ----------------

list_package tells you everything about a package's installed files. In
general you will want to execute something like 

   list_package $(whoami) >pkg.lst
   
right after installing a package and you can forget about the chronically
inaccurate content listings in the (B)LFS book. 
The following (shortened) output for util-linux speaks for itself:

PS1> list_package util-linux

EXECUTABLES (in */bin or */sbin)
  agetty, arch, blockdev, cal, cfdisk, [...] vidmode(->rdev), whereis, write

EXECUTABLES WITH NO MANPAGE (in */bin or */sbin)
  fsck.cramfs, mkfs.cramfs

MANPAGE SUMMARIES OF EXECUTABLES (in */bin or */sbin)
  agetty: alternative Linux getty
  arch: print machine architecture
  blockdev: call block device ioctls from the command line
  cal: displays a calendar
  cfdisk: Curses based disk partition table manipulator for Linux
  chkdupexe: find duplicate executables
  col: filter reverse line feeds from input
  [...]
  swapon: enable/disable devices and files for paging and swapping
  tailf: follow the growth of a log file
  tunelp: set various parameters for the lp device
  ul: do underlining
  umount: unmount file systems
  vidmode: query/set image root device, RAM disk size, or video mode
  whereis: locate the binary, source, and manual page files for a command
  write: send a message to another user

EXTRA MANPAGES
  /usr/share/man/man5/fstab.5
  /usr/share/man/man5/nfs.5
  /usr/share/man/man8/sln.8

EXTRA EXECUTABLES (not in */bin or */sbin)
  /usr/share/misc/getopt/getopt-parse.bash
  /usr/share/misc/getopt/getopt-parse.tcsh
  /usr/share/misc/getopt/getopt-test.bash
  /usr/share/misc/getopt/getopt-test.tcsh

ALL FILES
  /etc/fdprm
  /sbin/agetty
  /sbin/blockdev
  /sbin/cfdisk
  /sbin/ctrlaltdel
  /sbin/elvtune
  /sbin/fdisk
  /sbin/fsck.cramfs
  /sbin/fsck.minix
  /sbin/hwclock
  /sbin/losetup
  /sbin/mkfs
  /sbin/mkfs.bfs
  [...]
  /usr/share/man/man8/rootflags.8
  /usr/share/man/man8/setfdprm.8
  /usr/share/man/man8/setsid.8
  /usr/share/man/man8/sfdisk.8
  /usr/share/man/man8/sln.8
  /usr/share/man/man8/swapoff.8
  /usr/share/man/man8/swapon.8
  /usr/share/man/man8/tunelp.8
  /usr/share/man/man8/umount.8
  /usr/share/man/man8/vidmode.8
  /usr/share/misc/getopt
  /usr/share/misc/getopt/getopt-parse.bash
  /usr/share/misc/getopt/getopt-parse.tcsh
  /usr/share/misc/getopt/getopt-test.bash
  /usr/share/misc/getopt/getopt-test.tcsh

SETUID FILES
  -rwsr-xr-x "/usr/bin/mount"  root:util-linux
  -rwsr-xr-x "/usr/bin/umount"  root:util-linux

SETGID FILES
  -rwxr-sr-x "/usr/bin/write"  util-linux:tty

FILES WITH UNUSUAL PERMISSIONS
  -rwsr-xr-x "/usr/bin/mount"  root:util-linux
  -rwsr-xr-x "/usr/bin/umount"  root:util-linux
  -rwxr-sr-x "/usr/bin/write"  util-linux:tty


Note: list_package works regardless of the prefix you've installed the package
      with, so you can for instance configure with --prefix=/opt/package and
      list_package will work just fine (provided that /opt is on a
      filesystem configured to be scanned by forall_direntries_from and
      list_suspicious_files).

Note: list_package only considers manpages actually owned by the package to 
      list. It will not consider manpages installed by another package. This
      means that you may see executables identified as not having a manpage
      although they do have one courtesy of another package 
      (usually man-pages).


 5.8 grep_all_regular_files_for
 ------------------------------

This script is not really related to the package user system, but because of
its similarity to the other scripts I've included it anyway. The sole purpose
of this script is to identify files that store references to the build 
environment, specifically the /tools directory. Such references may point out
problems, since the /tools directory is supposed to be transient.
Don't forget that results for unstripped binaries and libraries are not 
reliable, because debugging information often includes references to the
build environment. These do not cause trouble (unless you're trying to debug
the objects in question after deleting /tools).
  
IMPORTANT NOTE: By default the grep_all_regular_files_for script will only scan
the / filesystem and will not traverse other filesystems. If you have
relevant directories that need to be scanned on other filesystems, you will 
need to edit the script and add the respective mount point(s) to the
fs_to_scan list at the beginning of the script. The most likely candidate for
addition is "/usr".  
  
  
 5.9 The etc Directory
 ---------------------
 
If you follow the instructions provided in the LFS-specific part of this hint,
the contents of the etc directory will be installed in /etc/pkgusr. The
directory contains a bashrc and bash_profile for package users that takes
care of package user specific details such as putting the wrappers directory
at the beginning of the PATH and calling cd, so that `su <package>' will
put you right into the package user's home directory. Also contained in the
etc directory is a skel-package directory as used by 
install_package/add_package_user to populate the home directories of newly
created package users.
 
 
 5.10 ldconfig.c
 --------------------

A lot of packages contain libraries. Having to manually call /sbin/ldconfig
as root after installing these packages can become annoying. It would be
much easier if one could grant package users permission to use /sbin/ldconfig.
Making ldconfig setuid root would be a simple and effective solution, but
there are some pitfalls. First of all it is imperative that ordinary users
be prohibited from executing ldconfig with elevated privileges. Otherwise
an ordinary user can overwrite and possibly read arbitrary files on the 
system. This can be prevented by making ldconfig owned by group install and
removing the o+x bit from the file mode. While this setup is no less secure
than running `make install' as root, one reason why we're using package users
is because we don't feel safe doing that. To protect against the (admittedly
very theoretical) danger of a malicious package user, the more_control_helpers
provide ldconfig.c. The only thing this program does is to call 
`/sbin/ldconfig -v' with an empty environment. Because it doesn't evaluate
any user input and doesn't pass any user-provided data to ldconfig, it can
safely be made setuid root. 


 5.11 Temporary Files 
 --------------------

3 files in the more_control_helpers archive are only used during the 
installation of the base LFS system and are not installed permanently.
The first of them is the installdirs.lst file that contains a list of 
directories that should be install directories. 
The second file is sbin/useradd, which is a very primitive shell script that
adds a new entry to /etc/passwd. It allows us to add package users before
we have installed shadow, which provides a real useradd.
Finally there is groupadd, which is like useradd, only for /etc/group.
Both scripts, useradd as well as groupadd, do very little error checking and
only support the syntax needed by install_package/add_package_user. So don't
try anything funky with them.

  
------------------------ PART 2: LFS Specifics ------------------------------


#############################################################################
 6. Pre-Chroot Phase (Chapter 6)
#############################################################################

Build Chapter 5 explained by the LFS book with the following changes:
 
more_control_helpers:
  When you have reached the end of Chapter 5, before you begin with Chapter 6 
  you will need to install the helper scripts in the /tools directory so that
  they are available once you've entered the chroot environment. Use the
  following commands to install the more_control_helpers in /tools:

    cd $LFS/tools &&
    tar xjf /path/to/more_control_helpers.tar.bz2 &&
    cd  more_control_helpers &&
    cp ./sbin/* $LFS/sbin


#############################################################################
 7. Chroot Phase (Chapter 7)
#############################################################################

 7.1 Preparations
 ----------------

Skip the chapter in which the ownership of installed files is changed. Currently
this is "7.2. Changing Ownership".

Enter the chroot environment and follow the instructions from the book up to
but *not* including the installation of the first package, unless is is
libstdc++

Install libstdc++ and then compile the shadow package (currently
chapter 8.25) and copy the su binary into its proper location:

    cp src/su /usr/bin

Now install the more_control_helpers files in their proper
locations on the new LFS system:

    cp -a /tools/more_control_helpers/etc /etc/pkgusr &&
    chown -R 0:0 /etc/pkgusr &&
    cp -a /tools/more_control_helpers/lib /usr/lib/pkgusr &&
    chown -R 0:0 /usr/lib/pkgusr &&
    cp /tools/more_control_helpers/bin/* /usr/bin &&
    cp /tools/more_control_helpers/sbin/* /usr/sbin &&
    rm /usr/sbin/{useradd,groupadd}

Note that the useradd and groupadd scripts are not installed on the new LFS 
system. These scripts are just temporary workarounds we will use as long as 
the real useradd and groupadd are not available. Therefore they should only 
be in /tools/bin.

ATTENTION! If you decide to use a different directory than /usr/lib/pkgusr
for the wrappers, you have to be careful, because at least the glibc
configure script ignores certain directories when looking for programs. The
list of ignored directories for glibc includes, among others, everything that 
starts with "/etc", "/usr/etc" and "/sbin". Wrappers put into a directory that
matches any of these patterns would be ineffective.

Now it's time to create the install group:

    groupadd -g 9999 install

The GID 9999 has been chosen because the default range used by 
add_package_user for package user GIDs starts at 10000. Choose whatever number
you like.

Once the install group has been created you have to create a temporary user
that owns all currently installed tools:

    useradd useradd -s /bin/bash -g lfs -m -k /dev/null -u 9999 lfs

The UID 9999 has been chosen because the default range used by
add_package_user for package user UIDs starts at 10000. Choose whatever number
you like. This user will be deleted after the install process is finished.

Now change the ownership of all currently installed tools and make them
group-writeable:

    chown -R lfs:install /{usr,lib,var,etc,bin,sbin,tools}
    chmod -R ug=rwx /{usr,lib,var,etc,bin,sbin,tools}
    case $(uname -m) in
      x86_64) chown -R lfs:install $LFS/lib64
      chmod -R ug=rwx $LFS/lib64
      ;;
    esac

After this, turn all the directories that packages will install files in into
install directories. To make this easier I have compiled a list of install
directories that can be found in the file
/tools/more_control_helpers/installdirs.lst. The following command
uses this list to assign the necessary directories to the install group.
Note that you will get several error messages because of non-existent
directories. This is because the list contains some directories not created
by LFS.

    chown 0:9999 $(cat /tools/more_control_helpers/installdirs.lst)

To be usable by package users, the directories will have to be group-writable
and should be sticky as has been explained in the beginning of this hint.
However, for now the sticky bit must not be set, as this would prevent the
temporary from being overwritten by the package users.
The following command sets the permissions appropriately.
You will get the same error messages as for the previous command.

    chmod ug=rwx,o=rx $(cat /tools/more_control_helpers/installdirs.lst)


 7.2 Walkthrough: Installing linux-libc-headers
 ----------------------------------------------

At this point everything has been set up for creating the first package
user. At the time of this writing the first package installed in the LFS
book is Linux-Libc-Headers, so this package will serve as an example for how 
things are done. The command

     install_package 'Linux Headers' linux-libc-headers linux-libc-headers

will create a package user with user and group name linux-libc-headers.
If you don't want to use the user=group scheme, change the last argument to
the desired group name. The description is arbitrary but needs to meet the
requirements for the description field of an /etc/passwd entry.

TIP:
  Remember that you can call install_package with just one argument, if you
  want user name, group name and description to be the same.

The directory /usr/src/linux-libc-headers will be set up as the home directory
for the package user, automatically populated with the contents of 
/etc/pkgusr/skel-package. The install_package command also issues the command
`su - linux-libc-headers' to assume the identity of the newly created package
user. If you're using the bashrc and bash_profile scripts from the
more_control_helpers archive, you will be put straight into the directory 
/usr/src/linux-libc-headers and your prompt will look like this

package linux-libc-headers:/usr/src/linux-libc-headers>
     
to show you that you're working as package user linux-libc-headers and
that your current working directory is /usr/src/linux-libc-headers.

Use the command 
   
     echo $PATH

to verify that your PATH starts with "/usr/lib/pkgusr", the directory that 
contains the wrappers, and ends with "/tools/bin".

Now everything is prepared for installing the package according to the
instructions in the LFS book. Note that at the time of this writing the 
LFS book tells you to execute a chown command to make sure that the headers 
are owned by root. This is just because the packager has made a very common 
mistake when creating the tarball for the headers: He has archived the files 
with a non-root user/group assignment. When unpacking such a tarball as root, 
the files end up being owned by some weird user/group combination, which may 
open a security hole. When you're working as a package user this can not 
happen and you don't want to chown the headers to root:root, because that 
would defeat the whole point of installing the headers with a package user.
This is one of the small points on which you will have to deviate from the 
standard LFS instructions when using package users. More package user related
issues with the current LFS book can be found in the next section.

After you've installed the headers, simply issue the command

    exit
    
to become root again. Now would be a good time to think about useful
customizations for /etc/pkgusr/{bash_profile,bashrc} and/or 
/etc/pkgusr/skel-package, if you've not already customized them.
Once you're satisfied with your setup, install the rest of the packages.
The following section will help you with some problems that you will run into.


 7.3 Cleaning up
 ---------------


When you reach the end of the chroot phase, you will need to do some manual
cleanup of file ownership. List all remaining files owned by the temporary lfs
user by using the following command:

    find / -user lfs

Many of the listed files belong to the linux api headers and glibc packages.
The following commands correct their ownership:

    chown -R linux: /usr/include/{asm,asm-generic,drm,linux,misc,mtd,rdma,scsi,\
    sound,video,xen}
    chown -R glibc: /usr/include/{arpa,bits,gnu,net,netash,netatalk,netax25,\
    neteconet,netinet,netipx,netiucv,nepacket,netrom,netrose,nfs,protocols,\
    rpc,sys,audit,gconv}

Use the apropriate package names for your install.

TIP:
    You will need to use chown -h to reassign symlinks.

Finally, add the sticky bit to all install directories
to complete the chroot stage:

    chmod o+t $(cat /tools/more_control_helpers/installdirs.lst)


 7.4 Known Issues with LFS Packages
 ----------------------------------

This section has details on the package user related problems you will face
when building your LFS system. You should copy the information from this
section to the INSTALL NOTES of the relevant .project files for the packages 
concerned, together with any of your own notes.

NOTE: If you're building by an LFS book later than 6.2 it is recommended that
      you read this complete chapter before you start building any packages.
      If your LFS version is 6.2 then it's fine to read this section package
      by package as you progress with your build.


linux-libc-headers:
    At the time of this writing the LFS book tells you to execute a chown 
    command to make sure that the headers are owned by root. This is just 
    because the packager has made a very common mistake when creating the 
    tarball for the headers: He has archived the files with a non-root 
    user/group assignment. When unpacking such a tarball as root, the files 
    end up being owned by some weird user/group combination, which may open 
    a security hole. When you're working as a package user this can not happen 
    and you don't want to chown the headers to root:root, because that would 
    defeat the whole point of installing the headers with a package user.
    
    There used to be another packaging error in the linux-libc-headers. 
    Version 2.6.12.0 (current as of this writing) doesn't have it anymore,
    but older versions used to contain files with permissions set incorrectly. 
    All headers are supposed to be world-readable, but they weren't. More about
    this later in the glibc notes.


man-pages:
    If the name you use for the man-pages package user is not exactly 
    "man-pages", then you will have to change the variable "manpagesowner"
    right at the beginning of the wrapper script `install'.

    Recent versions of man-pages contain POSIX manpages that the package
    tries to install in /usr/share/man/man{0,1,3}p. There's also a manpage
    that man-pages wants to install to /usr/share/man/man9.
    As /usr/share/man is
    not an install directory and the LFS book does not have instructions to
    create these directories at the time of this writing, the installation 
    will fail and the respective man-pages will not be installed.
    Possible remedies:
      1. Make /usr/share/man an install directory. 
         Consequence: All Packages will be able to create new subdirectories
         in /usr/share/man. I find this undesirable because there are packages
         that create directories for manpages in foreign languages that I
         don't want. YMMV.
      2. Ignore the problem and live without the POSIX manpages. Unless
         you are a developer (including script writer) who is interested
         in writing portable programs/scripts this is a good solution.
      3. Create the directories /usr/share/man/man{0,1,3}p and man9 as root
         prior to installing man-pages. You'll have to either chown them
         to the man-pages package user or make them install directories.
         This is my preferred solution.


glibc:
    It is kind of unfortunate that the packaging error of libc-linux-headers 
    concerning the permissions doesn't exist in the current version. It 
    provided for a great learning experience. I've kept the following section
    in the hint for this reason even though it doesn't apply anymore. Please
    take the time to read it.
    
--------------------- old stuff start ----------------------------------------
    Because of the error, the headers in /tools/include
    are not world-readable. Unfortunately the LFS book (as of this writing)
    does not correct this in Chapter 5 like it does in Chapter 6. For a
    standard LFS build this is no problem, because glibc is built as root and
    root can access everything regardless of permissions.
    The glibc package user, however, does not have permission to access
    these headers. This will cause several configure tests to fail, because
    the respective test programs can not be compiled.
    The end result is the error message "/lib/cpp fails sanity check", which
    is completely nonsensical as we don't have a /lib/cpp.
    
    This is the perfect opportunity to introduce rule #1 of error diagnostics:
    
       NEVER TRUST DIAGNOSTIC MESSAGES !
       
    There are 2 kinds of diagnostic messages:
    
      1. Those that are unnecessary, because once you see which component has
         failed, the source of the problem is obvious.
      2. Those that grossly misdiagnose the source of the problem and lead
         you to draw the wrong conclusions.
    
    No, there is no other kind. Trust me ;-)
    In this case, /lib/cpp has nothing to do with the problem. It doesn't
    exist and that's fine. The message just wants to trick you into doing
    something stupid such as create a symlink /lib/cpp -> /tools/bin/cpp.
    But that would be totally wrong. Before you jump to any premature 
    conclusions you should always try to get as much *low-level* information 
    as you can. Diagnostic messages are *high-level* information. They 
    represent a filtered view of the problem, which is usually of little help.
    Fortunately the message (the complete one, not the part quoted above) also
    points at the source for the necessary low-level information. In this 
    case that is the file config.log (not to be confused with configure.log, 
    the file created by the build script included in the more_control_helpers 
    archive).
    config.log is created by all autoconf-created configures (not just that 
    of glibc) and it contains, among other things, the test programs used by 
    configure and messages output while building and running them. Whenever a 
    configure script fails or gives weird results, check config.log. And 
    always remember rule #2 of error diagnostics
    
       ALWAYS START AT THE FIRST ERROR
       
    This seems pretty obvious, but nevertheless people commonly do the exact
    opposite. It's just too tempting to start at the point of the final 
    failure and try to work backwards. In this case many people would open 
    config.log and scroll to the point of the failed /lib/cpp sanity check.
    After all, that's what caused configure to abort and so that's what needs
    to be fixed, right?  WRONG! Someone who takes this approach just sees the 
    error message "/lib/cpp: No such file or directory" and is even more 
    convinced that a missing /lib/cpp symlink (or program) is the problem.
    
    The correct way to approach such a problem is to start at the beginning
    of config.log, to scroll down to first error message and to check if it
    is an issue that needs to be fixed (error messages in config.log are
    not always signs for a problem). If the issue needs to be fixed, then
    it needs to be fixed first, because all later errors could be rooted in
    this issue (even if, no, *especially* if you don't believe this is the 
    case).
    If we apply this advice to the problem at hand, we quickly get to the first
    serious error in config.log: 

      "/tools/include/linux/limits.h: Permission denied"
    
    A quick check with ls reveals that indeed the directory with the linux
    headers is not world-readable, which is obviously wrong. The fix is
    easy. Just make (as root) the header directories /tools/include/{linux,asm} 
    world-readable with commands similar to those the LFS book presents 
    in Chapter 6 for the installation of linux-libc-headers.
    Once this change has been made, glibc's configure succeeds.
--------------------- old stuff end -----------------------------------------    

TIP:
    Even when configure completes successfully, you should still check the
    output carefully to see if there is anything odd. E.g. if you're using
    the wrappers, you should check that configure outputs the line
    
       checking for a BSD-compatible install... /usr/lib/pkgusr/install -c
    
    If configure detects a different install, such as /tools/bin/install,
    something is wrong. Maybe there's a typo in the PATH for the package
    user, or you've put the wrappers into a directory that is ignored by
    configure.
    

    With the wrappers the glibc build and install should work smoothly. 
    The wrapper script for install makes sure that the /usr/share/locale/* 
    directories become install directories so that other programs can install 
    their localized messages. 
    One thing that the wrappers do not take care of, 
    however, is the file /usr/share/info/dir. Because in the current LFS build
    order glibc is the first package that installs info files, dir is owned by 
    and only writable by glibc. In order to allow other packages to install 
    info pages, execute the following commands as root:
    
       chown root:install /usr/share/info/dir &&
       chmod ug=rw,o=r /usr/share/info/dir

NOTE:
    glibc wants to install the program pt_chown as setuid root. If you install
    as a package user, the program will get installed but not given root
    privileges (because of the install wrapper). 
    The following info is from the glibc docs:
          
       One auxiliary program, `/usr/libexec/pt_chown', is installed setuid
       `root'.  This program is invoked by the `grantpt' function; it sets the
       permissions on a pseudoterminal so it can be used by the calling
       process.  This means programs like `xterm' and `screen' do not have to
       be setuid to get a pty.  (There may be other reasons why they need
       privileges.)  If you are using a 2.1 or newer Linux kernel with the
       `devptsfs' or `devfs' filesystems providing pty slaves, you don't need
       this program; otherwise you do.  The source for `pt_chown' is in
       `login/programs/pt_chown.c'.
                               
    So unless you're building a system that does not use devpts (which would
    be quite unusual), this does not need to concern you.

TIP:
    In case you were wondering if you should create /etc/nsswitch.conf and
    /etc/ld.so.conf as root or glibc, I recommend to assign all files that 
    you manually create or manually edit to the root account. That way you can
    distinguish between those files that can be regenerated automatically and
    those that can not. Assigning even automatically generated files to
    root once you make the first manual edit, ensures that a later 
    reinstallation of a package won't silently do away with your manual tweaks.

ldconfig:
    Now that glibc has installed /sbin/ldconfig you can activate the ldconfig
    wrapper if you want to. Perform the following steps as root 
    AFTER re-adjusting the toolchain, 
    just before starting with binutils:
    
       cd /usr/lib/pkgusr
       gcc -O2 -W -Wall -o ldconfig ldconfig.c
       chown root:install ldconfig
       chmod u=rwxs,g=rxs,o= ldconfig
  
    These instructions make the ldconfig wrapper setuid root and setgid install
    and prevent non-root users not in the install group from executing it. 
    The setuid root is required so that it can replace /etc/ld.so.cache. 
    The setgid install is not strictly required, but without it 
    /etc/ld.so.cache will end up with the group of the last package user that 
    touched it. If you use the user name=group name scheme this will cause the 
    more_control_helpers scripts to believe that /etc/ld.so.cache belongs to 
    the package in question which can be confusing.

binutils:
    Have you make /usr/share/info/dir group-writable as explained above in
    the glibc notes? If you've missed that part, go back and do it now.
    The installation of binutils should complete without problems. 
    It does however cause minor conflicts with autoconf (see later).
    
NOTE:
    At the time of this writing the version of bash used in the LFS book has
    a bug that causes the list_package script to spit out errors and to list
    all manpages of binutils (and other packages) as Broken. This bug is 
    already fixed by the bash patch used by the book but the patch is not
    applied in chapter 5. Since the manpage summary functionality of 
    list_package requires man which is not installed until after bash is
    rebuilt, this doesn't really matter, because while patching the
    chapter 5 bash would get rid of the error messages, it wouldn't result
    in usable manpage summaries.
    
    
gcc:
    Because the /usr/lib/libgcc_s.so* symlinks created at the beginning of
    Chapter 6 is owned by root, gcc's installation cannot remove it. 
    So you will have to remove it as root before `make install'.
    Alternatively use
    
       chown -h gcc: /usr/lib/libgcc*
       
    to change ownership of the files in question after creating the gcc
    package user. Note the -h option which has to be used to change 
    ownership of the symlinks themselves rather than their target files.

db:
    It should be obvious that you don't want to change the ownership of the
    installed files.
    
    
coreutils:
    Because the /bin/cat, /bin/pwd and /bin/stty symlinks are owned by root,
    coreutils' installation cannot remove them. So you will have to remove 
    them manually before `make install'. Alternatively use the command
    
       chown -h coreutils: /bin/{cat,pwd,stty}
       
    after creating the coreutils package user. Note the -h switch that makes
    chown change the ownership of the symlinks themselves rather than their 
    target files.
    
    The chapter 6 instructions move the coreutils binaries to /bin, including
    the mv binary itself. You need to make sure that hashing is turned off
    for this to work. The LFS book does this by putting `set +h' into the
    LFS user's .bashrc. If you're following this hint, you're likely using
    build scripts, so you need to put this command into the build script
    before the mv commands.

NOTE:
    The man-pages package has already installed manpages for the binaries
    from coreutils. The install wrapper will prevent coreutils from overwriting
    those. This is done because the manpages from the man-pages package are 
    of superior quality (although not necessarily uptodate). 
    It also prevents errors during `make install' that
    would otherwise occur because the coreutils package user cannot overwrite 
    manpages owned by another user.
    If you don't like the above behaviour and would rather have the original 
    package manpages (because they are uptodate), you can set the variable 
    manpagesowner at the beginning of the install wrapper to a string that
    doesn't correspond to a package user name (it must not be empty, though!).
    If you do this, you will have to resolve manpage conflicts in another way. 
    The easiest way to handle this is probably to not install the man-pages 
    package at the beginning of Chapter 6 but at the end, after all the other
    packages have already installed their manpages. Then you need only deal 
    with the conflicts once, when installing man-pages.


ncurses:
    The installation of ncurses (like that of other packages that include 
    libraries) wants to run /sbin/ldconfig to update /etc/ld.so.cache. 
    This fails because the package user doesn't have permission to replace 
    /etc/ld.so.cache.
    Making /etc/ld.so.cache group-writable by the install group doesn't help, 
    because the permissions would be reset on the next call to /sbin/ldconfig.
    This error will usually not abort the installation and you can just
    run /sbin/ldconfig manually as root afterwards.
    Alternatively you can use the ldconfig wrapper as described earlier.


aclocal directory:
    At the time of this writing the directory /usr/share/aclocal is
    created during the bison installation. This directory contains
    macros for autoconf. Other packages want to install 
    files into this directory, so you should make it writable by the install 
    group and sticky. 


perl:
    Before you do `make install', you will have to 
    `chown -h perl: /usr/bin/perl' so that the perl package user is allowed to
    remove the /usr/bin/perl symlink.
    
    If you will install add-on packages for perl as their own package users
    into /usr/lib/perl5/site_perl, then you will need to turn 
    /usr/lib/perl5/site_perl/ and its subdirectories into 
    install directories. You don't need to do this now as you'll notice it
    anyway when installing a perl add-on fails. 


autoconf:
    The autoconf package wants to install its own copy of standards.info,
    which fails because binutils has already installed this file. You can 
    either ignore the error or remove the binutils version of standards.info 
    before `make install'.


bash:
  configure:
    The bash configure script tests for the presence of the special devices
    /dev/std* and /dev/fd/*. Unfortunately at the time of this writing the
    test for /dev/fd/* is buggy (the test for /dev/stdin used to be broke, too
    in bash-2.x, but has been fixed since). It ends up testing read access to 
    standard input, 
    which is the (pseudo)terminal you're building your system in. 
    Unfortunately su doesn't change ownership of the terminal device, so when
    you're su'd to a package user account, the terminal still belongs to the
    login user. As the package user doesn't have read access to the device,
    the tests fail.
    
    There is a simple way to get around this. Simply run ./configure like this
    
      ./configure ....  </dev/null
    
    The trick here is to redirect standard input (note, that this is a '<' not
    a '>' !) to refer to /dev/null. Unlike the terminal device, /dev/null is
    world-readable and world-writable, so the tests succeed as they should.
    If you don't like this trick, you can also chown the terminal device in
    question (see `ls -la /dev/fd/0') to the package user before building
    bash.
    
  make check:
    When running the test suite as a package user, the test "run-test" will 
    fail with the output like this:

      33d32
      < *** chmod g+s /tmp/test.setgid
      35c34
      < 1
      ---
      > 0
      64d62
      < *** chmod u+s /tmp/test.setuid
      66c64
      < 1
      ---
      > 0
      154c152
      < 1
      ---
      > 0
      160c158
      < 1
      ---
      > 0
    
    The first 2 failures are caused by the chmod wrapper which prevents the
    test from setting the setuid and setgid bits and outputs the *** warning.
    The failures are harmless. You can get rid of them by removing the wrappers
    directory from the PATH before running the tests.
    
    The last 2 failures are not specific to package users but will occur 
    whenever you run the tests su'd to another user. The reasons are the same
    as for the configure problem described earlier. The same solutions apply.
    Either use chown (if you chowned before configure you're already 
    done, of course) or run make check like this
       
      make check </dev/null
    
  make install:
    Before you can `make install' you need to `chown -h bash: /bin/bash' as
    root so that the bash installation can replace the /bin/bash symlink that
    you manually created at the beginning of chapter 6.


pkgconfig directory:
    At the time of this writing the directory /usr/lib/pkgconfig is
    created during the e2fsprogs installation. This directory contains
    build information used by the pkg-config tool. Other packages want to 
    install files into this directory, so you should make it writable by the
    install group and sticky. 


grub:
    The commands to create and populate /boot/grub have to be executed as
    root.
    
    
grep:
    Before you can `make install' you need to `chown -h grep: /bin/grep' as
    root so that the grep installation can replace the /bin/grep symlink that
    you manually created at the beginning of chapter 6.
    

inetutils:
    This package contains some programs that it wants to be setuid root:
    rsh, rcp, rlogin and ping
    The install wrapper prevents these programs from being installed
    setuid root. You must decide which of these programs you want to be 
    setuid root and manually make them so. Be conservative. Don't make a
    binary setuid root unless you *know* that ordinary users can't live
    without it. Every setuid root binary is a potential security hole.


iproute2:
    This package tries to change the permissions of /usr/sbin and some man
    directories. The install wrappers take care of this.


man-db:
    Even after installing man-db you won't get manpage summaries from
    list_package, because the way list_package calls man it needs col
    to work and col is from util-linux. You may however install util-linux
    right away. The alphabetical sort is the only reason it is at the end
    of Chapter 6.


shadow:
    By default shadow wants to install non-English manpages. This fails
    because the /usr/share/man directory is not an install directory and
    therefore package users are not allowed to create new subdirectories in it.
    To solve this problem, before you `make install', open the file
    man/Makefile, find the line 
    
      SUBDIRS = cs de es fr hu id it ja ko pl pt_BR ru zh_CN zh_TW
    
    and remove all the languages that you don't want to install. For those
    languages that you do want to install, create directories with the
    respective names in /usr/share/man as root and make them install
    directories (i.e. group install, group-writable, sticky).
    
    There is yet another issue with shadow concerning manpages. The shadow
    package contains a passwd.5 and a getspnam.3 manpage. 
    Installation of these manpages is
    automatically suppressed by the install wrapper, because it would 
    overwrite the manpages provided by the man-pages package. As usual
    the man-pages version is better, so you can simply ignore this issue.

    shadow wants to install the programs su, chage, chfn, chsh, expiry, 
    gpasswd, newgrp and passwd as setuid root. You will need to decide which 
    of these programs you want to be setuid root and manually make them so. 


sysklogd:
    sysklogd's Makefile has /usr/bin/install hardwired as the install
    program, which circumvents the install wrapper. The wrapper is needed 
    for sysklogd because it tries to make its manpages owned by root 
    (which obviously a package user is not allowed to do). 
    Therefore, install with
    
      make INSTALL=install install


udev:
    udev wants to install files into the directory /usr/lib/pkgconfig. If
    you've followed the instructions given further above you've already made
    this an install directory. If you haven't, do so now or the udev 
    installation will fail.
    
    The LFS instructions for installing udev tell you to execute the command
      
      mknod -m0666 /lib/udev/devices/null c 1 3
      
    Because a package user is not allowed to create device nodes, execute this
    command as root.
    

util-linux:
    util-linux wants to install write as setgid tty and u/mount as
    setuid root. The wrappers catch this, so it doesn't cause the install to
    fail, but as usual you'll have to decide if you want these programs to
    have special privileges and take manual action as root if you do.


##########################################################################  
 8. Sanity Checks 
##########################################################################

 8.1 Suspicious Files
 --------------------
 
You probably ran the `list_package' command for each package and reviewed
the results which include the suspicious files owned by that package. But even
if you did that it's still a good idea to run the non-package specific 
`list_suspicious_files' command once your build is complete. There could be
something you overlooked the first time, or maybe you created a file as root
with the wrong permissions. It doesn't hurt to check again and this will also
give you the opportunity to review any setuid/setgid decisions you made with
respect to the installed binaries.

TIP: 
  When you check the list of setuid and setgid files, don't forget to
  look at the actual user or group ownership of the file. It's easy to forget
  that, especially in the setuid case, because we often equate setuid with
  setuid root since setuid is seldom used with other user accounts.

  
 8.2 References to Temporary Files
 ---------------------------------

One big concern when building an LFS system is independence of the new LFS
system from the files installed in /tools. The /tools directory is intended 
to be temporary and it should be possible to delete it after building your
LFS system with no adverse side effects. The `grep_all_regular_files_for'
script from the more_control_helpers package can help you verify that your
new LFS system is indeed clean. The command

     grep_all_regular_files_for /tools

will give you a list of all files that contain the string "/tools". Review the
files in the list to make sure that no dependencies on the temporary files in
/tools have crept in. But remember that results from binaries and libraries 
are only meaningful after stripping away the debug information, because
debug information necessarily includes references to the build environment.
Of course, if you are a developer who will potentially run gdb on system
libraries/binaries, your position will be that stripping away debug information
is the wrong way to do away with /tools references. The other way to deal with
them is to rebuild packages for which /tools references are reported. The new
build will not involve any files from /tools and so the new debug information
will not refer to /tools. Note that the LFS build instructions for glibc
make glibc compile against /tools/glibc-kernheaders. Unless you copy the
glibc-kernheaders directory to a location outside of /tools and compile glibc
against that copy, you won't get rid of the /tools references in glibc's 
debug information.
No matter what means you choose to deal with the debug information issue, in 
the end you should have a system where the above command produces only false 
positives (such as "perlfaq3.1", which includes the URL
"http://www.research.att.com/sw/tools/uwin/") and files that legitimately
refer to /tools (such as a copy of this hint file).


----------------------------- APPENDICES ----------------------------------


###########################################################################
 Appendix A: Security Issues
###########################################################################

 A.1 NFS
 -------

If you use the network filesystem NFS, there are some things you need to
look out for when using the package user system. A fundamental security 
problem with NFS is that it blindly trusts the UID and GID of the client. 
If an attacker can get access to the root account on a system in your network 
that is allowed to mount NFS shares from your server, or if the attacker can 
attach his own computer to your network, then this attacker can pretend to be 
anyone. NFS will happily allow the attacker to work in the NFS exported 
directory as any user he wants to be. The only exception is the root account. 
By default NFS exports directories with the root_squash option that maps all 
incoming requests from uid 0 to anonuid (65534 unless set in /etc/exports) 
and gid 0 to anongid (65534 unless set in /etc/exports). This protects files 
owned by root:root. On a normal system this includes most files in /bin, /etc,
/lib and most other directories except /home. If you use the package user 
scheme, however, most of these files are owned by package users. These files 
are not protected by the root_squash option. In order to make NFS exports 
secure, you have to add the option "all_squash" to every entry in /etc/exports 
that exports a directory that contains files owned by package users. Note that 
all_squash is always a good idea because even systems that don't use package 
users often have some programs owned by other users or groups, because they 
need to be setuid or setgid.


 A.2 Daemons
 -----------

It is a common practice to run daemons under special user accounts rather
than as root as a security measure. If you feel tempted to use a package 
user account for this purpose, resist the temptation. It would be a very
stupid idea. Although they are deliberately less powerful than root, package 
user accounts are still privileged and need to be considered as equivalent to 
root as far as security is concerned. Do not do anything with a package user 
that on a system without package users you would not do with the root account.


###########################################################################
 Appendix B: Package Categories
###########################################################################

Although the user name = group name scheme is recommended by this hint, it is
not the only possible one. Another scheme that has some appeal is to define
package categories and to use the group for the purpose of categorizing the
packages. Following is a suggested set of categories that can serve as a
guideline for implementing this scheme.

devel: development related stuff, e.g. compilers. This is not restricted to
       software development. TeX for instance would belong in this group.
       
utils: Most software fits into this category, even somewhat essential software 
       like grep or text editors.
      
net: network related stuff such as an ftp daemon or a web browser. This
     group overlaps with other groups to a large extent. It should be used
     in preference of the other groups whenever a package is clearly focused
     towards Internet, LAN, WWW,... A utility like wget for instance would
     go in net rather than utils. Exceptions from this rule are the groups
     docs, addons, games and mmedia. If a package fits into one of those 
     groups, use the respective group instead of net.
     
docs: Documentation related packages, such as a tarball with Linux howtos.
      Note that software to create documentation such as XML processors should
      probably go in devel and software to view or post-process documentation
      such as man or groff should probably go in utils.
      
system: important system software, such as bash. This group should be used
        only for really essential packages. Most packages you would put in 
        this group are better put in "utils". Vi for instance belongs in 
        utils. 
        It is unlikely that any package not part of basic LFS belongs in the
        system group.
        
libs: What utils is for executables, libs is for libraries. Libraries that are
      not strongly related to any of the other categories should go here, such
      as zlib or libpng.
      Essential system libraries such as glibc, ncurses or gettext should
      go in system instead.
      The libs group is also used for run-time environments such as the
      Java Virtual Machine, dosemu and wine. Other emulators like MAME for
      instance should probably go into games instead. 
     
games: what do you expect ;-)

mmedia: This is the group for audio and video editors, mp3 players etc.

apps: Applications such as spreadsheets and word processors (not text editors)
      but also CAD software and graphics software such as Gimp.
      The apps group is a bit like utils, but apps are usually more user 
      friendly and more streamlined and feel less nerdish than utils. 
      
addons: plugins, filters and similar that are meant to be used in conjunction
       with another package.
       
x: software that relates to the X Window System in general and does not fit
   into any of the other categories, such as the X server itself or window 
   managers.
   Most X software should be put into other more specific groups.
   A game like xmines would go in games for instance and a text editor for
   X would go in utils.
   
kde: Software that relates to KDE and does not fit into
     any other category. This group should be used with care. 
     Do *not* use it for all KDE software. K Office for instance belongs in
     apps. Konqueror belongs in net.
     
gnome: Software that relates to GNOME and does not fit into
       any other category. This group should be used with care. 
       Do *not* use it for all GNOME software. Gimp for instance belongs 
       in apps. A GNOME-aware window manager that works with plain X should
       go in the x group.


###########################################################################
 Appendix C: Acknowledgements and Changelog
###########################################################################

ACKNOWLEDGEMENTS:
  * Matthias Benkmann for writing the original version of this hint
  * Tushar Teredesai for suggesting the user=group scheme.
  * Markus Laire for reporting the 2005-01-01 build bug

CHANGELOG:

2007-10-20 Matthias Benkmann
           -relicensed under CC-BY-SA (previously CC-BY-ND).
           -added name tags to changelog entries in preparation for having the
            hint continued by different authors.
           -added workaround to list_package for bug in man-db that causes
            some manpages to show up as "Weird manpage" in the summary.
           -chmod wrapper now prevents shadow from installing files setuid 
            shadow.
           -added a wrapper to solve ldconfig issue.
           -install_package now works when called with just a single argument. 
            That argument is used for user name, group name and description.
           -bash_profile of more_control_helpers now has /sbin and /usr/sbin 
            in the PATH to match the PATH used by root when building.
           -install_package does su - <name> now (i.e. start a login shell).
           -build script now handles unpacking of tarballs and allows calling 
            the different stages individually.
           -useradd uses the -s provided shell and no longer hardwires bash.
           -chapter 6 bash notes now properly address the configure and 
            make check issues.

2007-03-21 Matthias Benkmann
           -changed forall_direntries_from to avoid warning message from find
            when -depth is used.
           -added 4.8 What Commands to Run as a Package User  

2005-12-22 Matthias Benkmann
           -added advice on how to cope with the moving mv problem to
            coreutils note.

2005-11-13 Matthias Benkmann
           -fixed list_suspicious_files and list_package to work with
            recent more POSIX-conforming versions of GNU find
           -released version 1.2 

2005-01-01 Matthias Benkmann
            -fixed bug in skel-package/build script that caused it to report
             all steps as successful, even if they failed 
            -released version 1.1

2004-11-01 Matthias Benkmann
            -capitalized title
            -released version 1.0
            
2004-10-14 Matthias Benkmann
            -started developing the more_control_helpers utilities
            
2004-08-14 Matthias Benkmann
            -started major rewrite (update for new LFS version, new hint 
             format, textual improvements,...)

2002-04-20 Matthias Benkmann
            -changed LFS VERSION header to be more conservative
            -added <br> tags to the synopsis for the sake of the hints 
             index
            -added group mmedia to the list of suggested groups 
            -submitted v0.8

2002-03-16 Matthias Benkmann  
            -added note, that on Linux make doesn't need to be setgid kmem

2002-02-18 Matthias Benkmann
            -added section "Security issues with NFS"
            -submitted v0.7

2002-01-30 Matthias Benkmann
            -added Changelog
            -moved "chown 0.10000 `cat /tmp/installdirs`" command up (before
             glibc package user is created)
            -add_package_user: create home directory with "mkdir -p"
                               use $grpfile everywhere instead of /etc/group
            -improved mammoth sentence in Introduction
            -added note about possibility to have user name==group name
            -source bashrc_basic in bashrc_package
            -minor textual changes