Fix EXECUTABLES symlink display.

[pkgusr] / usr / bin / list_package

#!/bin/bash
# Copyright © 2021 Steve Youngs. All rights reserved.
# SPDX-License-Identifier: BSD-3-Clause

# Author:     Steve Youngs <steve@sxemacs.org>
# Maintainer: Steve Youngs <steve@sxemacs.org>
# Created:    <2021-05-04>
# Time-stamp: <Thursday 27 May 2021 19:36:09 (steve)>
# Homepage:   https://git.sxemacs.org/pkgusr
# Keywords:   pkgusr package-management tools

## This file is part of pkgusr

### Commentary:
#
#     This script outputs the contents (filelist) of a package.  Along
#     with the raw list of files and directories, there are also sectioned
#     lists of binaries, libraries, man page summaries, and texinfo doc
#     top node entries.  If the package is holding some deprecated old
#     libs from a previous install they are listed separately, as are any
#     "suspicious" files/directories.  By "suspicious" we mean weird-arsed
#     permissions, broken symlinks, hardlinks, that sort of thing.
#
#     The output is suitable to redirect to the pkgusr's .project file.
#
#     The idea for this script comes from Matthias S. Benkmann's
#     script 'list_package' in his set of pkgusr tools.  In fact,
#     this script uses some of Matthias' other scripts.  So, to be
#     fair, I'm going to include the copyright notice from Matthias'
#     list_package script...

###[Matthias' list_package]
# Copyright (c) 2004 Matthias S. Benkmann <article AT winterdrache DOT de>
# You may do everything with this code except misrepresent its origin.
# PROVIDED `AS IS' WITH ABSOLUTELY NO WARRANTY OF ANY KIND!
###

### User-facing differences from Matthias' list_package
#
#     o Man pages:
#       This script lists summaries for all man pages of a package,
#       whereas Matt's script only lists summaries for the binaries.
#       Matt's script lists binaries that don't have man pages (many
#       false-positives), this script doesn't.  Matt's script claims
#       to have a bug where it can't list multiple pages that exist
#       with the same name in different topic sections.  No such bug
#       in this script.
#
#     o Texinfo docs:
#       This script -- yep; Matt's script -- nope.
#
#     o Group support:
#       Matt's script supports using a group name/gid, this script
#       doesn't.  The rationale is that it was too much of a PITA to
#       determine whether a specific group belonged to a pkgusr,
#       especially doing so without escalating privileges.  Instead
#       this script will mark any file that has a different group
#       from the pkgusr's primary group, but only if the script is
#       called with '--subgroups'. (when using group name/gid with
#       Matt's script no distinction is made in the output)
#
#     o Scary warnings:
#       Quoting part of the help output from Matt's script...
#       ,----
#       | WARNING! This program is for listing files from package users only!
#       |     Do NOT use it to list files from untrusted users!
#       |     An untrusted user could set up a manipulated manpage to exploit
#       |     a bug in man when it is used to extract the summary!
#       `----
#       This script does away with that by simply refusing to run if
#       asked to list a non-pkgusr's files.
# 
#     o Deprecated libs:
#       This script -- yep; Matt's script -- nope.
#
#     Other than that the output is pretty much the same and all other
#     differences are behind the scenes, hidden from the user.

### Todo:
#
#     o Write it.
#     o Make it work without the bugs.
#


### Code:
PKG="$1"
SGRP="$2"

# User help
usage()
{
    less<<EOF

Synopsis:
--------

    ${0##*/} --help
    ${0##*/} PKGUSR
    ${0##*/} PKGUSR --subgroups

In the first form, display this help and exit.
In the second, output filelist from PKGUSR package.
In the last, do the same, but mark any files belonging to a "sub-group"

Description:
-----------

${0##*/} is a script to list an installed package's content in a
formatted, human-readable manner.

Along with a complete file list with full pathnames, it also lists
executable files, libraries, manpage summaries, info document
summaries, and any extra executables (those residing outside of
/**/{,s}bin/).  It will then list anything odd or suspicious
(setuid/gid, world/group writable, sticky, broken links, hardlinks,
non-printable chars in filenames, etc.  basically, anything funky).
This is done by calling out to 'list_suspicious_files_from'.  Finally,
it lists anything from /usr/lib/deprecated (see "Deprecating
Libraries" below).

This script can take a very long time to complete, often several minutes,
and, depending on the size of the package, output many thousands of lines
(e.g. the output for a full KDE install is well over 60K lines).  Be
smart, redirect to a file.  Ideally ~pkgusr/.project, that way you can
view it later with 'pinky -l PKGUSR|less'.

Requirements:
------------

Nothing out of the ordinary, pretty much just basic UNIX commands.  This
script calls 'forall_direntries_from' and 'list_suspicious_files_from',
so anything they need which you would already have.  We do need to take
into account dealing with compressed .info files which could be gzip,
bzip2, lzip, lzma, or xz (I personally have never seen any compressed
with anything other than gzip, but the stand-alone reader supports all
of those so, yeah... there is that)

The script will run fine if you do not have those compression tools
installed, it will just skip the file if it cannot be decompressed.
Having said that, do yourself a favour and install lzip and zutils,
especially zutils (and then get rid of zcat from gzip, it blows)

Deprecating Libraries:
---------------------

The way I deal with library upgrades where the so version changes is
to put the old lib into /usr/lib/deprecated which is listed _LAST_ in
/etc/ld.so.conf.  The way it works is...

/usr/lib/deprecated is obviously not in ld search path so libs in
there would never be used at compile/link time, but dlopen() will
still find them because the directory is listed in /etc/ld.so.conf.
Stuff that is already linked will still be able to find what it needs.
Then it is simply a matter of rebuilding the packages that depend on
the upgraded library at your leisure.  Nuke the stuff in deprecated
once nothing needs it any longer.

EOF
exit 0
}

# This interesting construct will return true if $PKG is any of:
#  ? -? --? h -h --h help -help --help usage -usage --usage
[[ "$PKG" =~ ^-?-?(h(elp)?|usage|\?)$ ]] && usage

# Bail if $PKG isn't a real package.
if [[ -z ${PKG} || ! "$(id -Gn ${PKG})" =~ "install" ]]; then
    echo 1>&2 'Invalid or missing PKGUSR'
    echo 1>&2 Usage: ${0##*/} '[--help | PKGUSR [ --subgroups ] ]'
    exit 1
fi

# We should definitely try to traverse the filesystem as few times as
# possible to save on time and CPU cycles.  So lets store the complete
# filelist right from the get-go, but only if we need to.
if [ -z "${PKGASSETS[*]}" ]; then
    # Massage $IFS to cope with filenames with spaces
    _oldifs="${IFS}"
    IFS=$'\n'
    PKGASSETS=($(forall_direntries_from $PKG|sort))
    IFS="${_oldifs}"
fi

# We are going to want extended globbing
shopt -s extglob

typeset -a INFOS MANS LIBS UBINS EBINS AEXEC
categorise()
{
    # yeah, you can use multiple counters in a for-loop
    for ((i=0,I=0,M=0,L=0,U=0,E=0,A=0; i<${#PKGASSETS[@]}; i++)); do
        # texinfo (collect full pathnames)
        if [[ ${PKGASSETS[$i]} =~ /share/info/.*\.info(-[[:digit:]]+)? ]]; then
            INFOS[$I]=${PKGASSETS[$i]}
            I=$((($I + 1)))
            # done with this file, skip to the next
            continue
        fi
        # man (collect filenames sans extensions)
        if [[ ${PKGASSETS[$i]} =~ /share/man/ ]]; then
            MANS[$M]=$(basename ${PKGASSETS[$i]%%\.?(man|n|url|[[:digit:]]*)})
            M=$((($M + 1)))
            # done with this file, skip to the next
            continue
        fi
        # libraries (collect filenames sans extensions, not symlinks)
        if [[ ${PKGASSETS[$i]} =~ /lib(64)?/lib ]]; then
            if [[ ! -h ${PKGASSETS[$i]} && -f ${PKGASSETS[$i]} ]]; then
                LIBS[$L]=$(basename ${PKGASSETS[$i]%%\.?(a|so*)})
                L=$((($L + 1)))
                # add to the list of all executables (for later mining
                # for dependencies) skip static libs
                if [[ $(file -i ${PKGASSETS[$i]}) =~ x-sharedlib ]]; then
                    AEXEC[$A]=${PKGASSETS[$i]}
                    A=$((($A + 1)))
                fi
                # done with this file, skip to the next
                continue
            fi
        fi
        # binaries in */{,s}bin/
        if [[ ${PKGASSETS[$i]} =~ /s?bin/ ]]; then
            # look at symlinks first, we'll be massaging them a bit
            if [[ -h ${PKGASSETS[$i]} && -f ${PKGASSETS[$i]} ]]; then
                LTGT=$(stat --printf "%N" "${PKGASSETS[$i]}" |
                    awk -F\' '{print "(->"$4")"}')
                UBINS[$U]=$(basename ${PKGASSETS[$i]})${LTGT}
                U=$((($U + 1)))
                # done with this file, skip to the next
                continue
            fi
            # regular files
            if [[ -f ${PKGASSETS[$i]} && -x ${PKGASSETS[$i]} ]]; then
                UBINS[$U]=$(basename ${PKGASSETS[$i]})
                # add to the list of all execs (not scripts)
                if [[ $(file -i ${PKGASSETS[$i]}) =~ x-executable ]]; then
                    AEXEC[$A]=${PKGASSETS[$i]}
                    A=$((($A + 1)))
                fi
                U=$((($U + 1)))
                # done with this file, skip to the next
                continue
            fi
        fi
        # binaries outside of */{,s}bin/ (not symlinks)
        if [[ ! -h ${PKGASSETS[$i]} && -f ${PKGASSETS[$i]} &&
                    -x ${PKGASSETS[$i]} ]]; then
            EBINS[$E]=${PKGASSETS[$i]}
            # add to the list of all execs (not scripts)
            if [[ $(file -i ${EBINS[$E]}) =~ x-executable ]]; then
                AEXEC[$A]=${EBINS[$E]}
                A=$((($A + 1)))
            fi
            E=$((($E + 1)))
        fi
    done

    # sort the libs and binaries
    oldifs="${IFS}"
    IFS=$'\n'
    UBINS=($(sort <<<"${UBINS[*]}"))
    LIBS=($(sort -u <<<"${LIBS[*]}"))
    IFS="${oldifs}"

    # export AEXEC for further processing
    export AEXEC
}

## Process the docs
_have_docs=0                    # Keep note if the pkg has docs
# texinfo
procinfos()
{
    local cmd
    _have_docs=1
    echo -e '\nDOCUMENTATION (texinfo):'
    if [[ $(file -i $(type -Pp zcat)) =~ x-executable ]]; then
        # Sweet! we have zcat from zutils
        #
        # This will fail if it hits a .lzma or .zst but texinfo 6.7's
        # stand-alone info reader doesn't support zstd, and I have
        # never in my life ever seen an info file compressed with
        # lzma. tl;dr I'm not fixing it because you'll never see it
        for ((i=0; i<${#INFOS[@]}; i++)); do
            zcat ${INFOS[$i]}|sed -n /START-INFO-DIR-ENTRY/,/END-INFO-DIR-ENTRY/p |
                sed /DIR-ENTRY/d
        done | sort -u | sed 's/^/  /'
    else # fine, do it the hard way then
        for ((i=0; i<${#INFOS[@]}; i++)); do
            # Um, yeah.  Slow AF.  Install zutils.
            case $(file -Lb --mime-type ${INFOS[$i]}) in
                application/gzip)    cmd=(gzip -dc)  ;;
                application/x-bzip2) cmd=(bzip2 -dc) ;;
                application/x-xz)    cmd=(xz -dc)    ;;
                application/x-lzip)  cmd=(lzip -dc)  ;;
                application/x-lzma)  cmd=(lzma -dc)  ;;
                application/zstd)    cmd=(zstd -dc)  ;;
                *)                   cmd=(cat)
            esac
            type -Pp ${cmd[0]} > /dev/null || continue
            ${cmd[@]} ${INFOS[$i]}|sed -n /START-INFO-DIR-ENTRY/,/END-INFO-DIR-ENTRY/p |
                sed /DIR-ENTRY/d
        done | sort -u | sed 's/^/  /'
    fi
}

# man
procmans()
{
    _have_docs=1
    echo -e '\nDOCUMENTATION (man):'
    for ((i=0; i<${#MANS[@]}; i++)); do
        man --whatis ${MANS[$i]} 2>/dev/null
    done | sort -u | sed 's/^/  /'
}

# We ain't got no steenkin docs
nodocs()
{
    echo -e '\nDOCUMENTATION (lol, wut?):'
    echo No docs here, maybe try Google...
}

## Process the binaries
# bins in {,s}bin
procubins()
{
    echo -e '\nEXECUTABLES (in */{,s}bin/):'
    echo ${UBINS[@]}|sed -e 's/ /, /g' -e 's/^/  /'|fmt
}

# bins outside of {,s}bin
procebins()
{
    echo -e '\nEXTRA EXECUTABLES (outside of */{,s}bin/):'
    for ((i=0; i<${#EBINS[@]}; i++)); do
        echo "  ${EBINS[$i]}"
    done
}

# libs
proclibs()
{
    echo -e '\nLIBRARIES:'
    echo ${LIBS[@]}|sed -e 's/ /, /g' -e 's/^/  /'|fmt
}

# We need to do one more find, but it is short and sweet so won't add
# any noticable time to the process...
listdeprecated()
{
    DEPRECATED=$(find /usr/lib/deprecated -maxdepth 1 -type f \
        -user $PKG -printf "  %p\n")
    if [ -n "${DEPRECATED}" ]; then
        echo -e '\nDEPRECATED LIBRARIES (remember to audit):'
        echo ${DEPRECATED}
    fi
}

## The complete file list
filelist()
{
    local GRP
    echo -e '\nFILE LIST:'
    if [[ -n ${SGRP} && ${SGRP} =~ ^--?subgroups$ ]]; then
        for ((i=0; i<${#PKGASSETS[@]}; i++)); do
            GRP=$(stat --printf "%G" "${PKGASSETS[$i]}")
            if [[ ${GRP} != ${PKG} ]]; then
                echo "* ${PKGASSETS[$i]} [${GRP}]"
            else
                echo "  ${PKGASSETS[$i]}"
            fi
        done
    else
        for ((i=0; i<${#PKGASSETS[@]}; i++)); do
            echo "  ${PKGASSETS[$i]}"
        done
    fi
}

## Rock and Roll
categorise
# binaries and libraries
[[ ${#UBINS[@]} -gt 0 ]] && procubins
[[ ${#EBINS[@]} -gt 0 ]] && procebins
[[ ${#LIBS[@]} -gt 0 ]] && proclibs
# docs
[[ ${#MANS[@]} -gt 0 ]] && procmans
[[ ${#INFOS[@]} -gt 0 ]] && procinfos
[[ ${_have_docs} -eq 0 ]] && nodocs
# main file list
filelist
# sus files
list_suspicious_files_from ${PKG}
# deprecated
listdeprecated

## Return cleanly via return and not exit so any calling script won't
## exit
return 0

### list_package ends here
# Local variables:
# coding: utf-8
# end: