#T2HTML-OPTION --css-code-bg
Tinypath.el --- Manage Emacs startup dynamically

    Preface Feb 1999 - How it all begun
  
        When you have set up your Emacs installation to your liking, a day
        comes when you decide that it's time to seriously reconsider the
        directory structure of your installed lisp packages. At start, it
        is customary to use simple file hierarchy where all private
        packages are installed under:
  
            ~/elisp   ;; some people also prefer ~/lisp
  
        Complete kits are usually installed directly under the root:
  
            ~/elisp/packages/bbdb-2.00.06/
            ~/elisp/packages/psgml-1.0.3/
            ~/elisp/packages/pcl-cvs-2.9.2/
  
        A more sophisticated way is to use symlinks to the latest
        versions, so that you don't have to change `load-path' every
        time you install a new version. It is only matter of updating
        the symlink:
  
            ~/elisp/packages/pcl-cvs/  --> ~/elisp/packages/pcl-cvs-2.9.2/
            |
            This path is in the `load-path'
  
        In networked, where Windows ic oupled with Unix workstations via SAMBA,
        you may have mapped the _H:_ disk to you Unix _$HOME_:
  
            H:  --> Unix $HOME  \\SERVER\DIRECTORY\YOUR-LOGIN-DIR
  
        Now, there is a catch when you use Unix symlinks in `$HOME/elisp'
        and try to access the directories from Windows. Having set PC's
        HOME environment variable to point to H:, Emacs can start reading
        Unix `$HOME/.emacs' startup file, but you soon see simple
        error messages like "Can't load library xxx", which soon follow by
        bigger concerns: "autoloading xxx failed". The problem is the
        mounted H: disk. You see, PC's network mount can't distinguish
        symlinked directories from real directories, so all symlinked Unix
        directories in `load-path' are dead. And that's why most of the
        files cannot be found any more.
  
       The conclusions
  
        For cross platform solution it is best not to rely on symlinks,
        because they don't work over a Windows mount. Secondly,
        updating `load-path' should not be needed by hand after a new
        package installation, after a directory name change, after
        directory structure change, etc. A dream package would solve this
        all and do the hard work: "There, that is the root(s) of all Emacs
        lisp, go and sniff all the directories and update `load-path'"
  
        That was what this package originally was all about. Nowadays it
        does a little more than that. Your `load-path' is updated
        automatically without any manual work. You only have to give the
        start ROOT path(s) of your installed lisp hierarchies in the file
        system. This package is highly effective: scanning thousands of
        files in a matter of seconds and once the cache has been created,
        it takes only a snap to load it in next sessions. All `require' and
        `load' commands execute also faster than previously, because the
        information about existing files is immediately available. The
        speedup is helped through advised functions.
  
    Overview of features
  
       Automatic load-path configuration
  
        o   Define list of `root' directories of your Emacs lisp and this
            package will recursively add directories which contain .el or
            .elc files to `load-path'
        o   A cache is utilized to remember previous scan and
            expired periodically. Using cache speeds up loading files
            considerably if you have many directories. The number of lisp
            directories doesn't affect the load performance any more.
            This is accomplished by using extra advice code in functions:
            `load', `load-library', `require', `locate-library' and
            `autoload'.
        o   When Emacs becomes idle (some 15 minutes of idle time) the
            cache and `load-path' is validated for erroneous entries and
            rebuilt as needed. This feature should auto-detect changes in
            directory structure automatically and help semi auto-installing
            new directories (packages) for you.
        o   The `load-path' is optimized so, that users' files automatically
            take precedence first (~/elisp), next any other files found,
            and last the core Emacs files in the distribution.
            For example in previous Emacs versions, shipped
            speedbar.el used to be too outdated if used with JDE
            package. It had to be manually updated from the
            development site. That us the reason why core Emacs paths
            are at the then of `load-path'.
  
       Automatic Info-default-directory-list configuration
  
        o   If you download packages that include Emacs info files,
            the `Info-default-directory-list' is updated at the same time
            as the `load-path', when root directories are examined.
        o   No more manual updating of info files. The missing
            `dir' entry is created or updated as needed.
        o   You can update all _new_ info files in your system by calling
            M-x `tinypath-info-scan-Info-default-directory-list'
  
        If you have added new info files by hand, just call function
        `tinypath-info-handler' to update your Emacs and update the
        `dir' entry. After that reset infor with `M-x'
        `tinypath-info-initialize'
  
       Win32 automatic manpath configuration
  
        o   In Unix systems the MANPATH enavironment variable contains
            directories where to find manual pages, but in Win32,
            there is no default MANPATH and `M-x' `man' does not work.
        o   If package *woman.el* (Included in latest Emacs
            versions) is along `load-path', it is automatically
            configured to support to read manual pages. It replaces
            the `M-x' `man' command.
  
       Win32 Cygwin environment support
  
        o   If *cygwin1.dll* (<http://www.cygwin.com/>) is in your path,
            automatic detection tries to find the Cygwin root and scan
            manual and info pages for use with *woman.el*
            _Note:_ This feature s for native Win32 Emacs. Nowadays,
            there is also native Cygwin Emacs, which behaves just like
            the big brother *nix Emacs.
  
       Compressed lisp file support
  
        o   Overloads commands load, load-library, load-file, require
            and autoload to accept `jka-compr' compressed lisp .el files.
        o   Primarily meant to be used in low quota accounts.
        o   Compress or decompress lisp files. You don't have to change 
            a thing in your Emacs startup file, all will work as usual.
        o   Correctly handles aliased commands that turn out to be
            in `autoload' state.
  
    How to set up your load path
  
        The `tinypath-:load-hook' contains function `tinypath-setup' which
        starts examining all directories under `load-path' and
        `tinypath-:load-path-root' which is set to reasonable defaults of
        site wide and personal installations. If you keep all your lisp
        files under *$HOME/elisp*, then you do not need to configure anything
        for this package to work. Your `load-path' has been updated after
        this statement at the beginning of your *$HOME/.emacs*
  
            (load "~/elisp/tiny/tinypath") ;; Or anywhere you have it installed
  
        If you have _many_ separate Emacs lisp root directories, like one
        for *site-lisp* and one for *site-packages* and one for your
        *personal* *lisp* files, then you have add those to variable
        `tinypath-:load-path-root'. Below there is an example for PC users,
        where the E: partition replicates identical Unix tree structure. 
        We suppose for the moment that Cygwin is installed there.
        The following actually works for Unix too, because non-existing
        directories will be ignored:
  
            (setq tinypath-:load-path-root
              '("~/elisp"  "E:/usr/local/share/emacs/site-lisp/common"))
            (load "~/elisp/tiny/tinypath")
  
    XEmacs and Emacs specific directories
  
        In spite of great effort from developers to make packages
        compatible for both Emacs platforms, there is always some packages
        that only works with Emacs or XEmacs. It is assumed that the site
        admin has created directories like these to keep the *site-lisp*
        installation clean:
  
            ;;   This might be also under /opt/share/site-lisp
            ;;   Refer to file hierarchy standard at
            ;;   http://www.pathname.com/fhs/
  
            /usr/share/emacs/site-lisp/common/   .. XEmacs and Emacs
            /usr/share/emacs/site-lisp/emacs/    .. only for Emacs
            /usr/share/emacs/site-lisp/xemacs/   .. only for XEmacs
  
        To take care of the Emacs specific `load-path' setting, use code
        similar to this snippet. If you load the setup multiple times, the
        `pushnew' ensures that the directories are not added multiple
        times.
  
            (require 'cl)
            (dolist (path ("~/elisp"
                           "/usr/share/emacs/site-lisp/common"
                           ;;  Select Emacs or XEmacs specific installations
                           (if (boundp 'xemacs-logo)
                               "/usr/share/site-lisp/xemacs"
                             "/usr/share/site-lisp/emacs")))
              (when (stringp path)
                (pushnew path tinypath-:load-path-root :test 'string=)))
  
            ;; PLEASE COPY VERBATIM. THERE ARE OPTIMIZATIONS
            ;; THAT ACTIVATE If YOU ADD THE PATH
            (pushnew "~/elisp/tiny/lisp" load-path :test 'stringp=)
            (load "tinypath.el")
  
        In fact this package will check current emacs version and make
        sure that only correct directories are included to the
        `load-path'. If you simply instructed to search the whole
        site-lisp root `/usr/local/share/site-lisp', and current emacs
        binary is "emacs", then all directories that contain path
        portion `/xemacs'  woudl have been automatically ignored.
  
       Building part of site-lisp using Net
  
        If we continue talking a bit more about site-lisp, there is utility
        *mywebget.pl* at <http://perl-webget.sourceforge.net/>. It 
        includes a *mywebget-emacs.conf* which contains
        knowledge where the various lisp developers' home pages are and how
        to download all known lisp tools that does not come with Emacs. If
        you have lot of disk space and you're interested in getting more
        lisp tools to go with your Emacs, follow the instruction laid out
        in this project's sourceforge page.
  
        If you are further interested in Emacs packages, see CVS
        version control program available for Unix at
        <http://www.cvshome.com/> and for Win32 `cvs' will ship with
        the <http://cygwin.com> installation. With CVS you can track
        development of many Emacs projects including Gnus, BBDB,
        Mailcrypt etc.  Cvs is minimizing network traffic by
        transferring only changes.  Here is one suggestion where you
        could put all your Emacs Lisp Version control downloads:
  
             /usr/share/emacs/site-lisp/net/cvs-packages
  
        Now, the overall structure of whole site-lisp looks something like
        this:
  
                     ROOT/  ( /usr/share/emacs or equivalent )
                     |
                     +--site-lisp/
                        |
                        +--emacs/
                        |  |  ...Eamcs only files
                        |  +--packages/
                        |  |  +--pcl-cvs-2.9.9/
                        |  |  +-... and so on
                        |  +--win32/
                        |     +--gnuserv/
                        |     +-... and so on
                        +--net/
                        |  +--users/
                        |     +-LispDeveloperA
                        |     +-LispDeveloperB
                        |     +-... and so on
                        |  +--cvs-packages/
                        |     +--liece/
                        |     +--lookup/
                        |     +--ILISP/
                        |     +--jess-mode/
                        |     +--devel/
                        |     +--emacro/
                        |     +--tnt/
                        |     +--cc-mode/
                        |     +--mailcrypt/
                        |     +--bbdb/
                        |     +--gnus/
                        |     +-... and so on
                        +--common/
                        |     ...COMMON for both Emacs and XEmacs
                        |     =======================================
                        |     ...Packages that you find posted to the
                        |     ...gnu.emacs.sources and whose author's
                        |     ...do not have a homepage
                        +--xemacs/
                           |  ...XEamcs only files
                           +--cvs-packages/
                              +--xemacs-packages/
  
       XEmacs 21.2+ core packages
  
        The latest XEmacs versions ship with only the very basic
        installation. Lisp packages are distributed in separate archive
        *xemacs-packages* (nick named SUMO due to its huge size). There is
        also *mule-packages* and *site-packages* archives. A built-in
        heuristics tries to guess the location of these by looking under and
        near your XEmacs installation:
  
            .../XEmacs/XEmacs-NN.N/xemacs-packages
            .../XEmacs/xemacs-packages
  
        If the archives have been installed elsewhere, you have to tell the
        location by defining following variable prior loading TinyPath. You
        can't put these to `tinypath-:load-path-root' because this is
        special information that needs to present during the very initial
        boot-up to find crucial packages like *jka-compr.el*.
  
            (setq tinypath-:core-emacs-load-path-list
                  '("/usr/share/site-lisp/xemacs/xemacs-packages"
                    "/usr/share/site-lisp/xemacs/mule-packages"
                    "/usr/share/site-lisp/xemacs/site-packages"))
  
    Finding load-path directories
  
        If you only used default *$HOME/elisp* directory for your files, the
        `tinypath-:load-path-function' starts recursively searching all
        the directories under the root(s) `tinypath-:load-path-root'. Not all
        directories are counted in when the search descends below the root(s).
        Variable `tinypath-:load-path-ignore-regexp' decides if the directory
        should be ignored. By default:
  
        o   Package's additional subdirectories like texinfo, tex, doc, etc,
            misc, RCS, CVS, .svn (Subversion), MT (monotone version control), 
            zip are ignored.
        o   Any temporary directories named .../t/ .../T/ .../tmp* .../temp*
            are ignored.
        o   Directories that do not contain any files ending to .el or .elc are
            ignored.
  
    Gnus and other 3rd party packages
  
        _Note:_ In latest version of this utility *Gnus* is treated
        specially. All Gnus versions are detected along load-path and
        the very latest Gnus version is installed to your
        `load-path'. This is based on the knowledge in the
        `gnus-version' variable and the heuristics will pick the
        newest for you. You actually do not have to do anything else,
        but to drop latest Gnus somewhere, to be able to use it
        immediately.
  
         Under the hood - What follows, is old documentation.
  
        It is important to understand how this package works: It caches
        every possible lisp directory it can find. Now, if you have
        installed private copy of Gnus, say in `~/elisp/cvs-packages/gnus',
        there is a problem, because Emacs distribution also includes Gnus.
        There is NO WAY TO TELL OR CHANGE path order when the cache is in
        use. This is a design decision and cannot be changed. The old trick,
        where a new directory was added in front of `load-path', will not
        work because everything goes through cache. What you need to do
        instead, is to tell that the "other" Gnus should be ignored during
        cache creation, so that it is completely unknown.
  
       Solution: ignoring directories
  
        There is very simple way. Put your regular expression to
        `tinypath-:ignore-file-regexp-extra' and it will tell which
        directories to ignore.  Naturally you must put the lisp code
        _before_ you load package.
  
            (setq tinypath-:load-path-ignore-regexp-extra
                  "\\|[/\\]x?emacs[/\\0-9.]+[/\\]lisp[/\\]gnus")
            ;; PLEASE COPY VERBATIM. THERE ARE OPTIMIZATIONS
            ;; THAT ACTIVATE If YOU ADD THE PATH
            (require 'cl)
            (pushnew "~/elisp/tiny/lisp" load-path :test 'stringp=)
            (load "tinypath.el")
  
        [For advanced Lisp programmers] You can add ignored gnus directory
        to `tinypath-:load-path-ignore-regexp' via
        `tinypath-:load-path-ignore-regexp-hook'. When the hook is run, the
        default value for `tinypath-:load-path-ignore-regexp' is already
        available. In hook, append regular expression that excludes the
        Gnus directory. Here is an example; make sure that you don't add
        the regexp multiple times. The multiple invocations is protected by
        setting a plist property and checking it. The ugly [\\/] makes the
        regexp compatible with both Unix and win32 paths. System
        directories in Unix are typically /emacs/NN.NN/ and in win32
        /emacs-NN.NN/, that's why added "-".
  
            (add-hook 'tinypath-:load-path-ignore-regexp-hook
                      'my-tinypath-:load-path-ignore-regexp-hook)
  
            (defun my-tinypath-:load-path-ignore-regexp-hook ()
              ;;  Do this only once
              (unless (get 'my-tinypath-:load-path-ignore-regexp-hook 'set)
                ;; mark as done.
                (put 'my-tinypath-:load-path-ignore-regexp-hook 'set t)
                (setq tinypath-:load-path-ignore-regexp
                      (concat
                       tinypath-:load-path-ignore-regexp
                       "[/\\]x?emacs[/\\0-9.]+[/\\]lisp[/\\]gnus"))))
  
        #todo: What about XEmacs public/private Gnus installations?
  
    Updating new lisp packages
  
        Suppose you have installed a new version of a package:
  
            ~/elisp/gnus/pgnus-0.74/
            ~/elisp/gnus/pgnus-0.95/    ;; NEW
  
        Both these directories end up being added to the `load-path',
        but that is not preferable. It is the latest version that
        should be in the `load-path'. The solution is to move the old
        versions under some name that will be ignored by default. It
        is recommended that a backup of previous packages are renamed
        to start with a word "tmp-". All directories that start with
        prefix *tmp* are ignored.
  
            % mv ~/elisp/gnus/pgnus-0.74/ ~/elisp/gnus/tmp-pgnus-0.74/
                                                       ====
  
        However if you update package in a site-lisp directory, there
        may be a distant problem that somebody needs older version of
        the package. If you made the backup like above, that user
        cannot load the old package any more, because it doesn't show
        up in `load-path'
  
        There is no easy answer to keep old packages. Admin could
        announce that: "new version has been installed in DIR, the old
        one is in TMP-OLD-DIR" and have users manually arrange their
        `load-path' if needed. Following lisp command would solve
        their setup. The statement below adds the old directory to the
        *beginning* of `load-path' and thus load commands would find the
        old version of the package first.
  
            (load "~/elisp/tiny/tinypath")
            (pushnew "TMP-OLD-OLD-DIR" load-path :test 'string=)
            (tinypath-cache-regenerate)
  
        Remember to mention to users that they need to update cache with
        `tinypath-cache-regenerate' (called with prefix argument) to see
        the changes.
  
    Duplicate files in path
  
        If you have accustomed to putting your path to specific order, you
        have to rethink your strategy if you are going to use this package.
        The philosophy behind creating this utility was: YOU SHOULD NOT
        NEED TO DO MANUAL WORK TO UPDATE PATHS. This means that the order of
        the paths *must* not be significant. Now, you may face a situation
        where you install a library or package that contains a file, which
        has already been installed to your hierarchy. Take for example,
        *smtpmail.el*:
  
            /usr/bin/emacs-20.4/lisp/mail/smtpmail.el
            /usr/share/site-lisp/common/packages/semi/flim-1.12.1/smtpmail.el
  
        We have a problem here if FLIM's *smtpmail.el* is not compatible with
        the one in Emacs. If it is, then there is no problem. Either one can be
        loaded, and the `load-path' order does not matter. But you don't
        know that before you get error "function smtpmail-xxxx not defined"
        and you start investigating with (locate-library "smtpmail") which
        package is actually active.
  
        Acer activating this package, please investigate your path
        with [C-u] `M-x' `tinypath-cache-problem-report' and see if you
        find duplicate entries. Investigate each one and possibly move
        the file to another name or remove older ones. E.g. in the above
        situation, the cure might be moving FLIM's *smtpmail.el* under name
        *flim-smtpmail.el* so that it doesn't get loaded with (require
        'smtpmail). The best is to contact the maintainer(s) and tell them
        about conflicts. Here is a sample of one generated problem report:
  
            imenu.el
              323 34073 1998-05-07 16:28:08 /usr/share/site-lisp/common/other/
              910 37169 1999-12-04 02:47:58 /usr/share/site-lisp/common/programming/java/jde/jde-2.1.6beta13/lisp/
              1350 38663 1999-11-28 01:14:38 /usr/bin/emacs/gnu-emacs/emacs-20.4.1/lisp/
            base64.el
              515  9943 1999-12-11 19:15:20 /usr/share/site-lisp/common/packages/gnus-5.8.2/lisp/
              807  9892 1999-11-15 00:00:12 /usr/share/site-lisp/common/packages/w3-4.0pre.46/lisp/
  
        _Explanation:_ User had used *imenu* as a separate package
        since early Emacs versions in "other" directory. Now the
        latest Emacs ships one, so it is best to delete the previous
        one `other/imenu.el.' Keep on eye on the numbers here: The
        lower, the more close it is to the beginning of cache when the
        directories were searched. The package with lowest score will
        get loaded. Another package, *base64.el* seems to be
        problematic too. But because Gnus path has lowest score, it
        will get loaded before w3's base64.el. This is good, because
        Gnus contains the latest version of *base64.el*. In the buffer
        `tinypath-report-mode' is turned on to manipulate reported
        lines.  Unnecessary files can be deleted with
        `Control-shift-mouse-1' or `C-c' `C-d'.
  
    Symlinked directories are ignored
  
        It has been the tradition to use symlinks a lot in Unix to
        arrange easy access to versioned packages. Like how to
        ~/elisp/gnus/ no matter what version is currently installed.
  
            ln -s ~/elisp/packages/gnus-N.NN  ~/elisp/packages/gnus
  
        This package however *skips* those symlinks and records the
        absolute path name to the `load-path'. There are couple of points:
        a) it is more instructive to peek the `load-path' to actually see
        what versions have been installed to the Emacs b) The symlinks are
        error prone since there may be several symlinks that lead to same
        directory and c) symlinks don'e work in heterogenous environments
        where Win32 and Linux and Unix is networked together. To migrate to
        this package you need to examine your symlinks and remove them.
  
        If you have drawn a symlink to the the current directory from
        *SEPARATE* directory, then that directory will never be seen:
  
            ln -s ~/some-disk/elisp/artist-1.1/ ~/elisp/packages/artist-1.1
  
        To solve this, instead either _a)_ move the package physically
        under the ~/elisp/ from the *~/some-disk/elisp/* so that the
        recursive search will record it or _b)_ add the separate
        directory *~/some-disk/elisp* to the variable
        `tinypath-:load-path-root'.
  
    Using cache
  
        Now when you're freed from update burden of the directories in your
        disk, you can concentrate organizing the files under sensible
        directories. Here is an example how the organizing could go:
  
            ~/elisp/users/kevinr/       Kevin Rodger's files
            ~/elisp/users/ilya/         Ilya Zakharevich's files
            ..
            ~/elisp/packages/bbdb-2.00.06/  Version-ed packages
            ~/elisp/packages/psgml-1.0.3/
            ~/elisp/packages/pcl-cvs-2.9.2/
            ~/elisp/packages/tiny-19990215/
            ...
            ~/elisp/other/              All single add-on packages
  
        All these paths in `load-path' and you can imagine how slow a
        standard Emacs would become: it takes even more time to find some
        package xxx, when Emacs sees a call (require 'xxx), because Emacs
        must start looking into every single directory under `load-path'
        until it can determine if it can or cannot load the asked package.
        This utility will store all lisp files in cache, and it is
        activated by default. The variable `tinypath-:cache-expiry-days'
        controls the interval when it is concluded that a new tree
        recursion is needed. If you install new packages during those
        non-expiry days, it is best to call `C-u' `M-x'
        `tinypath-cache-regenerate' to build up to date image of your files
        and `load-path' directories.
  
          If you want one short advice: always call `tinypath-cache-regenerate'
          after any lisp file or directory update.
  
    Cache file and different Emacs versions
  
        It is important that each Emacs loads correct cache file. The cache
        file's name is derived from the emacs version and emacs type, which
        can be "xemacs", "win32-xemacs", "emacs" or "win32-emacs".
  
              tinypath-:cache-file-prefix
            + EMACS-TYPE
            + HOST
            + EMACS-VERSION
            + tinypath-:cache-file-postfix
  
            ~/elisp/config/emacs-config-tinypath-cache-win32-HOST-emacs-20.4.1.el.gz
            ==========================================                        ======
            prefix                                                           postfix
  
       Unix hosts and NFS mounts
  
        In Unix environment, it is also common that several hosts are NFS
        mounted so that your home disk is available from every server. The
        available programs are not usually NFS mounted, but programs are
        stored locally on each server's own disk. Now, there would be a
        problem if you logged to host *A* and started tinypath.el which had
        made cache in on host *B*, because *A* does not have the same
        directories as *B* did (site-lisp). This has been taken care of by
        including _hostname_ part in the cache file name. For each host, a
        separate cache file is created. Now, suppose all the Unix hosts are
        same brand, say Sun OS, Linux, or HP-UX and a good administrator has
        separated the programs and the data in their own directory
        structures. Furthermore, these structures are NFS mounted and thus
        visible to the remote machines. In this scenario, it would not
        really matter to which host you log into, because you would always
        see the same programs and site-lisp directories and there would not
        be need for host specific cache files. In that case, disable the
        *HOST* word by setting this variable to nil:
  
            (setq tinypath-:cache-file-hostname-function nil)
  
    Info file support
  
        In addition to updating the `load-path', the recursive function
        has a chance to search for installed info files as well. When you
        keep all your site lisp under one directory, it is not uncommon
        that the bigger packages include documentation files in info format
        as well. Like:
  
            /usr/share/site-lisp/emacs/pcl-cvs-2.9.9/
            /usr/share/site-lisp/common/packages/psgml-1.2.1/
  
        One possibility is that after you download and uncompress a
        package, you copy the info file to some central directory
        where you keep all you info files. This is lot of manual work.
        (Never mind that in Unix you might use Makefile to install
        everything, in Win32 it's all manual work). This package does the
        same job by looking for directories that either have info files or
        a central info repository called `dir'. If the `dir' file
        has all the info files up to date, nothing is done. In other cases:
  
        o   If the central `dir' in the directory does not exits,
            it is created.
        o   If `dir' does not contain entry for info file, it is added.
            The entry name is derived from the filename.
  
        The `Info-default-directory-list' is updated to include any new
        directory locations and they are saved to same cache file. When you
        call `C-h' `i' you will see the new info entries. Easy and
        maintenance friendly. No need to worry about supplied info files any
        more, they are automatically integrated to your Emacs. If you have
        installed any new packages to your system, Emacs packages or Unix
        packages that installed something with "install -c", it is best to
        update your info files with `M-x'
        `tinypath-info-scan-Info-default-directory-list'. This is also
        called if you call: `C-u' `M-x' `tinypath-cache-regenerate'
  
    Cygwin support (Win32 and woman.el)
  
        It is common that Emacs in Win32 environment is coupled with
        <http://www.cygwin.com> toolkit which contains all the manual pages
        for the unix commands and possibly new info pages. This package
        will locate `cygwin1.dll' file along PATH and recurse whole cygwin
        installation root to find new entries that can be used inside
        Emacs. In theory this all should happen automatically and the only
        thing you have to do is to ensure that you have proper PATH
        settings at your OS level before this package is started. If Cygwin
        /bin directory in in PATH, `tinypath-:extra-path-root' will get set
        to a correct value at boot time.
  
        If you have more places where you keep Unix tools which contain
        more manual or info pages, like Reed Kotler (old Unix-like env)
        http://www.reedkotler.com/ you _must_ manually set variable
        `tinypath-:extra-path-root' to the list of search root directories.
        If you set this yourself, you _must_ also include the cygwin
        installation root directory
  
            (setq tinypath-:extra-path-root
                  '("e:/unix-root/cygwin"
                    "e:/unix-root/reed-kotler"
                    ...))
  
        Package *woman.el* will be configured automatically if it is along
        `load-path' to handle manual page viewing with command `M-x'
        `man'. Please make sure that you do not destroy the pre-defined
        `woman-manpath' in your Emacs startup files with lisp commands or
        the efforts to find out new manual pages are thrown off the window.
        Search you startup files for anything that looks like `setq',
        `defvar', `defconst': (setq woman-manpath ... and change the code
        to _add_ to the variable instead:
  
            (require 'cl)
            (dolist (path '("one" "two" "three"))
              (pushnew (expand-file-name path) woman-manpath :test 'string))
  
    Faster Emacs configuration (Perl emacs-util.pl)
  
        Indication of this feature at startup is following message, where
        EXT refers to externally launched process which must be waited
        until further processing is done.
  
            TinyPath: EXT Process running ... [may take a while]
  
        As this package evolved and more support was added to various
        environments, like Cygwin, which requires traversing hundred of
        directories to find out if they contained info or manual pages,
        it came evident that Emacs Lisp method was too slow. An alternative
        method was developed using Perl language and written in *emacs-util.pl*
        which can traverse directory hierarchies to find relevant
        directories for the setup. This interface is automatically used
        if two conditions are met in current environment:
  
        o   Binary *perl* must be along PATH. (according  `executable-find')
        o   perl script *emacs-util.pl* must be along PATH. Either copy
            the file to suitable place or include Tiny Tool's `/bin'
            directory to your PATH (This is not the whole story, see
            "Internal optimizations").
  
        If all goes well, a `call-process' to the utility script will
        return the file hierarchies much faster than the Emacs Lisp ever
        could. The difference is that you don't see the traversing progress
        as you would if Emacs Lisp did the same thing. The command line
        arguments passed to the utility scripts can be found from the
        *Message* buffer and you can run the program yourself if you think
        that it returns incorrect listing. Print the script help with
        following command:
  
            % perl emacs-util.pl --help
  
        Here are some performance statistics of the perl script in action.
        (Use --verbose argument to see the statistics)
  
        o   Intel 400MHz, IBM GXP 80G IDE disk, whole Cygwin installation
            scan: 3 min 46 sec, dirs: 2373, files: 35 271
        o   Same PC, but this time site-lisp directory, subset of Cygwin
            hierarchy at /usr/share/site-lisp took:
            0 min 13 sec, dirs: 648, files: 8750
  
        Let's consider one scenario that you may encounter if you intend to
        use Cygwin similarly as the big brother Linux. Let's suppose that
        you have dedicated a disk portion where you intend to duplicate
        whole Linux-like directory hierarchy. You have ROOT under which you
        keep all the files, including anything that is Cygwin-related.
  
            E:/usr/share/site-lisp Emacs lisp as outlined earlier
            E:/usr/share/site-perl Perl packages and scripts
            E:/usr/share/site-php  PHP code
            E:/usr/share/site-cvs  Various other external CVS C-packages
  
        The default heuristics `ti::win32-cygwin-p' should find
        *cygwin1.dll* installed and report that Cygwin root is *E:/*
        This means that `tinypath-:extra-path-root' will get set for
        you when package loads. Suppose further that you have set
        variable `tinypath-:load-path-root' to point out suitable
        locations in *E:/usr/share/site-lisp*. It would seem
        that this combination means that the hierarchies would be
        traversed multiple times, since the Cygwin root already
        includes all the rest:
  
            E:/                             Cygwin root
            E:/usr/share/site-lisp/emacs    For this emacs...
            E:/usr/share/site-lisp/common   Emacs and XEmacs compatible tree
  
        Don't worry. The Perl utility is smart enough to reduce this to
        search only *E:/* and discard other roots as redundant. Hm, what if
        other lisp files are found _outside_ of the
        *E:/usr/share/site-lisp/*, because it searches every dir starting
        from *E:/* Say:
  
            E:/tmp/try/some-file.el
  
        Will the directory *E:/tmp/try/* reported as lisp `load-path'
        candidate and added to search list? Yes and no. Yes, it will be
        reported, but no, it will not be added to the `load-path' because it
        doesn't match the initial user's idea where to look for lisp files. If
        you pump up the `tinypath-:verbose' to level 5, you can see PATH-NOK
        messages labeled "candidate" to indicate those rejections. Only files
        that reside under `tinypath-:load-path-root' directories are counted
        in.
  
    Updating running Emacs
  
        Suppose you have downloaded the latest versions of packages X, Y and Z
        and you want your current emacs's paths updated, call this function:
  
            M-x tinypath-cache-regenerate
  
        Take a bit of skepticism: It is a fortunate event if it all
        worked that easily. You see, you already have several packages
        loaded in your Emacs and they are using the "old" code. Now
        you wiped the old directories away and told Emacs to look for
        only "new" directories.  After a while you may run into
        bizarre dependency problems. I recommend that after any major
        package update, which contains _several_ of files (like Gnus),
        you:
  
        o    Install package and regenerate cache in current Emacs session
             with `M-x' `tinypach-cache-regenerate'.
        o    Save your current Emacs buffers (see *desktop.el*, *tinydesk.el*)
        o    Quit, restart Emacs and restore your working desktop.
  
    Compressed lisp file support
  
        In order to use the full compression support for autoload functions
        as well, set variable `tinypath-:compression-support' to symbol
        `all'. the default support only handles `require' and `load' commands.
        the variable must be set before rhis utility package is loaded.
  
       Jka-compr and this package
  
        jka-compr has native support to un/compress any file that have
        specific extensions. The handling is done via
        `file-name-handler-alist' and commands like these will load
        properly including any autoloads.
  
            (load "my-el.gz")
  
        The obvious problem is that you have to manually go and change all
        you load commands so that they end in .gz so that jka-compr takes
        care of loading. What if you later uncompress the file? You have to
        go and update all the load commands in you Emacs startup files.
        This isn't very nice, since you should be able to un/compress elisp
        files whenever you wish and still have permanent statement like one
        below. Basically this is what the compression support here is all
        about; you don't have to worry if the file is compressed or not
        when these advised functions are in effect. The following statement
        will always work:
  
            (load "my-el")
  
       How the compressed loading works
  
        o   When user request `load' FILE, try to find some compressed file
            that JKA knows about by adding extensions ".gz" and ".Z" and
            whatever user has configured JKA to handle. _LIMITATION:_
            only .gz .bz2 and the like that compress one file at a time
            is currently supported. Don't try using .zip or similar.
        o   If the FILE is absolute path, then look from that
            directory only.
        o   If no directory is given, find the file along the path.
        o   If there was somewhere a compressed file, just load it (because JKA
            will transparently uncompress it), eval it, and kill the buffer.
        o   If NO COMPRESSED file was found, just follow normal
            emacs rules.
  
       Note: Why you should not prefer compressed .elc files
  
        The purpose of compression support is to make it possible to
        have more useful lisp files in an account that has a limited
        disk space (quota). Many Unicersity student accounts have this
        limitation. Keeping lisp files in compressed format
        saves quite a much disk space.
  
        o   Plain text, lisp `.el', files may compress better.
        o   The documentation in comments is important, e.g all the
            instruction to use the file are there. Byte compiling
            strips away documentation.
        o   In order to debug or send bug reports you have to use .el files,
            the .elc files are useless.
        o   The performance ratio that the .elc files offer may not
            be a crucial factor (many times you couldn't tell).
  
       Note: advised emacs commands
  
        The adviced functions can be further adviced, but
        if the redefined function uses `interactive-p' test, it will
        not indicate user call (like M-x load-library). The reason why
        the advised functions detect it, is that advice.el's
        `ad-do-it' macro cannot pass the interactive flag information
        to the original functions.
  
    Trouble shooting
  
        There is no denying it, this package is dangerous. When something
        goes wrong, it really goes wrong and your Emacs may be messed up
        completely. So, here are some trouble shooting tips, that you
        might want to try to rescue the situation or understand what is
        going on. The most usual blame is the *cache* content which does not
        contain the correct or up to date information.
  
       Package is not found or loaded?
  
        Please confirm that the file location is known and is in right
        directory by calling `M-x' `locate-library'. If the result is
        not correct, please check `tinypath-:load-path-root' and
        `tinypath-:extra-path-root'. Try to remedy the situation,
        regenerate cache with `M-x' `tinypath-cache-regenerate'.
  
       You don't know what particular package is causing troubles
  
        Go to the *Message* buffer and clear it (`C-x' `h' followed by
        `C-w'). Run the path generation engine with debug `M-x'
        `tinypath-debug-external-helper' and study the output. It may
        be ignoring some files that you think should be included. Please
        check content of `tinypath-:load-path-ignore-regexp' and
        `tinypath-:load-path-ignore-regexp-extra'.
  
       You need to see the internals
  
        Call function `tinypath-cache-file-find-file' to display the current
        cache and use `C-s' and `C-r' to search entries in the file. Remember
        that you must not modify this file, because any changes you do, will
        get overwritten next time the cache is regenerated. The problem is
        somewhere else if you can see incorrect settings in the cache file.
  
    Code note: General
  
        Because this package is among the first that is loaded from Emacs
        startup file, It contains copies of some functions from TinyLib;
       libraries, to make the package independent until the point where
        the `load-path' has been set up and other libraries are available.
        In the code you may find marks "#copy:" which indicate code that
        has been copied/simplified to be used here. Autoloads statements in
        this package defer loading functions until the end is reached and
        `load-path' is determined and the rest of the functions can be
        loaded from the libraries.
  
    Code note: Where is that emacs package
  
        If you ever need to know the location of a package that Emacs
        would load or has loaded, while this utility is in effect,
        use this call:
  
            (insert (tinypath-cache-p "gnus.el"))
  
        In fact the regular call yields same result, because
        `locate-library' is adviced:
  
            (insert (locate-library "gnus.el"))
  
        More easily, with *tinylisp.el*, which takes advantage of
        tinypath.el cache, you can load any emacs package for editing
        with command:
  
            M-x load-library RET tinylisp RET
            M-x tinylisp-library-find-file
            (tinypath cache)Lisp Library: gnus.el RET
  
        Alternatively there is mode hot-keys $ l f  and $ l p :
  
            M-x load-library RET tinylisp RET
            M-x tinylisp-install
            M-x tinylisp-mode  (in *scratch* buffer, see "E" in modeline)
            $ l f
            (tinypath cache)Lisp Library: gnus.el RET
  
    Code note: Internal optimizations
  
        In the installation section it is instructed that the location of the
        package is pushed into the `load-path' before the package is loaded:
  
            (require 'cl)
            (pushnew "~/elisp/tiny/lisp" load-path :test 'stringp=)
            (load "tinypath.el")
  
        Please follow this instruction. The reason is that program tries to
        use most efficient code to boot everything up and the first thing
        it can do is to check the location where it has been saved. This
        package will use this information to assume that the Perl program
        is available at *~/some/path/bin/emacs-util.pl*. If that fails, the
        Perl program is searched along `exec-path'. This is usually
        desirable, situation because every new installation includes newer
        version of Perl program and the one at `exec-path' may not be up
        to date. The perl code will speed up booting compared to pure Emacs
        Lisp implementation. In addition the Perl code section in this file
        (often referred as "external") has extra features included.
  
    Code note: *Messages*
  
        This package will print loads of messages to Emacs "*Message*" or
        XEmacs " *Message-Log*" buffer. This is a design decisions so that
        execution can be easily traced during Emacs load time. It also help
        reporting errors. The default `tinypath-:verbose' 3 will log the most
        important messages.  Even if you set the level to 0 or nil, still
        some messages are displayed. Have a look at Message buffer if you have
        not much used it before. You may find interesting information to
        debug some of your own mis-configurations, like stale directories
        in `exec-path'.
  
    Code note: Custom
  
        If you have Emacs that does not contain *custom.elc* (Yes, it must be
        in compiled format, be sure to check), you can download Noah
        Friedman's excellent custom emulation package *cust-stub.el* at
        http://www.splode.com/~friedman/software/emacs-lisp/ You have to load
        it from absolute location before loading this packages like this:
  
            (load "~/elisp/noah/cust-stub")
            (load "tinypath")
  
    Code note: Insinuating packages
  
        Some packages can be auto-configured when the perl script reads the
        contents of the directories. Like package *woman.el* which
        needs to know the location of man path directories. For other
        packages there are different "installations". Gnus is one
        interesting example: Every Emacs and XEmacs release comes with
        Gnus version, which is usually outdated and many install Gnus
        privately. The problem is multiple Gnus versions in the load
        paths and the wished situation is that there would be only the
        latest. there is experimental code to find out which of the
        Gnus packages along `load-path' is the latest and after making
        hopefully right decision (according to gnus-version-*
        variable) the other Gnus locations are hidden by modifying
        `load-path' and `tinypath-:load-path-ignore-regexp'. This is a
        complimentary method
        to that suggested in the manual section "3rd party packages".
  
    Code note: Elp profiling results 
  
        The profiling results were run using method below. It must be note,
        that the `tinypath-external-*' is the time when the external perl
        program examines all the directories, so EXT time is not significant
        because it varies from system to system. The
        `tinypath-external-setup-parse-data' is the actual time spent in
        parsing the returned data. The functions that are called most of the
        time are the ones that must be kept on eye on and they seem to
        perform very well. Immediate below are the most important functions
        that perform the Parsing after the perl has returned results (these
        are not from the total listing, but after tweaking). The listing
        below represents timing results somewhere around 2001:
  
            tinypath-external-output-parse                   1    4.89  4.89
              tinypath-external-output-parse-1               5    1.09  0.21
              tinypath-external-output-parse-1-cache         1    3.79  3.79
  
  
            tinypath-external-setup-parse-data               1    5.77  5.77
              tinypath-external-setup-1-load-path            249  0.70  0.002
              tinypath-external-setup-1-man-path             44   0.0   0.0
              tinypath-exec-path-append                      73   0.92  0.012
              tinypath-info-handler                          31   8.46  0.27
              tinypath-external-setup-cache                  1    0.0   0.0
  
        These timing results was taken 2003-05-18 running Cygwin
        XEmacs 21.4.10, Pentium 400 Mhz. These profiling results are
        from the initial boot phase, before cache is loaded. It's
        pretty fast.
  
            (setq tinypath-:install-flag nil)
            (load "elp"
            (load "tinypath")
            (elp-instrument-package "tinypath-")
  
            ;; Now run the boot phase ONLY
            (tinypath-load-path-initial-value
              tinypath-:core-emacs-load-path-list)
  
            Function Name                                   Count Elap   Ave
            =============================================== ===== =====  ===
            tinypath-load-path-initial-value                1     0.477  0.47
            tinypath-load-path-add-subdirs                  1     0.463  0.46
            tinypath-directory-subdirs                      1     0.451  0.45
            tinypath-emacs-root-directory                   1     0.008  0.00
            tinypath-emacs-root-by-load-path                1     0.008  0.00
            tinypath-emacs-core-path-p                      119   0.004  3.36
            tinypath-expand-file-name                       5     0.001  0.00
            tinypath-load-path-initial-value-xemacs         1     0.001  0.00
            tinypath-load-path-string-match                 1     0.001  0.00
            tinypath-win32-p                                5     0.0    0.0
            tinypath-emacs-versions                         1     0.0    0.0
  
  	Theses timing results was taken 2003-05-18 running Cygwin
  	XEmacs 21.4.10, Pentium 400 Mhz. The cache with 4500
  	directories was loaded from configuration file. In this case
  	`tinypath-:cache-file-postfix' value was '.el'. The timing
  	information was tested and generated with:
  
        o   `C-x' `C-f' tinypath.el RET  -- toad read tinypath.el to Emacs
        o   `M-x' `load-library' RET tinylisp.el RET
        o   `M-x' `turn-on-tinylisp-mode' RET
        o   $ -    to eval current buffer
        o   $ e I  to instrument everything (Wtih empty value, scan buffer)
        o   `M-x' `tinypath-cache-regenerate' RET
        o   $ e s  to show results
  
  
            Function Name                                   Count Elap   Ave
            =============================================== ===== =====  ===
            tinypath-install                                1     6.812  6.81
            tinypath-cache-setup                            1     5.76   5.76
            tinypath-setup                                  1     5.76   5.76
            tinypath-directory-list-clean                   7     3.756  0.53
            tinypath-cache-file-load                        1     2.552  2.55
            tinypath-load-path-clean                        1     2.272  2.27
            tinypath-cache-file-need-sync-p                 1     1.932  1.93
            tinypath-load-path-not-in-synch-p               1     1.932  1.93
            tinypath-exec-path-clean                        2     0.679  0.34
            tinypath-exec-path-check-verbose                2     0.597  0.298
            tinypath-exec-path-check                        2     0.594  0.297
            tinypath-load-path-merge                        1     0.364  0.364
            tinypath-Info-default-directory-list-clean      1     0.218  0.218
            tinypath-file-remove-trailing-slash             825   0.121  0.000
            tinypath-cache-mode                             1     0.082  0.082
            turn-on-tinypath-cache-mode                     1     0.082  0.082
            turn-on-tinypath-cache-mode-maybe               1     0.082  0.082
            tinypath-ti::advice-control                     1     0.081  0.081
            tinypath-install-timer                          1     0.007  0.007
            tinypath-cache-file-name                        2     0.006  0.003
            tinypath-exec-path-from-path                    2     0.006  0.003
            tinypath-ti::compat-timer-cancel-function       1     0.005  0.005
            tinypath-ti::compat-timer-elt                   2     0.004  0.002
            tinypath-cache-warn-if-not-exist                1     0.004  0.004
            tinypath-cache-file-old-p                       1     0.004  0.004
            tinypath-days-old                               1     0.002  0.002
            tinypath-cache-status-string                    2     0.002  0.001
            tinypath-cache-status-message                   1     0.002  0.002
            tinypath-advice-instantiate                     1     0.002  0.002
            tinypath-expand-file-name                       3     0.001  0.000
            tinypath-ti::compat-timer-cancel                1     0.001  0.001
            tinypath-use-compression-maybe                  2     0.001  0.0005
            tinypath-exec-path-append                       1     0.001  0.001
            tinypath-win32-p                                6     0.0    0.0
            tinypath-emacs-versions                         2     0.0    0.0
            tinypath-ti::date-time-difference               1     0.0    0.0
            tinypath-eval-after-load                        1     0.0    0.0
            tinypath-time-string                            2     0.0    0.0
            tinypath-file-compressed-p                      2     0.0    0.0
            tinypath-emacs-lisp-file-list-cache-clear       1     0.0    0.0
            tinypath-autoload-file-name                     1     0.0    0.0
            tinypath-autoload-require                       1     0.0    0.0
            tinypath-cache-p                                1     0.0    0.0
            tinypath-cache-file-hostname                    2     0.0    0.0
            tinypath-load-path-root-changed-p               1     0.0    0.0
  
  	Same timing test as above, but now using compiled cache file at
  	Emacs startup. In this case `tinypath-:cache-file-postfix' value was
  	'.elc'. The speedup is 50%, rducing the load time to mere 3-4
  	seconds. Notice the dramatic change in `tinypath-cache-file-load':
  	0.5 seconds vs. 2.5 seconds non-compiled.
  
            Function Name                                   Count Elap   Ave
            =============================================== ===== =====  ===
            tinypath-install                                1     3.305  3.30
            tinypath-cache-setup                            1     2.017  2.01
            tinypath-setup                                  1     2.017  2.01
            tinypath-directory-list-clean                   7     1.608  0.22
            tinypath-load-path-clean                        1     0.904  0.90
            tinypath-advice-instantiate                     1     0.784  0.78
            tinypath-cache-file-load                        1     0.549  0.54
            tinypath-exec-path-check                        2     0.506  0.25
            tinypath-exec-path-check-verbose                2     0.506  0.25
            tinypath-load-path-not-in-synch-p               1     0.368  0.36
            tinypath-cache-file-need-sync-p                 1     0.368  0.36
            tinypath-exec-path-clean                        2     0.326  0.16
            tinypath-exec-path-from-path                    2     0.154  0.07
  
    Todo
  
        o   In theory it is possible to load remote files with ange-ftp/EFS in
            manner of `load-library' RET /user@host:/path/to/file but this
            has never been tested.
        o   It theory it is possible to add /user@host:/path/to/dir/
            to `load-path', but that has never been tested.


