Package Management

Package management is the way to manage software packages on the system. For many people, when package management is mentioned what comes to mind is an application which they can use to download, install, and uninstall other software (software like synaptics, yum, apt-get, pacman, etc).

While this is not wrong, package management actually has a lot bigger scope than that. I will expound some of these issues which I think is the most important.

What is Package Management, really?

Package management is also known by its other trendy (at the time of writing) name, Application Lifecycle Management. It is the aspect of managing software (nee applications) which are installed in a system - from their creations, installations, updates, and de-installation (or removal) when their duties are over.

Its scope covers from:

  1. Creation of deliverable software packages
  2. Making available of the packaged software
  3. Delivery of the packaged software
  4. Installation of the packaged software
  5. Tracking and registration of installed software
  6. Checking the currency of the installed software
  7. Notification of available software updates
  8. Delivery of software updates
  9. Installation of software updates
  10. De-installation (removal) of software
  11. Cleanup and possibly archiving of user-generated data from removed applications.

And less obvious but no less important:

And in especially the enterprise environment:

And that's just to begin with. I may have missed a few others.

Why Package Management is important

Looking at the above aspects, it should be immediately clear why package management is important. It is one of the side-aspects that people often don't appreciate (like the janitorial or secretarial services) - the state of the package management in the system determines, in the long run, the health of the system.

Imagine a system where the package management software ("package manager" for short) cannot always successfully remove that it installed. Over time, cruft and garbage leftover by uninstalled applications will grow boundlessly, consuming precious system resources and slowing the system to the point it becomes unusable. When that happen, the only recourse is to re-install the entire operating system. Sounds familiar?

And like it or not, there is always de-installation process even if you don't uninstall anything - because every application update always involves removal either before or after the the new update is installed. A system with bad package management system will deteriorate over time as more and more updates are applied to it. Sounds familiar?

It is difficult to over-emphasise the need of a good package management.


That being said, there is no perfect package management system. Available package managers (both free and commercial) systems always have deficiencies in one way or another. That is not a reason not to choose a package management system altogether, because obviously some package managers are better than others, even though they aren't perfect.

Package management considerations for FatdogArm

For FatdogArm, the criteria I use will only meet a subset of those functionalities. The reason is simplicity and practicality - unlike many other operating systems; FatdogArm (and Fatdog64 and Puppy Linux in general) can be reset to its original pristine state in less than 1 minute; making mistakes in software installation lest costly than others.

They are grouped into two categories:


End-user consideration

The package manager should at least:

  1. have two versions; one graphical and one for command line use (so failure in starting graphical desktop does not preclude the use of the package manager);

  2. be remote repository aware: that is, it can list available packages from remote repository and can download/install packages from that repository;

  3. be able to show list of installed software (which may be different from the available software from the remote repository);

  4. be able to show "updates" or newer version of the installed software, if it is available on the remote repository;

  5. be able to detect integrity of software downloaded from remote repository;

  6. be able to install software downloaded by other means (not through the package manager itself);

  7. remove installed software.

Developer consideration

This is an oft-looked-over part of package management. Not only a package manager needs to provide ease-of-use to the end user; it must also help the developer or a packager to build, package, and deliver the software to them.

In general this means two things:

  1. A tool to automate the software packaging.
  2. A simple software distribution method.

Specifically, for FatdogArm, I'm looking for

  1. An automated method to build package.
  2. A remote repository which can be published on standard web-server without any need for CGI or other complex requirements (where publishing can be done just by FTP-ing the repository contents).

FatdogArm package manager

After various considerations, I have decided to adopt Slackware's package management software suite ("pkgtools"), complemented with Jason Woodward's excellent slapt-get and gslapt front-ends, as the package manager for FatdogArm.

pkgtools will provide local package management; while the front-ends provides access to remote repositories. (Slackware has its own front-end to access remote repository called slackpkg, but Jason's one is nicer).

pkgtools package format is simple (it's just a tarball), the package logging format is simple; it comes with tools to build packages (makepkg); and the package manager is text-based which means it can be used even when the graphical desktop isn't working.

slapt-get adds remote repository access (=listing of available files on repository servers); gslapt adds a nice graphical GUI on top of that. The repository format is simple (just a bunch of packages plus PACKAGES.TXT) which is publishable on any web server; integrity is maintained by using a centralised MD5 checksum and optionally, GPG signing.


As originally noted in BuildingApplications article, I have used paco as a temporary measure to log and capture the software being built. While paco never claims to be a package manager (the description on its homepage has this to say),

After the installation of a source package with "./configure && make && make install", one is usually left with having no idea of what it was installed and where it all went, making it difficult to uninstall the package in the future.

Paco was written to solve this problem in a quite simple fashion.

When installing a package from sources, paco wraps the "make install" command (or whatever command or group of commands are needed to install the files into the system), and saves installation information into a text database.

paco is actually a working and a worthwhile package manager in its own. Not only it can logs the installation, it can also list all previously installed software (and remove them if asked), as well as create tarballs of previously installed files (called pacoballs) which can be used to re-install the same software later (on the same or different machine). In addition it also has a nice graphical GUI interface (gpaco) for viewing installed packages, list the files, install pacoballs, etc.

So why not paco as the package manager? Indeed, I would have gone with paco were it not for one big gaping hole: paco only handles local files, and it does not have remote repository access; in a way it is very similar to pkgtools without slapt-get/gslapt (and I would argue paco is way better than pkgtools, but I digress). I could have written my own front-end to enable remote-repository access to paco; but it is easier to use existing tools.


Fatdog64 (on which FatdogArm is based) comes with its own original package manager called Fatdog Package Manager (FPM) written by me.

It meets the criteria too; and in some respects is better (e.g. FPM has categories; gslapt doesn't; FPM is layered-filesystem aware and can restore original file from the basesfs layer when a package is uninstalled).

So why not FPM instead of pkgtools/slapt-get/gslapt?

Bootstrapping FatdogArm package management

As paco already has all the package installation information, it is a matter of converting its logs into pkgtools log format. This is quite straightforward as both formats are simple tarballs adorned with text file descriptions. I wrote a script to do this, you can get it here. The script can automatically generate dependency information too.

After the conversion, all packages originally logged with paco will show up as installed packages in pkgtools/slapt-get/gslapt; and they can be removed and re-installed or updated as needed.

As part of the conversion, we can also create pkgtools tarballs - these will be used for build the initial repository. Once the pkgtools tarball it is a simple matter to use Jason's provided script to build repository for slapt-get/gslapt.

Conclusion

This article concludes the series of FatdogArm porting articles. By the time we reached this step, we already have a working operating system which is good enough for distribution to others.

From here onwards, it is just a matter of maintenance and optimising the system for specific purposes, such as size optimisation, making use of specific hardware features, etc.

There are of course large topics of information that I skip; not because they are not important but because they distract from the main purpose of these articles: illustrating the FatdogArm porting process. I may write them in the future as independent articles.

For now, though, this is The End (but don't forget to read the extras).



References: