Something I frequently do on whichever Linux distribution I find myself on (including Arch Linux, Fedora, Manjaro Linux, openSUSE and Sabayon Linux), is I build packages. It is infinitely rare, in fact so rare that I have not ever come across such a case, for a distribution to have every exact package (including package version) I want. Package development, for me, is done via one of four major methods:

  • Building the package using the standard tools on my own instance of the respective Linux distribution. For example, on Arch Linux this involves writing a PKGBUILD and running makepkg -s in the directory of said PKGBUILD. This method is the most risky of methods, as if there is something wrong with the build file (e.g., with the .ebuild or .spec file extension or PKGBUILD/rules file name), it is possible for this method to lead to system breakage. It is also the easiest and fastest method for building packages for the present OS.
  • Building the package in a chroot, using the standard tools. chroots should (but there are no guarantees) protect one’s system from any errors in the build file.
  • Building the package using Docker, using the standard tools. It is at least as safe as building in a chroot and it is more platform-independent as you can build packages for any of a variety of different distributions, on the one system.
  • Building the package using the Open Build Service (OBS). The OBS can be accessed from the command-line using the osc (OSC(1) man page) command and from a web-based interface. The package to which the osc command belongs, along with its various subcommands, can be challenging to install on non-openSUSE platforms. On Arch Linux this package is available from the Arch User Repository (AUR), so to install it with Yaourt run: user $  yaourt -S osc. Using the OBS to build packages also allows one to more easily distribute the packages one builds, as one can set up an OBS repository using this method. This repository can have packages for a variety of different distributions including Arch Linux, Debian, Fedora, Mageia, openSUSE and Ubuntu. There is one major caveat, however, which is that one cannot build packages that require an Internet connection during their build, using this method. This precludes me from creating a package for the Atom text editor using this method, except via using pre-built binaries which are only available for 64-bit platforms.

Fedora users also have another method available to them for building packages, Copr. Copr is similar to the OBS in that it builds packages and creates package repositories in which to store them, but unlike the OBS Copr can optionally provide packages with Internet access during the build and can only be used to build Fedora packages.

Terminology

In this post I will use several terms that are not explained in the glossary, this section will (hopefully) explain them so that everyone knows what I mean when I use each of them. If this section, or any other section of this post, is confusing please contact me and tell me. I am afraid I am not perfect, or a mind-reader, so I need you to tell me if something is wrong or if I omitted some important information.

Package Building

Building a package will be used to refer to the process of compiling, or otherwise making a package’s source code executable, usually so that some file inside (usually a shell script or file written in object code) can be executed in order to start the program the package is for.

Package Installation

The act of installing a software package involves moving the installed files of a package into a file system. Most of the time, when someone says that they are installing a package, what they are doing is they are installing the package’s file to the live file system (that is, the file system of the PC that is being used).

Package Preparation

An important step in package development is the preparation of the sources being used to build the package.

Source Compilation

When one is creating a software package, quite frequently one will need to compile the package’s source code, which means convert the source code written in some programming language into something that is machine readable and executable. This conversion is performed, using so called “compilers” like the GNU Compiler Collection (GCC) which creates object code from the source code.

Package Formats

In order to effectively build packages one must understand the basics of the package format one intends to build. There are four major types of Linux package format that I worked with:

  • Arch Linux packages (ALPs, file extension: .pkg.tar.xz), the package format used by Arch Linux, its derivatives and select “independent” distributions such as Frugalware Linux and KaOS. They are built based on the contents of PKGBUILDs, which are Bash scripts with build instructions for the package along with its associated metadata.
  • Debian packages (or Deb packages, file extension: .deb), the package format used by Debian and its derivatives such as Ubuntu and its derivative, Linux Mint. They are built based on the contents of a whole directory and its subdirectories. The build instructions are found within the rules file.
  • Gentoo packages (file extension: .tbz2), the package format used by Gentoo Linux and its derivatives like Sabayon Linux. tbz2 files are built based on the contents of a specialized Bash script called an ebuild (with the .ebuild file extension). ebuilds are stored within a set of directories and subdirectories (called overlays), usually managed by Git (git) or some other version control system (VCS) like Mercurial (hg) or Subversion (svn). They are essentially like more complicated (and hence more difficult to write) equivalents to PKGBUILDs and like PKGBUILDs they include package metadata and build instructions.
  • RPM packages (file extensions: .rpm, .src.rpm), a package format used by select distributions such as CentOS, Fedora, Mageia and openSUSE. They are built based on the contents of a whole directory, entitled rpmbuild, and its subdirectories. The most important file in the rpmbuild directory and its subdirectories is called a spec file, which has the .spec file extension. This spec file contains package metadata and build instructions, similarly to ebuilds and PKGBUILDs.

Arch Linux Packages

Arch Linux packages are xz-compressed tar archives that are built and installed using commands provided by the pacman package on Arch Linux. ALPs are the package format used by Arch Linux derivatives (like Manjaro Linux) along with the “independent” distributions, Frugalware Linux and KaOS, which also use the pacman package manager, so this information should be applicable to packaging on these distributions too.

ALP Contents

ALPs have the following contents:

$INSTALLED_FILES
.BUILDINFO
.INSTALL
.MTREE
.PKGINFO

where $INSTALLED_FILES are, of course, the installed files of the package with its respective file structure. For example, for the broadcom-wl package the $INSTALLED_FILES have the directory structure:

usr/
 - lib/
   - modprobe.d/
     - broadcom-wl.conf
   - modules/
     - extramodules-4.4-ARCH/
       - wl.ko.gz
 - share/
   - licenses/
     - broadcom-wl/
       - LICENSE

The package metadata (which is used by pacman when it installs new packages to check for file conflicts and such) is stored in the four hidden files (that is, those with . in their filename) in the package’s top-level directory.

PKGBUILD Structure

ALPs are built from PKGBUILDs using the makepkg command that comes bundled with the pacman package manager. They are the easiest packages to build, in my opinion. PKGBUILDs have the following general format (for more details see the PKGBUILD(5) man page):

# ~ Maintainer/Contributor name and email ~
pkgname=      # The package's name.
pkgver=       # The upstream package version, e.g., 1.5.0 for Atom 1.5.0.
pkgrel=       # The PKGBUILD revision number.
pkgdesc=      # The PKGBUILD's description.
arch=         # The architecture(s) on which the package is to be built.
url=          # The website of the package.
license=      # The legal license of the package.
depends=      # Runtime dependencies.
makedepends=  # Build dependencies.
optdepends=   # Optional dependencies.
provides=     # What the package provides.
conflicts=    # The package conflicts.
source=       # The source files required; also includes patches.
sha256sums=   # SHA256 sums of the source files.
md5sums=      # MD5 sums of the source files. Usually used INSTEAD of sha256sums.
install=      # Install files.

prepare() {   # Prepare the sources. Most commonly you will find sed functions
}             # and patches being applied here.

build() {     # Perform any compiling of the source code that may be necessary.
}             # You may also see configure scripts being run here.

package() {   # This will actually build the package. If more than one package is
}             # built from the one PKGBUILD then more than one package() function is provided.

the sha256sums can be replaced with sha512sums and sometimes GPG signatures are used also. For example, the Linux kernel PKGBUILD, in the core pacman repository, uses GPG and sha256sums to check package integrity and validity. The variable definition lines (that is, the pkgname line through to install line) provide mostly the package’s metadata and security checks (as well as variables that can be used in the following functions), while the prepare(), build() and package() functions are responsible for the actual building of the package. The install line defines the .install file that contains pre-, peri- and post-install checks and functions that need to be executed for the package. Here is an example PKGBUILD I have used to build gVim 7.4.1525:

# Maintainer: Brenton Horne <brentonhorne77 at gmail dot com>
# Contributor: Peter Mattern <pmattern at arcor dot de>

_pkgname=vim
pkgname="gvim"
pkgver=7.4.1525
pkgrel=1
pkgdesc="Vim the editor. CLI version and GTK2 GUI providing majority of features."
arch=("i686" "x86_64")
url="http://www.vim.org"
license=("custom:vim")
depends=("gtk2" "hicolor-icon-theme" "gtk-update-icon-cache" "desktop-file-utils")
optdepends=("lua: Lua interpreter" "perl: Perl interpreter" "python: Python 3 interpreter"
            "python2: Python 2 interpreter" "ruby: Ruby interpreter")
makedepends=("lua" "python" "python2" "ruby")
provides=("gvim" "xxd" "vim-runtime")
conflicts=("vim-minimal-git" "vim-git"
           "vim-minimal" "vim" "vim-python3" "gvim" "gvim-python3")
source=("https://github.com/vim/vim/archive/v$pkgver.tar.gz"
        "gvim.desktop")
sha256sums=('SKIP'
            'c346da4725b2db6f7b58c5b72bdf9e7efbba2a3275e97c17db48689e4de674ca')
install=gvim.install

prepare() {

    # set global configuration files to /etc/[g]vimrc
    sed -i 's|^.*\(#define SYS_.*VIMRC_FILE.*"\) .*$|\1|' ${srcdir}/${_pkgname}-${pkgver}/src/feature.h

}

build() {

    cd "${srcdir}/${_pkgname}-${pkgver}"
    ./configure \
      --enable-fail-if-missing \
      --with-compiledby='Arch Linux AUR' \
      --prefix=/usr \
      --enable-gui=gtk2 \
      --with-features=huge \
      --enable-cscope \
      --enable-multibyte \
      --enable-perlinterp=dynamic \
      --enable-pythoninterp=dynamic \
      --enable-python3interp=dynamic \
      --enable-rubyinterp=dynamic \
      --enable-luainterp=dynamic
    make

}

package() {

    # actual installation
    cd "${srcdir}/${_pkgname}-${pkgver}"
    make DESTDIR=$pkgdir install

    # desktop entry file and corresponding icon
    install -D -m644 ../gvim.desktop      $pkgdir/usr/share/applications/gvim.desktop
    install -D -m644 runtime/vim48x48.png $pkgdir/usr/share/icons/hicolor/48x48/apps/gvim.png

    # remove ex/view and man pages (normally provided by package 'vi' on Arch Linux)
    cd $pkgdir/usr/bin ; rm ex view
    find $pkgdir/usr/share/man -type d -name 'man1' 2>/dev/null | \
      while read _mandir; do
        cd ${_mandir}
        rm -f ex.1 view.1
      done

    # add license
    install -D -m644 ${srcdir}/${_pkgname}-${pkgver}/runtime/doc/uganda.txt \
      $pkgdir/usr/share/licenses/$pkgname/LICENSE
}

prepare() is used to prepare the source, which means if the source is compressed (like a gz-compressed tar archive) the prepare() function will exact its contents so that they are available for the build() and package() functions. build() is used to build, or compile, the source, that is if this needs to be done (for example, some PKGBUILDs actually build ALPs from Debian or RPM packages, so no source code compiling is required). package() is what builds a package from either the compiled source (that is, the source after the build() function is run) or the prepared pre-compiled sources (that is, the contents of Debian/RPM binaries).

The package() function is essentially where the objective of the game is to move all the files you wish to be in the end package from the products (whether it be compiled source code, or unpacked Debian package contents) of the build() function into the $pkgdir directory. The $pkgdir directory is meant to have the same internal file system structure as where the package will place its installed files, if installed on one’s file system. For example, GTK themes are usually installed to /usr/share/themes so this is an example package() function for such cases (this one is specifically taken from the osx-el-capitan-theme PKGBUILD):

package() {
  mkdir -p "$pkgdir/usr/share/themes/"
  cp -a "$srcdir/${_pkgname}-${pkgver}/OS X El Capitan" "$pkgdir/usr/share/themes/"
}

see the package’s contents are moved to ${pkgdir}/usr/share/themes/OS X El Capitan.

Building ALPs

To build an ALP you run:

user $  makepkg

from within the same directory, as the PKGBUILD you intend to build is located. You may not have the package’s build dependencies pre-installed so this command may return an error stating that you have missing build dependencies. To fix this (assuming all the dependencies are in the presently-enabled pacman repositories) by installing all required build dependencies prior to the build, run:

user $  makepkg -s

Debian Packages

The Debian package format (file extension: .deb) was one of the first Linux package formats developed. It was first developed by Ian Murdock and other members of the Debian development team. The package manager that was originally developed to work with Debian packages (installing, uninstalling, upgrading, etc. these packages) was called dpkg (invoked by the dpkg command), while APT, aptitude and Synaptic are front-ends that perform repository management, dependency resolution, etc. and then use dpkg to perform the actual installation of Debian packages. Debian packages are built based on the contents of several different files in a directory (with its own set file structure, including subdirectories and alike) entitled debian. Debian packages are ar archives, that is archives generated with the ar Unix utility. They are built using the debuild or dpkg-buildpackage commands that are provided by the devscripts package, which is separate from the package that provides the dpkg command.

Package Contents

As previously mentioned, Debian packages are ar archives and they have the following three files inside them:

debian-binary
control.tar.xx
data.tar.xx

where .xx denotes the compression file extension of the containing files. Most Debian packages use gz-compression for its control and data tar archives, so in this case .xx is replaced with .gz. Some Debian packages have xz-compressed control and data tar archives inside them. The debian-binary file is a plain text file containing the standard number of the Debian package (e.g., the latest is 3.0). The control.tar.xx archive contains the package’s metadata, while the data.tar.xx archive contains the package’s installed files.

Build Directory Structure

The debian directory used to build Debian packages, has the structure:

debian/
  - changelog
  - compat
  - control
  - copyright
  - rules
  - source/
    - format

The changelog and copyright files have pretty self-explanatory contents, so I will not bother describing their contents. The compat file has the number nine (9) in it, because allegedly it is a “magic number”. The control file contains the package metadata, like its description, name, version, dependencies, etc. The rules file contains the package build instructions. The format file contains the standard of the Debian package being described, for example, most packages at the moment will be using the 3.0 (quilt) standard.

Gentoo/Sabayon Packages

Gentoo packages (file extension: .tbz2) are bz2-compressed binary packages used by Gentoo Linux and its derivatives. They are produced and installed using the Portage package manager. Sabayon Linux’s Entropy package manager uses a slightly different package format (same file extension though, .tbz2), generated from the corresponding Gentoo packages using Entropy. Most Gentoo users will not install their software from tbz2 files, as Portage is a source code package manager (which is usually the reason why people use Gentoo in the first place, because they want to install packages from source code using Portage) and as a result most packages are built from source code and not installed from binary packages. The way that Portage installs software from source code is by following the instructions found in a specialized Bash script called an ebuild. Portage can be used to install tbz2 binary packages, however, and it can be configured to work with (that is, install, remove, upgrade, etc. packages in said repositories) binary package repositories. This is just an uncommon Portage configuration.

Package Contents

Running:

user $  qtbz2 $package.tbz2

where $package.tbz2 is a tbz2 binary, extracts an xpak file (file extension: .xpak; which contains the package metadata) and .tar.bz2 archive containing the installed files of the package.

ebuild Structure

Syntactically, I would say that ebuilds are most similar to PKGBUILDs, but there are several key differences. For one, they include eclasses, specialized Bash functions designed specifically for ebuilds, many of which are poorly documented, in my opinion. Secondly, PKGBUILDs are all named PKGBUILD, while ebuilds only share the same file extension .ebuild. Their name consists of the package’s name and its version, e.g., gVim 7.4.1342 would have an ebuild named gvim-7.4.1342.ebuild. ebuilds also come with manifests (files entitled Manifest) that include checksums for all the source files and the ebuilds themselves. Here is an ebuild for gVim that you can compare to the previously-provided PKGBUILD and spec file for gVim, it is over 400 lines long so I am not going to include it in this post. To build a Gentoo binary package from an ebuild run:

user $  ebuild $package.ebuild package

while to build a Sabayon binary package, one has to run one additional command:

root #  equo pkg inflate $package.tbz2

RPM Packages

RPM Packages (file extension: .rpm, source RPMs have the .src.rpm file extension) are the package format used by Red Hat Linux (RHL), its derivatives (such as CentOS, Fedora, Korora, Oracle Linux, Red Hat Enterprise Linux, Scientific Linux), openSUSE, SUSE Linux Enterprise, etc. They are built using the rpmbuild command provided by the rpmdevtools package on most distributions. From what I understand RPMs are a type of file archive (which can be extracted using the bsdtar or rpm2cpio commands). They are not ar archives, however. RPM is a binary package format, although a source code version also exists, which is called a SRPM. SRPMs can also be extracted using bsdtar.

RPM Contents

Decompressing RPM packages using user $  bsdtar -xf $package.rpm extracts just the package’s installed files. This might make it seem like RPM packages have no metadata, but they do, it is just not readily apparent by decompressing them using bsdtar. To show a summary of the metadata inside these packages you need to run user $  rpm -qip $package.rpm.

rpmbuild

rpmbuild needs to be run within a directory called rpmbuild within the current user’s home directory, with its own set of subdirectories, this is its general structure:

rpmbuild/
  - BUILD
  - BUILDROOT
  - RPMS
  - SOURCES
  - SPECS
  - SRPMS

The BUILD and BUILDROOT subdirectories are used for compiling the source code and collecting the necessary installed files for packaging, respectively. The SOURCES subdirectory contains the source files, including any patches, and SPECS contains the all-important spec files, which instruct the rpmbuild utility how to build the package and what metadata the RPM should contain. The RPM is stored in the RPMS subdirectory and the SRPM is stored in the SRPMS subdirectory.

spec files look sort of like PKGBUILDs, except they use macros instead of many of the variables and functions found in PKGBUILDs. I would provide an example here in this post of Vim’s spec file (the one I use to build Vim in the Open Build Service) but it is over 520 lines long (as opposed to 72 lines for the gVim PKGBUILD shown earlier). So to view it see here. I personally find writing spec files significantly more complicated than writing PKGBUILDs, as PKGBUILDs are written more like as if you were writing a shell script to install the software package locally on your machine. The use of macros can make things more complicated to follow for package development newcomers.

Build Methods

Chroot

Chroot (invoked by the chroot command) is a Unix command that is used to change the apparent root for a given process and its children (that is, processes started by this “given process”). This means, or so is the hope, that the file system outside this new root should be unaffected by any commands run in this new root. This “new root” is also known as the new apparent root, or guest file system; the file system outside this new root will be referred to as the host file system. This means that if one is building a package and the build commands being used1 are flawed and potentially system-damaging, they will not have access to the host file system and rather will only be able to cause damage to the guest file system.

The new root, used by the chroot command, is also frequently referred to by the term chroot, which can lead to some confusion. When one uses the chroot command to build packages one usually sets up a directory (which is used as the apparent root for the chroot command), with in it, the file system structure of the operating system one is intending to build packages for. Chroot is a basic form of virtualization, which is something I will leave you to read up about in this site’s glossary. I have fairly limited experience building packages in chroots and this is mostly limited to building in Gentoo / Sabayon chroots, so this section will mostly relate to building packages in Gentoo / Sabayon chroots.

For Gentoo / Sabayon chroots, I usually created a new directory /root2 wherein I would create a file system that resembles a minimal (with only the bare essential programs pre-installed) installation of Gentoo / Sabayon. I did this by extracting a stage3 tarball for the respective OS, into the /root2 directory. Then I bound up the necessary parts of my host system (like /dev, /proc, /sys and /tmp), copied my /etc/resolv.conf file, to their respective parts in the /root2 directory. This I did, by running:

root #  mount -t proc none /root2/proc
root #  mount -o bind /dev /root2/dev
root #  mkdir /root2/usr/portage
root #  mount -o bind /usr/portage /root2/usr/portage
root #  mkdir /root2/usr/src/linux
root #  mount -o bind /usr/src/linux /root2/usr/src/linux
root #  mkdir /root2/lib/modules
root #  mount -o bind /lib/modules /root2/lib/modules
root #  mount -o bind /sys /root2/sys
root #  cp /etc/resolv.conf /root2/etc/resolv.conf
root #  mount -o bind /tmp /root2/tmp
root #  mount --rbind /dev /root2/dev
root #  mount --rbind /sys /root2/sys

In order to use chroot to build packages one must have at least a basic understanding of the chroot command’s syntax. GNU’s chroot command has the following basic syntax (taken from the CHROOT(1) man page):

root #  chroot [OPTION] NEWROOT [COMMAND [ARG]...]

Now, the way you should interpret this basic syntax, is that what appears in square-brackets ([...]), are optional arguments or inputs. What is not in square brackets is a mandatory input (in this case only NEWROOT is a mandatory argument). For me, I have never had to concern myself with the [OPTION] part of this basic syntax, so I will ignore it in this post. NEWROOT, as you can probably guess, is the new apparent root being used by the process started by the chroot command, which in the aforementioned examples for Gentoo / Sabayon chroots would be /root2. [COMMAND] is the “given process” mentioned earlier in this section and is the first command that is to be run in the new root (any additional commands that are run in the new root have to be started by this command, as a child process) and [ARG]... refers to any arguments (like options) for this first command that the user wishes to specify. The most common choice of [COMMAND] is the default Unix shell of the host operating system, which for most Linux distributions is /bin/bash. So in the case of a Gentoo / Sabayon chroot I would normally use the command:

root #  chroot /root2 /bin/bash

to enter the chroot. Note no [ARG]... is specified in this line, I seldom ever specify an [ARG]... for [COMMAND].

Once you have entered the chroot, you can build packages the same way you would normally. For example, in a Gentoo / Sabayon chroot one would likely build tbz2 binaries by running ebuild $package.ebuild package.

Docker

Docker is an open-source virtualization program, that is written in Google’s Go programming language and that enables the automated deployment of software containers (which usually contain miniature operating systems) that can be used for a variety of different purposes. One such purpose is the building of software packages. Building packages using Docker is similar to building packages in a chroot, as it involves using the standard tools for building the package in question (e.g., makepkg for ALPs and ebuild for Gentoo / Sabayon packages), but it is done in a safer environment (in this case, a software container). This environment (which is called a container in the case of Docker) is designed so that any programs running inside it (including package-building commands) cannot access or modify anything outside the environment, including components of the host’s file system.

Alternatively, a more tedious and hard drive space-exhausting method, is using VirtualBox VMs for the distribution being packed for, instead of Docker containers. It is not covered in this post because it is not a method I would recommend using.

Setup

Docker can only be run on 64-bit Linux systems and is available from the official software repositories of many Linux distributions, including:

  • Arch Linux
  • Fedora
  • Gentoo Linux
  • Manjaro Linux
  • openSUSE / SUSE Linux Enterprise
  • Sabayon Linux

Docker can be installed on CentOS, Debian, Red Hat Enterprise Linux and Ubuntu, from unofficial repositories via methods outlined at the links provided. On most distributions after installing Docker you will need to add your user to the docker group, log out and log back in to this user account and start the Docker daemon using systemd. To add your user to the docker group run:

root #  gpasswd -a $USER docker

where $USER is the name of your user. And to start the Docker daemon with systemd one would run:

root #  systemctl start docker

While to get the Docker daemon to automatically start on startup one would run:

root #  systemctl enable docker

Pulling

To find pre-existing containers that you can use, search the Docker Hub.

To build Arch Linux packages you are probably better off pulling the base/archlinux container, with:

user $  docker pull base/archlinux

To build Debian packages you will probably be better off using the Debian container. To pull it (which makes it available for use on your current PC) run:

user $  docker pull debian:latest

To build Gentoo packages you can use either a Gentoo or Sabayon container, out of these I would personally recommend the sabayon/base-amd64 container. This is because Sabayon containers have the Entropy package manager available, which can be helpful in helping one more quickly install required build dependencies for any ebuilds you wish to build. This makes installing the build dependencies faster than if one was using a Gentoo container, as Gentoo containers have only the Portage package manager available for installing the required build dependencies which is quite often a tedious process. To pull the sabayon/base-amd64 container run:

user $  docker pull sabayon/base-amd64:latest

To build a RPM package one would probably be best using the latest official Fedora container, to pull it run:

user $  docker pull fedora:latest

alternatively, you can use the opensuse container. If you cannot find a suitable container for the distribution you wish to package for, you can write your own Dockerfile and build a container based on it. I have never written one so I am afraid I cannot give you any pointers on how to do this, besides directing you to the official documentation on writing Dockerfiles.

Running

Once you have chosen a Docker container to build your packages in and have pulled it to make it available for local use, then the next command you will need to run is docker run. It has the following general syntax (for further details see its official documentation):

user $  docker run [OPTION] IMAGE[:TAG|@DIGEST] [COMMAND] [ARG...]

as with the chroot command, square brackets ([...]) are used to denote optional arguments, with all remaining arguments being mandatory. The IMAGE argument as you can see is mandatory and refers to the Docker container’s image ID. The [COMMAND] argument, as with the chroot command, refers to an optional command you wish to start your container off with. Any extra commands or programs you wish to run in this container, will have to be started by this first command. This is why most people will choose the /bin/bash (or some other Unix shell) command as this argument, as additional commands can be easily started off it.

For example, to start the sabayon/base-amd64 container I have on my local machine, to build packages in it, I usually run:

user $  docker run -i -t 8bdbc44b6570 /bin/bash

Copying Files to Host

Once you have built a package with Docker you will likely want to copy it across to your host machine, using the docker cp command. Here is its basic syntax (for further details see its official documentation)

user $  docker cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH

CONTAINER is the container’s ID (which is distinct from its image ID, by the way!), SRC_PATH is the file’s path on the Docker container, while DEST_PATH is where on the host machine one wishes to copy the file to. To determine the container ID, I suggest you run:

user $  docker ps

which will show you all Docker containers presently running. To show every container, including those that are not running, run:

user $  docker ps -a

Committing Changes

If you have made some changes to a Docker container, like upgraded its software, installed some packages you need to build new packages, etc. you will probably not want to loose them. You will loose them, however, if you exit the machine normally (which you do by running exit inside the container, potentially repeatedly, until you are no longer in it). To save your changes you need to commit them (its official documentation is here), by running:

user $  docker commit [OPTIONS] CONTAINER [REPOSITORY[:TAG]]

I usually never have to concern myself with [OPTIONS] myself. CONTAINER is the container ID.

Open Build Service

The Open Build Service (OBS) is the only method outlined here that not only builds packages, but also sets up a repository with which you can distribute your packages. It has a command-line client invoked by the osc command, which is written in Python, and can be run locally to build packages. The problem with this command is that it is not available from the official repositories of most distributions that the OBS can build packages for. osc has several additional dependencies (like obs-build) that are not necessarily available from the official repositories of most distributions, too, which adds to the difficulty of obtaining it on these distributions. For details on installing the OBS on non-openSUSE platforms see the Installing section of this post. It can only be used to build packages for the following distributions:2

  • Arch Linux
  • CentOS / Red Hat Enterprise Linux / Scientific Linux
  • Debian
  • Fedora
  • Mandriva Linux
  • openSUSE / SUSE Linux Enterprise
  • Ubuntu

Installing

On openSUSE, installing osc and its dependencies is simple, merely run:

root #  zypper in osc

and it is installed! One way, that one can install osc and all its dependencies on non-openSUSE distributions is by manually compiling them all from source code. Now I am no expert, in fact, I have not tried installing these packages, manually from source code, on any distribution other than Arch Linux, so if there are gaps or flat out errors in this brief tutorial, please contact me or fork the fusion809.github.io GitHub repo, modify the _posts/2016/03/PKG/04-build-methods/04-obs.md file in your fork and start a pull request. Firstly, ensure you have the following packages installed on your system:3

  • bash
  • diffutils
  • libxml2
  • make
  • perl
  • perl-uri
  • python-keyring
  • python2
  • python2-m2crypto
  • rpm (if you are using a RPM-based distribution like CentOS, Fedora, Mageia, Red Hat Enterprise Linux, Scientific Linux, etc. this program should be pre-installed)
  • urlgrabber
  • wget
  • xz

The following is the procedure I recommend you follow to install each OBS-specific package:

  • Install obs-service-source_validator. To do this run:
    user $  git clone https://github.com/openSUSE/obs-service-source_validator
    user $  cd obs-service-source_validator
    user $  sudo make install
  • Install obs-service-format_spec_file. To do this run:
    user $  git clone https://github.com/openSUSE/obs-service-format_spec_file
    user $  cd obs-service-format_spec_file
    user $  sudo make install
  • Install obs-build. To do this run:
    user $  git clone https://github.com/openSUSE/obs-build
    user $  cd obs-build
    user $  sudo make install
  • Install obs-service-download_files. To do this run:
    user $  git clone https://github.com/openSUSE/obs-service-download_files
    user $  cd obs-service-download_files
    user $  sudo make install
  • Install osc. To do this run:
    user $  git clone https://github.com/openSUSE/osc
    user $  cd osc
    user $  sudo make install

These are the bare minimum you need in order to run the osc command, but you may wish to install a few extra packages, in order for osc to have its full capabilities. To see all available officially-supported OBS-related packages see this GitHub search. Now, I have not installed each of these packages of myself, so all I can say is that based on the contents of their respective GitHub repositories (most importantly, that they contain a Makefile) I think, you should be able to install them via following the same procedure used to install the packages above. That is, git clone their GitHub repository, cd into your local copy of their repository, and run user $  sudo make install.

Creating Packages

NOTE
  • An excellent Open Build Service tutorial can be found here at the openSUSE Wiki. While this tutorial is hosted by the openSUSE Wiki, it should work regardless of your platform. I recommend you read it as this post section is not designed to replace it! It merely contains a few, hopefully helpful, observations I have made about using the OBS.

Creating packages in the OBS, modifying their characteristics, etc. can be the done from either the command-line or from within a web browser (URL: https://build.opensuse.org). I would recommend you have access to both methods, as they both compliment each other, although you can use just one method by itself. Both require an Internet connection; the command-line means of accessing the OBS also requires you have osc installed, while the browser method merely requires that you have a functioning web browser available to you. I personally use both methods of accessing the OBS, with my preference for doing most OBS things being using osc. The OBS uses its own version control system (VCS), that is most similar to Subversion.

An OBS source repository (from which the OBS attempts to build your packages) should contain the following files:

  • Any necessary patch files. Many packages, especially smaller packages, will have no patches though, so this component is not always needed.

  • Any package format-specific files (PFSFs) required to build the package (e.g., for an RPM package this would be a spec file, while for a Arch Linux package this would be a PKGBUILD).

  • The package’s upstream source (or whatever you intend to build the package in question from, like a Debian or RPM binary will also do). The user can add the upstream source manually, the same way one adds patch files and other files to the OBS repository. This can be repetitive and annoying when upstream package updates come out frequently, so my preference is to add a _service file to the OBS repository. Most commonly, this is all you need in your _service file. This will then automatically download the upstream source(s) listed in the PFSFs mentioned earlier, when required. Like PKGBUILDs and spec files often include the upstream source’s download URL and this is where the _service file will attempt to download the source from. You need to be careful with this _service method, as sometimes the download URL will not be in an acceptable format for the _service file to download the upstream source from it. For example, for a PKGBUILD this source variable will not work:

    source=($pkgname-$pkgver.tar.gz::https://github.com/xylosper/bomi/archive/v$pkgver.tar.gz)
    

    while, this:

    source=(https://github.com/xylosper/bomi/archive/v$pkgver.tar.gz)
    

    will work.

Something that is important to note, however, is that a single OBS source repository can be used to build packages for several different Linux distributions, even distributions that have their own very different PFSFs. For example, my Vim OBS repository builds packages for Arch Linux, Fedora and openSUSE. Even though Arch Linux uses PKGBUILDs for its PFSFs and Fedora / openSUSE use spec files.

Something that is particularly important to note for users that want to build Arch Linux packages, is that any dependencies your packages have that are in the Arch Linux community repository, or AUR, will need to be added to your OBS Project too. See the OBS does not have the packages in the Arch [community]/[multilib] repositories, or AUR, available to it (for use satisfying build dependencies) only packages in the Arch [core] and [extra] repositories. I have requested that the [community] repository be added to the OBS, here at openFATE.

If the Build Service Tutorial linked previously is not sufficient to help you start building OBS packages then I recommend reading the openSUSE Wiki articles linked to at the aforementioned Build Service Tutorial article. It has most of the more helpful information for packagers, although there is also a free OBS manual available and here is a PDF copy. If you have specific queries related to the OBS, you may wish to ask a question on this board at the openSUSE Forums.

Footnotes

  1. Usually contained in an ebuild, PKGBUILD, rules or spec file, depending on the package being built. 

  2. Keep in mind, however, that packages built for one distribution, should be compatible and able to be installed on most, if not all, distributions derived from said distribution. For example, Arch Linux packages should be compatible with the Arch Linux derivative, Manjaro Linux. 

  3. If you are wondering about which components of each package you need, I suggest you look up the corresponding Arch Linux package and find equivalent packages for your distribution. See I am using the dependencies for the osc package in the AUR and its OBS-specific dependencies (like obs-build) to base this list on. 

Comments