About the Ravenports Universal Package System

Ravenports is an integrated system designed to build packages of complex software on all UNIX-like platforms. It is considered “universal” because Ravenports is not anchored to any single operating system; once the build procedure for a particular software is created (a.k.a “port”), that software is then available for all supported platforms. The simplest use case is to access the freely available prebuilt package repository for each supported platforms. More complex usage involves customizing builds to create private package repositories for specialized use. The product packages and repositories are manipulated with a package manager.

Notable advantages of Ravenports over other frameworks:

Inherent support of variant packages

Other frameworks may refer to these as “flavors” or “slave ports”. A variant package set is one that is built with a different configuration over the standard set. The use of variant packages eliminates the need for user-configurable options, meaning the consumer is likely to have access to a pre-built binary package and not have to build the software themselves.

Inherently higher quality build procedures

Most other frameworks are based on makefiles or shell scripts. Ravenports are compiled into a single “buildsheet”. Mistakes by developers are caught during the compilation process, so syntax and formatting are always correct. Unlike other frameworks, there’s exactly one way to build Ravenports, meaning that if a new port or modification builds successfully for a contributor, it will build identically for an evaluator. Submission checks are capable of being automated and performed against multiple platforms before a human ever sees them. This leds to much better submissions and eliminates unnecessary multiple review cycles.

Inherent support of subpackages

Other frameworks require separate ports if they want to split documentation, examples, libraries, or anything other components into separate packages to sanely satisfy dependency requirements. This requires redundant rebuilding and risks getting related ports out of sync. Ravenports allows many subpackages to be generated simultaneously by a single port and this capability is heavily used. This allows for fine-grain dependency specification (e.g. just pull a fortran runtime library instead of the entire GCC compiler collection.)

Multiple package formats possible

Currently only one package format is supported, which is the first version defined by pkg(8), the FreeBSD package manager. This package manager has been throughly tested on all platforms supported by Ravenports. Given interest, other package managers could be supported such as the Image Packaging System (IPS), Archlinux’s pacman, Debian DPKG family, etc.

Enterprise friendly

Ravenports was designed from the beginning to handle “custom” ports. These allow, among other things, to freeze software at a specific version. Companies are not forced to use the versions in quarterly sets or the bleeding-edge tree, nor is support tied to the End-of-Life (EOL) of any operating system release. Ports version tailoring can be provided as a service to the company which has the added benefit of receiving security updates on a weekly basis.

Using other frameworks can lead to frustration when help is urgently needed but all the developers contribute on a volunteer basis. Corporate users of Ravenports have the option of obtaining professional service when required.

Entire tree integrity

Other frameworks receive ports updates many times a day, sometimes numbering in the hundreds. Coupled with a much lower quality (a result of a wide range of volunteer developer skills coupled with little or no quality assurance checks and impact assessments), ports trees break frequently and often without addressing downstream breaking. Ravenports stages the changes for several days. A complete build of the ports tree is done to ensure tree integrity. The ports tree is published on weekly after any fallout issues are address to avoid surprises. This also prevents temptation by users to update and rebuild daily, which would result in constant and frustrating build churn. Finally, a weekly publishing schedule allows the ports tree and provided binary packages to remain in sync, making it possible to safely mix ports and packages.

Build performance

This advantage does not benefit system adminstrators that exclusively use prebuilt binary packages, but it is greatly appreciated by those that build their own packages. Ravenports can scan and process the ports tree hundreds to thousands of times faster than similar frameworks. Unlike the others, Ravenports has an administration/build utility that’s fully integrated and not a bolt-on tool. The utility, ravenadm, is written in Ada and capable of high concurrency to take full advantage of powerful build machines. Full tree scans which may take minutes (up to an hour on pkgsrc) are accomplished in less than 1 second by ravenadm. Every aspect of the framework is geared towards build quality and performance, resulting in significantly shorter build times over comparable frameworks.

Concurrently installed versions

Several popular projects maintain several branch releases. For example, the Perl project has 3 concurrent versions. Python, Ruby, most database servers and other projects do as well. Ravenports always supports two versions of perl and provides packages for both which can be installed concurrently. The same case exists for Python except three branches are supported (2.7, and the last two 3.x releases). The PHP branches have unique installations so even multiple versions of PHP can be installed together. Default database server versions can be specified per repository, but currently only a single version can be installed at a time.

Supported Platforms

Ravenports has been designed to support any POSIX-compliant operating system outside of Microsoft Windows. While it self-identifies as “universal”, Ravenports must be bootstrapped to the point of self-hosting on each supported architecture of each supported operating system. For the core developers, the main architectures targeted are the x86_64 and ARM Aarch64. New architecture support from serious contributors capable of performing the initial bootstrap and periodic quality-assurance builds of the entire Ravenports tree will be accepted into the project.

Current

Currently supported platforms

  • DragonFly 4.9 and later
  • FreeBSD/amd64 Release 11 and later
  • Linux/x86-64 (glib 2.6.32-based)

Short Term

Platform support planned

  • Solaris 10+ / Illumos
  • FreeBSD/ARM64
  • Linux/AArch64

Long Term

Support increasingly unlikely

  • Darwin/Mac OSX/x86_64
  • OpenBSD/amd64
  • NetBSD/amd64
  • i386 (Linux and *BSD)

A major requirement on platforms that are candidates for Ravenports support is that GCC 7 must be available on the platform, and must include fully functional C++, Fortran and Ada front-ends. For exotic platform/architecture combinations, this may require cross-compilation bootstrapping along with new support patches to bring Ada tasking capability to the candidate platform.

Quick Start

The section will be re-worked. The main User Guide has not been written. The quickstart guides linked below tell how to install the Ravenports Administration tool (ravenadm) with a host-specific system root and toolchain.

There are complete repositories for each of the supported platforms, so anyone familiar with pkg(8) on FreeBSD or DragonFly BSD will understand how to install the prebuilt binaries immediately. Linux users may require finding FreeBSD-based tutorials to grasp it better. This should be addressed in the upcoming user guides (TBW).

The ravenadm tool does have a complete set of man pages though. ravenadm help will show those pages. Those familiar with the FreeBSD ports >synth port building tool will quickly adapt to ravenadm, as it has the same look and feel, and also has the web-based dynamic build report.

Update Schedule

The source tree for Ravenports is under constant update. However, the Ravenports tree is only generated on a weekly basis. Minor updates to heavily used ports can cause the majority of the tree to require a rebuild, so the policy is to provide all the updates in one step each Tuesday.

Perl ports

With many thousand perl modules with CPAN, updates are seen many times per day. Perl alone causes serious build churn based on these daily updates, and that’s without any kind of integrity checking. Therefore the policy is to update perl modules to the latest versions on a monthly basis. After the update, a full build check is performed internally to ensure the perl ecosystem remains functional. Perl is updated on the second Tuesday of each month.

Python ports

Python modules are similar to perl, but are more volatile. Therefore the same monthly policy is in place, and the same delivery schedule is used: the second Tuesday of each month.

Security and vulnerability updates

For non-commercial users, security updates appear as regular updates every Tuesday. These updates may appear as version updates. Commercial service users that have specified and locked down software versions (older than the current Ravenports tree) will receive security as port revisions accompanied by patches each Tuesday. If patches aren’t feasible, the client will be informed as soon as possible to determine how best to address newly-known security issue.)

Binary package updates

As soon as the Ravenports tree is generated, incremental building of packages will commence on all supported repositories. Depending on the extent of the changes, new packages should be available within a few hours to a couple of days after the generation of the source tree. For commercial service clients, a notice will be sent when new packages are ready for download.

Volunteer Contributions

Every leg of the Ravenports Triad consists of Free and Open-Source Software (FOSS), released under the permissive Internet Systems Consortium (ISC) License. Contributions are welcome and may appear in many forms, such as:

  • bug reports
  • patch submissions
  • port adoption and maintenance
  • Binary package and repository generation
  • Security vulnerability reports
  • Submission of new ports

Bug and Security vulnerability reports

In the future, we may host a dedicated issue tracking system. For the time being, the Github issue tracking system is available for users to report bugs and any security issues that may not have been known or addressed yet.

Report here: https://github.com/jrmarino/ravensource/issues

Submission of patches and new ports

In the future, submission of patches and new ports will be done through a dedicated interface that will put the submission in a build queue and test build the affected ports on all supported platforms automatically. If the build tests pass automated tested, the submissions will be forward for human inspection and inclusion. However, we aren’t there yet.

All patches must be build tested locally. If they pass the submitter’s own testing, the best approach is to submit the contribution as a github pull request here: https://github.com/jrmarino/ravensource/pulls

The process is the same when submitted an entirely new port, with one additional requirement. The person that creates and submits the port agrees to be listed as the port’s maintainer for at least 6 months. It is expected that submitter fully tests the new port on at least one platform.

The pull request description for a new port should provide the follow information:

  • Synopsis of what functionality the new port provides (what the software does)
  • Impact of not approving the port for inclusion
  • Optionally: what type of user is likely to benefit from its inclusion
  • Which platform(s) the port was tested on

Port adoption

Each port may have one or more maintainers. Most ports have no listed maintainer or are generated by a script. There are benefits to having a specific person or team of people responsible for maintenance and update of a port. Ideally the maintainer should be a regular user of the port’s software. This helps porting errors to be detected quickly and also allows for quick confirmation of reported bugs. A real user is more motivated to fix bugs and keep up with the latest software upgrades.

Those volunteering to be the primary maintainer of a port are expected to be familiar with the Ravenporter's Guide and have the technical skills necessary for basic port maintenance and troubleshooting. They are also expected to handle bug reports to their natural conclusion. If multiple maintainers are listed on a port, they are expected to communicate internally (each speak for the others on bug trackers, etc.).

Requests to be adopt ports can be made either through the Github bug tracker or submitted as changes via a github pull request. The latter method is preferred.

Binary package and repository generation

This contribution forms when somebody wishes to extend Ravenports support to lesser-used platforms or architectures. The new platform support requires the contributor to have the necessary hardware to accomplish the system root and toolchain bootstrapping. Additionally the contributor is expected to build a complete set of binary packages and upload them at regular intervals. That will likely require per-platform adjustments to existing and incoming ports, so this is not a trivial contribution. Serious contributions are more than welcome, but there will be some vetting to ascertain the skill level and the commitment level of any person or team offering to extend Ravenports for non-commercial support use.

Latest News

2017-09-01 Website online. User guides still being developed. User feedback welcome.

Professional Services

Software Set Maintenance (source)

This service allows clients to specify the exact version or range of versions of software on a per-port basis they require. When the port is upgraded on the public Ravenports tree to a version newer than the client specifies, the client remains with the older specified version. Security sites are scanned daily for vulnerability announcements and patches for all ports used by the client. The client is sent a weekly report on all discovered vulnerabilities since the previous report. The client’s private Ravenports tree is updated every Tuesday. The client’s IT department uses the Ravenadm tool and the updated tree to build its own software packages.

Port creation service

If required software is not yet available in the Ravenports tree, it may be possible to pay to bring it in. The cost will vary depending on porting difficulty, and if it has been successfully ported to other package systems such as FreeBSD ports, pkgsrc, archlinux packages, etc. Keeping the new port limited to the client’s private ports tree (in other words, not placing the new port in the public Ravenports tree for others to enjoy) is possible but may affect the final cost. It’s recognized that legal requirements may require privacy, but companies are encouraged to contribute their new ports whenever possible.

Software Set Maintenance (source + binary packages)

This service maintains software to the client’s specifications as above, but also rebuilds the package repository. The client is sent a notification when the new packages and the changes that occurred over the previous week. This report is provided concurrently with the vulnerability briefing. The client’s IT department has the option to sync a local repository to the updated remote repository, or just directly update its machines from the remote repository.

Provision of new platform/architecture

A significant amount of time and work is required to bootstrap Ravenports on a new platform and/or architecture. Should a client find it a worthy investment, this support can be provided. The effort is a one-time cost; any required maintenance is considered standard work and not charged to a client.

Contact for Professional Services

Contact John Marino
email ravenports@ironwolf.systems
telephone +1 (479) 306-8850