diff --git a/.gitattributes b/.gitattributes deleted file mode 100644 index 4065553..0000000 --- a/.gitattributes +++ /dev/null @@ -1,5 +0,0 @@ -*.sh text eol=lf -*.php text eol=lf -*.xml text eol=lf -*.volt text eol=lf -Makefile text eol=lf diff --git a/.gitignore b/.gitignore deleted file mode 100644 index c49bf4e..0000000 --- a/.gitignore +++ /dev/null @@ -1,10 +0,0 @@ -/dist/ -/.build/ -/.cache/ -/app/ -/opnsense-plugin/www/rustproxy/work/ -/opnsense-plugin/www/rustproxy/src/sbin/rust-proxy -*.log -*.key -.env -builder.env diff --git a/README.md b/README.md index 0e3a939..260b0bb 100644 --- a/README.md +++ b/README.md @@ -1,66 +1,82 @@ # RustProxy OPNsense packaging repository -This repository contains only the OPNsense plugin and build/release automation. The RustProxy application source is **not vendored here**. The Debian scheduler downloads it dynamically from: +Docker-only build pipeline for RustProxy on Debian. No FreeBSD VM is required. + +RustProxy itself is not stored here. Every scheduled run fetches it independently from: `https://git.linuxiarz.pl/gru/RustProxy` -## What is included - -- `opnsense-plugin/www/rustproxy/` - OPNsense plugin source. -- `builder/debian/` - hourly scheduler and optional cross-build helper. -- `builder/freebsd/` - native FreeBSD release builder and repository signing helper. -- `ci/build-release.sh` - Debian-to-FreeBSD orchestration and publishing. -- `deploy/opnsense/` - custom package repository install/update helpers. -- `docs/` - setup and architecture notes. - -## Source/update flow +## Flow ```text -Packaging repository RustProxy upstream -(plugin + scripts) https://git.linuxiarz.pl/gru/RustProxy - | | - +------------------+-------------------+ - v - Debian cron, hourly - | - detects either commit change - | - v - FreeBSD 15.1 amd64 builder - | - cargo test + cargo build - | - OPNsense make package - | - v - signed pkg repository - | - v - OPNsense +rustproxy-opnsense-repo RustProxy upstream + | | + +--------------+---------------+ + v + Debian cron hourly + | + v + Docker cross-build image + Rust 1.88 + clang/lld + FreeBSD 15.1 sysroot + | + v + x86_64-unknown-freebsd binary + | + v + Linux-side .pkg assembly + | + v + dist/os-rustproxy-latest.pkg + | + v + pkg add -f on OPNsense ``` -By default `APP_REPO_REF=HEAD`, so every cron run follows the current default branch of the RustProxy repository. You can instead pin a branch/tag/ref in `/etc/rustproxy-opnsense-builder.env`. +The Docker container does not emulate FreeBSD. It runs on the Linux kernel and uses the FreeBSD 15.1 headers/libraries only as a cross-compilation sysroot. -A release rebuild is triggered when either the packaging repository commit or the RustProxy upstream commit changes. The package version is read at build time from the dynamically downloaded `Cargo.toml`. +## Repository contents -## Recommended release build +- `opnsense-plugin/www/rustproxy/` - OPNsense plugin source. +- `builder/docker/Dockerfile` - Rust/Clang FreeBSD 15.1 cross toolchain. +- `builder/debian/hourly-build.sh` - fetch/change detection scheduler. +- `builder/debian/build-docker-release.sh` - Docker build orchestration. +- `builder/debian/package-freebsd.sh` - creates a FreeBSD `.pkg` archive on Linux. +- `ci/build-release.sh` - build/publish/test-deploy entrypoint. +- `deploy/opnsense/install-direct.sh` - direct package installer for OPNsense. +- `docs/FIRST_RUN.md` - exact first-run commands. -Use the Debian VM only as scheduler/orchestrator and a FreeBSD 15.1 amd64 VM as compiler/packager. The included Docker/cross path (`BUILD_MODE=cross-binary`) is for development validation only and does not publish production `.pkg` files. +## Build trigger -## Quick validation +A build starts when either: + +- this packaging repository changes, or +- the fetched RustProxy commit changes. + +`APP_REPO_REF=HEAD` follows the current default branch. A tag or branch can be pinned in `/etc/rustproxy-opnsense-builder.env`. + +## Output + +After a successful run: + +```text +dist/ +├── BUILD-INFO.txt +├── SHA256 +├── os-rustproxy-latest.pkg +├── docker-build/rust-proxy +└── packages/26.7/amd64/os-rustproxy-_.pkg +``` + +This version intentionally produces an installable `.pkg`, not a full `pkg repo` catalogue. Installation/update is currently done with `pkg add -f`. + +## OPNsense UI + +The first version exposes only **Services -> RustProxy -> Enabled**. RustProxy runs as the unprivileged `rustproxy` user. + +## Validation ```sh ./scripts/check.sh ``` -## Quick setup - -Read `docs/FIRST_RUN.md`, copy `builder.env.example` to `/etc/rustproxy-opnsense-builder.env` on Debian and fill in your packaging-repository, SSH and publication settings. `APP_REPO_URL` is already set to your RustProxy repository. - -## OPNsense UI scope - -The plugin currently exposes only Enabled / Disabled. RustProxy is supervised by FreeBSD `daemon(8)`, runs as user `rustproxy`, and uses `/var/db/rustproxy` as its private data directory. - -## License - -RustProxy's `Cargo.toml` currently does not declare a license and the supplied source snapshot did not contain LICENSE/COPYING, so the plugin manifest remains `UNKNOWN`. Set the correct license before public redistribution. +See `docs/FIRST_RUN.md` for Debian setup. diff --git a/builder.env.example b/builder.env.example index f77274e..b2946fd 100644 --- a/builder.env.example +++ b/builder.env.example @@ -1,39 +1,35 @@ -# This repository: OPNsense plugin + packaging/build scripts. -# Push this generated repository to your own Git server and set its URL here. -PACKAGING_REPO_URL=git@github.com:YOUR_ORG/rustproxy-opnsense.git -PACKAGING_REPO_REF=main -PACKAGING_WORKDIR=/srv/rustproxy-opnsense/packaging +# Packaging/build repository. +PACKAGING_REPO_URL=https://git.linuxiarz.pl/gru/rustproxy-opnsense-repo +# HEAD follows the repository default branch. You may pin main/master/tag instead. +PACKAGING_REPO_REF=HEAD +PACKAGING_WORKDIR=/opt/opn-rustproxy/packaging -# RustProxy application source. This is fetched independently on every cron run. +# RustProxy application source. Fetched independently on every hourly run. APP_REPO_URL=https://git.linuxiarz.pl/gru/RustProxy -# HEAD follows the default branch of the upstream repository. You can pin a branch, -# tag or other fetchable ref instead, for example: main, stable, v0.21.0. APP_REPO_REF=HEAD -APP_WORKDIR=/srv/rustproxy-opnsense/source/RustProxy +APP_WORKDIR=/opt/opn-rustproxy/source/RustProxy -# Local Debian state. +# Local Debian state and Docker cache. STATE_DIR=/var/lib/rustproxy-opnsense-builder +DOCKER_CACHE_DIR=/opt/opn-rustproxy/cache -# Release builder. Use a FreeBSD 15.1 amd64 host for production packages. -BUILD_MODE=native-freebsd -FREEBSD_HOST=freebsd-builder.example.net -FREEBSD_USER=builder -FREEBSD_SSH_KEY=/root/.ssh/rustproxy_freebsd_builder -FREEBSD_REMOTE_DIR=/usr/local/rustproxy-build/project - -# OPNsense branch/ABI targeted by the plugin package. -OPNSENSE_PLUGINS_REF=stable/26.7 +# Docker-only FreeBSD cross build. +BUILD_MODE=docker-freebsd +FREEBSD_RELEASE=15.1-RELEASE +FREEBSD_ABI=FreeBSD:15:amd64 +FREEBSD_ARCH=freebsd:15:x86:64 OPNSENSE_ABI=26.7 +DOCKER_IMAGE=rustproxy-freebsd15-builder:15.1-rust1.88 -# Paths on the FreeBSD builder. Leave empty for an unsigned development repo. -PKG_REPO_SIGN_KEY=/usr/local/rustproxy-build/keys/rustproxy-repo.key -PKG_REPO_PUBLIC_KEY=/usr/local/rustproxy-build/keys/rustproxy-repo.pub +# Run native Linux tests before the FreeBSD cross build (1=yes, 0=no). +RUN_LINUX_TESTS=1 -# Publish dist/ over SSH/rsync. This should be the document root for RustProxy packages. -PUBLISH_TARGET=repo-user@packages.example.net:/srv/www/rustproxy/ -PUBLISH_SSH_KEY=/root/.ssh/rustproxy_package_publisher +# Optional rsync publication target for dist/. Example: +# PUBLISH_TARGET=repo-user@packages.example.net:/srv/www/rustproxy/ +PUBLISH_TARGET= +PUBLISH_SSH_KEY= -# Optional automatic upgrade of explicitly listed TEST firewalls after a successful publish. -# Disabled by default. Space-separated SSH targets, for example: root@fw-test-01 root@fw-test-02 +# Optional automatic installation on explicitly listed TEST OPNsense hosts. +# Space-separated SSH targets, e.g. root@fw-test-01 root@fw-test-02. AUTO_DEPLOY_HOSTS= -AUTO_DEPLOY_SSH_KEY=/root/.ssh/rustproxy_opnsense_test +AUTO_DEPLOY_SSH_KEY= diff --git a/builder/debian/build-docker-release.sh b/builder/debian/build-docker-release.sh new file mode 100755 index 0000000..f0dcf66 --- /dev/null +++ b/builder/debian/build-docker-release.sh @@ -0,0 +1,62 @@ +#!/bin/bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +SOURCE_DIR="${RUSTPROXY_SOURCE_DIR:?RUSTPROXY_SOURCE_DIR is required}" +FREEBSD_RELEASE="${FREEBSD_RELEASE:-15.1-RELEASE}" +DOCKER_IMAGE="${DOCKER_IMAGE:-rustproxy-freebsd15-builder:15.1-rust1.88}" +DOCKER_CACHE_DIR="${DOCKER_CACHE_DIR:-/opt/opn-rustproxy/cache}" +RUN_LINUX_TESTS="${RUN_LINUX_TESTS:-1}" + +command -v docker >/dev/null 2>&1 || { echo "Docker is required" >&2; exit 1; } +[[ -f "$SOURCE_DIR/Cargo.toml" ]] || { echo "Invalid RustProxy source: $SOURCE_DIR" >&2; exit 1; } + +mkdir -p "$DOCKER_CACHE_DIR/cargo" "$DOCKER_CACHE_DIR/target" "$ROOT/dist/docker-build" +uid="$(id -u)" +gid="$(id -g)" + +# Rebuild is cheap after the first run because the FreeBSD sysroot layer is cached. +docker build \ + --build-arg "FREEBSD_RELEASE=$FREEBSD_RELEASE" \ + -t "$DOCKER_IMAGE" \ + -f "$ROOT/builder/docker/Dockerfile" \ + "$ROOT/builder/docker" + +run_args=( + --rm + --user "$uid:$gid" + -e CARGO_HOME=/cargo + -e CARGO_TARGET_DIR=/target + -v "$SOURCE_DIR:/src:ro" + -v "$DOCKER_CACHE_DIR/cargo:/cargo" + -v "$DOCKER_CACHE_DIR/target:/target" + -v "$ROOT/dist/docker-build:/out" + -w /src +) + +if [[ "$RUN_LINUX_TESTS" == "1" ]]; then + echo "Running RustProxy tests in Linux builder container..." + docker run "${run_args[@]}" "$DOCKER_IMAGE" \ + cargo test --locked --release +fi + +echo "Cross-compiling RustProxy for x86_64-unknown-freebsd..." +docker run "${run_args[@]}" "$DOCKER_IMAGE" bash -lc ' + set -euo pipefail + cargo build --locked --release --target x86_64-unknown-freebsd + install -m 0555 /target/x86_64-unknown-freebsd/release/rust-proxy /out/rust-proxy + desc="$(file -b /out/rust-proxy)" + echo "$desc" + header="$(llvm-readelf -h /out/rust-proxy)" + printf "%s\n" "$header" | grep -E "Class:|Machine:|OS/ABI:" || true + printf "%s\n" "$header" | grep -qi FreeBSD || { echo "Built ELF is not marked for FreeBSD" >&2; exit 1; } + dynamic="$(llvm-readelf -d /out/rust-proxy || true)" + printf "%s\n" "$dynamic" | grep NEEDED || true + if printf "%s\n" "$dynamic" | grep -q "libc\.so\.6"; then + echo "Linux libc detected in FreeBSD target binary" >&2 + exit 1 + fi +' + +RUSTPROXY_BINARY="$ROOT/dist/docker-build/rust-proxy" \ + "$ROOT/builder/debian/package-freebsd.sh" diff --git a/builder/debian/build-freebsd-cross.sh b/builder/debian/build-freebsd-cross.sh deleted file mode 100755 index 9ee9226..0000000 --- a/builder/debian/build-freebsd-cross.sh +++ /dev/null @@ -1,27 +0,0 @@ -#!/bin/bash -set -euo pipefail - -ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" -SOURCE_DIR="${RUSTPROXY_SOURCE_DIR:-$ROOT/app}" -TARGET="x86_64-unknown-freebsd" - -if [[ ! -f "$SOURCE_DIR/Cargo.toml" ]]; then - echo "RustProxy source not found. Set RUSTPROXY_SOURCE_DIR to the dynamically fetched checkout." >&2 - exit 1 -fi -if ! command -v docker >/dev/null 2>&1; then - echo "Docker is required for cross mode." >&2 - exit 1 -fi -if ! command -v cross >/dev/null 2>&1; then - echo "cross is not installed. Install it with: cargo install cross --locked" >&2 - exit 1 -fi - -cd "$SOURCE_DIR" -cross build --locked --release --target "$TARGET" -mkdir -p "$ROOT/dist/cross/$TARGET" -cp "target/$TARGET/release/rust-proxy" "$ROOT/dist/cross/$TARGET/rust-proxy" - -echo "Cross-built binary: $ROOT/dist/cross/$TARGET/rust-proxy" -echo "Do not publish this as the production OPNsense package without validation on the target FreeBSD ABI." diff --git a/builder/debian/hourly-build.sh b/builder/debian/hourly-build.sh index a12a52b..5962615 100755 --- a/builder/debian/hourly-build.sh +++ b/builder/debian/hourly-build.sh @@ -10,11 +10,11 @@ fi source "$CONFIG_FILE" : "${PACKAGING_REPO_URL:?PACKAGING_REPO_URL is required}" -: "${PACKAGING_REPO_REF:=main}" -: "${PACKAGING_WORKDIR:=/srv/rustproxy-opnsense/packaging}" +: "${PACKAGING_REPO_REF:=HEAD}" +: "${PACKAGING_WORKDIR:=/opt/opn-rustproxy/packaging}" : "${APP_REPO_URL:=https://git.linuxiarz.pl/gru/RustProxy}" : "${APP_REPO_REF:=HEAD}" -: "${APP_WORKDIR:=/srv/rustproxy-opnsense/source/RustProxy}" +: "${APP_WORKDIR:=/opt/opn-rustproxy/source/RustProxy}" : "${STATE_DIR:=/var/lib/rustproxy-opnsense-builder}" mkdir -p "$STATE_DIR" "$(dirname "$PACKAGING_WORKDIR")" "$(dirname "$APP_WORKDIR")" diff --git a/builder/debian/install-cross.sh b/builder/debian/install-cross.sh deleted file mode 100755 index 1e5ae3e..0000000 --- a/builder/debian/install-cross.sh +++ /dev/null @@ -1,22 +0,0 @@ -#!/bin/bash -set -euo pipefail - -if [[ ${EUID} -ne 0 ]]; then - echo "Run as root" >&2 - exit 1 -fi - -apt-get update -DEBIAN_FRONTEND=noninteractive apt-get install -y docker.io rustup build-essential pkg-config -systemctl enable --now docker - -TARGET_USER="${SUDO_USER:-root}" -if [[ "$TARGET_USER" == "root" ]]; then - rustup default stable - cargo install cross --locked -else - sudo -u "$TARGET_USER" -H sh -c 'rustup default stable && cargo install cross --locked' -fi - -echo "Docker + rustup + cross installed for $TARGET_USER." -echo "Use BUILD_MODE=cross-binary only for development validation." diff --git a/builder/debian/install.sh b/builder/debian/install.sh index 179ecbb..88cc1c0 100755 --- a/builder/debian/install.sh +++ b/builder/debian/install.sh @@ -9,21 +9,30 @@ fi REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" apt-get update -DEBIAN_FRONTEND=noninteractive apt-get install -y \ - ca-certificates cron git openssh-client rsync util-linux curl jq +DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ + ca-certificates cron git openssh-client rsync util-linux curl jq \ + docker.io zstd file python3 -install -d -m 0755 /srv/rustproxy-opnsense +install -d -m 0755 /opt/opn-rustproxy +install -d -m 0755 /opt/opn-rustproxy/source +install -d -m 0755 /opt/opn-rustproxy/cache/cargo install -d -m 0755 /var/lib/rustproxy-opnsense-builder install -m 0755 "$REPO_ROOT/builder/debian/hourly-build.sh" /usr/local/sbin/rustproxy-opnsense-hourly install -m 0644 "$REPO_ROOT/builder/debian/rustproxy-opnsense.cron" /etc/cron.d/rustproxy-opnsense if [[ ! -e /etc/rustproxy-opnsense-builder.env ]]; then install -m 0600 "$REPO_ROOT/builder.env.example" /etc/rustproxy-opnsense-builder.env - echo "Created /etc/rustproxy-opnsense-builder.env. Edit it before the first cron run." + echo "Created /etc/rustproxy-opnsense-builder.env" +else + echo "Keeping existing /etc/rustproxy-opnsense-builder.env" + echo "Compare it with: $REPO_ROOT/builder.env.example" fi touch /var/log/rustproxy-opnsense-builder.log chmod 0640 /var/log/rustproxy-opnsense-builder.log +systemctl enable --now docker systemctl enable --now cron -echo "Debian scheduler installed." +echo "Debian Docker builder installed." +echo "Edit /etc/rustproxy-opnsense-builder.env and run:" +echo " /usr/local/sbin/rustproxy-opnsense-hourly" diff --git a/builder/debian/package-freebsd.sh b/builder/debian/package-freebsd.sh new file mode 100755 index 0000000..312e748 --- /dev/null +++ b/builder/debian/package-freebsd.sh @@ -0,0 +1,150 @@ +#!/bin/bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +SOURCE_DIR="${RUSTPROXY_SOURCE_DIR:?RUSTPROXY_SOURCE_DIR is required}" +BINARY="${RUSTPROXY_BINARY:?RUSTPROXY_BINARY is required}" +OPNSENSE_ABI="${OPNSENSE_ABI:-26.7}" +FREEBSD_ABI="${FREEBSD_ABI:-FreeBSD:15:amd64}" +FREEBSD_ARCH="${FREEBSD_ARCH:-freebsd:15:x86:64}" +SOURCE_REVISION="${RUSTPROXY_SOURCE_REVISION:-0}" +SOURCE_COMMIT="${RUSTPROXY_SOURCE_COMMIT:-unknown}" +SOURCE_URL="${RUSTPROXY_SOURCE_URL:-https://git.linuxiarz.pl/gru/RustProxy}" +SOURCE_REF="${RUSTPROXY_SOURCE_REF:-HEAD}" + +for tool in python3 sha256sum tar zstd file; do + command -v "$tool" >/dev/null 2>&1 || { echo "Missing tool: $tool" >&2; exit 1; } +done + +[[ -f "$SOURCE_DIR/Cargo.toml" ]] || { echo "Missing Cargo.toml in $SOURCE_DIR" >&2; exit 1; } +[[ -f "$BINARY" ]] || { echo "Missing cross-built binary: $BINARY" >&2; exit 1; } + +binary_desc="$(file -b "$BINARY")" +echo "$BINARY: $binary_desc" +if [[ "${ALLOW_NON_FREEBSD_BINARY:-0}" != "1" ]] && [[ "$binary_desc" != *FreeBSD* ]]; then + echo "Refusing to package a binary that is not identified as FreeBSD." >&2 + exit 1 +fi + +app_version="$(awk -F '"' '/^version = "/ {print $2; exit}' "$SOURCE_DIR/Cargo.toml")" +[[ -n "$app_version" ]] || { echo "Cannot read app version" >&2; exit 1; } + +if [[ "$SOURCE_REVISION" == "0" ]]; then + pkg_version="$app_version" +else + pkg_version="${app_version}_${SOURCE_REVISION}" +fi + +work="$ROOT/dist/.package-work" +stage="$work/stage" +meta="$work/meta" +repo_dir="$ROOT/dist/packages/$OPNSENSE_ABI/amd64" +package_file="$repo_dir/os-rustproxy-${pkg_version}.pkg" + +rm -rf "$work" "$ROOT/dist/packages" +mkdir -p "$stage/usr/local" "$meta" "$repo_dir" + +# OPNsense plugin files map directly below /usr/local. +cp -a "$ROOT/opnsense-plugin/www/rustproxy/src/." "$stage/usr/local/" +mkdir -p "$stage/usr/local/sbin" +install -m 0555 "$BINARY" "$stage/usr/local/sbin/rust-proxy" + +# Ensure executable hooks/scripts keep executable mode in the package. +chmod 0555 "$stage/usr/local/etc/rc.d/rustproxy" +chmod 0555 "$stage/usr/local/etc/rc.syshook.d/10-rustproxy.start" +chmod 0555 "$stage/usr/local/opnsense/scripts/rustproxy/service.sh" + +POST_INSTALL_FILE="$ROOT/opnsense-plugin/www/rustproxy/+POST_INSTALL.pre" +PRE_DEINSTALL_FILE="$ROOT/opnsense-plugin/www/rustproxy/+PRE_DEINSTALL.pre" + +python3 - "$stage" "$meta" "$pkg_version" "$FREEBSD_ABI" "$FREEBSD_ARCH" \ + "$POST_INSTALL_FILE" "$PRE_DEINSTALL_FILE" <<'PY' +import hashlib +import json +import os +from pathlib import Path +import sys + +stage = Path(sys.argv[1]) +meta_dir = Path(sys.argv[2]) +version = sys.argv[3] +abi = sys.argv[4] +arch = sys.argv[5] +post_install = Path(sys.argv[6]).read_text(encoding="utf-8") +pre_deinstall = Path(sys.argv[7]).read_text(encoding="utf-8") + +files = {} +flatsize = 0 +for p in sorted(stage.rglob("*")): + if p.is_symlink(): + rel = "/" + p.relative_to(stage).as_posix() + files[rel] = "-" + elif p.is_file(): + rel = "/" + p.relative_to(stage).as_posix() + files[rel] = hashlib.sha256(p.read_bytes()).hexdigest() + flatsize += p.stat().st_size + +common = { + "name": "os-rustproxy", + "version": version, + "origin": "opnsense/os-rustproxy", + "comment": "RustProxy reverse proxy service", + "desc": "RustProxy packaged as an OPNsense plugin. The initial UI exposes Enabled/Disabled only.", + "maintainer": "local@localhost", + "www": "https://git.linuxiarz.pl/gru/RustProxy", + "prefix": "/usr/local", + "abi": abi, + "arch": arch, + "flatsize": flatsize, + "licenselogic": "single", + "licenses": ["UNKNOWN"], + "categories": ["www"], + "deps": {}, + "options": {}, +} +manifest = dict(common) +manifest["files"] = files +manifest["scripts"] = { + "post-install": post_install, + "pre-deinstall": pre_deinstall, +} + +(meta_dir / "+COMPACT_MANIFEST").write_text( + json.dumps(common, separators=(",", ":"), ensure_ascii=False) + "\n", + encoding="utf-8", +) +(meta_dir / "+MANIFEST").write_text( + json.dumps(manifest, separators=(",", ":"), ensure_ascii=False) + "\n", + encoding="utf-8", +) +PY + +archive="$work/os-rustproxy.tar" +rm -f "$archive" "$package_file" +tar --format=ustar --owner=0 --group=0 --numeric-owner \ + -cf "$archive" -C "$meta" +COMPACT_MANIFEST +MANIFEST +tar --format=ustar --owner=0 --group=0 --numeric-owner \ + -rf "$archive" -C "$stage" usr +zstd -q -f -19 -T0 "$archive" -o "$package_file" + +# Convenience stable path for direct deployment. +cp -f "$package_file" "$ROOT/dist/os-rustproxy-latest.pkg" +(cd "$repo_dir" && sha256sum "$(basename "$package_file")") > "$ROOT/dist/SHA256" + +cat > "$ROOT/dist/BUILD-INFO.txt" </dev/null +rm -rf "$work" +echo "Package: $package_file" +echo "Latest: $ROOT/dist/os-rustproxy-latest.pkg" diff --git a/builder/docker/Dockerfile b/builder/docker/Dockerfile new file mode 100644 index 0000000..703bc17 --- /dev/null +++ b/builder/docker/Dockerfile @@ -0,0 +1,41 @@ +# syntax=docker/dockerfile:1 + +# FreeBSD 15.1 sysroot used only for headers and target libraries. +FROM ubuntu:24.04 AS sysroot +ARG FREEBSD_RELEASE=15.1-RELEASE +ARG FREEBSD_MIRROR=https://download.freebsd.org/releases/amd64/amd64 +RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ + ca-certificates wget xz-utils \ + && rm -rf /var/lib/apt/lists/* +WORKDIR /tmp/freebsd +RUN set -eux; \ + url="${FREEBSD_MIRROR}/${FREEBSD_RELEASE}"; \ + wget -q "${url}/base.txz" -O base.txz; \ + wget -q "${url}/MANIFEST" -O MANIFEST; \ + expected="$(awk '$1 == "base.txz" { print $2; exit }' MANIFEST)"; \ + test -n "$expected"; \ + echo "$expected base.txz" | sha256sum -c -; \ + mkdir -p /freebsd-sysroot; \ + tar -xJf base.txz -C /freebsd-sysroot ./lib ./usr/include ./usr/lib + +# RustProxy requires rust-version = 1.88. +FROM rust:1.88-slim-bookworm AS builder +RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ + ca-certificates clang lld llvm file git perl pkg-config python3 \ + && rm -rf /var/lib/apt/lists/* + +COPY --from=sysroot /freebsd-sysroot /opt/freebsd-sysroot +RUN rustup target add x86_64-unknown-freebsd + +ENV CARGO_TARGET_X86_64_UNKNOWN_FREEBSD_LINKER=clang \ + CC_x86_64_unknown_freebsd=clang \ + CXX_x86_64_unknown_freebsd=clang++ \ + AR_x86_64_unknown_freebsd=llvm-ar \ + RANLIB_x86_64_unknown_freebsd=llvm-ranlib \ + RUST_PROXY_PYTHON=python3 \ + CFLAGS_x86_64_unknown_freebsd="--target=x86_64-unknown-freebsd15.1 --sysroot=/opt/freebsd-sysroot" \ + CXXFLAGS_x86_64_unknown_freebsd="--target=x86_64-unknown-freebsd15.1 --sysroot=/opt/freebsd-sysroot" \ + CARGO_TARGET_X86_64_UNKNOWN_FREEBSD_RUSTFLAGS="-C link-arg=--target=x86_64-unknown-freebsd15.1 -C link-arg=--sysroot=/opt/freebsd-sysroot -C link-arg=-fuse-ld=lld" + +WORKDIR /src +CMD ["bash"] diff --git a/builder/freebsd/bootstrap.sh b/builder/freebsd/bootstrap.sh deleted file mode 100755 index 772c55c..0000000 --- a/builder/freebsd/bootstrap.sh +++ /dev/null @@ -1,35 +0,0 @@ -#!/bin/sh -set -eu - -if [ "$(id -u)" -ne 0 ]; then - echo "Run as root" >&2 - exit 1 -fi - -BUILDER_USER="${1:-builder}" -BUILDER_GROUP="$BUILDER_USER" - -ASSUME_ALWAYS_YES=yes pkg bootstrap -f -pkg update -pkg install -y git rust python3 pkgconf cmake gmake perl5 rsync - -if ! pw groupshow "$BUILDER_GROUP" >/dev/null 2>&1; then - pw groupadd "$BUILDER_GROUP" -fi -if ! pw usershow "$BUILDER_USER" >/dev/null 2>&1; then - pw useradd "$BUILDER_USER" -g "$BUILDER_GROUP" -m -s /bin/sh -c "RustProxy package builder" -fi - -install -d -o "$BUILDER_USER" -g "$BUILDER_GROUP" -m 0755 /usr/local/rustproxy-build -install -d -o "$BUILDER_USER" -g "$BUILDER_GROUP" -m 0700 /usr/local/rustproxy-build/keys - -echo "FreeBSD builder dependencies installed." -echo "Builder account: $BUILDER_USER" -echo "Add its SSH public key to /home/$BUILDER_USER/.ssh/authorized_keys." -echo "Then log in as $BUILDER_USER and create the repository signing key." -rust_version="$(rustc --version | awk '{print $2}')" -if [ "$(pkg version -t "$rust_version" 1.88)" = "<" ]; then - echo "RustProxy requires Rust >= 1.88, found $rust_version" >&2 - exit 1 -fi -echo "Rust version OK: $rust_version" diff --git a/builder/freebsd/build-release.sh b/builder/freebsd/build-release.sh deleted file mode 100755 index 18e3eb7..0000000 --- a/builder/freebsd/build-release.sh +++ /dev/null @@ -1,108 +0,0 @@ -#!/bin/sh -set -eu - -ROOT="${1:-$(pwd)}" -APP_DIR="$ROOT/app" -PLUGIN_SOURCE="$ROOT/opnsense-plugin/www/rustproxy" -DIST_DIR="$ROOT/dist" -CACHE_DIR="${RUSTPROXY_BUILD_CACHE:-$HOME/.cache/rustproxy-opnsense}" -PLUGINS_DIR="$CACHE_DIR/opnsense-plugins" -OPNSENSE_PLUGINS_REF="${OPNSENSE_PLUGINS_REF:-stable/26.7}" -OPNSENSE_ABI="${OPNSENSE_ABI:-26.7}" -SOURCE_REVISION="${SOURCE_REVISION:-0}" -SOURCE_COMMIT="${RUSTPROXY_SOURCE_COMMIT:-unknown}" -SOURCE_URL="${RUSTPROXY_SOURCE_URL:-https://git.linuxiarz.pl/gru/RustProxy}" -SOURCE_REF="${RUSTPROXY_SOURCE_REF:-HEAD}" -RUN_TESTS="${RUN_TESTS:-1}" - -if [ "$(uname -s)" != "FreeBSD" ]; then - echo "This release builder must run on FreeBSD." >&2 - exit 1 -fi -case "$(uname -m)" in - amd64|x86_64) ;; - *) echo "Only amd64/x86_64 is supported by this pipeline." >&2; exit 1 ;; -esac - -for tool in cargo git make pkg rsync; do - command -v "$tool" >/dev/null 2>&1 || { echo "Missing tool: $tool" >&2; exit 1; } -done - -rust_version="$(rustc --version | awk '{print $2}')" -echo "Using rustc $rust_version on $(freebsd-version -ku | tr '\n' ' ')" - -app_version="$(awk -F '"' '/^version = "/ {print $2; exit}' "$APP_DIR/Cargo.toml")" -if [ -z "$app_version" ]; then - echo "Cannot read app version from Cargo.toml" >&2 - exit 1 -fi - -mkdir -p "$CACHE_DIR" -if [ ! -d "$PLUGINS_DIR/.git" ]; then - git clone https://github.com/opnsense/plugins.git "$PLUGINS_DIR" -fi -git -C "$PLUGINS_DIR" fetch --prune origin "$OPNSENSE_PLUGINS_REF" -git -C "$PLUGINS_DIR" checkout -f -B rustproxy-build FETCH_HEAD -git -C "$PLUGINS_DIR" clean -ffd - -cd "$APP_DIR" -if [ "$RUN_TESTS" = "1" ]; then - cargo test --locked --release -fi -cargo build --locked --release - -plugin_dir="$PLUGINS_DIR/www/rustproxy" -rm -rf "$plugin_dir" -cp -R "$PLUGIN_SOURCE" "$plugin_dir" -mkdir -p "$plugin_dir/src/sbin" -install -m 0555 "$APP_DIR/target/release/rust-proxy" "$plugin_dir/src/sbin/rust-proxy" - -cd "$plugin_dir" -make PLUGIN_VERSION="$app_version" PLUGIN_REVISION="$SOURCE_REVISION" PLUGIN_ABI="$OPNSENSE_ABI" package - -package_file="$(find "$plugin_dir/work/pkg" -maxdepth 1 -type f -name 'os-rustproxy-*.pkg' | sort | tail -n 1)" -if [ -z "$package_file" ] || [ ! -f "$package_file" ]; then - echo "Package was not produced." >&2 - exit 1 -fi - -repo_dir="$DIST_DIR/repository/$OPNSENSE_ABI/amd64" -rm -rf "$DIST_DIR" -mkdir -p "$repo_dir" -cp "$package_file" "$repo_dir/" - -if [ -n "${PKG_REPO_SIGN_KEY:-}" ]; then - if [ ! -r "$PKG_REPO_SIGN_KEY" ]; then - echo "Repository signing key is not readable: $PKG_REPO_SIGN_KEY" >&2 - exit 1 - fi - pkg repo "$repo_dir" "$PKG_REPO_SIGN_KEY" -else - echo "WARNING: creating unsigned development repository" >&2 - pkg repo "$repo_dir" -fi - -if [ -n "${PKG_REPO_PUBLIC_KEY:-}" ]; then - if [ ! -r "$PKG_REPO_PUBLIC_KEY" ]; then - echo "Repository public key is not readable: $PKG_REPO_PUBLIC_KEY" >&2 - exit 1 - fi - cp "$PKG_REPO_PUBLIC_KEY" "$DIST_DIR/rustproxy-repo.pub" -fi - -cat > "$DIST_DIR/BUILD-INFO.txt" < "$DIST_DIR/SHA256" -echo "Release artifacts created in $DIST_DIR" diff --git a/builder/freebsd/create-repo-key.sh b/builder/freebsd/create-repo-key.sh deleted file mode 100755 index d03f26a..0000000 --- a/builder/freebsd/create-repo-key.sh +++ /dev/null @@ -1,21 +0,0 @@ -#!/bin/sh -set -eu - -KEY_DIR="${1:-/usr/local/rustproxy-build/keys}" -KEY="$KEY_DIR/rustproxy-repo.key" -PUB="$KEY_DIR/rustproxy-repo.pub" - -umask 077 -mkdir -p "$KEY_DIR" -if [ -e "$KEY" ] || [ -e "$PUB" ]; then - echo "Key already exists in $KEY_DIR" >&2 - exit 1 -fi - -openssl genrsa -out "$KEY" 2048 -chmod 0400 "$KEY" -openssl rsa -in "$KEY" -out "$PUB" -pubout -chmod 0444 "$PUB" - -echo "Private key: $KEY" -echo "Public key: $PUB" diff --git a/ci/build-release.sh b/ci/build-release.sh index 35b6881..46f1bc2 100755 --- a/ci/build-release.sh +++ b/ci/build-release.sh @@ -8,35 +8,24 @@ if [[ -r "$CONFIG_FILE" ]]; then source "$CONFIG_FILE" fi -: "${BUILD_MODE:=native-freebsd}" -: "${STATE_DIR:=/var/lib/rustproxy-opnsense-builder}" -: "${OPNSENSE_PLUGINS_REF:=stable/26.7}" +: "${BUILD_MODE:=docker-freebsd}" : "${OPNSENSE_ABI:=26.7}" +: "${FREEBSD_RELEASE:=15.1-RELEASE}" +: "${FREEBSD_ABI:=FreeBSD:15:amd64}" +: "${FREEBSD_ARCH:=freebsd:15:x86:64}" : "${RUSTPROXY_SOURCE_DIR:?RUSTPROXY_SOURCE_DIR must point to a checked-out RustProxy repository}" : "${RUSTPROXY_SOURCE_URL:=https://git.linuxiarz.pl/gru/RustProxy}" : "${RUSTPROXY_SOURCE_REF:=HEAD}" : "${RUSTPROXY_SOURCE_COMMIT:=$(git -C "$RUSTPROXY_SOURCE_DIR" rev-parse HEAD)}" : "${RUSTPROXY_SOURCE_REVISION:=$(git -C "$RUSTPROXY_SOURCE_DIR" rev-list --count "$RUSTPROXY_SOURCE_COMMIT")}" -if [[ ! -f "$RUSTPROXY_SOURCE_DIR/Cargo.toml" ]]; then - echo "RustProxy source directory is invalid: $RUSTPROXY_SOURCE_DIR" >&2 - exit 1 -fi - -revision="$RUSTPROXY_SOURCE_REVISION" - -safe_value() -{ - [[ "$1" =~ ^[A-Za-z0-9_./:@+-]*$ ]] -} - publish_dist() { if [[ -z "${PUBLISH_TARGET:-}" ]]; then echo "PUBLISH_TARGET is empty; artifacts remain in $ROOT/dist" return 0 fi - local ssh_cmd=(ssh) + local ssh_cmd=(ssh -o BatchMode=yes) if [[ -n "${PUBLISH_SSH_KEY:-}" ]]; then ssh_cmd+=( -i "$PUBLISH_SSH_KEY" ) fi @@ -46,72 +35,36 @@ publish_dist() auto_deploy_tests() { [[ -n "${AUTO_DEPLOY_HOSTS:-}" ]] || return 0 + local package="$ROOT/dist/os-rustproxy-latest.pkg" + [[ -f "$package" ]] || { echo "Missing package for deployment: $package" >&2; exit 1; } + for host in $AUTO_DEPLOY_HOSTS; do - echo "Upgrading RustProxy on test firewall: $host" - ssh_args=( -o BatchMode=yes ) + echo "Deploying RustProxy to test firewall: $host" + local ssh_args=( -o BatchMode=yes ) if [[ -n "${AUTO_DEPLOY_SSH_KEY:-}" ]]; then ssh_args+=( -i "$AUTO_DEPLOY_SSH_KEY" ) fi - ssh "${ssh_args[@]}" "$host" \ - 'pkg update -f -r RustProxy && pkg install -y -r RustProxy os-rustproxy' + scp "${ssh_args[@]}" "$package" "$host:/tmp/os-rustproxy-latest.pkg" + ssh "${ssh_args[@]}" "$host" ' + set -e + pkg add -f /tmp/os-rustproxy-latest.pkg + rm -f /tmp/os-rustproxy-latest.pkg + service configd restart + /usr/local/sbin/pluginctl -cq cache_flush >/dev/null 2>&1 || true + /usr/local/sbin/configctl template reload OPNsense/RustProxy >/dev/null 2>&1 || true + ' done } case "$BUILD_MODE" in - native-freebsd) - : "${FREEBSD_HOST:?FREEBSD_HOST is required}" - : "${FREEBSD_USER:?FREEBSD_USER is required}" - : "${FREEBSD_REMOTE_DIR:?FREEBSD_REMOTE_DIR is required}" - for value in "$FREEBSD_HOST" "$FREEBSD_USER" "$FREEBSD_REMOTE_DIR" \ - "$OPNSENSE_PLUGINS_REF" "$OPNSENSE_ABI" \ - "${PKG_REPO_SIGN_KEY:-}" "${PKG_REPO_PUBLIC_KEY:-}" \ - "$RUSTPROXY_SOURCE_URL" "$RUSTPROXY_SOURCE_REF" \ - "$RUSTPROXY_SOURCE_COMMIT" "$revision"; do - if ! safe_value "$value"; then - echo "Unsafe character in builder configuration value: $value" >&2 - exit 1 - fi - done - - ssh_args=( -o BatchMode=yes ) - if [[ -n "${FREEBSD_SSH_KEY:-}" ]]; then - ssh_args+=( -i "$FREEBSD_SSH_KEY" ) - fi - remote="${FREEBSD_USER}@${FREEBSD_HOST}" - ssh "${ssh_args[@]}" "$remote" "mkdir -p '$FREEBSD_REMOTE_DIR' '$FREEBSD_REMOTE_DIR/app'" - - # Send packaging/plugin sources separately from the application source. - rsync -az --delete \ - --exclude .git/ --exclude app/ --exclude dist/ --exclude .cache/ \ - -e "ssh ${ssh_args[*]}" \ - "$ROOT/" "$remote:$FREEBSD_REMOTE_DIR/" - - rsync -az --delete \ - --exclude .git/ --exclude target/ \ - -e "ssh ${ssh_args[*]}" \ - "$RUSTPROXY_SOURCE_DIR/" "$remote:$FREEBSD_REMOTE_DIR/app/" - - remote_command="OPNSENSE_PLUGINS_REF=$OPNSENSE_PLUGINS_REF OPNSENSE_ABI=$OPNSENSE_ABI RUSTPROXY_SOURCE_URL=$RUSTPROXY_SOURCE_URL RUSTPROXY_SOURCE_REF=$RUSTPROXY_SOURCE_REF RUSTPROXY_SOURCE_COMMIT=$RUSTPROXY_SOURCE_COMMIT SOURCE_REVISION=$revision" - if [[ -n "${PKG_REPO_SIGN_KEY:-}" ]]; then - remote_command+=" PKG_REPO_SIGN_KEY=$PKG_REPO_SIGN_KEY" - fi - if [[ -n "${PKG_REPO_PUBLIC_KEY:-}" ]]; then - remote_command+=" PKG_REPO_PUBLIC_KEY=$PKG_REPO_PUBLIC_KEY" - fi - remote_command+=" sh '$FREEBSD_REMOTE_DIR/builder/freebsd/build-release.sh' '$FREEBSD_REMOTE_DIR'" - ssh "${ssh_args[@]}" "$remote" "$remote_command" - - rm -rf "$ROOT/dist" - mkdir -p "$ROOT/dist" - rsync -az -e "ssh ${ssh_args[*]}" "$remote:$FREEBSD_REMOTE_DIR/dist/" "$ROOT/dist/" - ;; - cross-binary) - RUSTPROXY_SOURCE_DIR="$RUSTPROXY_SOURCE_DIR" "$ROOT/builder/debian/build-freebsd-cross.sh" - echo "Cross mode builds a binary only and is not accepted as a release package pipeline." - exit 0 + docker-freebsd) + export OPNSENSE_ABI FREEBSD_RELEASE FREEBSD_ABI FREEBSD_ARCH + export RUSTPROXY_SOURCE_DIR RUSTPROXY_SOURCE_URL RUSTPROXY_SOURCE_REF + export RUSTPROXY_SOURCE_COMMIT RUSTPROXY_SOURCE_REVISION + "$ROOT/builder/debian/build-docker-release.sh" ;; *) - echo "Unknown BUILD_MODE: $BUILD_MODE" >&2 + echo "Unsupported BUILD_MODE: $BUILD_MODE. Use docker-freebsd." >&2 exit 1 ;; esac diff --git a/deploy/opnsense/install-direct.sh b/deploy/opnsense/install-direct.sh new file mode 100755 index 0000000..a97902e --- /dev/null +++ b/deploy/opnsense/install-direct.sh @@ -0,0 +1,43 @@ +#!/bin/sh +set -eu + +if [ "$(id -u)" -ne 0 ]; then + echo "Run as root on OPNsense" >&2 + exit 1 +fi + +source_pkg="${1:-}" +if [ -z "$source_pkg" ]; then + echo "usage: $0 " >&2 + exit 64 +fi + +tmp="" +case "$source_pkg" in + http://*|https://*) + tmp="/tmp/os-rustproxy-install.pkg" + fetch -o "$tmp" "$source_pkg" + package="$tmp" + ;; + *) + package="$source_pkg" + ;; +esac + +if [ ! -f "$package" ]; then + echo "Package not found: $package" >&2 + exit 1 +fi + +pkg add -f "$package" + +# New configd action files are loaded only after configd is restarted. +service configd restart +/usr/local/sbin/pluginctl -cq cache_flush >/dev/null 2>&1 || true +/usr/local/sbin/configctl template reload OPNsense/RustProxy >/dev/null 2>&1 || true + +if [ -n "$tmp" ]; then + rm -f "$tmp" +fi + +echo "RustProxy package installed. Open Services -> RustProxy." diff --git a/deploy/opnsense/install-repo.sh b/deploy/opnsense/install-repo.sh deleted file mode 100755 index 8a44df1..0000000 --- a/deploy/opnsense/install-repo.sh +++ /dev/null @@ -1,36 +0,0 @@ -#!/bin/sh -set -eu - -if [ "$(id -u)" -ne 0 ]; then - echo "Run as root on OPNsense" >&2 - exit 1 -fi - -REPO_URL="${1:-}" -PUBLIC_KEY_FILE="${2:-}" - -if [ -z "$REPO_URL" ] || [ -z "$PUBLIC_KEY_FILE" ]; then - echo "usage: $0 https://packages.example.net/rustproxy/26.7/amd64 /path/to/rustproxy-repo.pub" >&2 - exit 64 -fi -if [ ! -r "$PUBLIC_KEY_FILE" ]; then - echo "Public key not readable: $PUBLIC_KEY_FILE" >&2 - exit 1 -fi - -install -d -m 0755 /usr/local/etc/pkg/repos /usr/local/etc/pkg/keys -install -m 0444 "$PUBLIC_KEY_FILE" /usr/local/etc/pkg/keys/rustproxy-repo.pub - -cat > /usr/local/etc/pkg/repos/RustProxy.conf < RustProxy in OPNsense." diff --git a/deploy/opnsense/remove-repo.sh b/deploy/opnsense/remove-repo.sh deleted file mode 100755 index 833f686..0000000 --- a/deploy/opnsense/remove-repo.sh +++ /dev/null @@ -1,6 +0,0 @@ -#!/bin/sh -set -eu - -rm -f /usr/local/etc/pkg/repos/RustProxy.conf -rm -f /usr/local/etc/pkg/keys/rustproxy-repo.pub -pkg update -f diff --git a/deploy/opnsense/uninstall.sh b/deploy/opnsense/uninstall.sh new file mode 100755 index 0000000..e4e381c --- /dev/null +++ b/deploy/opnsense/uninstall.sh @@ -0,0 +1,14 @@ +#!/bin/sh +set -eu + +if [ "$(id -u)" -ne 0 ]; then + echo "Run as root on OPNsense" >&2 + exit 1 +fi + +/usr/local/sbin/configctl rustproxy stop >/dev/null 2>&1 || true +pkg delete -y os-rustproxy || true +service configd restart +/usr/local/sbin/pluginctl -cq cache_flush >/dev/null 2>&1 || true + +echo "RustProxy package removed." diff --git a/deploy/opnsense/update.sh b/deploy/opnsense/update.sh deleted file mode 100755 index 8bb4687..0000000 --- a/deploy/opnsense/update.sh +++ /dev/null @@ -1,5 +0,0 @@ -#!/bin/sh -set -eu - -pkg update -f -r RustProxy -pkg install -y -r RustProxy os-rustproxy diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 5332cb1..c466255 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -1,46 +1,37 @@ # Architecture -## Release path +## Components -1. Debian cron runs once per hour. -2. It refreshes the packaging repository containing this plugin/build logic. -3. Independently, it fetches `APP_REPO_REF` from `https://git.linuxiarz.pl/gru/RustProxy` into `APP_WORKDIR`. -4. It compares both current commits with the last successful build state. -5. If either commit changed, Debian rsyncs the packaging tree and the RustProxy checkout separately to a FreeBSD 15.1 amd64 builder. -6. FreeBSD reads the application version from the downloaded `Cargo.toml`, runs Cargo tests and a release build. -7. The binary is injected into `opnsense-plugin/www/rustproxy/src/sbin/rust-proxy` inside a clean checkout of the official OPNsense plugins tree. -8. `make package` creates `os-rustproxy-_.pkg`. -9. `pkg repo` creates a signed package repository. -10. Debian downloads `dist/` and optionally publishes it over rsync/SSH. -11. OPNsense consumes only the custom RustProxy package repository. +1. **Debian host** - Git polling, Docker engine, package assembly and optional publication. +2. **Docker builder** - Rust 1.88, Clang/LLD and a verified FreeBSD 15.1 sysroot extracted from official `base.txz`. +3. **OPNsense package** - `os-rustproxy-*.pkg`, installed directly with `pkg add -f`. -The packaging repository deliberately contains no `app/` directory. This prevents stale application code from accidentally being packaged. +No FreeBSD VM is used. -## Upstream selection +## Why the sysroot is enough for the binary build -Default configuration: +Rust provides `x86_64-unknown-freebsd`. Clang receives `--target=x86_64-unknown-freebsd15.1` and `--sysroot=/opt/freebsd-sysroot`, so Rust and C/assembly dependencies such as `ring` link against FreeBSD headers and libraries instead of Debian ones. + +The container is still Linux. It cannot execute the generated FreeBSD binary, so the pipeline performs Linux tests before cross-compilation and ELF inspection after cross-compilation. Runtime validation must happen on an OPNsense/FreeBSD test firewall. + +## Package construction + +FreeBSD pkg archives are compressed tar archives with metadata files beginning with `+` before package data. The Linux packager creates: ```text -APP_REPO_URL=https://git.linuxiarz.pl/gru/RustProxy -APP_REPO_REF=HEAD ++COMPACT_MANIFEST ++MANIFEST +usr/local/... ``` -`HEAD` follows the repository's default branch. To freeze releases, set `APP_REPO_REF` to a branch, tag or other fetchable ref. The exact commit used for each package is written to `dist/BUILD-INFO.txt`. +and compresses it with zstd to `.pkg`. -## Why native FreeBSD for releases +## Source isolation -Rust can cross-compile to `x86_64-unknown-freebsd`, but a release also needs the target ABI and FreeBSD/OPNsense packaging behavior. A native FreeBSD builder makes both the Rust link step and `pkg` metadata/catalog generation match the target platform. The Docker/cross path is intended for fast developer validation only. +```text +/opt/opn-rustproxy/packaging packaging repository +/opt/opn-rustproxy/source/RustProxy dynamically fetched application +/opt/opn-rustproxy/cache Docker/Cargo cache +``` -## OPNsense plugin behavior - -The first UI version exposes only `general.enabled`. - -- Enabled: the service controller renders configuration and starts RustProxy through configd. -- Disabled: the service controller stops RustProxy. -- Boot: an OPNsense syshook regenerates configuration and starts the service only when enabled. - -RustProxy runs as the unprivileged `rustproxy` user. Its data is stored in `/var/db/rustproxy` and logs in `/var/log/rustproxy/rustproxy.log`. - -## Ports 80 and 443 - -The MVP does not run RustProxy as root. Keep initial listeners above port 1023. If public 80/443 is required later, prefer an explicit PF redirect from 80/443 to high RustProxy listener ports instead of running the whole proxy as root. +The exact RustProxy SHA and package version are written to `dist/BUILD-INFO.txt`. diff --git a/docs/FIRST_RUN.md b/docs/FIRST_RUN.md index c22e492..7778fee 100644 --- a/docs/FIRST_RUN.md +++ b/docs/FIRST_RUN.md @@ -1,109 +1,138 @@ -# First run +# First run on Debian -## 1. Push this packaging repository +Everything below assumes the working root is `/opt/opn-rustproxy`. -Create a repository for the generated OPNsense plugin/build files and push this tree there. RustProxy itself must not be copied into this repository; it will be fetched dynamically from: - -```text -https://git.linuxiarz.pl/gru/RustProxy -``` - -After pushing, set that new repository address as `PACKAGING_REPO_URL` in the Debian configuration. - -## 2. Prepare the FreeBSD release builder - -Use FreeBSD 15.1 amd64. +## 1. Checkout ```sh -sudo sh builder/freebsd/bootstrap.sh builder -sudo -u builder sh builder/freebsd/create-repo-key.sh +sudo mkdir -p /opt/opn-rustproxy +sudo chown -R "$USER:$USER" /opt/opn-rustproxy +cd /opt/opn-rustproxy +git clone https://git.linuxiarz.pl/gru/rustproxy-opnsense-repo packaging +cd packaging ``` -The bootstrap creates a non-root `builder` account and gives it `/usr/local/rustproxy-build`. +RustProxy is intentionally not cloned manually. The scheduler fetches it to `/opt/opn-rustproxy/source/RustProxy`. -## 3. Prepare Debian - -From a checkout of this packaging repository: +## 2. Install Debian dependencies ```sh sudo bash builder/debian/install.sh +``` + +This installs Docker, cron, Git, zstd and the scheduler, then starts `docker` and `cron`. + +## 3. Configure + +```sh sudo editor /etc/rustproxy-opnsense-builder.env ``` -At minimum configure: +For the basic local build, use: ```text -PACKAGING_REPO_URL= -PACKAGING_REPO_REF=main +PACKAGING_REPO_URL=https://git.linuxiarz.pl/gru/rustproxy-opnsense-repo +PACKAGING_REPO_REF=HEAD +PACKAGING_WORKDIR=/opt/opn-rustproxy/packaging APP_REPO_URL=https://git.linuxiarz.pl/gru/RustProxy APP_REPO_REF=HEAD +APP_WORKDIR=/opt/opn-rustproxy/source/RustProxy -FREEBSD_HOST= -FREEBSD_USER=builder -FREEBSD_REMOTE_DIR=/usr/local/rustproxy-build/project +STATE_DIR=/var/lib/rustproxy-opnsense-builder +DOCKER_CACHE_DIR=/opt/opn-rustproxy/cache + +BUILD_MODE=docker-freebsd +FREEBSD_RELEASE=15.1-RELEASE +FREEBSD_ABI=FreeBSD:15:amd64 +FREEBSD_ARCH=freebsd:15:x86:64 +OPNSENSE_ABI=26.7 +DOCKER_IMAGE=rustproxy-freebsd15-builder:15.1-rust1.88 +RUN_LINUX_TESTS=1 + +PUBLISH_TARGET= +PUBLISH_SSH_KEY= +AUTO_DEPLOY_HOSTS= +AUTO_DEPLOY_SSH_KEY= ``` -`APP_REPO_REF=HEAD` means "build the current default branch". If you want controlled releases, change it to a specific branch or tag. - -Run the first cycle manually: +## 4. Validate repository scripts ```sh -sudo /usr/local/sbin/rustproxy-opnsense-hourly +cd /opt/opn-rustproxy/packaging +./scripts/check.sh ``` -Cron runs at minute 17 of every hour. Each run fetches the RustProxy upstream. A new build starts when either the upstream RustProxy commit or this packaging repository commit changed. A failed build does not update `last-successful-build`, so it is retried on the next run. +## 5. First build -## 4. Build state +```sh +sudo /usr/local/sbin/rustproxy-opnsense-hourly 2>&1 | sudo tee -a /var/log/rustproxy-opnsense-builder.log +``` -Debian stores checkouts separately: +The first run downloads the RustProxy Git checkout and the FreeBSD 15.1 sysroot, then builds the Docker image. Later runs reuse Docker/Cargo caches. RustProxy's build also downloads crates and pinned Swagger UI assets, so the builder needs outbound HTTPS access. + +Follow the log with: + +```sh +sudo tail -f /var/log/rustproxy-opnsense-builder.log +``` + +## 6. Verify output + +```sh +cd /opt/opn-rustproxy/packaging +cat dist/BUILD-INFO.txt +cat dist/SHA256 +file dist/docker-build/rust-proxy +zstd -dc dist/os-rustproxy-latest.pkg | tar -tf - | head -50 +``` + +The binary should identify as an x86-64 FreeBSD ELF executable, not a Linux executable. + +## 7. Install on a test OPNsense + +Copy the package and installer: + +```sh +scp dist/os-rustproxy-latest.pkg root@OPNSENSE_IP:/tmp/ +scp deploy/opnsense/install-direct.sh root@OPNSENSE_IP:/tmp/ +``` + +Then: + +```sh +ssh root@OPNSENSE_IP +sh /tmp/install-direct.sh /tmp/os-rustproxy-latest.pkg +``` + +Open **Services -> RustProxy**, enable it and click **Apply**. + +Useful checks on OPNsense: + +```sh +pkg info os-rustproxy +file /usr/local/sbin/rust-proxy +service rustproxy status +tail -f /var/log/rustproxy/rustproxy.log +``` + +## 8. Hourly scheduler + +`/etc/cron.d/rustproxy-opnsense` runs at minute 17 each hour: ```text -/srv/rustproxy-opnsense/packaging/ # this repository -/srv/rustproxy-opnsense/source/RustProxy # dynamically fetched app -/var/lib/rustproxy-opnsense-builder/ # lock + last successful commit pair +17 * * * * root /usr/local/sbin/rustproxy-opnsense-hourly >> /var/log/rustproxy-opnsense-builder.log 2>&1 ``` -The FreeBSD build machine receives the two trees separately. Its temporary `$FREEBSD_REMOTE_DIR/app/` always comes from the upstream checkout, never from the packaging repository. +No rebuild occurs when both Git SHAs are unchanged. A failed build does not advance the successful-state marker and will be retried later. -## 5. Publish `dist/` +## 9. Optional test-firewall auto deploy -Set `PUBLISH_TARGET`. The published layout is similar to: +Only for explicitly named test firewalls: ```text -/rustproxy/ - BUILD-INFO.txt - SHA256 - rustproxy-repo.pub - repository/ - 26.7/ - amd64/ - os-rustproxy-_.pkg - meta.conf - data.pkg - packagesite.pkg +AUTO_DEPLOY_HOSTS="root@10.0.0.10" +AUTO_DEPLOY_SSH_KEY=/root/.ssh/opnsense_test ``` -`BUILD-INFO.txt` contains the RustProxy source URL, ref and exact commit used to create the package. - -## 6. Install on OPNsense - -Copy `rustproxy-repo.pub` to the firewall and run: - -```sh -sh deploy/opnsense/install-repo.sh \ - https://packages.example.net/rustproxy/repository/26.7/amd64 \ - /tmp/rustproxy-repo.pub -``` - -Then open `Services -> RustProxy`, enable it and click Apply. - -## 7. Update - -Publishing a new package does not automatically upgrade production firewalls. Run: - -```sh -sh deploy/opnsense/update.sh -``` - -`AUTO_DEPLOY_HOSTS` exists only for explicitly listed test firewalls and is empty by default. +Leave both empty for manual deployment. diff --git a/opnsense-plugin/www/rustproxy/Makefile b/opnsense-plugin/www/rustproxy/Makefile index 1c826f7..d45cdfa 100644 --- a/opnsense-plugin/www/rustproxy/Makefile +++ b/opnsense-plugin/www/rustproxy/Makefile @@ -3,7 +3,7 @@ PLUGIN_VERSION?= 0.20.0 PLUGIN_REVISION?= 0 PLUGIN_COMMENT= RustProxy reverse proxy service PLUGIN_MAINTAINER?= local@localhost -PLUGIN_WWW?= https://example.invalid/rustproxy +PLUGIN_WWW?= https://git.linuxiarz.pl/gru/RustProxy PLUGIN_LICENSE= UNKNOWN .include "../../Mk/plugins.mk" diff --git a/scripts/check.sh b/scripts/check.sh index b10dcf9..53c58f3 100755 --- a/scripts/check.sh +++ b/scripts/check.sh @@ -40,4 +40,16 @@ if [[ -e "$ROOT/opnsense-plugin/www/rustproxy/src/sbin/rust-proxy" ]]; then exit 1 fi +[[ -f "$ROOT/builder/docker/Dockerfile" ]] +grep -q '15.1-RELEASE' "$ROOT/builder/docker/Dockerfile" +grep -q 'rust:1.88-' "$ROOT/builder/docker/Dockerfile" +grep -q 'x86_64-unknown-freebsd' "$ROOT/builder/docker/Dockerfile" +grep -q 'BUILD_MODE=docker-freebsd' "$ROOT/builder.env.example" +grep -q 'PACKAGING_WORKDIR=/opt/opn-rustproxy/packaging' "$ROOT/builder.env.example" + +if [[ -d "$ROOT/builder/freebsd" ]]; then + echo "Obsolete native FreeBSD builder directory is still present." >&2 + exit 1 +fi + echo "Repository checks passed."