v0.25.0

docs/WEB_SERVER.md

@@ -0,0 +1,320 @@
+# Web Server - RustProxy 0.25
+
+RustProxy 0.25 provides a Web Server source inside the existing vhost/location pipeline. A Web Service is a reusable configuration resource; vhosts and locations select it by ID and do not duplicate filesystem/PHP settings.
+
+## Architecture
+
+```text
+listener -> vhost -> location -> shared ACL/auth/policy
+                              -> source
+                                 |- Reverse Proxy
+                                 `- Web Server -> Web Service
+                                      |- static file
+                                      `- FastCGI -> PHP-FPM
+                              -> shared response/error/header/compression pipeline
+```
+
+A Web Service can be reused by several vhosts and one vhost can mix Web Server and Reverse Proxy locations. Web Services live in the normal revisioned RustProxy configuration (`web_services`); there is no separate Web Server configuration database.
+
+## Dashboard configuration
+
+The Web Service editor is split into stable configuration tabs so later features do not create one long form:
+
+- **General** - optional one-shot application preset, ID, name, document root, ordered index files, directory listing and quick hardening switches.
+- **Routing** - `try_files` and Web Service rewrite rules.
+- **.htaccess** - bounded migration-oriented Apache compatibility, recursive scan and strict-mode behavior.
+- **MIME** - per-extension MIME overrides and fallback MIME type.
+- **Cache & headers** - default `Cache-Control`, ordered cache rules and fixed response headers.
+- **Delivery** - precompressed `.br`/`.gz` sidecars, forced-download extensions and Range/streaming information.
+- **PHP-FPM** - Unix/loopback-TCP endpoint, executable extensions, connect/request timeout, bounded concurrency, Authorization forwarding, request/header/stderr limits, custom FastCGI parameters and controlled `PHP_VALUE` overrides.
+- **Security** - symlink policy, request/response hardening summary and filesystem-confinement notes.
+
+After the Web Service is saved, attach it under **Vhosts** by selecting `Web Server` and the service ID for the main source or a custom location.
+
+### Application presets
+
+The editor provides one-shot presets for **Static site, SPA, PHP generic, WordPress, Drupal, Laravel, Symfony, Joomla, PrestaShop and Downloads/file repository**. Applying a preset writes normal Web Service fields; it does not store a hidden CMS mode and every generated field remains editable. Service ID/name, document root and the deployment-specific PHP-FPM endpoint/tuning are preserved. Laravel/Symfony operators should point `document_root` at the application's `public/` directory.
+
+The preset control also exposes two independent hardening switches. They are saved as normal Web Service configuration and remain active even after the generated preset settings are edited manually.
+
+## Configuration example
+
+```json
+{
+  "id": "main-site",
+  "name": "Main site",
+  "document_root": "/var/www/main",
+  "index": ["index.html", "index.php"],
+  "directory_listing": false,
+  "htaccess": {
+    "enabled": true,
+    "filename": ".htaccess",
+    "scan_subdirectories": true,
+    "strict": false
+  },
+  "try_files": ["$uri", "$uri/", "/index.php?$query_string"],
+  "rewrites": [
+    {"pattern": "^/old/(.*)$", "replacement": "/new/$1", "status": 301}
+  ],
+  "mime_types": {"avif": "image/avif"},
+  "default_mime_type": "application/octet-stream",
+  "cache_control": "public, max-age=3600",
+  "cache_rules": [
+    {"matcher": "ext:html", "cache_control": "no-cache"},
+    {"matcher": "mime:image/", "cache_control": "public, max-age=604800"}
+  ],
+  "response_headers": {"x-content-type-options": "nosniff"},
+  "precompressed": {"enabled": true, "brotli": true, "gzip": true},
+  "download_extensions": ["zip", "iso"],
+  "symlink_policy": "deny",
+  "hardening": {
+    "block_common_attacks": true,
+    "strip_unnecessary_headers": true
+  },
+  "php": {
+    "enabled": true,
+    "extensions": ["php"],
+    "endpoint": "unix:/run/php/php-fpm.sock",
+    "connect_timeout_ms": 1000,
+    "request_timeout_ms": 30000,
+    "max_concurrent_requests": 64
+  }
+}
+```
+
+Older configurations remain compatible because all newly added settings have safe defaults.
+
+## Request routing inside a Web Service
+
+The outer vhost/location router always runs first. Inside the selected Web Service the file-serving sequence is:
+
+1. validate and decode the already-selected public path;
+2. apply configured Web Service rewrite rules in order;
+3. apply the compiled `.htaccess` rewrite subset when enabled;
+4. resolve `try_files` candidates when configured;
+5. open the selected filesystem object within the document-root boundary;
+6. execute configured PHP extensions through PHP-FPM or serve a static file;
+7. for a directory, redirect to the slash form if required, then test the effective index list;
+8. if no index exists, generate directory listing when effectively enabled or return the normal customizable 403 response.
+
+This is Web Service file resolution, not a replacement for vhost/location routing.
+
+### `try_files`
+
+One candidate is configured per line. Supported substitutions are `$uri` and `$query_string`.
+
+Example for an SPA:
+
+```text
+$uri
+$uri/
+/index.html
+```
+
+Example for a PHP front controller:
+
+```text
+$uri
+$uri/
+/index.php?$query_string
+```
+
+A final `=4xx`/`=5xx` candidate can return an error status, for example `=404`. Non-final candidates are tested for existence; the final path candidate is the fallback target.
+
+### Rewrite rules
+
+Rules use a regular-expression pattern and a replacement path. The replacement must start with `/`.
+
+- without a status: internal rewrite;
+- with `301`, `302`, `307` or `308`: client redirect.
+
+The current query string is retained unless the replacement supplies its own query. Rewritten paths still pass normal Web Server path validation and filesystem confinement.
+
+## `.htaccess` compatibility
+
+RustProxy implements a **bounded compatibility subset for migration**, not a complete Apache runtime. The files are read and compiled when the Web Service is validated/saved/applied. They are **not parsed on every request**. After changing `.htaccess` on disk, apply the Web Service again.
+
+Supported directives in 0.25 development:
+
+- `RewriteEngine`, `RewriteBase`, `RewriteCond`, `RewriteRule`;
+- `DirectoryIndex`;
+- `Options +Indexes`, `Options -Indexes` and the common `-MultiViews` no-op;
+- `AddType`;
+- `AddHandler` when the handler is PHP-related;
+- `Header set`, `Header add`, `Header unset`, including `always`/`onsuccess`;
+- `ExpiresActive`, `ExpiresDefault`, `ExpiresByType`;
+- `Require all granted` / `Require all denied` outside `Files` blocks.
+
+Rewrite compatibility covers the common front-controller patterns used by WordPress/Laravel-style deployments. Conditions can inspect `REQUEST_FILENAME` with `-f`, `-d` and `-s`, plus `REQUEST_URI`, `REQUEST_METHOD`, `QUERY_STRING`, `HTTP_HOST`, `HTTPS`, `REQUEST_SCHEME`, `DOCUMENT_ROOT`, `THE_REQUEST`, selected server/client variables and request headers. Supported rewrite flags are `L`, `END`, `R`, `R=301/302/307/308`, `QSA`, `NC`, `F` and condition `OR`. `PT`/`NE` are accepted as no-ops. `E=` is accepted as a compatibility no-op because HTTP request headers are already forwarded to PHP-FPM.
+
+Example front controller:
+
+```apache
+RewriteEngine On
+RewriteCond %{REQUEST_FILENAME} !-f
+RewriteCond %{REQUEST_FILENAME} !-d
+RewriteRule . /index.php [L]
+```
+
+`DirectoryIndex`, `Options ±Indexes`, MIME, header and Expires directives are scoped by the directory containing the compiled `.htaccess`. Subdirectory files can therefore override the effective index/listing/MIME/header/cache behavior for their subtree when recursive scanning is enabled.
+
+PHP remains intentionally controlled by RustProxy: `AddHandler` may extend the set of PHP file extensions, but the PHP-FPM endpoint, timeouts and concurrency are configured only in the Web Service PHP-FPM tab. Apache `SetHandler` socket/proxy syntax cannot choose an arbitrary FastCGI endpoint.
+
+The compatibility scanner is bounded to 128 files, depth 16, 128 KiB per file and 1 MiB total. Symlinked directories are not traversed. `.htaccess`, `.htpasswd` and `.ht*` control files are blocked from HTTP delivery. Unsupported non-security directives are logged and ignored by default; **Strict** mode rejects them instead. General `Files`/`FilesMatch`, unsupported authentication/authorization directives, non-PHP handlers and `SetHandler` are rejected even with Strict off because ignoring them could weaken access control or expose source. `ErrorDocument`, `SetEnvIf`, `Redirect`/`RedirectMatch` and `FallbackResource` are not part of this subset. Existing RustProxy vhost/location ACL, authentication, custom error pages and response rules remain authoritative.
+
+## Index and directory behavior
+
+`index` is an ordered list and can contain static files and, when PHP is enabled, `index.php`. For every requested directory RustProxy checks the list from the beginning. If no configured index exists, directory listing is used only when enabled.
+
+Directory listing is disabled by default. A generated listing:
+
+- opens a fresh directory handle for every request, so repeated refreshes remain correct;
+- HTML-escapes names and percent-encodes URL path segments;
+- distinguishes files/directories without revealing local filesystem paths;
+- sends `Cache-Control: no-store`.
+
+If listing is disabled and no index exists, RustProxy returns the standard generic `403 Access denied` response. Existing vhost/global custom error pages may replace it.
+
+## MIME handling
+
+Static `Content-Type` is selected in this order:
+
+1. Web Service extension override;
+2. built-in MIME table;
+3. `default_mime_type` (default `application/octet-stream`).
+
+The built-in table covers common HTML/CSS/JS/data formats, archives, images, audio/video, fonts and common Office file formats. Extension override keys are normalized to lowercase without a leading dot.
+
+## Cache policy and Web Service headers
+
+RustProxy still does **not** add a second object cache for local files. Static content uses filesystem/OS page cache, validators and normal HTTP client/intermediary caching.
+
+A Web Service can define:
+
+- default `Cache-Control`;
+- ordered `cache_rules` using `ext:<extension>` or `mime:<prefix>`;
+- fixed response headers.
+
+Rules are evaluated before the default. Example:
+
+```text
+ext:html = no-cache
+mime:image/ = public, max-age=604800, immutable
+```
+
+Vhost/location response-header rules still execute later in the shared pipeline and can impose URL-specific policy. Framing/transport headers such as `Content-Length`, `Content-Encoding`, `Content-Range`, `Transfer-Encoding`, connection headers and `Vary` cannot be replaced by Web Service fixed headers.
+
+## Precompressed static files
+
+Optional precompressed delivery checks sidecar files only after the original resource has been resolved safely:
+
+- `file.js.br` for Brotli;
+- `file.js.gz` for gzip.
+
+The client must advertise the matching coding in `Accept-Encoding`. Brotli is preferred when both are available. The response keeps the original file's MIME type and adds the appropriate `Content-Encoding` and `Vary: Accept-Encoding`.
+
+Validators, streaming and a single Range request operate on the selected representation. RustProxy does not compress the file on every request in this mode.
+
+## Range and downloads
+
+Static and precompressed responses support:
+
+- single `Range: bytes=...`;
+- `If-Range`;
+- `206 Partial Content`;
+- `416 Range Not Satisfiable`;
+- `Accept-Ranges` and `Content-Range`;
+- streaming rather than whole-file buffering.
+
+Multipart ranges remain intentionally unsupported.
+
+Configured `download_extensions` add `Content-Disposition: attachment` using an encoded filename. This is useful for storage/download services without changing the MIME type.
+
+## Validators
+
+Static files emit metadata-based `ETag` and `Last-Modified` and handle `If-None-Match` / `If-Modified-Since` with `304 Not Modified`. Large files are not hashed on each request.
+
+## PHP-FPM / FastCGI
+
+PHP execution is optional per Web Service. RustProxy speaks FastCGI directly to a long-running PHP-FPM process; it never starts `php`, `php-cgi` or a shell command per request.
+
+Supported endpoint forms:
+
+```text
+unix:/run/php/php-fpm.sock
+tcp:127.0.0.1:9000
+tcp:[::1]:9000
+```
+
+TCP endpoints are deliberately restricted to literal loopback addresses. The connector has explicit connect/request timeouts and a bounded semaphore. If all PHP execution slots are busy the request receives `503`; connection failures/timeouts become gateway errors and PHP source is never returned as a fallback.
+
+The PHP-FPM tab also exposes an optional streaming request-body limit (`0` means no additional RustProxy limit), response-header and stderr bounds, an Authorization-forwarding switch, static custom FastCGI parameters and php.ini-style `PHP_VALUE` overrides. Custom parameters use `NAME = value` per line; PHP settings use `directive = value` per line. Core CGI/HTTP/PHP control parameter names cannot be overridden. The in-panel **FastCGI syntax and safety** dialog contains the exact syntax and examples.
+
+Configured PHP extensions are executable rather than downloadable source. `index.php` and `try_files` front-controller patterns use the same Web Service root and path validation. See [PHP_FPM.md](PHP_FPM.md) for the operator/runtime details.
+
+## Optional Web Service hardening
+
+`hardening.block_common_attacks` is a deterministic high-confidence protection layer for the local Web Server. It is **not a generic WAF** and deliberately avoids broad SQL-injection/XSS regex matching that would create application false positives. When enabled it:
+
+- denies TRACE, CONNECT and TRACK at the Web Service source;
+- blocks hidden dot/control paths except `.well-known`;
+- blocks common secret/configuration targets such as `.env`, `wp-config.php`, Joomla `configuration.php`, Drupal `settings.php`, Composer auth/config metadata and private-key file names;
+- blocks common backup/database suffixes (`.bak`, `.old`, `.sql`, `.sqlite`, `.db`, editor swap files and `~` backups);
+- blocks high-confidence exploit-probe paths such as exposed PHPUnit `eval-stdin`, Laravel Ignition execute-solution, Apache server-status/server-info and `cgi-bin`;
+- suppresses the incoming `Proxy` header when building FastCGI parameters, mitigating the CGI/FastCGI `HTTP_PROXY`/httpoxy class.
+
+Existing decoded-path/traversal validation, document-root confinement, `.ht*` protection, vhost ACL/authentication and PHP source protection continue to run independently of this switch. A denied hardening request uses the normal customizable 403 pipeline.
+
+`hardening.strip_unnecessary_headers` removes common implementation/fingerprint response headers after Web Service and `.htaccess` response processing: `Server`, `X-Powered-By`, ASP.NET version headers, `X-Runtime`, `X-Generator` and Drupal cache fingerprints. Vhost/location response rules run later and can intentionally add an operator-controlled header if required.
+
+## Filesystem security and symlinks
+
+The request path rejects malformed encoding, encoded separators, backslashes, control characters and `.`/`..` traversal before filesystem access.
+
+On Unix, filesystem lookup starts from an opened document-root descriptor and walks relative components with `openat`.
+
+Two symlink policies are available:
+
+- `deny` (default) - every path component is opened with `O_NOFOLLOW`; symlinks are rejected.
+- `inside_root` - Linux-only; symlinks may be followed only when the opened descriptor resolves back inside the canonical document root through `/proc/self/fd` verification after every component.
+
+`inside_root` is not available with PHP-FPM in 0.25 because PHP-FPM reopens `SCRIPT_FILENAME` itself. PHP therefore requires the safer `deny` policy.
+
+The secure Web Server implementation remains Unix-only until an equivalent race-resistant confinement implementation exists for non-Unix systems.
+
+## Shared RustProxy pipeline
+
+Web Server dispatch occurs after the existing vhost/location selection and access controls. Responses continue through the existing common mechanisms, including:
+
+- ACL, authentication and authorization;
+- rate/access policy;
+- vhost/location custom and security headers;
+- custom error pages;
+- access logging and per-vhost Traffic metrics;
+- response processing/compression where applicable;
+- TLS and existing HTTP/1.1, HTTP/2 and HTTP/3 listeners.
+
+A local Web Server request does not create a reverse-proxy backend connection. WebSocket upgrades remain unsupported for a Web Server source.
+
+## Validation limits
+
+Current configuration limits include:
+
+- index files: maximum 16;
+- MIME overrides: maximum 64;
+- cache rules: maximum 64;
+- fixed Web Service headers: maximum 32;
+- `try_files`: maximum 16 candidates;
+- rewrites: maximum 32 rules;
+- download extensions: maximum 64;
+- PHP extensions: maximum 16;
+- PHP concurrency: 1..4096;
+- PHP connect timeout: 50..60000 ms;
+- PHP request timeout: 100..3600000 ms;
+- `.htaccess`: maximum 128 files, depth 16, 128 KiB per file and 1 MiB total.
+
+Document root must be absolute, exist and be a directory at configuration compile time. References to unknown Web Services are rejected and deletion of a referenced service is blocked/caught by validation.
+
+## Remaining follow-up
+
+The current implementation intentionally leaves multipart Range, named locations, `PATH_INFO` script splitting, full Apache `.htaccess` semantics and non-Unix confinement for later work. These additions must preserve the existing Web Service / vhost-source separation.