Commit Graph
738 Commits
Author SHA1 Message Date
Jörg Thalheim 56f07573ea Avoid shadowing internal run function by renaming it to runCommand
see https://github.com/NixOS/hydra/issues/1520
2025-09-12 21:45:58 +02:00
Jörg ThalheimandJörg Thalheim 29734ae51f replace backtick operator with run3 2025-08-28 13:08:59 +02:00
Janne Heß fd0b8ec8e0 Fix too much XSS protections
- Fixes build graphs
- Fixes pagination
- Fixes pressure of new queue runner
2025-08-14 12:25:17 +02:00
Jörg ThalheimandJanne Heß b47b187553 webhooks: implement authentication for GitHub and Gitea
- Add HMAC-SHA256 signature verification for webhooks
- Support multiple secrets for rotation
- Add security logging for authentication events
- Maintain backward compatibility (auth optional during migration)
- Add comprehensive test coverage

Without authentication, anyone could trigger job evaluations by sending
POST requests to webhook endpoints. This could lead to resource exhaustion
through repeated requests or manipulation of build scheduling. While not
a data breach risk, it allows unauthorized control over CI/CD operations.
2025-08-10 12:41:47 +02:00
Janne Heß 7096ae3a5b machine-status: Fixup double localhost during development 2025-08-02 14:05:23 +02:00
Janne Heß 632a59172a machine-status: Make new runner status prettier
- Remove bottom margin
- Properly format memory in human format
- Calculate free memory
- Format the load with 2 digits after comma
- Lpad pressure percentages
- Use a macro to render pressure
- Score -> Scheduling Score
- More spacing in the load
- Add IRQ pressure
2025-08-01 11:25:14 +02:00
Janne HeßandGitHub 61573c71d1 Merge pull request #1497 from helsinki-systems/feat/show-new-q-runner-status
Show queue runner v2 status
2025-07-17 21:30:36 +00:00
Janne HeßandGitHub 2fcfa969b8 Merge branch 'master' into fix/local-store-detection 2025-07-16 18:25:54 +02:00
Janne Heß d0008d4238 Show queue runner v2 status
This is guarded behind a setting and will overwrite everything that was
learned from the machines file. Also drops `sshKeys` since that wasn't
used anyway.
2025-07-16 17:39:06 +02:00
Pierre Bourdonandzowoq 0ab357e435 jobset-eval: fix actions not showing up sometimes for new jobs
New jobs have their "new" status take precedence over them being
"failed" or "queued", which means actions that can act on "failed" or
"queued" jobs weren't shown to the user when they could only act on
"new" jobs.

(cherry picked from commit https://git.lix.systems/lix-project/hydra/commit/9a4a5dd624a1cedc7cdc40687815739b228e5c77)
2025-04-16 09:50:32 +10:00
Pierre BourdonandJohn Ericson b4322edd05 web: replace 'errormsg' with 'errormsg IS NULL' in most cases
This is implement in an extremely hacky way due to poor DBIx feature
support. Ideally, what we'd need is a way to tell DBIx to ignore the
errormsg column unless explicitly requested, and to automatically add a
computed 'errormsg IS NULL' column in others. Since it does not support
that, this commit instead hacks some support via method overrides while
taking care to not break anything obvious.
2025-04-07 14:48:07 -04:00
git@71rd.netandJohn Ericson abe35881e4 Stream files from store instead of buffering them
When an artifact is requested from hydra the output is first copied
from the nix store into memory and then sent as a response, delaying
the download and taking up significant amounts of memory.

As reported in https://github.com/NixOS/hydra/issues/1357

Instead of calling a command and blocking while reading in the entire
output, this adds read_into_socket(). the function takes a
command, starting a subprocess with that command, returning a file
descriptor attached to stdout.
This file descriptor is then by responsebuilder of Catalyst to steam
the output directly

(cherry picked from commit 459aa0a5983a0bd546399c08231468d6e9282f54)
2025-04-07 11:59:49 -04:00
ajs124andJohn Ericson 99359c251a lazy-load evaluation errors
Closes #1362
2025-04-07 11:54:47 -04:00
Maximilian BoschandJohn Ericson 33b982f408 Running builds view: show build step names
When using Hydra to build machine configurations, you'll often see
"nixosConfigurations.foo" five times, i.e. for each build step being
run. This isn't very helpful I think because in such a case, a single
build step can also be compiling the Linux kernel.

This change also fetches the `drvpath` and `type` from the `buildsteps`
relation. We're already joining it, so this doesn't make much difference
(confirmed via query logging that this doesn't cause extra SQL queries).

Unfortunately build steps don't have a human readable name, so I'm
deriving it from the drvpath by stripping away the hash (assuming that
it'll never contain a `-` and that `/nix/store/` is used as prefix). I
decided against using the Nix bindings for that to avoid too much
overhead due to store operations for each build step.
2025-04-07 11:54:47 -04:00
John Ericson 85383b9522 Render the nix-eval-jobs version too 2025-02-07 16:55:28 -05:00
John Ericson 647191cd4a Merge remote-tracking branch 'upstream/nix-2.21' 2024-09-24 11:23:21 -04:00
Michael Bishop cc1b6d394e fix the binary cache calling isValidPath, and 2 other files with the same issue 2024-09-20 04:08:22 -03:00
Martin Weinelt f730433789 Create eval-jobset role and guard /api/push route 2024-08-27 19:49:05 +02:00
Janne HeßandMartin Weinelt 916531dc9c api: Require POST for /api/push 2024-08-27 17:52:13 +02:00
John EricsonandGitHub d7986226f0 Merge pull request #1227 from SuperSandro2000/gitea-push-hook
Add gitea push hook
2024-07-09 14:31:10 -04:00
Sandro Jäckel 1b8154e67f Fix doi resolution after #1375
This fixes:

> Caught exception in Hydra::Controller::Root->realisations "Undefined subroutine &Hydra::Controller::Root::queryRawRealisation called at /nix/store/v842xb35ph8ka1yi1xanjhk4xh1pn5nm-hydra-2024-04-22/libexec/hydra/lib/Hydra/Controller/Root.pm line 371."
2024-05-03 14:31:34 +02:00
Pierre BourdonandJanne Heß b72528be50 web: serveFile: also serve a CSP putting served HTML in its own origin 2024-04-22 16:28:50 +02:00
John EricsonandGitHub ef7bf1e67b Merge pull request #1375 from NixOS/nix-2.21
Nix 2.21
2024-04-12 17:28:37 -04:00
K900 9db5d0a88d urlencode drv names when fetching logs
Otherwise names with special characters like + break things.
2024-02-26 22:48:16 +03:00
Maximilian Bosch e499509595 Switch to new Nix bindings, update Nix for that
Implements support for Nix's new Perl bindings[1]. The current state
basically does `openStore()`, but always uses `auto` and doesn't support
stores at other URIs.

Even though the stores are cached inside the Perl implementation, I
decided to instantiate those once in the Nix helper module. That way
store openings aren't cluttered across the entire codebase. Also, there
are two stores used later on - MACHINE_LOCAL_STORE for `auto`,
BINARY_CACHE_STORE for the one from `store_uri` in `hydra.conf` - and
using consistent names should make the intent clearer then.

This doesn't contain any behavioral changes, i.e. the build product
availability issue from #1352 isn't fixed. This patch only contains the
migration to the new API.

[1] https://github.com/NixOS/nix/pull/9863
2024-02-12 18:50:56 +01:00
323b556dc8 Minimal CA support
This verison has a worse UI, but also chnages the schema less: One
non-null constraint is removed, but no new columns are added.

Co-Authored-By: Andrea Ciceri <andrea.ciceri@autistici.org>
Co-Authored-By: regnat <rg@regnat.ovh>
2024-01-26 00:34:58 -05:00
John Ericson fcde5908d8 More CA derivations prep
Again, with care not to change the schema in any way.
2024-01-25 21:32:22 -05:00
Josef Kemetmüller ad99d3366f Fix MIME types when serving .js and .css
To correctly render HTML reports we make sure to return the following MIME
types instead of "text/plain"

- *.css: "text/css"
- *.js: "application/javascript"

Fixes: #1267
2022-12-29 22:26:59 +01:00
Graham ChristensenandGitHub 01802efc17 Merge pull request #1263 from Ma27/fix-my-jobs-tab
Fix "My Jobs" tab in user dashboard
2022-12-05 01:55:49 +01:00
Eelco DolstraandGitHub 14d4624dc2 Merge pull request #1215 from knedlsepp/fix-hydra-with-latest-git
scmdiff: Hardcode `--git-dir`
2022-11-24 12:04:23 +01:00
Maximilian Bosch fd765bc97a Fix "My Jobs" tab in user dashboard
Nowadays `Builds` doesn't reference `Project` directly anymore. This
means that simply resolving both `jobset` and `project` with a single
JOIN from `Builds` doesn't work anymore. Instead we need to resolve the
relation to `jobset` first and then the relation to `project`.

For similar fixes see e.g. c7c4759600.
2022-11-22 20:54:51 +01:00
Maximilian Bosch d3fe4ffbf6 Job: expose closuresize and size (output size in the UI) as prometheus metrics 2022-09-22 10:47:22 +02:00
Marco Rebhan a58e2f1a64 Use libmagic for better output MIME detection 2022-08-04 22:34:58 +02:00
a81c6a3a80 Match URIs that don't end in .git
Co-authored-by: Charlotte <lotte@chir.rs>
2022-07-01 22:21:32 +02:00
ajs124 bab671124d replace nix cat-store with nix store cat
the former was deprecated in favor of the latter
2022-06-30 00:24:09 +02:00
Sandro Jäckel 750978a192 Add gitea push hook 2022-06-18 13:22:42 +02:00
Josef Kemetmüller b6ea85a601 scmdiff: Hardcode --git-dir
The newest version of git refuses to work on repositories not owned by
the current user. This leads to issues with the /api/scmdiff endpoint:

May 27 11:16:05 myhydra hydra-server[923698]: fatal: unsafe repository ('/var/lib/hydra/scm/git/57ea036ec7ecd85c8dd085e02ecc6f12dd5c079a6203d16aea49f586cadfb2be' is owned by someone else)
May 27 11:16:05 myhydra hydra-server[923698]: To add an exception for this directory, call:
May 27 11:16:05 myhydra hydra-server[923698]:         git config --global --add safe.directory /var/lib/hydra/scm/git/57ea036ec7ecd85c8dd085e02ecc6f12dd5c079a6203d16aea49f586cadfb2be
May 27 11:16:05 myhydra hydra-server[923701]: warning: Not a git repository. Use --no-index to compare two paths outside a working tree
May 27 11:16:05 myhydra hydra-server[923701]: usage: git diff --no-index [<options>] <path> <path>

I used the same solution that was used in NixOS/nix#6440.

Fixes #1214
2022-05-27 12:12:44 +02:00
Graham ChristensenandGitHub 5c90edd19f Merge pull request #1103 from DeterminateSystems/runcommand/dynamic
Dynamic RunCommand
2022-04-19 10:09:47 -04:00
Janne Heß 54675a0d94 Fix local store detection and related issues
- Add localStore into the stash because it's used in templates
- Hide the Channels button for non-local stores because the link 404s
  anyway
- Fix a style issue when having popovers in dark mode
2022-02-13 14:24:36 +01:00
Graham Christensen 27ddde1e9e dynamic runcommand: print a notice on the build page if it is disabled 2022-02-11 15:04:54 -05:00
Cole HelblingandGraham Christensen 928ba9e854 Controller/{Jobset,Project}: error when enabling dynamic runcommand but it's disabled elsewhere 2022-02-11 14:35:52 -05:00
Graham Christensen f07fb7d279 LDAP support: include BC support for the YAML based loading
Includes a refactoring of the configuration loader.
2022-02-11 10:49:38 -05:00
Janne HeßandGraham Christensen 61d74a7194 Redo LDAP config in the main configuration and add role mappings 2022-02-11 10:49:38 -05:00
Graham Christensen 8a96f07f58 Project: enable enabling dynamic runcommand per project 2022-02-01 10:58:54 -05:00
Graham Christensen 726ea80e99 HTTP/Jobset: support setting / reading enable_dynamic_run_command 2022-02-01 10:58:54 -05:00
Cole Helbling 61189ecca9 Helper/Nix: constructRunCommandLogPath: verify uuid is valid
This shouldn't be possible normally, but it is possible to:

    $db->resultset('RunCommandLogs')->new({ uuid => "../etc/passwd" });

if you have access to the `$db`.
2022-01-31 08:58:33 -08:00
Cole Helbling e381751564 Helper/Nix: constructRunCommandLogPath: return undef in case of an error
This allows us to give a web request to an invalid UUID a 404.
2022-01-31 08:58:33 -08:00
Cole Helbling 8eab7b8543 Helper/Nix: constructRunCommandLogPath: take RunCommandLog as input
This way we ensure that it actually exists in the database, rather than
blindly trusting user-generated input.
2022-01-31 08:58:33 -08:00
Cole Helbling 3594ba942a Controller/Build: use showLog in view_runcommandlog
This also adds the `runcommandlog` object to the stash so that we can
access its uuid as well as command run in order to display more useful
and specific information on the webpage.
2022-01-31 08:58:33 -08:00
Cole Helbling 1d0076408b Controller/Build: pass log_uri to showLog in place of drvPath
This way, we can reuse the `showLog` sub for other things, such as
`view_runcommandlog` (which doesn't have a drvPath attached).
2022-01-31 08:58:33 -08:00