Compare commits

..

36 Commits

Author SHA1 Message Date
John Ericson
60dd7ec187 Merge branch 'nix-2.21' into nix-2.22 2024-10-21 11:23:30 -04:00
John Ericson
53b04ddf74 Merge branch 'nix-2.20' into nix-2.21 2024-10-21 11:23:20 -04:00
Martin Weinelt
4e2c06ec2c queue-runner: don't decode base64 hostkey in hydra
Nix expects a base64 encoded hostkey in SSHMaster, so make sure we don't
decode this prematurely in hydra.

Reported-By: Puck Meerburg <puck@puck.moe>
2024-10-21 11:22:44 -04:00
John Ericson
108e409559 Merge branch 'nix-2.22' 2024-09-24 11:25:33 -04:00
John Ericson
1a5bd9e103 Merge remote-tracking branch 'upstream/nix-2.21' into nix-2.22 2024-09-24 11:25:10 -04:00
John Ericson
647191cd4a Merge remote-tracking branch 'upstream/nix-2.21' 2024-09-24 11:23:21 -04:00
Jörg Thalheim
73e51b94b1 Merge pull request #1414 from cleverca22/fix-binary-cache
fix nar endpoint for binary cache hosting
2024-09-20 13:24:47 +02:00
hacker1024
1ef6b5e7b4 Use Nix::Store and Nix::Utils in NARInfo.pm
These are required for the `signString` and `readFile` subroutines used when signing NARs.

(cherry picked from commit b94a7b6d5c)
2024-09-20 08:04:23 -03:00
Jörg Thalheim
44248d3cf4 Merge pull request #1412 from NixOS/hydra-compress-logs
set a default for hydra-compress-logs service
2024-09-20 09:21:34 +02:00
Michael Bishop
cc1b6d394e fix the binary cache calling isValidPath, and 2 other files with the same issue 2024-09-20 04:08:22 -03:00
zowoq
b472f55563 set a default for hydra-compress-logs service
follow up from 99ca560d58
2024-09-20 09:04:15 +02:00
Jörg Thalheim
c61bdd2c28 Merge pull request #1410 from NixOS/compiler-warnings
hydra-queue-runner: fix compilation warning
2024-09-20 09:00:55 +02:00
Jörg Thalheim
2dad87ad89 hydra-queue-runner: fix compilation warning
instead of converting to double, we can convert to float right away.
2024-09-20 07:50:24 +02:00
Jörg Thalheim
b6f44b5cd0 Merge pull request #1402 from NixOS/like-sub
tests: use `like` for testing regexes
2024-09-15 23:50:13 +02:00
Janne Heß
c8b7a0fea9 Merge pull request #1403 from NixOS/docs
Devdocs: mention nix develop and nproc
2024-09-03 15:03:01 +02:00
Jörg Thalheim
2d79b0a4da Merge pull request #1406 from NixOS/fix/remove-url-literal
default.nix: Drop URL literal
2024-08-27 21:41:42 +02:00
Martin Weinelt
f730433789 Create eval-jobset role and guard /api/push route 2024-08-27 19:49:05 +02:00
Janne Heß
916531dc9c api: Require POST for /api/push 2024-08-27 17:52:13 +02:00
Janne Heß
0ead8dc65c default.nix: Drop URL literal 2024-08-27 17:44:36 +02:00
Jörg Thalheim
b1a0501520 Merge pull request #1405 from hacker1024/patch-2
Use Nix::Store and Nix::Utils in NARInfo.pm
2024-08-27 17:09:37 +02:00
hacker1024
b94a7b6d5c Use Nix::Store and Nix::Utils in NARInfo.pm
These are required for the `signString` and `readFile` subroutines used when signing NARs.
2024-08-25 17:25:08 +10:00
Jörg Thalheim
9ee3c6aea2 Merge pull request #1400 from SuperSandro2000/feat/buildlogs-zstd
CompressLog: Add zstd compression
2024-08-21 09:43:39 +02:00
Jörg Thalheim
02a514234b hacking.md: make build parallel 2024-08-21 08:42:22 +02:00
Jörg Thalheim
54a9729a0f hacking.md: mention nix develop 2024-08-21 08:42:22 +02:00
Jörg Thalheim
4bb2f08be1 Merge pull request #1396 from nh2/hash-length-12
renderInputDiff: Increase git hash length 8 -> 12
2024-08-20 09:55:04 +02:00
Jörg Thalheim
c23973785f Merge pull request #1399 from Mindavi/bugfix/too-strict-timeouts
Looser timeouts, disable broken test, less verbose output
2024-08-20 09:54:40 +02:00
Sandro Jäckel
b2b2d6e26c Expand docs with new compression options 2024-08-18 17:59:36 +02:00
Sandro Jäckel
99ca560d58 Use configured compression in hydra-compress-logs service 2024-08-18 17:59:36 +02:00
Janne Heß
2c886f51d3 CompressLog: Add zstd compression 2024-08-09 18:52:03 +02:00
Rick van Schijndel
54002f0fcf t/evaluator/evaluate-oom-job.t: always skip, the test always fails
We should look into how to resolve this, but I tried some things and nothing really worked.
Let's put it skipped for now until someone comes along to improve it.
2024-07-31 17:15:02 +02:00
Rick van Schijndel
a6b14369ee t/test.pl: increase event-timeout, set qvf
Only log issues/failures when something's actually up.
It has irked me for a long time that so much output came
out of running the tests, this seems to silence it.
It does hide some warnings, but I think it makes the output
so much more readable that it's worth the tradeoff.

Helps for highly parallel running of jobs, sometimes they'd not give output for a while.
Setting this timeout higher appears to help.
Not completely sure if this is the right place to do it, but it works fine for me.
2024-07-31 17:15:02 +02:00
Rick van Schijndel
578a3d2292 t: increase timeouts for slow commands with high load
We've seen many fails on ofborg, at lot of them ultimately appear to come down to
a timeout being hit, resulting in something like this:

Failure executing slapadd -F /<path>/slap.d -b dc=example -l /<path>/load.ldif.

Hopefully this resolves it for most cases.
I've done some endurance testing and this helps a lot.
some other commands also regularly time-out with high load:

- hydra-init
- hydra-create-user
- nix-store --delete

This should address most issues with tests randomly failing.

Used the following script for endurance testing:

```

import os
import subprocess

run_counter = 0
fail_counter = 0

while True:
    try:
        run_counter += 1
        print(f"Starting run {run_counter}")
        env = os.environ
        env["YATH_JOB_COUNT"] = "20"
        result = subprocess.run(["perl", "t/test.pl"], env=env)
        if (result.returncode != 0):
            fail_counter += 1
        print(f"Finish run {run_counter}, total fail count: {fail_counter}")
    except KeyboardInterrupt:
        print(f"Finished {run_counter} runs with {fail_counter} fails")
        break
```

In case someone else wants to do it on their system :).
Note that YATH_JOB_COUNT may need to be changed loosely based on your
cores.
I only have 4 cores (8 threads), so for others higher numbers might
yield better results in hashing out unstable tests.
2024-07-31 17:13:28 +02:00
Niklas Hambüchen
bc19e7cd65 renderInputDiff: Increase git hash length 8 -> 12
See investigation on lengths required to be conflict-free in practice:

https://github.com/NixOS/hydra/pull/1258#issuecomment-1321891677
2024-07-20 23:45:12 +02:00
John Ericson
879ceb5cdc flake.lock: Update
Flake lock file updates:

• Updated input 'nix':
    'github:NixOS/nix/60824fa97c588a0faf68ea61260a47e388b0a4e5' (2024-04-11)
  → 'github:NixOS/nix/1ebc34e9c54b740ea4f4466443047d709dccf5b2' (2024-05-16)
2024-05-21 18:23:35 -04:00
John Ericson
898ca2f600 Merge branch 'nix-2.20' into nix-2.21 2024-05-21 18:23:04 -04:00
John Ericson
21044bc4d9 flake.lock: Update
Flake lock file updates:

• Updated input 'nix':
    'github:NixOS/nix/8f42912c80c0a03f62f6a3d28a3af05a9762565d' (2024-01-30)
  → 'github:NixOS/nix/ab48ea416a203e9ccefb70aa634e27477e4c1ac4' (2024-05-15)
2024-05-21 18:01:30 -04:00
31 changed files with 148 additions and 40 deletions

View File

@@ -1,6 +1,6 @@
# The `default.nix` in flake-compat reads `flake.nix` and `flake.lock` from `src` and
# returns an attribute set of the shape `{ defaultNix, shellNix }`
(import (fetchTarball https://github.com/edolstra/flake-compat/archive/master.tar.gz) {
(import (fetchTarball "https://github.com/edolstra/flake-compat/archive/master.tar.gz") {
src = ./.;
}).defaultNix

View File

@@ -208,7 +208,8 @@ Example configuration:
<role_mapping>
# Make all users in the hydra_admin group Hydra admins
hydra_admin = admin
# Allow all users in the dev group to restart jobs and cancel builds
# Allow all users in the dev group to eval jobsets, restart jobs and cancel builds
dev = eval-jobset
dev = restart-jobs
dev = cancel-build
</role_mapping>

View File

@@ -15,12 +15,18 @@ and dependencies can be found:
$ nix-shell
```
of when flakes are enabled:
```console
$ nix develop
```
To build Hydra, you should then do:
```console
[nix-shell]$ autoreconfPhase
[nix-shell]$ configurePhase
[nix-shell]$ make
[nix-shell]$ make -j$(nproc)
```
You start a local database, the webserver, and other components with

View File

@@ -42,7 +42,7 @@ Sets CircleCI status.
## Compress build logs
Compresses build logs after a build with bzip2.
Compresses build logs after a build with bzip2 or zstd.
### Configuration options
@@ -50,6 +50,14 @@ Compresses build logs after a build with bzip2.
Enable log compression
- `compress_build_logs_compression`
Which compression format to use. Valid values are bzip2 (default) and zstd.
- `compress_build_logs_silent`
Whether to compress logs silently.
### Example
```xml

View File

@@ -70,7 +70,7 @@ paths:
$ref: '#/components/examples/projects-success'
/api/push:
put:
post:
summary: trigger jobsets
parameters:
- in: query

View File

@@ -408,6 +408,7 @@ in
requires = [ "hydra-init.service" ];
after = [ "hydra-init.service" ];
restartTriggers = [ hydraConf ];
path = [ pkgs.zstd ];
environment = env // {
PGPASSFILE = "${baseDir}/pgpass-queue-runner"; # grrr
HYDRA_DBI = "${env.HYDRA_DBI};application_name=hydra-notify";
@@ -458,10 +459,17 @@ in
# logs automatically after a step finishes, but this doesn't work
# if the queue runner is stopped prematurely.
systemd.services.hydra-compress-logs =
{ path = [ pkgs.bzip2 ];
{ path = [ pkgs.bzip2 pkgs.zstd ];
script =
''
find ${baseDir}/build-logs -type f -name "*.drv" -mtime +3 -size +0c | xargs -r bzip2 -v -f
set -eou pipefail
compression=$(sed -nr 's/compress_build_logs_compression = ()/\1/p' ${baseDir}/hydra.conf)
if [[ $compression == "" ]]; then
compression="bzip2"
elif [[ $compression == zstd ]]; then
compression="zstd --rm"
fi
find ${baseDir}/build-logs -type f -name "*.drv" -mtime +3 -size +0c | xargs -r "$compression" --force --quiet
'';
startAt = "Sun 01:45";
};

View File

@@ -164,14 +164,14 @@ void State::parseMachines(const std::string & contents)
? string2Int<MaxJobs>(tokens[3]).value()
: 1,
// `speedFactor`
atof(tokens[4].c_str()),
std::stof(tokens[4].c_str()),
// `supportedFeatures`
std::move(supportedFeatures),
// `mandatoryFeatures`
std::move(mandatoryFeatures),
// `sshPublicHostKey`
tokens[7] != "" && tokens[7] != "-"
? base64Decode(tokens[7])
? tokens[7]
: "",
});

View File

@@ -95,6 +95,7 @@ sub get_legacy_ldap_config {
"hydra_bump-to-front" => [ "bump-to-front" ],
"hydra_cancel-build" => [ "cancel-build" ],
"hydra_create-projects" => [ "create-projects" ],
"hydra_eval-jobset" => [ "eval-jobset" ],
"hydra_restart-jobs" => [ "restart-jobs" ],
},
};
@@ -159,6 +160,7 @@ sub valid_roles {
"bump-to-front",
"cancel-build",
"create-projects",
"eval-jobset",
"restart-jobs",
];
}

View File

@@ -239,6 +239,8 @@ sub triggerJobset {
sub push : Chained('api') PathPart('push') Args(0) {
my ($self, $c) = @_;
requirePost($c);
$c->{stash}->{json}->{jobsetsTriggered} = [];
my $force = exists $c->request->query_params->{force};
@@ -246,19 +248,24 @@ sub push : Chained('api') PathPart('push') Args(0) {
foreach my $s (@jobsets) {
my ($p, $j) = parseJobsetName($s);
my $jobset = $c->model('DB::Jobsets')->find($p, $j);
requireEvalJobsetPrivileges($c, $jobset->project);
next unless defined $jobset && ($force || ($jobset->project->enabled && $jobset->enabled));
triggerJobset($self, $c, $jobset, $force);
}
my @repos = split /,/, ($c->request->query_params->{repos} // "");
foreach my $r (@repos) {
triggerJobset($self, $c, $_, $force) foreach $c->model('DB::Jobsets')->search(
my @jobsets = $c->model('DB::Jobsets')->search(
{ 'project.enabled' => 1, 'me.enabled' => 1 },
{
join => 'project',
where => \ [ 'exists (select 1 from JobsetInputAlts where project = me.project and jobset = me.name and value = ?)', [ 'value', $r ] ],
order_by => 'me.id DESC'
});
foreach my $jobset (@jobsets) {
requireEvalJobsetPrivileges($c, $jobset->project);
triggerJobset($self, $c, $jobset, $force)
}
}
$self->status_ok(

View File

@@ -330,7 +330,7 @@ sub nar :Local :Args(1) {
else {
$path = $Nix::Config::storeDir . "/$path";
gone($c, "Path " . $path . " is no longer available.") unless isValidPath($path);
gone($c, "Path " . $path . " is no longer available.") unless $MACHINE_LOCAL_STORE->isValidPath($path);
$c->stash->{current_view} = 'NixNAR';
$c->stash->{storePath} = $path;

View File

@@ -15,6 +15,7 @@ our @EXPORT = qw(
forceLogin requireUser requireProjectOwner requireRestartPrivileges requireAdmin requirePost isAdmin isProjectOwner
requireBumpPrivileges
requireCancelBuildPrivileges
requireEvalJobsetPrivileges
trim
getLatestFinishedEval getFirstEval
paramToList
@@ -186,6 +187,27 @@ sub isProjectOwner {
defined $c->model('DB::ProjectMembers')->find({ project => $project, userName => $c->user->username }));
}
sub hasEvalJobsetRole {
my ($c) = @_;
return $c->user_exists && $c->check_user_roles("eval-jobset");
}
sub mayEvalJobset {
my ($c, $project) = @_;
return
$c->user_exists &&
(isAdmin($c) ||
hasEvalJobsetRole($c) ||
isProjectOwner($c, $project));
}
sub requireEvalJobsetPrivileges {
my ($c, $project) = @_;
requireUser($c);
accessDenied($c, "Only the project members, administrators, and accounts with eval-jobset privileges can perform this operation.")
unless mayEvalJobset($c, $project);
}
sub hasCancelBuildRole {
my ($c) = @_;
return $c->user_exists && $c->check_user_roles('cancel-build');
@@ -272,7 +294,7 @@ sub requireAdmin {
sub requirePost {
my ($c) = @_;
error($c, "Request must be POSTed.") if $c->request->method ne "POST";
error($c, "Request must be POSTed.", 405) if $c->request->method ne "POST";
}

View File

@@ -174,6 +174,9 @@ sub getDrvLogPath {
for ($fn . $bucketed, $fn . $bucketed . ".bz2") {
return $_ if -f $_;
}
for ($fn . $bucketed, $fn . $bucketed . ".zst") {
return $_ if -f $_;
}
return undef;
}

View File

@@ -9,11 +9,24 @@ use Hydra::Helper::CatalystUtils;
sub stepFinished {
my ($self, $step, $logPath) = @_;
my $doCompress = $self->{config}->{'compress_build_logs'} // "1";
my $doCompress = $self->{config}->{'compress_build_logs'} // '1';
my $silent = $self->{config}->{'compress_build_logs_silent'} // '0';
my $compression = $self->{config}->{'compress_build_logs_compression'} // 'bzip2';
if ($doCompress eq "1" && -e $logPath) {
print STDERR "compressing $logPath...\n";
system("bzip2", "--force", $logPath);
if (not -e $logPath or $doCompress ne "1") {
return;
}
if ($silent ne '1') {
print STDERR "compressing '$logPath' with $compression...\n";
}
if ($compression eq 'bzip2') {
system('bzip2', '--force', $logPath);
} elsif ($compression eq 'zstd') {
system('zstd', '--rm', '--quiet', '-T0', $logPath);
} else {
print STDERR "unknown compression type '$compression'\n";
}
}

View File

@@ -92,7 +92,7 @@ sub buildFinished {
my $hash = substr basename($path), 0, 32;
my ($deriver, $narHash, $time, $narSize, $refs) = queryPathInfo($path, 0);
my $system;
if (defined $deriver and isValidPath($deriver)) {
if (defined $deriver and $MACHINE_LOCAL_STORE->isValidPath($deriver)) {
$system = derivationFromPath($deriver)->{platform};
}
foreach my $reference (@{$refs}) {

View File

@@ -46,7 +46,7 @@ sub fetchInput {
$MACHINE_LOCAL_STORE->addTempRoot($cachedInput->storepath) if defined $cachedInput;
if (defined $cachedInput && isValidPath($cachedInput->storepath)) {
if (defined $cachedInput && $MACHINE_LOCAL_STORE->isValidPath($cachedInput->storepath)) {
$storePath = $cachedInput->storepath;
$sha256 = $cachedInput->sha256hash;
} else {

View File

@@ -6,6 +6,8 @@ use File::Basename;
use Hydra::Helper::CatalystUtils;
use MIME::Base64;
use Nix::Manifest;
use Nix::Store;
use Nix::Utils;
use Hydra::Helper::Nix;
use base qw/Catalyst::View/;

View File

@@ -16,7 +16,10 @@ sub process {
my $tail = int($c->stash->{tail} // "0");
if ($logPath =~ /\.bz2$/) {
if ($logPath =~ /\.zst$/) {
my $doTail = $tail ? "| tail -n '$tail'" : "";
open($fh, "-|", "zstd -dc < '$logPath' $doTail") or die;
} elsif ($logPath =~ /\.bz2$/) {
my $doTail = $tail ? "| tail -n '$tail'" : "";
open($fh, "-|", "bzip2 -dc < '$logPath' $doTail") or die;
} else {

View File

@@ -374,7 +374,7 @@ BLOCK renderInputDiff; %]
[% ELSIF bi1.uri == bi2.uri && bi1.revision != bi2.revision %]
[% IF bi1.type == "git" %]
<tr><td>
<b>[% bi1.name %]</b></td><td><tt>[% INCLUDE renderDiffUri contents=(bi1.revision.substr(0, 8) _ ' to ' _ bi2.revision.substr(0, 8)) %]</tt>
<b>[% bi1.name %]</b></td><td><tt>[% INCLUDE renderDiffUri contents=(bi1.revision.substr(0, 12) _ ' to ' _ bi2.revision.substr(0, 12)) %]</tt>
</td></tr>
[% ELSE %]
<tr><td>

View File

@@ -205,6 +205,7 @@
if (!c) return;
requestJSON({
url: "[% HTML.escape(c.uri_for('/api/push', { jobsets = project.name _ ':' _ jobset.name, force = "1" })) %]",
type: 'POST',
success: function(data) {
bootbox.alert("The jobset has been scheduled for evaluation.");
}

View File

@@ -91,6 +91,7 @@
[% INCLUDE roleoption mutable=mutable role="restart-jobs" %]
[% INCLUDE roleoption mutable=mutable role="bump-to-front" %]
[% INCLUDE roleoption mutable=mutable role="cancel-build" %]
[% INCLUDE roleoption mutable=mutable role="eval-jobset" %]
</p>
</div>
</div>

View File

@@ -57,6 +57,7 @@ subtest "getLDAPConfig" => sub {
"hydra_cancel-build" => [ "cancel-build" ],
"hydra_create-projects" => [ "create-projects" ],
"hydra_restart-jobs" => [ "restart-jobs" ],
"hydra_eval-jobset" => [ "eval-jobset" ],
}
},
"The empty file and set env var make legacy mode active."
@@ -177,6 +178,7 @@ subtest "get_legacy_ldap_config" => sub {
"hydra_cancel-build" => [ "cancel-build" ],
"hydra_create-projects" => [ "create-projects" ],
"hydra_restart-jobs" => [ "restart-jobs" ],
"hydra_eval-jobset" => [ "eval-jobset" ],
}
},
"Legacy, default role maps are applied."

View File

@@ -22,9 +22,24 @@ sub is_json {
}
my $ctx = test_context();
Catalyst::Test->import('Hydra');
# Create a user to log in to
my $user = $ctx->db->resultset('Users')->create({ username => 'alice', emailaddress => 'alice@example.com', password => '!' });
$user->setPassword('foobar');
$user->userroles->update_or_create({ role => 'admin' });
# Login and save cookie for future requests
my $req = request(POST '/login',
Referer => 'http://localhost/',
Content => {
username => 'alice',
password => 'foobar'
}
);
is($req->code, 302, "The login redirects");
my $cookie = $req->header("set-cookie");
my $finishedBuilds = $ctx->makeAndEvaluateJobset(
expression => "one-job.nix",
build => 1
@@ -109,7 +124,10 @@ subtest "/api/push" => sub {
my $jobsetName = $jobset->name;
is($jobset->forceeval, undef, "The existing jobset is not set to be forced to eval");
my $response = request(GET "/api/push?jobsets=$projectName:$jobsetName&force=1");
my $response = request(POST "/api/push?jobsets=$projectName:$jobsetName&force=1",
Cookie => $cookie,
Referer => 'http://localhost/',
);
ok($response->is_success, "The API enpdoint for triggering jobsets returns 200.");
my $data = is_json($response);
@@ -128,7 +146,10 @@ subtest "/api/push" => sub {
print STDERR $repo;
my $response = request(GET "/api/push?repos=$repo&force=1");
my $response = request(POST "/api/push?repos=$repo&force=1",
Cookie => $cookie,
Referer => 'http://localhost/',
);
ok($response->is_success, "The API enpdoint for triggering jobsets returns 200.");
my $data = is_json($response);

View File

@@ -24,6 +24,7 @@ $ldap->add_group("hydra_create-projects", $users->{"many_roles"}->{"username"});
$ldap->add_group("hydra_restart-jobs", $users->{"many_roles"}->{"username"});
$ldap->add_group("hydra_bump-to-front", $users->{"many_roles"}->{"username"});
$ldap->add_group("hydra_cancel-build", $users->{"many_roles"}->{"username"});
$ldap->add_group("hydra_eval-jobset", $users->{"many_roles"}->{"username"});
my $hydra_ldap_config = "${\$ldap->tmpdir()}/hydra_ldap_config.yaml";
LDAPContext::write_file($hydra_ldap_config, <<YAML);
@@ -68,7 +69,7 @@ subtest "Valid login attempts" => sub {
unrelated => [],
admin => ["admin"],
not_admin => [],
many_roles => [ "create-projects", "restart-jobs", "bump-to-front", "cancel-build" ],
many_roles => [ "create-projects", "restart-jobs", "bump-to-front", "cancel-build", "eval-jobset" ],
);
for my $username (keys %users_to_roles) {
my $user = $users->{$username};

View File

@@ -24,6 +24,7 @@ $ldap->add_group("hydra_create-projects", $users->{"many_roles"}->{"username"});
$ldap->add_group("hydra_restart-jobs", $users->{"many_roles"}->{"username"});
$ldap->add_group("hydra_bump-to-front", $users->{"many_roles"}->{"username"});
$ldap->add_group("hydra_cancel-build", $users->{"many_roles"}->{"username"});
$ldap->add_group("hydra_eval-jobset", $users->{"many_roles"}->{"username"});
my $ctx = test_context(
@@ -76,10 +77,12 @@ my $ctx = test_context(
hydra_cancel-build = cancel-build
hydra_bump-to-front = bump-to-front
hydra_restart-jobs = restart-jobs
hydra_eval-jobset = eval-jobset
hydra_one_group_many_roles = create-projects
hydra_one_group_many_roles = cancel-build
hydra_one_group_many_roles = bump-to-front
hydra_one_group_many-roles = eval-jobset
</role_mapping>
</ldap>
CFG
@@ -92,7 +95,7 @@ subtest "Valid login attempts" => sub {
unrelated => [],
admin => ["admin"],
not_admin => [],
many_roles => [ "create-projects", "restart-jobs", "bump-to-front", "cancel-build" ],
many_roles => [ "create-projects", "restart-jobs", "bump-to-front", "cancel-build", "eval-jobset" ],
many_roles_one_group => [ "create-projects", "bump-to-front", "cancel-build" ],
);
for my $username (keys %users_to_roles) {

View File

@@ -31,6 +31,10 @@ if ($sd_res != 0) {
skip_all("`systemd-run` returned non-zero when executing `true` (expected 0)");
}
# XXX(Mindavi): We should think about how to fix this.
# Note that it was always skipped on ofborg/h.n.o (nixos hydra) since systemd-run is not present in the ambient environment there.
skip_all("Always fails, an error about 'oom' being a string is logged and the process never OOMs. Needs a way to use more memory.");
my $ctx = test_context();
# Contain the memory usage to 25 MegaBytes using `systemd-run`

View File

@@ -92,7 +92,7 @@ sub new {
$opts{'before_init'}->($self);
}
expectOkay(5, ("hydra-init"));
expectOkay(30, ("hydra-init"));
return $self;
}

View File

@@ -70,7 +70,7 @@ sub add_user {
my $email = $opts{'email'} // "$name\@example";
my $password = $opts{'password'} // rand_chars();
my ($res, $stdout, $stderr) = captureStdoutStderr(1, ("slappasswd", "-s", $password));
my ($res, $stdout, $stderr) = captureStdoutStderr(5, ("slappasswd", "-s", $password));
if ($res) {
die "Failed to execute slappasswd ($res): $stderr, $stdout";
}
@@ -178,7 +178,7 @@ sub start {
sub validateConfig {
my ($self) = @_;
expectOkay(1, ("slaptest", "-u", "-F", $self->{"_slapd_dir"}));
expectOkay(5, ("slaptest", "-u", "-F", $self->{"_slapd_dir"}));
}
sub _spawn {
@@ -218,7 +218,7 @@ sub load_ldif {
my $path = "${\$self->{'_tmpdir'}}/load.ldif";
write_file($path, $content);
expectOkay(1, ("slapadd", "-F", $self->{"_slapd_dir"}, "-b", $suffix, "-l", $path));
expectOkay(5, ("slapadd", "-F", $self->{"_slapd_dir"}, "-b", $suffix, "-l", $path));
$self->validateConfig();
}

View File

@@ -39,7 +39,7 @@ subtest "Building, caching, and then garbage collecting the underlying job" => s
ok(unlink(Hydra::Helper::Nix::gcRootFor($path)), "Unlinking the GC root for underlying Dependency succeeds");
(my $ret, my $stdout, my $stderr) = captureStdoutStderr(5, "nix-store", "--delete", $path);
(my $ret, my $stdout, my $stderr) = captureStdoutStderr(15, "nix-store", "--delete", $path);
is($ret, 0, "Deleting the underlying dependency should succeed");
};

View File

@@ -9,7 +9,7 @@ my $db = $ctx->db();
subtest "Handling password and password hash creation" => sub {
subtest "Creating a user with a plain text password (insecure) stores the password securely" => sub {
my ($res, $stdout, $stderr) = captureStdoutStderr(5, ("hydra-create-user", "plain-text-user", "--password", "foobar"));
my ($res, $stdout, $stderr) = captureStdoutStderr(15, ("hydra-create-user", "plain-text-user", "--password", "foobar"));
is($res, 0, "hydra-create-user should exit zero");
like($stderr, qr/Submitting plaintext passwords as arguments is deprecated and will be removed/, "Submitting a plain text password is deprecated.");
@@ -23,7 +23,7 @@ subtest "Handling password and password hash creation" => sub {
};
subtest "Creating a user with a sha1 password (still insecure) stores the password as a hashed sha1" => sub {
my ($res, $stdout, $stderr) = captureStdoutStderr(5, ("hydra-create-user", "old-password-hash-user", "--password-hash", "8843d7f92416211de9ebb963ff4ce28125932878"));
my ($res, $stdout, $stderr) = captureStdoutStderr(15, ("hydra-create-user", "old-password-hash-user", "--password-hash", "8843d7f92416211de9ebb963ff4ce28125932878"));
is($res, 0, "hydra-create-user should exit zero");
my $user = $db->resultset('Users')->find({ username => "old-password-hash-user" });
@@ -36,7 +36,7 @@ subtest "Handling password and password hash creation" => sub {
};
subtest "Creating a user with an argon2 password stores the password as given" => sub {
my ($res, $stdout, $stderr) = captureStdoutStderr(5, ("hydra-create-user", "argon2-hash-user", "--password-hash", '$argon2id$v=19$m=262144,t=3,p=1$tMnV5paYjmIrUIb6hylaNA$M8/e0i3NGrjhOliVLa5LqQ'));
my ($res, $stdout, $stderr) = captureStdoutStderr(15, ("hydra-create-user", "argon2-hash-user", "--password-hash", '$argon2id$v=19$m=262144,t=3,p=1$tMnV5paYjmIrUIb6hylaNA$M8/e0i3NGrjhOliVLa5LqQ'));
is($res, 0, "hydra-create-user should exit zero");
my $user = $db->resultset('Users')->find({ username => "argon2-hash-user" });
@@ -50,7 +50,7 @@ subtest "Handling password and password hash creation" => sub {
subtest "Creating a user by prompting for the password" => sub {
subtest "with the same password twice" => sub {
my ($res, $stdout, $stderr) = captureStdoutStderrWithStdin(5, ["hydra-create-user", "prompted-pass-user", "--password-prompt"], "my-password\nmy-password\n");
my ($res, $stdout, $stderr) = captureStdoutStderrWithStdin(15, ["hydra-create-user", "prompted-pass-user", "--password-prompt"], "my-password\nmy-password\n");
is($res, 0, "hydra-create-user should exit zero");
my $user = $db->resultset('Users')->find({ username => "prompted-pass-user" });
@@ -62,7 +62,7 @@ subtest "Handling password and password hash creation" => sub {
};
subtest "With mismatched password confirmation" => sub {
my ($res, $stdout, $stderr) = captureStdoutStderrWithStdin(5, ["hydra-create-user", "prompted-pass-user", "--password-prompt"], "my-password\nnot-my-password\n");
my ($res, $stdout, $stderr) = captureStdoutStderrWithStdin(15, ["hydra-create-user", "prompted-pass-user", "--password-prompt"], "my-password\nnot-my-password\n");
isnt($res, 0, "hydra-create-user should exit non-zero");
};
};
@@ -76,7 +76,7 @@ subtest "Handling password and password hash creation" => sub {
);
for my $case (@cases) {
my ($res, $stdout, $stderr) = captureStdoutStderr(5, (
my ($res, $stdout, $stderr) = captureStdoutStderr(15, (
"hydra-create-user", "bogus-password-options", @{$case}));
like($stderr, qr/please specify only one of --password-prompt or --password-hash/, "We get an error about specifying the password");
isnt($res, 0, "hydra-create-user should exit non-zero with conflicting " . join(" ", @{$case}));
@@ -84,7 +84,7 @@ subtest "Handling password and password hash creation" => sub {
};
subtest "A password is not required for creating a Google-based account" => sub {
my ($res, $stdout, $stderr) = captureStdoutStderr(5, (
my ($res, $stdout, $stderr) = captureStdoutStderr(15, (
"hydra-create-user", "google-account", "--type", "google"));
is($res, 0, "hydra-create-user should exit zero");
};

View File

@@ -28,7 +28,7 @@ subtest "hydra-init upgrades user's password hashes from sha1 to sha1 inside Arg
$janet->setPassword("foobar");
is($alice->password, "8843d7f92416211de9ebb963ff4ce28125932878", "Alices's sha1 is stored in the database");
my ($res, $stdout, $stderr) = captureStdoutStderr(5, ("hydra-init"));
my ($res, $stdout, $stderr) = captureStdoutStderr(30, ("hydra-init"));
if ($res != 0) {
is($stdout, "");
is($stderr, "");
@@ -55,7 +55,7 @@ subtest "hydra-init upgrades user's password hashes from sha1 to sha1 inside Arg
};
subtest "Running hydra-init don't break Alice or Janet's passwords" => sub {
my ($res, $stdout, $stderr) = captureStdoutStderr(5, ("hydra-init"));
my ($res, $stdout, $stderr) = captureStdoutStderr(30, ("hydra-init"));
is($res, 0, "hydra-init should exit zero");
my $updatedAlice = $db->resultset('Users')->find({ username => "alice" });

View File

@@ -21,7 +21,7 @@ if (defined($ENV{"NIX_BUILD_CORES"})
print STDERR "test.pl: Defaulting \$YATH_JOB_COUNT to \$NIX_BUILD_CORES (${\$ENV{'NIX_BUILD_CORES'}})\n";
}
system($^X, find_yath(), '-D', 'test', '--default-search' => './', @ARGV);
system($^X, find_yath(), '-D', 'test', '--qvf', '--event-timeout', 240, '--default-search' => './', @ARGV);
my $exit = $?;
# This makes sure it works with prove.