Compare commits

...

20 Commits

Author SHA1 Message Date
John Ericson
de3646cb13 Merge pull request #1488 from NixOS/nixpkgs-25.05
flake.lock: Update Nixpkgs to 25.05
2025-05-26 01:11:40 +00:00
John Ericson
8a50488f6c flake.lock: Update Nixpkgs to 25.05
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/eea3403f7ca9f9942098f4f2756adab4ec924b2b?narHash=sha256-JT1wMjLIypWJA0N2V27WpUw8feDmTok4Dwkb0oYXDS4%3D' (2025-04-23)
  → 'github:NixOS/nixpkgs/db1aed32009f408e4048c1dd0beaf714dd34ed93?narHash=sha256-8A7HjmnvCpDjmETrZY1QwzKunR63LiP7lHu1eA5q6JI%3D' (2025-05-24)
2025-05-25 20:51:14 -04:00
Martin Weinelt
35c9264306 Merge pull request #1484 from NixOS/nix-keep-options
Migrate from "gc-" prefixed nix options
2025-05-23 22:19:15 +00:00
Martin Weinelt
da1aebe970 Migrate from "gc-" prefixed nix options
These have been deprecated, e.g. gc-keep-outputs is now just
keep-outputs.
2025-05-15 04:08:57 +02:00
Jörg Thalheim
183bc39d1a Merge pull request #1483 from SuperSandro2000/patch-2
Add missing slash
2025-05-09 15:49:33 +00:00
Sandro
2ae27dd20d Add missing slash
error: access to absolute path '/nix/store/sai35xfsrba2a2vasmzxakmn54wdfa13-sourcepackaging' is forbidden in pure evaluation mode (use '--impure' to override)
2025-05-05 00:10:59 +02:00
Jörg Thalheim
1b5c2fb747 Merge pull request #1479 from qowoz/queue-runner
queue runner: attempt at slightly smarter scheduling criteria
2025-05-01 07:02:54 +00:00
Jörg Thalheim
8d068fea3e Merge pull request #1482 from NixOS/hydra-passthru
Expose nix package in hydra package
2025-04-29 18:35:50 +00:00
Jörg Thalheim
8218a9ad1b hydra: expose nix-cli package
This makes it easier in other packages to get the nix version used to
build Hydra.
2025-04-29 20:27:30 +02:00
John Ericson
455f1a0665 Merge pull request #1481 from NixOS/nix-flake-false
Use Nix without the flake
2025-04-23 22:04:54 +00:00
John Ericson
89fcb931ce Use Nix without the flake
This is what we do for `nix-eval-jobs` already. It allows for more
fine-grained control over dependencies.
2025-04-23 17:58:52 -04:00
Martin Weinelt
b023cc8f87 Merge pull request #1480 from NixOS/update-nix
flake.lock: Update
2025-04-23 18:05:14 +00:00
Martin Weinelt
23755bf001 flake.lock: Update
Flake lock file updates:

• Updated input 'nix':
    'github:NixOS/nix/a4962f73b5fc874d4b16baef47921daf349addfc' (2025-04-07)
  → 'github:NixOS/nix/70921714cb3b5e6041b7413459541838651079f3' (2025-04-23)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/db8f4fe18ce772a9c8f3adf321416981c8fe9371' (2025-04-07)
  → 'github:NixOS/nixpkgs/eea3403f7ca9f9942098f4f2756adab4ec924b2b' (2025-04-23)
2025-04-23 18:27:14 +02:00
Pierre Bourdon
720db63d52 queue runner: attempt at slightly smarter scheduling criteria
Instead of just going for "whatever is the oldest build we know of",
use the following first:

- Is the step more constrained? If so, schedule it first to avoid
  filling up "more desirable" build slots with less constrained builds.

- Does the step have more dependents? If so, schedule it first to try
  and maximize open parallelism and breadth of scheduling options.

(cherry picked from commit b8d03adaf4)
2025-04-20 13:44:06 +10:00
John Ericson
bdde73acbd Merge pull request #1478 from qowoz/fix-actions
jobset-eval: fix actions not showing up sometimes for new jobs
2025-04-16 16:49:31 +00:00
Pierre Bourdon
0ab357e435 jobset-eval: fix actions not showing up sometimes for new jobs
New jobs have their "new" status take precedence over them being
"failed" or "queued", which means actions that can act on "failed" or
"queued" jobs weren't shown to the user when they could only act on
"new" jobs.

(cherry picked from commit 9a4a5dd624)
2025-04-16 09:50:32 +10:00
Jörg Thalheim
6fcfa9e796 Merge commit from fork
Re-enable restrict-eval for non-flakes
2025-04-15 06:48:18 +02:00
Martin Weinelt
ffbde9c9e3 Merge pull request #1474 from NixOS/machine-status-colspan
web: increase colspan for machine row in machine status
2025-04-13 14:37:52 +00:00
Martin Weinelt
cf33a9158a web: increase colspan for machine row in machine status 2025-04-13 08:29:01 +02:00
Jörg Thalheim
8d75026513 re-enable restrict-eval for non-flakes 2025-04-11 13:42:55 +02:00
11 changed files with 93 additions and 68 deletions

31
flake.lock generated
View File

@@ -1,22 +1,13 @@
{
"nodes": {
"nix": {
"inputs": {
"flake-compat": [],
"flake-parts": [],
"git-hooks-nix": [],
"nixpkgs": [
"nixpkgs"
],
"nixpkgs-23-11": [],
"nixpkgs-regression": []
},
"flake": false,
"locked": {
"lastModified": 1744030329,
"narHash": "sha256-r+psCOW77vTSTNbxTVrYHeh6OgB0QukbnyUVDwg8s4I=",
"lastModified": 1745420957,
"narHash": "sha256-ZbB3IH9OlJvo14GlQZbYHzJojf/HCDT38GzYTod8DaU=",
"owner": "NixOS",
"repo": "nix",
"rev": "a4962f73b5fc874d4b16baef47921daf349addfc",
"rev": "70921714cb3b5e6041b7413459541838651079f3",
"type": "github"
},
"original": {
@@ -29,11 +20,11 @@
"nix-eval-jobs": {
"flake": false,
"locked": {
"lastModified": 1744018595,
"narHash": "sha256-v5n6t49X7MOpqS9j0FtI6TWOXvxuZMmGsp2OfUK5QfA=",
"lastModified": 1744370057,
"narHash": "sha256-n220U5pjzCtTtOJtbga4Xr/PyllowKw9anSevgCqJEw=",
"owner": "nix-community",
"repo": "nix-eval-jobs",
"rev": "cba718bafe5dc1607c2b6761ecf53c641a6f3b21",
"rev": "1260c6599d22dfd8c25fea6893c3d031996b20e1",
"type": "github"
},
"original": {
@@ -44,16 +35,16 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1743987495,
"narHash": "sha256-46T2vMZ4/AfCK0Y2OjlFzJPxmdpP8GtsuEqSSJv3oe4=",
"lastModified": 1748124805,
"narHash": "sha256-8A7HjmnvCpDjmETrZY1QwzKunR63LiP7lHu1eA5q6JI=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "db8f4fe18ce772a9c8f3adf321416981c8fe9371",
"rev": "db1aed32009f408e4048c1dd0beaf714dd34ed93",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-24.11-small",
"ref": "nixos-25.05-small",
"repo": "nixpkgs",
"type": "github"
}

View File

@@ -1,18 +1,12 @@
{
description = "A Nix-based continuous build system";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.11-small";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.05-small";
inputs.nix = {
url = "github:NixOS/nix/2.28-maintenance";
inputs.nixpkgs.follows = "nixpkgs";
# hide nix dev tooling from our lock file
inputs.flake-parts.follows = "";
inputs.git-hooks-nix.follows = "";
inputs.nixpkgs-regression.follows = "";
inputs.nixpkgs-23-11.follows = "";
inputs.flake-compat.follows = "";
# We want to control the deps precisely
flake = false;
};
inputs.nix-eval-jobs = {
@@ -30,10 +24,27 @@
# A Nixpkgs overlay that provides a 'hydra' package.
overlays.default = final: prev: {
nix-eval-jobs = final.callPackage nix-eval-jobs {};
nixDependenciesForHydra = final.lib.makeScope final.newScope
(import (nix + "/packaging/dependencies.nix") {
pkgs = final;
inherit (final) stdenv;
inputs = {};
});
nixComponentsForHydra = final.lib.makeScope final.nixDependenciesForHydra.newScope
(import (nix + "/packaging/components.nix") {
officialRelease = true;
inherit (final) lib;
pkgs = final;
src = nix;
maintainers = [ ];
});
nix-eval-jobs = final.callPackage nix-eval-jobs {
nixComponents = final.nixComponentsForHydra;
};
hydra = final.callPackage ./package.nix {
inherit (nixpkgs.lib) fileset;
inherit (final.lib) fileset;
rawSrc = self;
nixComponents = final.nixComponentsForHydra;
};
};
@@ -73,24 +84,26 @@
});
packages = forEachSystem (system: let
nixComponents = {
inherit (nix.packages.${system})
nix-util
nix-store
nix-expr
nix-fetchers
nix-flake
nix-main
nix-cmd
nix-cli
nix-perl-bindings
;
};
inherit (nixpkgs) lib;
pkgs = nixpkgs.legacyPackages.${system};
nixDependencies = lib.makeScope pkgs.newScope
(import (nix + "/packaging/dependencies.nix") {
inherit pkgs;
inherit (pkgs) stdenv;
inputs = {};
});
nixComponents = lib.makeScope nixDependencies.newScope
(import (nix + "/packaging/components.nix") {
officialRelease = true;
inherit lib pkgs;
src = nix;
maintainers = [ ];
});
in {
nix-eval-jobs = nixpkgs.legacyPackages.${system}.callPackage nix-eval-jobs {
nix-eval-jobs = pkgs.callPackage nix-eval-jobs {
inherit nixComponents;
};
hydra = nixpkgs.legacyPackages.${system}.callPackage ./package.nix {
hydra = pkgs.callPackage ./package.nix {
inherit (nixpkgs.lib) fileset;
inherit nixComponents;
inherit (self.packages.${system}) nix-eval-jobs;

View File

@@ -228,8 +228,8 @@ in
nix.settings = {
trusted-users = [ "hydra-queue-runner" ];
gc-keep-outputs = true;
gc-keep-derivations = true;
keep-outputs = true;
keep-derivations = true;
};
services.hydra-dev.extraConfig =

View File

@@ -277,5 +277,8 @@ stdenv.mkDerivation (finalAttrs: {
dontStrip = true;
meta.description = "Build of Hydra on ${stdenv.system}";
passthru = { inherit perlDeps; };
passthru = {
inherit perlDeps;
nix = nixComponents.nix-cli;
};
})

View File

@@ -134,6 +134,8 @@ system_time State::doDispatch()
comparator is a partial ordering (see MachineInfo). */
int highestGlobalPriority;
int highestLocalPriority;
size_t numRequiredSystemFeatures;
size_t numRevDeps;
BuildID lowestBuildID;
StepInfo(Step::ptr step, Step::State & step_) : step(step)
@@ -142,6 +144,8 @@ system_time State::doDispatch()
lowestShareUsed = std::min(lowestShareUsed, jobset->shareUsed());
highestGlobalPriority = step_.highestGlobalPriority;
highestLocalPriority = step_.highestLocalPriority;
numRequiredSystemFeatures = step->requiredSystemFeatures.size();
numRevDeps = step_.rdeps.size();
lowestBuildID = step_.lowestBuildID;
}
};
@@ -194,6 +198,8 @@ system_time State::doDispatch()
a.highestGlobalPriority != b.highestGlobalPriority ? a.highestGlobalPriority > b.highestGlobalPriority :
a.lowestShareUsed != b.lowestShareUsed ? a.lowestShareUsed < b.lowestShareUsed :
a.highestLocalPriority != b.highestLocalPriority ? a.highestLocalPriority > b.highestLocalPriority :
a.numRequiredSystemFeatures != b.numRequiredSystemFeatures ? a.numRequiredSystemFeatures > b.numRequiredSystemFeatures :
a.numRevDeps != b.numRevDeps ? a.numRevDeps > b.numRevDeps :
a.lowestBuildID < b.lowestBuildID;
});

View File

@@ -76,7 +76,9 @@ sub view_GET {
$c->stash->{removed} = $diff->{removed};
$c->stash->{unfinished} = $diff->{unfinished};
$c->stash->{aborted} = $diff->{aborted};
$c->stash->{failed} = $diff->{failed};
$c->stash->{totalAborted} = $diff->{totalAborted};
$c->stash->{totalFailed} = $diff->{totalFailed};
$c->stash->{totalQueued} = $diff->{totalQueued};
$c->stash->{full} = ($c->req->params->{full} || "0") eq "1";

View File

@@ -32,7 +32,12 @@ sub buildDiff {
removed => [],
unfinished => [],
aborted => [],
failed => [],
# These summary counters cut across the categories to determine whether
# actions such as "Restart all failed" or "Bump queue" are available.
totalAborted => 0,
totalFailed => 0,
totalQueued => 0,
};
my $n = 0;
@@ -80,8 +85,15 @@ sub buildDiff {
} else {
push @{$ret->{new}}, $build if !$found;
}
if (defined $build->buildstatus && $build->buildstatus != 0) {
push @{$ret->{failed}}, $build;
if ($build->finished != 0 && $build->buildstatus != 0) {
if ($aborted) {
++$ret->{totalAborted};
} else {
++$ret->{totalFailed};
}
} elsif ($build->finished == 0) {
++$ret->{totalQueued};
}
}

View File

@@ -48,16 +48,16 @@ c.uri_for(c.controller('JobsetEval').action_for('view'),
<a class="nav-link dropdown-toggle" data-toggle="dropdown" href="#">Actions</a>
<div class="dropdown-menu">
<a class="dropdown-item" href="[% c.uri_for(c.controller('JobsetEval').action_for('create_jobset'), [eval.id]) %]">Create a jobset from this evaluation</a>
[% IF unfinished.size > 0 %]
[% IF totalQueued > 0 %]
<a class="dropdown-item" href="[% c.uri_for(c.controller('JobsetEval').action_for('cancel'), [eval.id]) %]">Cancel all scheduled builds</a>
[% END %]
[% IF aborted.size > 0 || stillFail.size > 0 || nowFail.size > 0 || failed.size > 0 %]
[% IF totalFailed > 0 %]
<a class="dropdown-item" href="[% c.uri_for(c.controller('JobsetEval').action_for('restart_failed'), [eval.id]) %]">Restart all failed builds</a>
[% END %]
[% IF aborted.size > 0 %]
[% IF totalAborted > 0 %]
<a class="dropdown-item" href="[% c.uri_for(c.controller('JobsetEval').action_for('restart_aborted'), [eval.id]) %]">Restart all aborted builds</a>
[% END %]
[% IF unfinished.size > 0 %]
[% IF totalQueued > 0 %]
<a class="dropdown-item" href="[% c.uri_for(c.controller('JobsetEval').action_for('bump'), [eval.id]) %]">Bump builds to front of queue</a>
[% END %]
</div>

View File

@@ -17,7 +17,7 @@
[% name = m.key ? stripSSHUser(m.key) : "localhost" %]
<thead>
<tr>
<th colspan="6">
<th colspan="7">
<tt [% IF m.value.disabled %]style="text-decoration: line-through;"[% END %]>[% INCLUDE renderMachineName machine=m.key %]</tt>
[% IF m.value.systemTypes %]
<span class="muted" style="font-weight: normal;">

View File

@@ -372,6 +372,7 @@ sub evalJobs {
or die "cannot find the input containing the job expression\n";
@cmd = ("nix-eval-jobs",
"--option", "restrict-eval", "true",
"<" . $nixExprInputName . "/" . $nixExprPath . ">",
inputsToArgs($inputInfo));
}

View File

@@ -25,7 +25,10 @@ subtest "empty diff" => sub {
removed => [],
unfinished => [],
aborted => [],
failed => [],
totalAborted => 0,
totalFailed => 0,
totalQueued => 0,
},
"empty list of jobs returns empty diff"
);
@@ -48,12 +51,7 @@ subtest "2 different jobs" => sub {
"succeed_with_failed is a new job"
);
is(scalar(@{$ret->{failed}}), 1, "list of failed jobs is 1 element long");
is(
$ret->{failed}[0]->get_column('id'),
$builds->{"succeed_with_failed"}->get_column('id'),
"succeed_with_failed is a failed job"
);
is($ret->{totalFailed}, 1, "total failed jobs is 1");
is(
$ret->{removed},
@@ -70,9 +68,9 @@ subtest "2 different jobs" => sub {
subtest "failed job with no previous history" => sub {
my $ret = buildDiff([$builds->{"fails"}], []);
is(scalar(@{$ret->{failed}}), 1, "list of failed jobs is 1 element long");
is($ret->{totalFailed}, 1, "total failed jobs is 1");
is(
$ret->{failed}[0]->get_column('id'),
$ret->{new}[0]->get_column('id'),
$builds->{"fails"}->get_column('id'),
"fails is a failed job"
);
@@ -93,7 +91,6 @@ subtest "not-yet-built job with no previous history" => sub {
is($ret->{removed}, [], "removed");
is($ret->{unfinished}, [], "unfinished");
is($ret->{aborted}, [], "aborted");
is($ret->{failed}, [], "failed");
is(scalar(@{$ret->{new}}), 1, "list of new jobs is 1 element long");
is(