ahtlon/hydra
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,609 Commits 36 Branches 0 Tags
0e337e6f9c7ea9edfc64e9975b35faac1bd2b708
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Eelco DolstraandGitHub be2791330e Merge pull request #641 from basvandijk/fix-linux-build-on-darwin 
Fix Linux build on Darwin
 2019-03-15 21:33:00 +08:00
Eelco DolstraandGitHub 8b5948f4cf Merge pull request #634 from NixOS/fix-build 
tests: disable sandbox
 2019-02-01 09:17:39 +01:00
Eelco DolstraandGitHub 5dd4736aed Merge pull request #632 from nlewo/localhost-error-message 
hydra-queue-runner: better error message if nix-store can not be started
 2019-01-23 10:46:30 +01:00
Eelco DolstraandGitHub f5686e0958 Merge pull request #633 from samueldr/feature/dev-server 
Add hydra-dev-server which uses the classic Catalyst server
 2019-01-23 10:39:47 +01:00
Eelco DolstraandGitHub 804c29703f Merge pull request #631 from samueldr/feature/dates 
Enhances user-facing dates
 2019-01-23 10:36:40 +01:00
Eelco DolstraandGitHub e0d8dcfe2d Merge pull request #619 from samueldr/feature/lazy_errors 
Adds error messages to lazy tabs
 2019-01-20 23:22:17 +01:00
Eelco Dolstra 423c0440ea Typo 2018-12-20 12:07:02 +01:00
Eelco DolstraandGitHub cd234f6a14 Merge pull request #529 from bennofs/feat-all-builds 
feat: add /eval/<id>/builds endpoint
 2018-11-19 17:36:19 +01:00
Eelco DolstraandGitHub f9870b5fce Merge pull request #612 from input-output-hk/devops-1126 
[DEVOPS-1126] throttle github status calls to remain under api ratelimits
 2018-11-12 10:09:38 +01:00
Eelco DolstraandGitHub 1c44de1779 Merge pull request #567 from phile314/bitbucket_pulls 
Add BitBucket pull request support
 2018-11-06 10:48:37 +01:00
Eelco Dolstra 8d26144121 Fix building against nix master 2018-10-30 14:41:21 +01:00
Eelco DolstraandGitHub e0f204f3da Merge pull request #599 from tomberek/19.03_fixes 
submodule fix
 2018-10-03 09:20:02 +02:00
Eelco DolstraandGitHub 0bc548ee2d Merge pull request #595 from kquick/darcs_cache_dir 
Fix darcs input to use darcs-specific SCM cache dir.
 2018-09-10 10:51:08 +02:00
Eelco DolstraandGitHub b7c42a38a3 Merge pull request #575 from Ma27/fix-tab-jobset-view 
jobset view: jump back to the proper tab when displaying all failed jobsets
 2018-09-06 21:11:59 +02:00
Eelco DolstraandGitHub b15288bea4 Merge pull request #590 from Taneb/one-runcommand 
Allow for precisely one instance of RunCommand plugin
 2018-08-21 17:00:23 +02:00
Eelco Dolstra b542b73ced hydra-eval-jobs: Reinitialize Boehm GC in the child 
This is necessary to prevent hangs in the garbage collector.
 2018-08-10 20:25:43 +02:00
Eelco Dolstra 7876cf677c Increase maxHeapSize more gradually 2018-08-10 19:23:31 +02:00
Eelco Dolstra 4dca8fe14d Gradually increase maxHeapSize 2018-08-07 11:47:53 +02:00
Eelco Dolstra e122d3bef3 RunCommand: Return metrics as a float 
Apparently, DBIx::Class doesn't handle columns with type 'double
precision' properly.
 2018-08-02 12:31:28 +02:00
Eelco Dolstra 6db2cbf094 Add a plugin to execute arbitrary commands when a build finishes 
The plugin can be configured using one or more <runcommand> sections
in hydra.conf, e.g.

<runcommand>
  command = echo Build finished
</runcommand>

Optionally, the command can be executed for specific
projects/jobsets/jobs:

  job = patchelf:master:tarball

or

  job = patchelf:*:*

The default is *:*:*.

The command is executed with the environment variable $HYDRA_JSON
pointing to a JSON file containing info about the build, e.g.

  {
    "build": 3772978,
    "buildStatus": 0,
    "drvPath": "/nix/store/9y4h1fyx9pl3ic08i2f09239b90x1lww-patchelf-tarball-0.8pre894_ed92f9f.drv",
    "event": "buildFinished",
    "finished": 1,
    "job": "tarball",
    "jobset": "master",
    "metrics": [
      {
        "name": "random1",
        "unit": null,
        "value": "20282"
      },
      {
        "name": "random2",
        "unit": "KiB",
        "value": "6664"
      }
    ],
    "outputs": [
      {
        "name": "out",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f"
      }
    ],
    "products": [
      {
        "defaultPath": "",
        "fileSize": 148216,
        "name": "patchelf-0.8pre894_ed92f9f.tar.gz",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f/tarballs/patchelf-0.8pre894_ed92f9f.tar.gz",
        "productNr": 4,
        "sha1hash": "9f27d18382436a7f743f6c2f6ad66e1b536ab4c8",
        "sha256hash": "b04faef2916c411f10711b58ea26965df7cb860ca33a87f1e868051b874c44b3",
        "subtype": "source-dist",
        "type": "file"
      },
      {
        "defaultPath": "",
        "fileSize": 121279,
        "name": "patchelf-0.8pre894_ed92f9f.tar.bz2",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f/tarballs/patchelf-0.8pre894_ed92f9f.tar.bz2",
        "productNr": 3,
        "sha1hash": "7a664841fb779dec19023be6a6121e0398067b7c",
        "sha256hash": "c81e36099893f541a11480f869fcdebd2fad3309900519065c8745f614dd024a",
        "subtype": "source-dist",
        "type": "file"
      },
      {
        "defaultPath": "README",
        "fileSize": null,
        "name": "",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f",
        "productNr": 2,
        "sha1hash": null,
        "sha256hash": null,
        "subtype": "readme",
        "type": "doc"
      },
      {
        "defaultPath": "",
        "fileSize": 6230,
        "name": "README",
        "path": "/nix/store/39h5xciz5pnh1aypmr3rpdx0536y5s2w-patchelf-tarball-0.8pre894_ed92f9f/README",
        "productNr": 1,
        "sha1hash": "dc6bb09093183ab52d7e6a35b72d179869bd6fbf",
        "sha256hash": "5371aee9de0216b3ea2d5ea869da9d5ee441b99156a99055e7e11e7a705f7920",
        "subtype": "readme",
        "type": "doc"
      }
    ],
    "project": "patchelf",
    "startTime": 1533137091,
    "stopTime": 1533137094,
    "timestamp": 1533136076
  }

So for example, the following command:

  command = echo Build $(jq -r .build $HYDRA_JSON) \($(jq -r .project $HYDRA_JSON):$(jq -r .jobset $HYDRA_JSON):$(jq -r .job $HYDRA_JSON)\) finished, metrics: $(jq -r '.metrics[].value' $HYDRA_JSON).

will print

  Build 3772978 (patchelf:master:tarball) finished, metrics: 20282 6664.
 2018-08-01 19:43:50 +02:00
Eelco Dolstra 8e17a413f5 hydra-notify step-finished: Don't barf if the step has no log file 2018-08-01 17:17:46 +02:00
Eelco Dolstra dca6c5a983 hydra-eval-jobs: Filter ANSI escape sequences 2018-08-01 17:06:32 +02:00
Eelco Dolstra 1898e31ed8 hydra-eval-jobset: Respect email_notification 2018-08-01 16:56:53 +02:00
Eelco Dolstra a71e6301d2 Fix Hydra VM tests 2018-07-31 21:49:05 +02:00
Eelco Dolstra 1a899890ac Use nixUnstable 2018-07-31 21:19:10 +02:00
Eelco Dolstra 162d671c48 Revert "Fix broken build.x86_64-linux job (#573)" 
This reverts commit c2e2b42877, which
breaks building with nixUnstable.
 2018-07-31 10:23:41 +02:00
Eelco Dolstra b11789f399 Add boost dependency 2018-06-05 13:24:23 +02:00
Eelco Dolstra 0882519b10 hydra-eval-jobs: Ugly hackery to reduce memory usage 
You can now set 'evaluator_max_heap_size' to make hydra-eval-jobs
restart itself if the Boehm heap exceeds the specified size.

For example, with 'evaluator_max_heap_size = 256000000',

  $ hydra-eval-jobs '<nixpkgs/pkgs/top-level/release.nix>' -I nixpkgs=channel:nixos-17.09

has a max RSS of .56 GiB rather than 4.7 GiB.

Unfortunately it doesn't help much for the NixOS jobsets because of
the "tested" job which requires a huge amount of memory all by itself.
 2018-06-05 13:24:23 +02:00
Eelco Dolstra 821f8bf5a0 release.nix: Use fetchGit to get Nixpkgs 2018-06-05 13:24:23 +02:00
Eelco Dolstra 39eca8bef1 Fix build against Nix master 2018-06-05 13:24:23 +02:00
Eelco Dolstra 09a4c9fad2 hydra-evaluator: Allow setting the maximum number of concurrent evaluations 2018-05-16 14:22:00 +02:00
Eelco Dolstra 4e27796eba Allow setting GC_INITIAL_HEAP_SIZE for hydra-eval-jobs 
This cannot be done in the hydra-evaluator systemd unit, since then
every other Nix process (e.g. hydra-evaluator and nix-prefetch-*) will
also allocate the specified heap size, probably leading to OOM.
 2018-05-16 14:14:53 +02:00
Eelco Dolstra 691f7e168c Fix building on Nixpkgs 17.09 2018-05-10 15:11:24 +02:00
Eelco Dolstra 215b724e44 Fix passing arguments to the reproduction script 2018-05-10 15:11:08 +02:00
Eelco Dolstra 85524b2d0f Disable jobset deletion 
This is a good way to make Hydra hang. (E.g. we had a deletion of
nixos:gcc-7 running for > 12 hours and blocking UPDATE statements from
hydra-queue-runner.) Generally it's better to just disable/hide an old
jobset anyway.
 2018-03-13 14:05:55 +01:00
Eelco Dolstra 02888105a8 Add a hydra.conf option to enable email notification 
Note that it's disabled by default.
 2018-03-12 14:04:23 +01:00
Eelco Dolstra c0fac52872 Add some debug code 2018-03-07 10:23:43 +01:00
Eelco Dolstra 5a1f2a50e5 Handle derivations with system type 'builtin' 
Fixes #540.
 2018-03-07 10:22:35 +01:00
Eelco Dolstra 68afa2bf6f Dump more system info in /queue-runner-status 2018-03-07 10:06:56 +01:00
Eelco Dolstra cb43da06f3 Use Nix 2.0 and default GCC 2018-03-07 09:52:18 +01:00
Eelco Dolstra 3ec73c93e2 Remove unused dependency on aws-sdk-cpp 2018-02-23 16:47:24 +01:00
Eelco Dolstra fd463ff403 Remove nix-install-package support 
'nix-install-package' no longer exists in Nix 1.12.

Closes #268.
 2018-01-15 14:27:58 +01:00
Eelco Dolstra 8913c682cf Doh 2017-12-25 15:02:17 +01:00
Eelco Dolstra 0d79f16fd5 git-export -> source 2017-12-25 14:31:30 +01:00
Eelco Dolstra 7ccec0900b Fix links to steps in the machines page 2017-12-07 16:20:23 +01:00
Eelco Dolstra e9670641ec Distinguish build step states 
The web interface now shows whether a build step is connecting,
copying inputs/outputs, building, etc.
 2017-12-07 15:35:31 +01:00
Eelco Dolstra 457483ba0e Don't lock the BuildSteps table when inserting 
Instead, optimistically insert a row and retry if there is a conflict.
 2017-12-07 14:41:29 +01:00
Eelco Dolstra 2660c3cc54 Remove redundant check 2017-12-05 16:00:18 +01:00
Eelco Dolstra 364e21919a hydra-eval-{jobs,jobset}: Pass file name as <...> 2017-11-28 16:51:00 +01:00
Eelco Dolstra b7bc4384b7 Fix serving plain text files 2017-11-21 17:55:52 +01:00
Eelco Dolstra 5986aae704 Add option eval_substituter 
When using the "build" or "sysbuild" jobset input types in conjunction
with a binary cache store, the evaluator needs to be able to fetch
store paths from the binary cache. Typical usage:

  store_uri = s3://nix-test-cache?secret-key=...
  eval_substituter = s3://nix-test-cache

Also, the public key of the binary cache must be added to
binary-cache-public-keys in nix.conf, otherwise the local nix-daemon
won't allow the store paths to be copied over.
 2017-11-20 16:12:20 +01:00
Eelco Dolstra 7857f83251 Serve directories and symlinks correctly 2017-11-14 17:17:07 +01:00
Eelco Dolstra 100249c066 Use git rev-list --count 2017-11-14 16:18:54 +01:00
Eelco Dolstra 5216b7eb2c Don't rely on ${LOGNAME:-foo} evaluating to a string 2017-11-14 15:11:14 +01:00
Eelco Dolstra 2cdc84f34f Fix build against Nix master 
Also, remove support in hydra-eval-jobs for multiple jobset input
alternatives. The web interface hasn't supported this in a long
time. Thus we can use the regular "--arg" handler.
 2017-10-26 13:10:14 +02:00
Eelco Dolstra eef0d8861b Remove test line 2017-10-19 13:17:29 +02:00
Eelco Dolstra b04dc6c76e Fix root creation when the root already exists but is owned by another user 2017-10-19 12:28:38 +02:00
Eelco Dolstra b904f25247 Hide contents links 2017-10-18 15:57:43 +02:00
Eelco Dolstra d8d71ea4a3 hydra-server: Use server_store_uri instead of store_uri, if set 
The store_uri may contain parameters that cause Nix to fail when
running under the hydra-server user (e.g. the signing key).
 2017-10-18 15:33:55 +02:00
Eelco Dolstra f39c13b2ff hydra-server: Set XDG_CACHE_HOME, other Nix barfs 2017-10-18 15:33:46 +02:00
Eelco Dolstra 3e7938c61a hydra-module: Include "=" in hydra.conf 
hydra-queue-runner is more anal about the format than Catalyst, so it
was misinterpreting gc_roots_dir.
 2017-10-18 14:23:10 +02:00
Eelco Dolstra 62c6cca937 Make one-click installs work for binary cache stores 
This requires setting binary_cache_public_uri in hydra.conf.
 2017-10-18 14:09:28 +02:00
Eelco Dolstra 44ef743d84 Hide some things that require a local store 2017-10-18 13:44:41 +02:00
Eelco Dolstra 40a9b70df2 Shut up warning 2017-10-18 13:39:39 +02:00
Eelco Dolstra 30943dd217 Detect MIME type 2017-10-18 13:38:34 +02:00
Eelco Dolstra 42fbde0383 Use "nix cat-store" to serve files from the Nix store 
This makes downloading/viewing build results work with binary cache
stores. For good performance, this should be used in conjunction with
https://github.com/NixOS/nix/commit/ca580bec35ea4d1984e36864158d7be99cfcb34b,
i.e. you should set store_uri to something like

  s3://my-cache?local-nar-cache=/tmp/nar-cache

to cache NARs between requests.
 2017-10-18 12:48:31 +02:00
Eelco Dolstra bc60fccf78 Remove remaining references to store_mode etc. 2017-10-18 12:23:07 +02:00
Eelco Dolstra cc64e51f75 USER -> LOGNAME for consistency 
Don't remember why we use LOGNAME. Also ensure that it's set.
 2017-10-18 11:23:00 +02:00
Eelco Dolstra 45b138373b hydra-queue-runner: Write GC roots for outputs paths 
We lost this behaviour somewhere. So build outputs could be GC'ed when
running the collector with --option gc-keep-outputs false.
 2017-10-12 18:55:38 +02:00
Eelco Dolstra 27103398c9 Make maxLogSize configurable 2017-09-22 15:23:58 +02:00
Eelco Dolstra d9cbf8cf01 Use PostgreSQL 9.5 
9.2 is no longer in Nixpkgs.
 2017-09-22 15:18:22 +02:00
Eelco Dolstra b828224fee Periodically close RemoteStore connections 
This prevents an accumulation of temproots. See
https://github.com/NixOS/nix/commit/89dc62c1745950748084a763ea325bc427045e07.
 2017-09-14 18:16:33 +02:00
Eelco Dolstra 6517446c34 Update to latest nixUnstable 2017-09-14 17:22:48 +02:00
Eelco Dolstra d676c91bd5 Return Builds.drvpath in the JSON API 
E.g.

  curl -H 'Accept: application/json' http://127.0.0.4/build/3772858 | jq -r .drvpath
  /nix/store/97vla79lavlcsk8milskr06rfm34bskx-patchelf-0.8pre843_f4522f4.drv
 2017-09-12 18:42:18 +02:00
Eelco Dolstra 4af97c57f5 Acquire the send lock only while actually sending 
Thus, we no longer hold the send lock while substituting missing paths
on the build machine. This is a good thing in particular for macOS
builders which have a tendency to hang forever in curl downloads.
 2017-09-01 16:28:49 +02:00
Eelco Dolstra 50ab80caf2 Don't wait forever to acquire the send lock 2017-09-01 15:29:06 +02:00
Eelco Dolstra 337a72ef4d Fix displaying errors from JSON requests 2017-08-17 15:32:46 +02:00
Eelco Dolstra 7c976d2aec hydra-queue-runner: Make build notification more reliable 
Previously, when hydra-queue-runner was restarted, any pending "build
finished" notifications were lost. Now hydra-queue-runner marks
finished but unnotified builds in the database and uses that to run
pending notifications at startup.
 2017-07-26 15:17:51 +02:00
Eelco Dolstra e117d85c2a hydra-queue-runner: Set a thread title for the builder threads 
This should make debugging slightly easier.
 2017-07-25 15:59:41 +02:00
Eelco Dolstra e78b9fd4ee hydra-queue-runner: Allow concurrent notifications 
The queue runner can now run up to ‘max-concurrent-notifications’ in
parallel (default is 2). This is useful when some hydra-notify
invocations can take a long time to complete (e.g. because they need
to compress a giant build log) and we don't want this to block all
other notifications.
 2017-07-24 16:35:34 +02:00
Eelco Dolstra bba383bf1b hydra-queue-runner: Keep some notification statistics 2017-07-24 16:26:44 +02:00
Eelco Dolstra f46a21e16e Slight cleanup 2017-07-21 17:22:11 +02:00
Eelco Dolstra dc5e0b120a Fix a race that can cause hydra-queue-runner to ignore newly added builds 
As @dtzWill discovered, with the concurrent hydra-evaluator, there can
be multiple active transactions adding builds to the database. As a
result, builds can become visible in a non-monotonically increasing
order, breaking the queue monitor's assumption that build IDs only go
up.

The fix is to have hydra-eval-jobset provide the lowest build ID it
just added in the builds_added notification, and have the queue
monitor check from there.

Fixes #496.
 2017-07-21 14:34:48 +02:00
Eelco Dolstra b0432c7762 Sync with nixUnstable 2017-07-21 13:09:08 +02:00
Eelco Dolstra 6fc851d376 Improve erorr message 2017-07-17 14:10:34 +02:00
Eelco Dolstra 66ae66024e Sync with latest Nix 2017-07-17 11:38:58 +02:00
Eelco Dolstra 803833aba7 Disable the build-queued hook 
This can take an excessive amount of time. For example, on
hydra.nixos.org, a call to hydra-notify takes 0.7s even if there are
no plugins. So for an eval with ~45K new builds, the calls to
hydra-notify add up to about 9 hours.

The proper fix would be to pass a list of build IDs, or an eval ID.
 2017-06-22 15:09:04 +02:00
Eelco Dolstra f0d24af940 Fix "Evaluation pending since..." message even though the jobset has been evaluated 2017-06-12 14:15:00 +02:00
Eelco Dolstra 941665044e /api/scmdiff: Set correct MIME type 
It was set to application/json but should be text/plain.
 2017-05-05 15:58:38 +02:00
Eelco Dolstra 8abc595d1c preHook -> shellHook 
Adding the build directory to $PERL5LIB causes it to end up in the
wrapper scripts, which is bad.
 2017-05-04 15:25:22 +02:00
Eelco Dolstra 1f94f03699 Fix build 2017-04-26 15:11:12 +02:00
Eelco Dolstra cc85208fe4 Fix build 2017-04-18 20:50:18 +02:00
Eelco Dolstra 468b048fa2 Revert "Move build logic for the core hydra package into its own default.nix." 
This reverts commit 949e5865c6. This
makes release.nix harder to read/maintain IMHO. There already is a Nix
expression for Hydra in Nixpkgs that can be used for this purpose.
 2017-04-12 13:10:01 +02:00
Eelco Dolstra a0376a92e5 Add dependency on nix-perl 
Fixes #459.
 2017-04-11 15:48:22 +02:00
Eelco Dolstra c2c2633e50 Fix link to cached build log 
Fixes #462.
 2017-04-11 14:25:48 +02:00
Eelco Dolstra 433b94ee68 Fix Mercurial diff 
It only showed the log messages of $rev1 and $rev2. Now it shows the
range $rev1::$rev2, excluding $rev1.
 2017-04-11 13:39:54 +02:00
Eelco Dolstra 585d3274e3 Better error message if jobset doesn't evaluate to an attrset 2017-04-11 13:01:15 +02:00
Eelco Dolstra 426aea1236 hydra-queue-runner: Allow multiple concurrent daemon connections 2017-04-06 18:50:53 +02:00
Eelco Dolstra 5810042a3b Periodically clear Store's path info cache 
Otherwise the queue runner can consider paths as valid that have been
garbage-collected since the first time it queried them.
 2017-04-06 17:20:23 +02:00
Eelco Dolstra 8364f4ec70 Upload log files to the right location 
We were mixing up builds and steps. So for example

  https://cache.nixos.org/log/2w66a98iqbjdppc5s2b8qvhi3gprvy45-freecell-solver-4.8.0.drv

at the moment contains the log for
/nix/store/442r9d5ihbcpgq8q9dhijhvhlmplzp96-perl-namespace-autoclean-0.28.drv
because the latter is a step in http://hydra.nixos.org/build/51300420.
Oops.
 2017-04-06 13:05:30 +02:00