List of commits:
Subject Hash Author Date (UTC)
Mostly worker stuff ccf3a8d8da2ad1b0d97418fabb5028b40721835f Catalin(ux) M. BOIE 2022-10-22 05:50:04
wh: lambda: seems we cannot pass x-Amz-Client-Context header empty anymore 7f065b0fb6ceed5d2339afd7590f5a795ed3582e Catalin(ux) M. BOIE 2022-10-21 19:33:58
workers: wrong URL for delete a2b2ff5925b1ee9b4a033da93084c008b7af8c64 Catalin(ux) M. BOIE 2022-10-21 16:03:56
Typo 4557595fb985fb2a0a482a387ef0a61293b511ed Catalin(ux) M. BOIE 2022-03-29 17:06:57
Builder improvements 2c27620922c4990454dc3039b2f1c4a86388501f Catalin(ux) M. BOIE 2022-03-29 06:28:16
Extra space in debian/conffiles preventing Debian build 416ed9995151c29bffb4ca3f0f6901aab7cbaa8e Catalin(ux) M. BOIE 2022-03-28 19:36:06
Show global pkg sub repos; cosmetic 43f60158b760b2789261e703ff2fa0781f590e04 Catalin(ux) M. BOIE 2022-03-28 18:21:07
Cosmetic a60d33914b8a1754c84ddc5440533df9b89337ad Catalin(ux) M. BOIE 2022-03-16 07:40:37
Cosmetic 8bb684cb885e0ad0f24519990a564031ea0e43a9 Catalin(ux) M. BOIE 2022-02-19 08:12:45
Set install_id under lock to not get errors 46904e58621450cc164afe769a2aeb597fbaecd8 Catalin(ux) M. BOIE 2022-02-17 18:08:59
Cosmetic bc38d3b0196d5701f75209a3125543ba096fe340 Catalin(ux) M. BOIE 2022-02-16 19:45:29
Bumped version to 0.75 1341db08f1342e7a6838305e94d31849d6b583c4 Catalin(ux) M. BOIE 2022-02-16 18:58:45
Misc small fixes 7bc12d7ee158d99979dbb91f293162472917a2e2 Catalin(ux) M. BOIE 2022-02-16 18:58:18
We did not validated the plan id - now is fixed. 3786d759107c4028f181c7de77d39382d9f572bf Catalin(ux) M. BOIE 2022-02-16 18:55:05
Added build stats 3f574c94ad35d6f85ee37a4da410499f268e608f Catalin(ux) M. BOIE 2022-02-16 18:53:54
Cosmetic 548c8a909c994ef528c9910c625e6afb88beeead Catalin(ux) M. BOIE 2022-02-16 18:44:12
.deb support 722ca672abf6f28109e35f3573dca7e367d3b51a Catalin(ux) M. BOIE 2022-02-16 18:39:19
compare: added rpm/deb lines 1356fb5c8e5d82356776561af4a2dcaec0d19c14 Catalin(ux) M. BOIE 2022-02-16 07:30:06
debian folder updates 45e86104b565e508c514804910cfda87a3c1239c Catalin(ux) M. BOIE 2022-02-16 07:26:09
Fix rate limit for IPv6 336d34f7eaac356688ba5c251f027aff947ce9e4 Catalin(ux) M. BOIE 2022-02-09 14:39:28
Commit ccf3a8d8da2ad1b0d97418fabb5028b40721835f - Mostly worker stuff
Author: Catalin(ux) M. BOIE
Author date (UTC): 2022-10-22 05:50
Committer name: Catalin(ux) M. BOIE
Committer date (UTC): 2022-10-22 05:50
Parent(s): 7f065b0fb6ceed5d2339afd7590f5a795ed3582e
Signer:
Signing key:
Signing status: N
Tree: 84a408fe3e7aba30789df1734779d787f2433577
File Lines added Lines deleted
README 17 8
TODO 61 1
debian/postinst 1 1
inc/sql.inc.php 8 29
inc/user/packages.inc.php 1 1
inc/user/pkg_subrepo.inc.php 1 1
inc/util.inc.php 18 7
rocketgit.spec 5 2
root/index.php 4 8
root/themes/default/hints/repo/clone_owner.html 2 1
samples/nginx.conf 7 0
scripts/remote.php 2 2
scripts/worker.php 234 237
tests/http_settings.php 2 0
tests/prof.php 1 1
tests/wh_build.php 2 2
tests/wh_lambda.php 2 2
File README changed (mode: 100644) (index 726b89f..96c40d8)
42 42 - allow enough RAM and execution time - allow enough RAM and execution time
43 43 - fix timezone (date.timezone = UTC, for example) - fix timezone (date.timezone = UTC, for example)
44 44 You may want to activate an op cache to speed up the PHP scripts: You may want to activate an op cache to speed up the PHP scripts:
45 yum/dnf install php-opcache
45 apt/yum/dnf install php-opcache
46 46 Also, we recommend to activate opcache also for cli: Also, we recommend to activate opcache also for cli:
47 47 change opcache.enable_cli to 1 in /etc/php.d/10-opcache.ini and change opcache.enable_cli to 1 in /etc/php.d/10-opcache.ini and
48 48 /etc/php-zts.d/10-opcache.ini. /etc/php-zts.d/10-opcache.ini.
49 49
50 50 . Install and prepare a web server: nginx/apache: . Install and prepare a web server: nginx/apache:
51 51 nginx is recommended because of the chunked encoding in POST requests. nginx is recommended because of the chunked encoding in POST requests.
52 yum/dnf install nginx
52 apt/yum/dnf install nginx
53 53 or or
54 yum/dnf install httpd mod_ssl
54 apt/yum/dnf install httpd mod_ssl
55 55
56 Copy rocketgit.conf.sample from /etc/nginx/conf.d
57 (or /etc/httpd/conf.d) into rocketgit.conf and edit it.
56 Edit /etc/nginx/conf.d/rocketgit.conf
57 (or /etc/httpd_or_apache2/conf.d/rocketgit.com).
58 Pay attention to the path to the certificate:
59 Debian like: use /etc/ssl/certs/ssl-cert-snakeoil.pem and
60 /etc/ssl/private/ssl-cert-snakeoil.key for a self signed cert.
58 61
59 62 Activate web server (nginx) Activate web server (nginx)
60 63 systemctl enable nginx.service systemctl enable nginx.service
 
71 74 systemctl enable --now rocketgit-git.socket systemctl enable --now rocketgit-git.socket
72 75
73 76 . Prepare PostgreSQL server . Prepare PostgreSQL server
74 yum/dnf install postgresql-server
77 RedHat like: yum/dnf install postgresql-server
78 Debian like: apt install postgresql
75 79
76 80 Add the following lines, before wildcard matches, in Add the following lines, before wildcard matches, in
77 /var/lib/pgsql/data/pg_hba.conf:
81 RedHat like: /var/lib/pgsql/data/pg_hba.conf:
82 Debian like: /etc/postgresql/*/main/pg_hba.conf:
83
78 84 local rocketgit rocketgit trust local rocketgit rocketgit trust
79 85 host rocketgit rocketgit 127.0.0.1/32 trust host rocketgit rocketgit 127.0.0.1/32 trust
80 86 host rocketgit rocketgit ::1/128 trust host rocketgit rocketgit ::1/128 trust
81 87
82 systemctl enable postgresql.service
88 RedHat like:
83 89 export PGSETUP_INITDB_OPTIONS="--data-checksums" # recommended export PGSETUP_INITDB_OPTIONS="--data-checksums" # recommended
84 90 postgresql-setup --initdb # (TAKE CARE! YOU MAY DESTROY ALL YOUR DATA!) postgresql-setup --initdb # (TAKE CARE! YOU MAY DESTROY ALL YOUR DATA!)
91 systemctl enable postgresql.service
85 92 systemctl start postgresql.service systemctl start postgresql.service
86 93
94 Debian like: TODO
95
87 96 Notes: Notes:
88 97 - Check also the config file (/etc/rocketgit/config.php) and set - Check also the config file (/etc/rocketgit/config.php) and set
89 98 correctly the rg_sql string. correctly the rg_sql string.
File TODO changed (mode: 100644) (index 8be3495..a60015c)
18 18 [ ] Announce build jobs users that now is working. [ ] Announce build jobs users that now is working.
19 19 [ ] Partner with Solo Keys? [ ] Partner with Solo Keys?
20 20 [ ] Enable disabled environments. [ ] Enable disabled environments.
21 [ ] We should not disable CentOS7. Only CentOS8! At least on r1 it is disabled!
21 22 [ ] [ ]
22 23
23 24 == next release == == next release ==
25 [ ] When a new distro is added, the dotrepo is not built automatically!
26 I think this is done at push time.
27 [ ] We need a 'latest' for http://rocketgit.com/op/pkgrepo/main/global/testing
28 /fedora/36/x86_64/os/rocketgit-global-testing-1.1-1.noarch.rpm
29 [ ] Prod: https://rocketgit.com/user/catalinux/rgfs/pkg
30 "Major version: rawhide (0)"
31 [ ] Cannot delete jobs from https://rocketgit.com/op/settings/workers/queue
32 [ ] Seems the update of keys still happens with key_id as string, even from the code I apply intval().
33 Probably I need to do correct binding of parameters.
34 [ ] https://rocketgit.com/op/settings/workers/queue shows '@@x@@' output!
35 env=rocky-8-x86_64
36 url=ssh://rocketgit@ssh.rocketgit.com/user/catalinux/ninedogs
37 head=0cf2be651bd2a2f93fe0af1c4bd56a11910791c5
38 hook_id=4
39 Machine started at @@status::vm_start_nice@@
40 Clone took @@status::clone_elap_nice@@
41 Script started at @@status::build_sh_start_nice@@
42 @@status_packages@@
43 [ ] workers: instead of showing the hook_id, we should put a likn to it!
44 [ ] When you delete an account, we must also clean the cache.
45 [ ] Seems we are not matching pushes by tag!
46 Should we push in 'stable' branch?
47 We should not build by tag AND by 'push commit'.
24 48 [ ] Seems that for Debian, a new VM is started for building the repo, which is not correct! [ ] Seems that for Debian, a new VM is started for building the repo, which is not correct!
25 49 See job 2009 (orig job_id 2007)! See job 2009 (orig job_id 2007)!
50 It is about .dev signing with global key.
51 But, it seems nothing happens in 2009!
52 Because +pending/2007 is missing, because those files where already processed.
53 Should we put the "dirty" debs into another dir? Or we should not remove them
54 when processing?
55 I hope the pkg repo id 1 key is not provided in job 2007!
56
57 So, the files are creted in 1/1/debian/bullseye/main/source/+pending/2007
58 I just have to be sure that the after build event does not process dirty
59 repos.
60 I will search the event to see what pkg repo ids I processed (events-20220316.log)!
61 07:43:32.043 DEBUG: pkg_subrepo_id 2
62 07:43:32.043 DEBUG: subrepo is marked dirty, skip this step
63 So, I am doing this!
64
65 It is possible job 2007 is cleaning also global +pending folders! I need to log the removing!
66 Loghez si stergerea folder-elor, asa ca sper sa vad cine sterge +pending pentru global!
67
68 Solved! It was the thing that the build of deb failed because of an extra space in conffiles!
69
70 [ ] Set a bigger receive buffer on the virtio port to not lose messages?
26 71 [ ] [ ]
27 72
28 73 == Next next release == == Next next release ==
74 [ ] When looking for distinct user agent, do limit the search in the past?
75 Now it takes around 30s.
76 [ ] SELinux Relabel only on install, not on update?
77 Or do not relabel big directories?
78 [ ] We should not pack .gitignore files! At least on Debian.
79 [ ] On Debian, php-fpm is versioned! Use a helper to execute the best php-fpm!
80 [ ] On Debian, the user is www-data, not apache. [php-fpm conf]
81 If I comment listen.acl_users, only www-data can connect, which is fine.
82 What about fedora?
83 [ ] If we cannot build the deb package, we should not start a second job for
84 signing global debs. Seems we have the test, but somehow it is not working.
85 Seems we have the source built.
86 [ ] We may want to emulate the dpkg-sig command, to not start a new VM.
87 [ ] In general, if a user has access in an area, we do not list those areas.
88 Example: pkg (sub)repo.
29 89 [ ] Reorder db columns to be 8 bytes aligned to save space. [ ] Reorder db columns to be 8 bytes aligned to save space.
30 90 [ ] When showing last events for a repo, add also the builds. [ ] When showing last events for a repo, add also the builds.
31 91 Same for bug's note list. Maybe other places? Same for bug's note list. Maybe other places?
 
41 101 Seen on VM stop, when making links to the image. Seen on VM stop, when making links to the image.
42 102 [ ] List also pkg repos where a user has access! [ ] List also pkg repos where a user has access!
43 103 [ ] Show what wh will be run when pushing. [ ] Show what wh will be run when pushing.
44 [ ] Allow a push to control environment for wh.
104 [ ] Allow a push (not commit) to control environment for wh.
45 105 [ ] Add a wh for "delete repo"? [ ] Add a wh for "delete repo"?
46 106 [ ] Allow individual package files to be downloaded. [ ] Allow individual package files to be downloaded.
47 107 [ ] Add info for deb/rpm + debug packages. [ ] Add info for deb/rpm + debug packages.
File debian/postinst changed (mode: 100644) (index 313e41f..c2086ec)
... ... case "${1}" in
17 17 # Set correct rights # Set correct rights
18 18 chown rocketgit:rocketgit /var/log/rocketgit chown rocketgit:rocketgit /var/log/rocketgit
19 19 chmod 0770 /var/log/rocketgit chmod 0770 /var/log/rocketgit
20 chown rocketgit:rocketgit /var/lib/rocketgit
20 chown -R rocketgit:rocketgit /var/lib/rocketgit
21 21 chmod 0771 /var/lib/rocketgit chmod 0771 /var/lib/rocketgit
22 22 chmod 0770 /var/lib/rocketgit/tmp chmod 0770 /var/lib/rocketgit/tmp
23 23 ;; ;;
File inc/sql.inc.php changed (mode: 100644) (index f35949c..5963d38)
... ... function rg_sql_escape($h, $str)
170 170 return pg_escape_string($db, $str); return pg_escape_string($db, $str);
171 171 } }
172 172
173 /*
174 * Returns the last error codes
175 */
176 function rg_sql_last_error_code($res)
177 {
178 return @pg_result_error_field($res, PGSQL_DIAG_SQLSTATE);
179 }
180
173 181 /* /*
174 182 * Helper for sql_query and sql_query_params * Helper for sql_query and sql_query_params
175 183 */ */
 
... ... function rg_sql_close($h)
365 373 */ */
366 374 function rg_sql_free_result($res) function rg_sql_free_result($res)
367 375 { {
368 if (!is_resource($res)) {
369 rg_internal_error("res is not resource!");
370 return;
371 }
372
373 376 pg_free_result($res); pg_free_result($res);
374 377 } }
375 378
 
... ... function rg_sql_free_result($res)
378 381 */ */
379 382 function rg_sql_fetch_array($res) function rg_sql_fetch_array($res)
380 383 { {
381 if (!is_resource($res)) {
382 rg_internal_error("res is not resource!");
383 return FALSE;
384 }
385
386 384 return pg_fetch_assoc($res); return pg_fetch_assoc($res);
387 385 } }
388 386
 
... ... function rg_sql_last_id($h)
400 398
401 399 function rg_sql_num_rows($res) function rg_sql_num_rows($res)
402 400 { {
403 if (!is_resource($res)) {
404 rg_internal_error("res is not resource!");
405 return FALSE;
406 }
407
408 401 return pg_num_rows($res); return pg_num_rows($res);
409 402 } }
410 403
411 404 function rg_sql_affected_rows($res) function rg_sql_affected_rows($res)
412 405 { {
413 if (!is_resource($res)) {
414 rg_internal_error("res is not resource!");
415 return FALSE;
416 }
417
418 406 return pg_affected_rows($res); return pg_affected_rows($res);
419 407 } }
420 408
 
... ... function rg_sql_fields($h, $table)
488 476 return $ret; return $ret;
489 477 } }
490 478
491 /*
492 * Returns the last error codes
493 */
494 function rg_sql_last_error_code($res)
495 {
496 return @pg_result_error_field($res, PGSQL_DIAG_SQLSTATE);
497 }
498
499 ?>
File inc/user/packages.inc.php changed (mode: 100644) (index 53020f4..5e4818c)
... ... function rg_packages_web($db, $rg, $paras, $pass)
852 852 //rg_log_debug('di: ' . print_r($di, TRUE)); //rg_log_debug('di: ' . print_r($di, TRUE));
853 853
854 854 if (strcmp($di['pkg_type'], 'rpm') == 0) { if (strcmp($di['pkg_type'], 'rpm') == 0) {
855 $a['major'] = preg_replace('/[^0-9]/', '', @array_shift($paras));
855 $a['major'] = preg_replace('/[^0-9a-zA-Z]/', '', @array_shift($paras)); // we need 'rawhide' to work!
856 856 $a['arch'] = preg_replace('/[^-_0-9a-zA-Z]/', '', @array_shift($paras)); $a['arch'] = preg_replace('/[^-_0-9a-zA-Z]/', '', @array_shift($paras));
857 857 $a['type'] = @array_shift($paras); $a['type'] = @array_shift($paras);
858 858 $a['path'] = implode('/', $paras); $a['path'] = implode('/', $paras);
File inc/user/pkg_subrepo.inc.php changed (mode: 100644) (index 97a4a92..ade0403)
... ... function rg_pkg_subrepo_event_edit($db, $ev)
69 69 . ' dotrepo because ver was incremented'); . ' dotrepo because ver was incremented');
70 70 if (!isset($per_env[$env])) if (!isset($per_env[$env]))
71 71 $per_env[$env] = array(); $per_env[$env] = array();
72 $per_env[$env] = $pkg_repo_id;
72 $per_env[$env][] = $pkg_subrepo_id;
73 73 } }
74 74 } }
75 75
File inc/util.inc.php changed (mode: 100644) (index eebb4ab..c05e780)
... ... function rg_id($len)
163 163 { {
164 164 rg_prof_start('id'); rg_prof_start('id');
165 165
166 $id = rg_random_bytes(($len + 1) / 2);
166 $len2 = intval(($len + 1) / 2);
167 $id = rg_random_bytes($len2);
167 168 $id = bin2hex($id); $id = bin2hex($id);
168 169 $id = substr($id, 0, $len); $id = substr($id, 0, $len);
169 170
 
... ... function rg_array2string_avoid($a, $avoid)
1926 1927 foreach ($a as $k => $v) { foreach ($a as $k => $v) {
1927 1928 if (in_array($k, $avoid)) { if (in_array($k, $avoid)) {
1928 1929 $s = '-'; $s = '-';
1929 } else if (is_array($v))
1930 } else if (is_array($v)) {
1930 1931 $s = rg_array2string_short($v); $s = rg_array2string_short($v);
1931 else if (is_resource($v))
1932 } else if (is_resource($v)) {
1932 1933 $s = 'RES'; $s = 'RES';
1933 else
1934 } else if (is_object($v)) {
1935 $s = 'OBJ';
1936 } else if (is_string($v)) {
1934 1937 $s = preg_replace_callback($what, 'rg_callback_hexa', $v); $s = preg_replace_callback($what, 'rg_callback_hexa', $v);
1938 } else if (is_int($v)) {
1939 $s = $v;
1940 } else {
1941 $s = '?';
1942 }
1935 1943
1936 1944 $ret .= $add . $k . '=[' . $s . ']'; $ret .= $add . $k . '=[' . $s . ']';
1937 1945 $add = ' '; $add = ' ';
 
... ... function rg_valid_referer()
2423 2431 if (empty($ref0)) if (empty($ref0))
2424 2432 return TRUE; return TRUE;
2425 2433
2426 $ref = preg_replace('|http(s)?://|', '', $ref0);
2427 $ref = preg_replace('|/.*|', '', $ref);
2434 // TODO: are we sure we want to strip the port?
2435 // TODO: are we sure we want to check the referer?
2436 $ref = preg_replace('|^http(s)?://|', '', $ref0);
2437 $ref = preg_replace('|/.*$|', '', $ref); // remove URI
2438 $ref = preg_replace('|:.*$|', '', $ref); // remove port
2428 2439
2429 2440 $we = isset($_SERVER['HTTP_HOST']) ? $_SERVER['HTTP_HOST'] : ""; $we = isset($_SERVER['HTTP_HOST']) ? $_SERVER['HTTP_HOST'] : "";
2430 2441
 
... ... function rg_web_output($c, $disable)
2850 2861 $comp_len = 0; $comp_len = 0;
2851 2862
2852 2863 if ($disable === FALSE) { if ($disable === FALSE) {
2853 $acc = @$_SERVER['HTTP_ACCEPT_ENCODING'];
2864 $acc = isset($_SERVER['HTTP_ACCEPT_ENCODING']) ? $_SERVER['HTTP_ACCEPT_ENCODING'] : '';
2854 2865 if (stristr($acc, 'gzip')) { if (stristr($acc, 'gzip')) {
2855 2866 $c = rg_gzencode($c, $orig_len, $comp_len); $c = rg_gzencode($c, $orig_len, $comp_len);
2856 2867 header('Content-Encoding: gzip'); header('Content-Encoding: gzip');
File rocketgit.spec changed (mode: 100644) (index efd9f54..595afaa)
... ... Requires: util-linux, qrencode, fuse, openssh-server, gpg
28 28
29 29 # SELinux stuff # SELinux stuff
30 30 # https://fedoraproject.org/wiki/SELinux_Policy_Modules_Packaging_Draft?rd=PackagingDrafts/SELinux/PolicyModules # https://fedoraproject.org/wiki/SELinux_Policy_Modules_Packaging_Draft?rd=PackagingDrafts/SELinux/PolicyModules
31 BuildRequires: checkpolicy, selinux-policy-devel, systemd, php-cli
32 BuildRequires: util-linux, tidy, systemd-rpm-macros, openldap-servers
31 BuildRequires: checkpolicy, selinux-policy-devel
33 32 # This is unknown yet. TODO # This is unknown yet. TODO
34 33 %if 0%{?rhel} < 9 %if 0%{?rhel} < 9
35 34 BuildRequires: hardlink BuildRequires: hardlink
36 35 %endif %endif
36
37 BuildRequires: util-linux, systemd-rpm-macros, openldap-servers
38 BuildRequires: systemd, php-cli, tidy
39
37 40 Requires(pre): shadow-utils Requires(pre): shadow-utils
38 41 Requires(post): /usr/sbin/semodule, /sbin/restorecon, /sbin/fixfiles Requires(post): /usr/sbin/semodule, /sbin/restorecon, /sbin/fixfiles
39 42 Requires(postun): /usr/sbin/semodule, /sbin/restorecon, /sbin/fixfiles Requires(postun): /usr/sbin/semodule, /sbin/restorecon, /sbin/fixfiles
File root/index.php changed (mode: 100644) (index e8f7d83..8aaa579)
... ... if ((rg_load() > 50) && rg_is_bot($rg['ua'])) {
94 94 } }
95 95 $rg['referer'] = isset($_SERVER['HTTP_REFERER']) ? $_SERVER['HTTP_REFERER'] : ''; $rg['referer'] = isset($_SERVER['HTTP_REFERER']) ? $_SERVER['HTTP_REFERER'] : '';
96 96 $req = $_REQUEST; $req = $_REQUEST;
97 if (isset($req['pass']) || isset($req['init::pass']) || isset($req['init::pass2']))
98 $req['pass'] = '*****';
99 if (isset($req['old_pass']))
100 $req['old_pass'] = '*****';
101 if (isset($req['pass1']))
102 $req['pass1'] = '*****';
103 if (isset($req['pass2']))
104 $req['pass2'] = '*****';
97 foreach ($req as $k => $v) {
98 if (stristr($k, 'pass'))
99 $req[$k] = 'redacted';
100 }
105 101 if (!empty($req)) if (!empty($req))
106 102 rg_log('_REQUEST: ' . rg_array2string($req)); rg_log('_REQUEST: ' . rg_array2string($req));
107 103 if (!empty($_COOKIE)) if (!empty($_COOKIE))
File root/themes/default/hints/repo/clone_owner.html changed (mode: 100644) (index 4597308..5f001ea)
... ... git init<br />
6 6 git remote add origin @@ri::clone_url_http@@<br /> git remote add origin @@ri::clone_url_http@@<br />
7 7 # or<br /> # or<br />
8 8 git remote add origin @@ri::clone_url_ssh@@<br /> git remote add origin @@ri::clone_url_ssh@@<br />
9 # commit something<br />
9 # commit...<br />
10 git push --set-upstream origin @@ri::main_branch@@<br />
10 11 git push origin --all<br /> git push origin --all<br />
11 12 git push origin --tags<br /> git push origin --tags<br />
12 13 </div> </div>
File samples/nginx.conf changed (mode: 100644) (index c45d123..c115778)
... ... server {
86 86 } }
87 87
88 88 ssl_protocols TLSv1.2 TLSv1.3; ssl_protocols TLSv1.2 TLSv1.3;
89
90 # RedHat like:
89 91 ssl_certificate /etc/pki/tls/certs/localhost.crt; ssl_certificate /etc/pki/tls/certs/localhost.crt;
90 92 ssl_certificate_key /etc/pki/tls/private/localhost.key; ssl_certificate_key /etc/pki/tls/private/localhost.key;
93
94 # Debian like:
95 #ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
96 #ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
97
91 98 # Specify the cyphers to get an A+ on Qualys (ssllabs.com); recommended # Specify the cyphers to get an A+ on Qualys (ssllabs.com); recommended
92 99 # https://www.digicert.com/ssl-support/ssl-enabling-perfect-forward-secrecy.htm # https://www.digicert.com/ssl-support/ssl-enabling-perfect-forward-secrecy.htm
93 100 # Generated by: https://ssl-config.mozilla.org # Generated by: https://ssl-config.mozilla.org
File scripts/remote.php changed (mode: 100644) (index 1bff76e..9a99362)
... ... if (isset($_SERVER['SSH_CONNECTION'])) {
93 93 $host = ''; $host = '';
94 94
95 95 // first parameter must be uid of the user // first parameter must be uid of the user
96 $login_uid = isset($_SERVER['argv'][1]) ? $_SERVER['argv'][1] : 0;
96 $login_uid = isset($_SERVER['argv'][1]) ? intval($_SERVER['argv'][1]) : 0;
97 97 rg_log('uid is ' . $login_uid); rg_log('uid is ' . $login_uid);
98 98
99 99 // second parameter must be the ssh key id // second parameter must be the ssh key id
100 $key_id = isset($_SERVER['argv'][2]) ? $_SERVER['argv'][2] : 0;
100 $key_id = isset($_SERVER['argv'][2]) ? intval($_SERVER['argv'][2]) : 0;
101 101 // TODO: because of build system, 0 may be valid. // TODO: because of build system, 0 may be valid.
102 102 //if ($key_id == 0) //if ($key_id == 0)
103 103 // fatal('key_id not provided!'); // fatal('key_id not provided!');
File scripts/worker.php changed (mode: 100644) (index 17055d9..5f6ba09)
... ... function load_config($file)
248 248 $i['pkg_cmd'] = 'dnf -y install'; $i['pkg_cmd'] = 'dnf -y install';
249 249 } else if (strcasecmp($x, 'centos_stream') == 0) { } else if (strcasecmp($x, 'centos_stream') == 0) {
250 250 $i['pkg_cmd'] = 'dnf -y install'; $i['pkg_cmd'] = 'dnf -y install';
251 } else if (strcasecmp($x, 'alma') == 0) {
252 $i['pkg_cmd'] = 'dnf -y install';
251 253 } else if (strcasecmp($x, 'centos') == 0) { } else if (strcasecmp($x, 'centos') == 0) {
252 254 if ($y > 7) if ($y > 7)
253 255 $i['pkg_cmd'] = 'dnf -y install'; $i['pkg_cmd'] = 'dnf -y install';
 
... ... function start_worker_build_tools($job, &$reason, &$reason2)
435 437 $s .= 'pri2uid[' . $pkg_repo_id . ']=' . $info['uid'] . "\n"; $s .= 'pri2uid[' . $pkg_repo_id . ']=' . $info['uid'] . "\n";
436 438 } }
437 439 } }
438 $s .= 'export pkg_repos_list="' . $list . '"' . "\n";
440 $s .= 'export pkg_repos_list="' . $list . '"' . "\n"
441 . '# Export ends' . "\n\n";
439 442
440 443 $s .= '' $s .= ''
441 . '# Export ends' . "\n"
442 . "\n"
443 444 . 'E=0; E_CMD=""' . "\n" . 'E=0; E_CMD=""' . "\n"
444 445 . "\n" . "\n"
445 446 . 'rg_log_prefix=""' . "\n" . 'rg_log_prefix=""' . "\n"
446 . 'function xecho()' . "\n"
447 . 'rg_log_level=0' . "\n"
448 . "\n"
449 . 'function rg_log()' . "\n"
447 450 . '{' . "\n" . '{' . "\n"
448 . ' echo "${rg_log_prefix}${@}"' . "\n"
451 . ' local type=${1}; shift' . "\n" // N - normal, W - warn, E - error
452 . ' if [ "${1}" = "" ]; then' . "\n"
453 . ' echo 1>&2' . "\n"
454 . ' else' . "\n"
455 . ' echo "${rg_log_prefix}${type} ${@} [$(date)]" 1>&2' . "\n"
456 . ' fi' . "\n"
457 . ' local d=$(date +%s.%3N)' . "\n"
458 . ' echo "v1 ${rg_log_level} ${d} ${type} ${@}" > /dev/virtio-ports/rgw' . "\n" // TODO: CentOS 7, for example, returns "Resource busy"; not only CentOS7. Also Debian 11
459 . ' echo "v1 ${rg_log_level} ${d} ${type} ${@}" >> /mnt/status/history' . "\n"
449 460 . '}' . "\n" . '}' . "\n"
450 461 . "\n" . "\n"
451 462 . 'function rg_log_push()' . "\n" . 'function rg_log_push()' . "\n"
452 463 . '{' . "\n" . '{' . "\n"
453 . ' [ "${1}" = "" ] || rg_notify "${1}"' . "\n"
464 . ' [ "${1}" = "" ] || rg_log N "${1}"' . "\n"
454 465 . ' rg_log_prefix+=" "' . "\n" . ' rg_log_prefix+=" "' . "\n"
466 . ' rg_log_level=$(( rg_log_level + 1 ))' . "\n"
455 467 . '}' . "\n" . '}' . "\n"
456 468 . "\n" . "\n"
457 469 . 'function rg_log_pop()' . "\n" . 'function rg_log_pop()' . "\n"
458 470 . '{' . "\n" . '{' . "\n"
459 . ' xecho "Done [E=${E}] [`date`]"' . "\n"
471 . ' rg_log N "Done [E=${E}] [$(date)]"' . "\n"
460 472 . ' rg_log_prefix=${rg_log_prefix:2}' . "\n" . ' rg_log_prefix=${rg_log_prefix:2}' . "\n"
473 . ' rg_log_level=$(( rg_log_level - 1 ))' . "\n"
461 474 . '}' . "\n" . '}' . "\n"
462 475 . "\n" . "\n"
463 . 'function rg_notify()' . "\n"
464 . '{' . "\n"
465 . ' xecho "${@} [$(date)]"' . "\n"
466 . ' echo "M${@} [$(date)]" > /dev/virtio-ports/rgw 2>/dev/null' . "\n" // TODO: CentOS 7, for example, returns "Resource busy"; not only CentOS7. Also Debian 11
467 . ' echo "M${@} [$(date)]" >> /mnt/status/history' . "\n"
468 . '}' . "\n"
469 . "\n"
470 . 'function rg_notify_err()' . "\n"
471 . '{' . "\n"
472 . ' xecho "${@} [$(date)]"' . "\n"
473 . ' echo "E${@} [$(date)]" > /dev/virtio-ports/rgw 2>/dev/null' . "\n"
474 . ' echo "E${@} [$(date)]" >> /mnt/status/history' . "\n"
475 . '}' . "\n"
476 . "\n"
477 . 'function rg_notify_warn()' . "\n"
478 . '{' . "\n"
479 . ' xecho "${@}"' . "\n"
480 . ' echo "W${@} [$(date)]" > /dev/virtio-ports/rgw 2>/dev/null' . "\n"
481 . ' echo "W${@} [$(date)]" >> /mnt/status/history' . "\n"
482 . '}' . "\n"
483 . "\n"
484 . 'function rg_op_basic()' . "\n" // TODO: we have a problem here: errors are not sent to rg_notify!
476 . 'function rg_op_basic()' . "\n"
485 477 . '{' . "\n" . '{' . "\n"
486 478 . ' E_CMD=${@}' . "\n" . ' E_CMD=${@}' . "\n"
487 . ' rg_notify " X: ${E_CMD}" 1>&2' . "\n" // we do not want to interfere with user redirection (rg_op cmd > out)
488 . ' "${@}" 2>/tmp/rg_op_${USER}.err' . "\n"
479 . ' rg_log N " X: ${E_CMD}"' . "\n"
480 . ' "${@}"' . "\n"
489 481 . ' E=${?}' . "\n" . ' E=${?}' . "\n"
490 . ' (' . "\n"
491 . ' if [ "${E}" != "0" ]; then' . "\n"
492 . ' rg_notify_err " Cannot execute [E=${E}]"' . "\n"
493 . ' rg_notify_err $(sed -e "s|^|${rg_log_prefix} |" /tmp/rg_op_${USER}.err)' . "\n"
494 . ' fi' . "\n"
495 . ' ) 1>&2' . "\n"
482 . ' [ "${E}" != "0" ] && rg_log E " Cannot execute [E=${E}]"' . "\n"
496 483 . ' return ${E}' . "\n" . ' return ${E}' . "\n"
497 484 . '}' . "\n" . '}' . "\n"
498 485 . "\n" . "\n"
499 . 'function rg_op()' . "\n" // The output will be captured by the caller
486 . 'function rg_op()' . "\n" // The standard output will BE captured by the caller
500 487 . '{' . "\n" . '{' . "\n"
501 488 . ' rg_op_basic "${@}"' . "\n" . ' rg_op_basic "${@}"' . "\n"
502 489 . ' return ${E}' . "\n" . ' return ${E}' . "\n"
503 490 . '}' . "\n" . '}' . "\n"
504 491 . "\n" . "\n"
505 . 'function rg_op_s()' . "\n" // The output will not be captured by the caller' . "\n"
492 . 'function rg_op_s()' . "\n" // The output will NOT be captured by the caller' . "\n"
506 493 . '{' . "\n" . '{' . "\n"
507 . ' rg_op_basic "${@}" | sed -e "s|^|${rg_log_prefix} |"' . "\n"
508 . ' rg_notify "rg_op_s: rg_op_basic returned E ${E}."' . "\n"
494 . ' rg_op_basic "${@}" 1>&2' . "\n"
509 495 . ' return ${E}' . "\n" . ' return ${E}' . "\n"
510 496 . '}' . "\n" . '}' . "\n"
511 497 . "\n" . "\n"
 
... ... function start_worker_build_tools($job, &$reason, &$reason2)
528 514 . ' local dir=${1}; shift' . "\n" . ' local dir=${1}; shift' . "\n"
529 515 . ' local pkg_list_file=${1}; shift' . "\n" . ' local pkg_list_file=${1}; shift' . "\n"
530 516 . "\n" . "\n"
531 . ' rg_notify "rg_rpm_sign: pkg_list_file=${pkg_list_file} key=repo_id-${pkg_repo_id} dir=${dir}"' . "\n"
517 . ' rg_log N "rg_rpm_sign: pkg_list_file=${pkg_list_file} key=repo_id-${pkg_repo_id} dir=${dir}"' . "\n"
532 518 . ' if [ ! -s "${pkg_list_file}" ]; then' . "\n" . ' if [ ! -s "${pkg_list_file}" ]; then' . "\n"
533 . ' rg_notify " No packages available."' . "\n"
519 . ' rg_log N " No packages available."' . "\n"
534 520 . ' return 0' . "\n" . ' return 0' . "\n"
535 521 . ' fi' . "\n" . ' fi' . "\n"
536 522 . ' if [ ! -d "${dir}" ]; then' . "\n" . ' if [ ! -d "${dir}" ]; then' . "\n"
537 . ' rg_notify " Dir ${dir} not found."' . "\n"
523 . ' rg_log N " Dir ${dir} not found."' . "\n"
538 524 . ' return 0' . "\n" . ' return 0' . "\n"
539 525 . ' fi' . "\n" . ' fi' . "\n"
540 526 . "\n" . "\n"
541 527 . ' local old_dir=${PWD}' . "\n" . ' local old_dir=${PWD}' . "\n"
542 528 . ' cd "${dir}"' . "\n" . ' cd "${dir}"' . "\n"
543 . ' rg_notify "Signing in dir ${dir}:" $(< "${pkg_list_file}")' . "\n"
529 . ' rg_log N "Signing in dir ${dir}:" $(< "${pkg_list_file}")' . "\n"
544 530 . ' rg_op_s rpmsign --define "_gpg_name repo_id-${pkg_repo_id}" \\' . "\n" . ' rg_op_s rpmsign --define "_gpg_name repo_id-${pkg_repo_id}" \\' . "\n"
545 531 . ' --define "_gpg_sign_cmd_extra_args --no-autostart --batch" \\' . "\n" . ' --define "_gpg_sign_cmd_extra_args --no-autostart --batch" \\' . "\n"
546 532 . ' --resign $(< "${pkg_list_file}") </dev/null' . "\n" . ' --resign $(< "${pkg_list_file}") </dev/null' . "\n"
547 533 . ' cd "${old_dir}"' . "\n" . ' cd "${old_dir}"' . "\n"
548 . ' rg_notify "Signing done (E=${E})"' . "\n"
534 . ' rg_log N "Signing done (E=${E})"' . "\n"
549 535 . ' return ${E}' . "\n" . ' return ${E}' . "\n"
550 536 . '}' . "\n" . '}' . "\n"
551 537 . "\n" . "\n"
 
... ... function start_worker_build_tools($job, &$reason, &$reason2)
553 539 . '{' . "\n" . '{' . "\n"
554 540 . ' local dir=${1}; shift' . "\n" . ' local dir=${1}; shift' . "\n"
555 541 . "\n" . "\n"
556 . ' rg_notify "rg_deb_extract_control: dir=[${dir}]"' . "\n"
542 . ' rg_log N "rg_deb_extract_control: dir=[${dir}]"' . "\n"
557 543 . ' if [ ! -d "${dir}" ]; then' . "\n" . ' if [ ! -d "${dir}" ]; then' . "\n"
558 . ' rg_notify " dir does not exists."; return 0' . "\n"
544 . ' rg_log N " dir does not exists."; return 0' . "\n"
559 545 . ' fi' . "\n" . ' fi' . "\n"
560 546 . ' local old_dir=${PWD}' . "\n" . ' local old_dir=${PWD}' . "\n"
561 547 . ' cd "${dir}"' . "\n" . ' cd "${dir}"' . "\n"
562 548 . ' while read file; do' . "\n" . ' while read file; do' . "\n"
563 549 . ' [ "${file}" = "" ] && continue' . "\n" . ' [ "${file}" = "" ] && continue' . "\n"
564 . ' rg_notify " Extracting control from ${file}"' . "\n"
550 . ' rg_log N " Extracting control from ${file}"' . "\n"
565 551 . ' if [ "${env_distro}" = "ubuntu" ]; then' . "\n" . ' if [ "${env_distro}" = "ubuntu" ]; then' . "\n"
566 552 . ' cf="control.tar.zst"' . "\n" . ' cf="control.tar.zst"' . "\n"
567 553 . ' tar_opts="--zstd"' . "\n" . ' tar_opts="--zstd"' . "\n"
 
... ... function start_worker_build_tools($job, &$reason, &$reason2)
571 557 . ' fi' . "\n" . ' fi' . "\n"
572 558 . ' ar x "${file}" ${cf}' . "\n" . ' ar x "${file}" ${cf}' . "\n"
573 559 . ' if [ "${?}" != "0" ]; then' . "\n" . ' if [ "${?}" != "0" ]; then' . "\n"
574 . ' rg_notify_err " Cannot extract control.tar.xz"' . "\n"
560 . ' rg_log E " Cannot extract control.tar.xz"' . "\n"
575 561 . ' E=1; break' . "\n" . ' E=1; break' . "\n"
576 562 . ' fi' . "\n" . ' fi' . "\n"
577 563 . ' [ "${E}" = "0" ] || break' . "\n" . ' [ "${E}" = "0" ] || break' . "\n"
578 564 . "\n" . "\n"
579 565 . ' tar xf ${cf} ${tar_opts} ./control' . "\n" . ' tar xf ${cf} ${tar_opts} ./control' . "\n"
580 566 . ' if [ "${?}" != "0" ]; then' . "\n" . ' if [ "${?}" != "0" ]; then' . "\n"
581 . ' rg_notify_err " Cannot extract ./control"' . "\n"
567 . ' rg_log E " Cannot extract ./control"' . "\n"
582 568 . ' E=1; break' . "\n" . ' E=1; break' . "\n"
583 569 . ' fi' . "\n" . ' fi' . "\n"
584 570 . ' [ "${E}" = "0" ] || break' . "\n" . ' [ "${E}" = "0" ] || break' . "\n"
 
... ... function start_worker_build_tools($job, &$reason, &$reason2)
586 572 . ' rm -f ${cf}' . "\n" . ' rm -f ${cf}' . "\n"
587 573 . ' mv control "${file}c"' . "\n" // here is build .debc and .ddebc files . ' mv control "${file}c"' . "\n" // here is build .debc and .ddebc files
588 574 . ' if [ "${?}" != "0" ]; then' . "\n" . ' if [ "${?}" != "0" ]; then' . "\n"
589 . ' rg_notify_err " Cannot rename"' . "\n"
575 . ' rg_log E " Cannot rename"' . "\n"
590 576 . ' E=1; break' . "\n" . ' E=1; break' . "\n"
591 577 . ' fi' . "\n" . ' fi' . "\n"
592 578 . ' [ "${E}" = "0" ] || break' . "\n" . ' [ "${E}" = "0" ] || break' . "\n"
 
... ... function start_worker_build_tools($job, &$reason, &$reason2)
601 587 . ' local dir=${1}; shift' . "\n" . ' local dir=${1}; shift' . "\n"
602 588 . ' local pkg_list_file=${1}; shift' . "\n" . ' local pkg_list_file=${1}; shift' . "\n"
603 589 . "\n" . "\n"
604 . ' rg_notify "rg_deb_sign: pkg_list_file=${pkg_list_file} key=repo_id-${pkg_repo_id} dir=${dir}"' . "\n"
590 . ' rg_log N "rg_deb_sign: pkg_list_file=${pkg_list_file} key=repo_id-${pkg_repo_id} dir=${dir}"' . "\n"
605 591 . ' if [ ! -s "${pkg_list_file}" ]; then' . "\n" . ' if [ ! -s "${pkg_list_file}" ]; then' . "\n"
606 . ' rg_notify " No packages available."' . "\n"
592 . ' rg_log N " No packages available."' . "\n"
607 593 . ' return 0' . "\n" . ' return 0' . "\n"
608 594 . ' fi' . "\n" . ' fi' . "\n"
609 595 . ' if [ ! -d "${dir}" ]; then' . "\n" . ' if [ ! -d "${dir}" ]; then' . "\n"
610 . ' rg_notify " Dir ${dir} not found."' . "\n"
596 . ' rg_log N " Dir ${dir} not found."' . "\n"
611 597 . ' return 0' . "\n" . ' return 0' . "\n"
612 598 . ' fi' . "\n" . ' fi' . "\n"
613 599 . "\n" . "\n"
614 600 . ' local old_dir=${PWD}' . "\n" . ' local old_dir=${PWD}' . "\n"
615 601 . ' cd "${dir}"' . "\n" . ' cd "${dir}"' . "\n"
616 . ' rg_notify "Signing in dir ${dir}:" $(< "${pkg_list_file}")' . "\n"
602 . ' rg_log N "Signing in dir ${dir}:" $(< "${pkg_list_file}")' . "\n"
617 603 . ' rg_op_s dpkg-sig --verbose -k repo_id-${pkg_repo_id} \\' . "\n" . ' rg_op_s dpkg-sig --verbose -k repo_id-${pkg_repo_id} \\' . "\n"
618 604 . ' --sign rg --verbose \\' . "\n" . ' --sign rg --verbose \\' . "\n"
619 605 . ' $(< "${pkg_list_file}") </dev/null' . "\n" . ' $(< "${pkg_list_file}") </dev/null' . "\n"
620 606 . ' cd "${old_dir}"' . "\n" . ' cd "${old_dir}"' . "\n"
621 . ' rg_notify "Signing done (E=${E})"' . "\n"
607 . ' rg_log N "Signing done (E=${E})"' . "\n"
622 608 . ' return ${E}' . "\n" . ' return ${E}' . "\n"
623 609 . '}' . "\n" . '}' . "\n"
624 610 . "\n" . "\n"
 
... ... function start_worker_build_tools($job, &$reason, &$reason2)
627 613 . ' local dir=${1}; shift' . "\n" . ' local dir=${1}; shift' . "\n"
628 614 . ' local pkg_list_file=${1}; shift' . "\n" . ' local pkg_list_file=${1}; shift' . "\n"
629 615 . "\n" . "\n"
630 . ' rg_notify "rg_rpm_create_list: pkg_list_file=${pkg_list_file} dir=${dir}"' . "\n"
616 . ' rg_log N "rg_rpm_create_list: pkg_list_file=${pkg_list_file} dir=${dir}"' . "\n"
631 617 . ' if [ ! -d "${dir}" ]; then' . "\n" . ' if [ ! -d "${dir}" ]; then' . "\n"
632 . ' rg_notify_err " Dir ${dir} not found."' . "\n"
618 . ' rg_log E " Dir ${dir} not found."' . "\n"
633 619 . ' return 0' . "\n" . ' return 0' . "\n"
634 620 . ' fi' . "\n" . ' fi' . "\n"
635 621 . ' (cd "${dir}"; find . -type f -name "*.rpm") > "${pkg_list_file}"' . "\n" . ' (cd "${dir}"; find . -type f -name "*.rpm") > "${pkg_list_file}"' . "\n"
636 622 . ' if [ ! -s "${pkg_list_file}" ]; then' . "\n" . ' if [ ! -s "${pkg_list_file}" ]; then' . "\n"
637 . ' rg_notify " No files!"' . "\n"
623 . ' rg_log N " No files!"' . "\n"
638 624 . ' return 0' . "\n" . ' return 0' . "\n"
639 625 . ' fi' . "\n" . ' fi' . "\n"
640 . ' rg_notify " content:" $(< "${pkg_list_file}")' . "\n"
626 . ' rg_log N " content:" $(< "${pkg_list_file}")' . "\n"
641 627 . '}' . "\n" . '}' . "\n"
642 628 . "\n" . "\n"
643 629 . 'function rg_deb_create_deb_list()' . "\n" . 'function rg_deb_create_deb_list()' . "\n"
 
... ... function start_worker_build_tools($job, &$reason, &$reason2)
645 631 . ' local dir=${1}; shift' . "\n" . ' local dir=${1}; shift' . "\n"
646 632 . ' local pkg_list_file=${1}; shift' . "\n" . ' local pkg_list_file=${1}; shift' . "\n"
647 633 . "\n" . "\n"
648 . ' rg_notify "rg_deb_create_deb_list: pkg_list_file=${pkg_list_file} dir=${dir}"' . "\n"
634 . ' rg_log N "rg_deb_create_deb_list: pkg_list_file=${pkg_list_file} dir=${dir}"' . "\n"
649 635 . ' if [ ! -d "${dir}" ]; then' . "\n" . ' if [ ! -d "${dir}" ]; then' . "\n"
650 . ' rg_notify " Dir ${dir} not found."' . "\n"
636 . ' rg_log N " Dir ${dir} not found."' . "\n"
651 637 . ' return 0' . "\n" . ' return 0' . "\n"
652 638 . ' fi' . "\n" . ' fi' . "\n"
653 639 . ' (cd "${dir}"; find . -type f -name "*.deb" -o -name "*.ddeb") > "${pkg_list_file}"' . "\n" . ' (cd "${dir}"; find . -type f -name "*.deb" -o -name "*.ddeb") > "${pkg_list_file}"' . "\n"
654 640 . ' if [ ! -s "${pkg_list_file}" ]; then' . "\n" . ' if [ ! -s "${pkg_list_file}" ]; then' . "\n"
655 . ' rg_notify " No files!"' . "\n"
641 . ' rg_log N " No files!"' . "\n"
656 642 . ' return 0' . "\n" . ' return 0' . "\n"
657 643 . ' fi' . "\n" . ' fi' . "\n"
658 . ' rg_notify " content:" $(< "${pkg_list_file}")' . "\n"
644 . ' rg_log N " content:" $(< "${pkg_list_file}")' . "\n"
659 645 . '}' . "\n" . '}' . "\n"
660 646 . "\n" . "\n"
661 647 . 'function rg_create_list()' . "\n" . 'function rg_create_list()' . "\n"
 
... ... function start_worker_build_tools($job, &$reason, &$reason2)
663 649 . ' local dir=${1}; shift' . "\n" . ' local dir=${1}; shift' . "\n"
664 650 . ' local file=${1}; shift' . "\n" . ' local file=${1}; shift' . "\n"
665 651 . "\n" . "\n"
666 . ' rg_notify "rg_create_list: file=${file} dir=${dir}"' . "\n"
652 . ' rg_log N "rg_create_list: file=${file} dir=${dir}"' . "\n"
667 653 . ' if [ ! -d "${dir}" ]; then' . "\n" . ' if [ ! -d "${dir}" ]; then' . "\n"
668 . ' rg_notify " Dir ${dir} not found."' . "\n"
654 . ' rg_log N " Dir ${dir} not found."' . "\n"
669 655 . ' return 0' . "\n" . ' return 0' . "\n"
670 656 . ' fi' . "\n" . ' fi' . "\n"
671 657 . ' (cd "${dir}"; find . -type f | grep -v "\.list\$") > "${file}"' . "\n" . ' (cd "${dir}"; find . -type f | grep -v "\.list\$") > "${file}"' . "\n"
672 658 . ' if [ ! -s "${file}" ]; then' . "\n" . ' if [ ! -s "${file}" ]; then' . "\n"
673 . ' rg_notify " No files!"' . "\n"
659 . ' rg_log N " No files!"' . "\n"
674 660 . ' return 0' . "\n" . ' return 0' . "\n"
675 661 . ' fi' . "\n" . ' fi' . "\n"
676 . ' rg_notify " content:" $(< "${file}")' . "\n"
662 . ' rg_log N " content:" $(< "${file}")' . "\n"
677 663 . '}' . "\n" . '}' . "\n"
678 664 . "\n" . "\n"
679 665 . 'function rg_rpm_createrepo()' . "\n" . 'function rg_rpm_createrepo()' . "\n"
 
... ... function start_worker_build_tools($job, &$reason, &$reason2)
683 669 . ' local pkg_list_file=${1}; shift' . "\n" . ' local pkg_list_file=${1}; shift' . "\n"
684 670 . ' local oldpackagedirs=${1}; shift' . "\n" . ' local oldpackagedirs=${1}; shift' . "\n"
685 671 . "\n" . "\n"
686 . ' rg_notify "rg_rpm_createrepo dir=${dir} pkg_list_file=${pkg_list_file} oldpackagedirs=${oldpackagedirs}"' . "\n"
672 . ' rg_log N "rg_rpm_createrepo dir=${dir} pkg_list_file=${pkg_list_file} oldpackagedirs=${oldpackagedirs}"' . "\n"
687 673 . ' rm -rf "${dir}/.repodata"' . "\n" . ' rm -rf "${dir}/.repodata"' . "\n"
688 674 . "\n" . "\n"
689 675 . ' local extra_paras=""' . "\n" . ' local extra_paras=""' . "\n"
 
... ... function start_worker_build_tools($job, &$reason, &$reason2)
693 679 . "\n" . "\n"
694 680 . ' local old_dir="${PWD}"' . "\n" . ' local old_dir="${PWD}"' . "\n"
695 681 . ' cd "${dir}"' . "\n" . ' cd "${dir}"' . "\n"
696 . ' rg_notify " Switch to dir ${dir}"' . "\n"
697 . ' rg_notify "DEBUG: Files:"' . "\n"
682 . ' rg_log N " Switch to dir ${dir}"' . "\n"
683 . ' rg_log N "DEBUG: Files:"' . "\n"
698 684 . ' find .' . "\n" . ' find .' . "\n"
699 685 . ' rg_op_s createrepo -v --update ${extra_paras} \\' . "\n" . ' rg_op_s createrepo -v --update ${extra_paras} \\' . "\n"
700 686 . ' --recycle-pkglist \\' . "\n" . ' --recycle-pkglist \\' . "\n"
 
... ... function start_worker_build_tools($job, &$reason, &$reason2)
713 699 . '{' . "\n" . '{' . "\n"
714 700 . ' local dir=${1}; shift' . "\n" . ' local dir=${1}; shift' . "\n"
715 701 . "\n" . "\n"
716 . ' rg_notify "Extracting .spec info"' . "\n"
717 . ' rg_notify "rg_rpm_extract_info: dir=${dir}"' . "\n"
702 . ' rg_log N "Extracting .spec info"' . "\n"
703 . ' rg_log N "rg_rpm_extract_info: dir=${dir}"' . "\n"
718 704 . ' spec_file=`find "${dir}" -iname \'*.spec\' -type f -printf "%d %p\n" | sort -n | head -n1 | awk \'{print $2}\'`' . "\n" . ' spec_file=`find "${dir}" -iname \'*.spec\' -type f -printf "%d %p\n" | sort -n | head -n1 | awk \'{print $2}\'`' . "\n"
719 705 . ' if [ "${spec_file}" = "" ]; then' . "\n" . ' if [ "${spec_file}" = "" ]; then' . "\n"
720 . ' rg_notify " Cannot find any .spec file in ${dir}!"' . "\n"
706 . ' rg_log N " Cannot find any .spec file in ${dir}!"' . "\n"
721 707 . ' return' . "\n" . ' return' . "\n"
722 708 . ' fi' . "\n" . ' fi' . "\n"
723 709 . "\n" . "\n"
724 710 . ' spec_file_base=`basename "${spec_file}"`' . "\n" . ' spec_file_base=`basename "${spec_file}"`' . "\n"
725 . ' rg_notify " .spec file: ${spec_file} basename=[${spec_file_base}]."' . "\n"
711 . ' rg_log N " .spec file: ${spec_file} basename=[${spec_file_base}]."' . "\n"
726 712 . "\n" . "\n"
727 . ' rg_notify " Extracting info..."' . "\n"
713 . ' rg_log N " Extracting info..."' . "\n"
728 714 . ' pkg_name=`grep ^Name: "${spec_file}" | cut -d: -f2| tr -d " \t\r\n"`' . "\n" . ' pkg_name=`grep ^Name: "${spec_file}" | cut -d: -f2| tr -d " \t\r\n"`' . "\n"
729 715 . ' pkg_ver=`grep ^Version: "${spec_file}" | cut -d: -f2 | tr -d " \t\r\n"`' . "\n" . ' pkg_ver=`grep ^Version: "${spec_file}" | cut -d: -f2 | tr -d " \t\r\n"`' . "\n"
730 716 . ' pkg_rel=`grep ^Release: "${spec_file}" | cut -d: -f2 | tr -d " \t\r\n"`' . "\n" . ' pkg_rel=`grep ^Release: "${spec_file}" | cut -d: -f2 | tr -d " \t\r\n"`' . "\n"
731 717 . ' pkg_deps=`rpmspec --parse "${spec_file}" | grep ^BuildRequires | cut -d: -f2- | tr ",\t\r\n" " "`' . "\n" . ' pkg_deps=`rpmspec --parse "${spec_file}" | grep ^BuildRequires | cut -d: -f2- | tr ",\t\r\n" " "`' . "\n"
732 . ' rg_notify " DEBUG: name=${pkg_name}, ver=${pkg_ver}, rel=${pkg_rel}"' . "\n"
718 . ' rg_log N " DEBUG: name=${pkg_name}, ver=${pkg_ver}, rel=${pkg_rel}"' . "\n"
733 719 . ' echo' . "\n" . ' echo' . "\n"
734 720 . '}' . "\n" . '}' . "\n"
735 721 . "\n" . "\n"
 
... ... function start_worker_build_tools($job, &$reason, &$reason2)
737 723 . '{' . "\n" . '{' . "\n"
738 724 . ' local dir=${1}; shift' . "\n" . ' local dir=${1}; shift' . "\n"
739 725 . "\n" . "\n"
740 . ' rg_notify "Extracting deb info dir=${dir}"' . "\n"
726 . ' rg_log N "Extracting deb info dir=${dir}"' . "\n"
741 727 . ' control_file=`find "${dir}" -iname control -type f -printf "%d %p\n" | sort -n | head -n1 | awk \'{print $2}\'`' . "\n" . ' control_file=`find "${dir}" -iname control -type f -printf "%d %p\n" | sort -n | head -n1 | awk \'{print $2}\'`' . "\n"
742 728 . ' if [ "${control_file}" = "" ]; then' . "\n" . ' if [ "${control_file}" = "" ]; then' . "\n"
743 . ' rg_notify_err " Cannot find any control file in ${dir}!"' . "\n"
729 . ' rg_log E " Cannot find any control file in ${dir}!"' . "\n"
744 730 . ' return' . "\n" . ' return' . "\n"
745 731 . ' fi' . "\n" . ' fi' . "\n"
746 732 . "\n" . "\n"
747 . ' rg_notify " Extracting info..."' . "\n"
733 . ' rg_log N " Extracting info..."' . "\n"
748 734 . ' pkg_name=$(grep ^Source: "${control_file}" | cut -d: -f2| tr -d " \t\r\n")' . "\n" . ' pkg_name=$(grep ^Source: "${control_file}" | cut -d: -f2| tr -d " \t\r\n")' . "\n"
749 735 . ' pkg_deps=$(grep ^Build-Depends: "${control_file}" | cut -d: -f2- | tr ",\t\r\n" " " | sed -e \'s/\${[^}]*}//g\')' . "\n" . ' pkg_deps=$(grep ^Build-Depends: "${control_file}" | cut -d: -f2- | tr ",\t\r\n" " " | sed -e \'s/\${[^}]*}//g\')' . "\n"
750 . ' rg_notify " DEBUG: name=${pkg_name}"' . "\n"
736 . ' rg_log N " DEBUG: name=${pkg_name}"' . "\n"
751 737 . ' echo' . "\n" . ' echo' . "\n"
752 738 . '}' . "\n" . '}' . "\n"
753 739 . "\n" . "\n"
 
... ... function start_worker_build_tools($job, &$reason, &$reason2)
759 745 . ' while [ 1 ]; do' . "\n" . ' while [ 1 ]; do' . "\n"
760 746 . ' rg_rpm_extract_info "${dir}"' . "\n" . ' rg_rpm_extract_info "${dir}"' . "\n"
761 747 . ' if [ "${pkg_name}" = "" ]; then' . "\n" . ' if [ "${pkg_name}" = "" ]; then' . "\n"
762 . ' rg_notify "Malformed .spec file?"' . "\n"
748 . ' rg_log N "Malformed .spec file?"' . "\n"
763 749 . ' E=1' . "\n" . ' E=1' . "\n"
764 750 . ' break' . "\n" . ' break' . "\n"
765 751 . ' fi' . "\n" . ' fi' . "\n"
766 752 . "\n" . "\n"
767 753 . ' local dst_tar_gz="/mnt/rpmbuild/SOURCES/${pkg_name}-${pkg_ver}.tar.gz"' . "\n" . ' local dst_tar_gz="/mnt/rpmbuild/SOURCES/${pkg_name}-${pkg_ver}.tar.gz"' . "\n"
768 754 . ' local dst_spec="/mnt/rpmbuild/SPECS/${spec_file_base}"' . "\n" . ' local dst_spec="/mnt/rpmbuild/SPECS/${spec_file_base}"' . "\n"
769 . ' local rg_notify "DEBUG: dst_spec=${dst_spec}."' . "\n"
755 . ' local rg_log N "DEBUG: dst_spec=${dst_spec}."' . "\n"
770 756 . "\n" . "\n"
771 . ' rg_notify "Creating tar archive [${dst_tar_gz}]"' . "\n"
772 . ' rg_notify "Creating tar"' . "\n"
757 . ' rg_log N "Creating tar archive [${dst_tar_gz}]"' . "\n"
758 . ' rg_log N "Creating tar"' . "\n"
773 759 . ' rg_op_s tar czf "${dst_tar_gz}" --transform "s:^:./${pkg_name}-${pkg_ver}/:g" \\' . "\n" . ' rg_op_s tar czf "${dst_tar_gz}" --transform "s:^:./${pkg_name}-${pkg_ver}/:g" \\' . "\n"
774 760 . ' -C "${dir}" . || break' . "\n" . ' -C "${dir}" . || break' . "\n"
775 761 . "\n" . "\n"
776 . ' rg_notify "Copying spec file [${spec_file}] -> [${dst_spec}]"' . "\n"
762 . ' rg_log N "Copying spec file [${spec_file}] -> [${dst_spec}]"' . "\n"
777 763 . ' rg_cp "${spec_file}" "${dst_spec}"' . "\n" . ' rg_cp "${spec_file}" "${dst_spec}"' . "\n"
778 764 . "\n" . "\n"
779 . ' rg_notify "Making only binary packages"' . "\n"
780 . ' rg_notify "Creating rpm"' . "\n"
765 . ' rg_log N "Making only binary packages"' . "\n"
766 . ' rg_log N "Creating rpm"' . "\n"
781 767 . ' rg_op_s rpmbuild --define "_topdir /mnt/rpmbuild" \\' . "\n" . ' rg_op_s rpmbuild --define "_topdir /mnt/rpmbuild" \\' . "\n"
782 768 . ' -bb "${dst_spec}" || break' . "\n" . ' -bb "${dst_spec}" || break' . "\n"
783 769 . ' break' . "\n" . ' break' . "\n"
 
... ... function start_worker_build_tools($job, &$reason, &$reason2)
789 775 . 'function rg_build_rpm()' . "\n" . 'function rg_build_rpm()' . "\n"
790 776 . '{' . "\n" . '{' . "\n"
791 777 . ' local dir=${1}; shift' . "\n" . ' local dir=${1}; shift' . "\n"
792 . "\n"
778 . "\n"
793 779 . ' rg_log_push "rg_build_rpm: dir=${dir}"' . "\n" . ' rg_log_push "rg_build_rpm: dir=${dir}"' . "\n"
794 780 . ' while [ 1 ]; do' . "\n" . ' while [ 1 ]; do' . "\n"
795 781 . ' rg_rpm_extract_info "${dir}"' . "\n" . ' rg_rpm_extract_info "${dir}"' . "\n"
796 782 . ' if [ "${pkg_name}" = "" ]; then' . "\n" . ' if [ "${pkg_name}" = "" ]; then' . "\n"
797 . ' rg_notify_err "Malformed .spec file?"' . "\n"
783 . ' rg_log E "Malformed .spec file?"' . "\n"
798 784 . ' E=1' . "\n" . ' E=1' . "\n"
799 785 . ' break' . "\n" . ' break' . "\n"
800 786 . ' fi' . "\n" . ' fi' . "\n"
801 787 . ' local dst_tar_gz="/mnt/rpmbuild/SOURCES/${repo_username}+${pkg_name}-${pkg_ver}.tar.gz"' . "\n" . ' local dst_tar_gz="/mnt/rpmbuild/SOURCES/${repo_username}+${pkg_name}-${pkg_ver}.tar.gz"' . "\n"
802 788 . ' local dst_spec="/mnt/rpmbuild/SPECS/${repo_username}+${spec_file_base}"' . "\n" . ' local dst_spec="/mnt/rpmbuild/SPECS/${repo_username}+${spec_file_base}"' . "\n"
803 . "\n"
804 . ' rg_notify "Creating archive from git repo [${dst_tar_gz}]"' . "\n"
805 . ' rg_notify "Creating tar"' . "\n"
789 . "\n"
790 . ' rg_log N "Creating tar archive from git repo [${dst_tar_gz}]"' . "\n"
806 791 . ' rg_op_s git -C "${dir}" archive -v \\' . "\n" . ' rg_op_s git -C "${dir}" archive -v \\' . "\n"
807 792 . ' --prefix "./${repo_username}+${pkg_name}-${pkg_ver}/" \\' . "\n" . ' --prefix "./${repo_username}+${pkg_name}-${pkg_ver}/" \\' . "\n"
808 793 . ' -o "${dst_tar_gz}" rgw || break' . "\n" . ' -o "${dst_tar_gz}" rgw || break' . "\n"
809 . "\n"
810 . ' rg_notify "Processing spec file [${spec_file}] -> [${dst_spec}]"' . "\n"
794 . "\n"
795 . ' rg_log N "Processing spec file [${spec_file}] -> [${dst_spec}]"' . "\n"
811 796 . ' rg_op sed -r -e "s|^Name:(\s*)(.*)(\s*)|Name:\1${repo_username}+\2\3|" \\' . "\n" . ' rg_op sed -r -e "s|^Name:(\s*)(.*)(\s*)|Name:\1${repo_username}+\2\3|" \\' . "\n"
812 797 . ' "${spec_file}" > "${dst_spec}" || break' . "\n" . ' "${spec_file}" > "${dst_spec}" || break' . "\n"
813 . "\n"
814 . ' rg_notify "Making source and binary packages"' . "\n"
815 . ' rg_notify "Creating source and binary packages"' . "\n"
798 . "\n"
799 . ' rg_log N "Creating source and binary packages"' . "\n"
816 800 . ' rg_op_s rpmbuild --define "_topdir /mnt/rpmbuild" \\' . "\n" . ' rg_op_s rpmbuild --define "_topdir /mnt/rpmbuild" \\' . "\n"
817 801 . ' -ba "${dst_spec}" || break' . "\n" . ' -ba "${dst_spec}" || break' . "\n"
818 . ' rg_notify "DEBUG: rpmbuild error code: E=${E}"' . "\n"
819 802 . ' break' . "\n" . ' break' . "\n"
820 803 . ' done' . "\n" . ' done' . "\n"
821 804 . ' rg_log_pop' . "\n" . ' rg_log_pop' . "\n"
 
... ... function start_worker_build_tools($job, &$reason, &$reason2)
832 815 . ' while [ 1 ]; do' . "\n" . ' while [ 1 ]; do' . "\n"
833 816 . ' rg_deb_extract_info "${dir}"' . "\n" . ' rg_deb_extract_info "${dir}"' . "\n"
834 817 . ' if [ "${pkg_name}" = "" ]; then' . "\n" . ' if [ "${pkg_name}" = "" ]; then' . "\n"
835 . ' rg_notify_err "Malformed debian/control file?"' . "\n"
818 . ' rg_log E "Malformed debian/control file?"' . "\n"
836 819 . ' E=1' . "\n" . ' E=1' . "\n"
837 820 . ' break' . "\n" . ' break' . "\n"
838 821 . ' fi' . "\n" . ' fi' . "\n"
839 822 . "\n" . "\n"
840 . ' rg_notify "Processing control file"' . "\n"
823 . ' rg_log N "Processing control file"' . "\n"
841 824 . ' # Changing Source: and Package: to prepend the user name' . "\n" . ' # Changing Source: and Package: to prepend the user name' . "\n"
842 825 . ' rg_op sed -i -r \\' . "\n" . ' rg_op sed -i -r \\' . "\n"
843 826 . ' -e "s#(Source|Package):(\s*)(.*)#\1:\2${repo_username_allowed}+\3#" \\' . "\n" . ' -e "s#(Source|Package):(\s*)(.*)#\1:\2${repo_username_allowed}+\3#" \\' . "\n"
 
... ... function start_worker_build_tools($job, &$reason, &$reason2)
846 829 . ' -e "s#^(\S+.*)\$#${repo_username_allowed}+\1#" \\' . "\n" . ' -e "s#^(\S+.*)\$#${repo_username_allowed}+\1#" \\' . "\n"
847 830 . ' "debian/changelog" || break' . "\n" . ' "debian/changelog" || break' . "\n"
848 831 . "\n" . "\n"
849 . ' rg_notify "Creating source and binary packages"' . "\n"
832 . ' rg_log N "Creating source and binary packages"' . "\n"
850 833 . ' rg_op_s dpkg-buildpackage --no-check-builddeps \\' . "\n" . ' rg_op_s dpkg-buildpackage --no-check-builddeps \\' . "\n"
851 834 . ' -uc -us || break' . "\n" . ' -uc -us || break' . "\n"
852 835 . ' # Because Ubuntu generates .ddeb files we need to rename them' . "\n" . ' # Because Ubuntu generates .ddeb files we need to rename them' . "\n"
853 836 . ' cd ..' . "\n" . ' cd ..' . "\n"
854 837 . ' while read f; do' . "\n" . ' while read f; do' . "\n"
855 . ' rg_notify "Renaming ${f} to ${f%.ddeb}.deb..."' . "\n"
838 . ' rg_log N "Renaming ${f} to ${f%.ddeb}.deb..."' . "\n"
856 839 . ' mv -fv "${f}" "${f%.ddeb}.deb"' . "\n" . ' mv -fv "${f}" "${f%.ddeb}.deb"' . "\n"
857 840 . ' done < <(ls *.ddeb 2>/dev/null)' . "\n" . ' done < <(ls *.ddeb 2>/dev/null)' . "\n"
858 841 . ' break' . "\n" . ' break' . "\n"
859 842 . ' done' . "\n" . ' done' . "\n"
860 843 . ' cd "${old_dir}"' . "\n" . ' cd "${old_dir}"' . "\n"
861 . ' rg_notify "Done E=${E}"' . "\n"
862 844 . ' rg_log_pop' . "\n" . ' rg_log_pop' . "\n"
863 845 . ' return ${E}' . "\n" . ' return ${E}' . "\n"
864 846 . '}' . "\n" . '}' . "\n"
 
... ... function start_worker_build_rgfs($job, &$reason, &$reason2)
992 974 . "\n" . "\n"
993 975 . 'set -e' . "\n" . 'set -e' . "\n"
994 976 . "\n" . "\n"
995 . 'rg_notify "Starting rgfs building"' . "\n"
977 . 'rg_log N "Starting rgfs building"' . "\n"
996 978 . "\n" . "\n"
997 979 . 'if [ -x /usr/bin/rgfs ]; then' . "\n" . 'if [ -x /usr/bin/rgfs ]; then' . "\n"
998 . ' rg_notify "rgfs binary already exists. Exit."' . "\n"
980 . ' rg_log N "rgfs binary already exists. Exit."' . "\n"
999 981 . ' exit 0' . "\n" . ' exit 0' . "\n"
1000 982 . 'fi' . "\n" . 'fi' . "\n"
1001 983 . "\n" . "\n"
 
... ... function start_worker_build_rgfs($job, &$reason, &$reason2)
1004 986 . './configure' . "\n" . './configure' . "\n"
1005 987 . 'make' . "\n" . 'make' . "\n"
1006 988 . 'make install' . "\n" . 'make install' . "\n"
1007 . 'rg_notify "rgfs building done"' . "\n";
989 . 'rg_log N "rgfs building done"' . "\n";
1008 990
1009 991 $f = $job['main'] . '/root/build_rgfs.sh'; $f = $job['main'] . '/root/build_rgfs.sh';
1010 992 $r = @file_put_contents($f, $s); $r = @file_put_contents($f, $s);
 
... ... function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1082 1064 . 'source /mnt/build_tools.sh' . "\n" . 'source /mnt/build_tools.sh' . "\n"
1083 1065 . 'id' . "\n" . 'id' . "\n"
1084 1066 . "\n" . "\n"
1085 . 'rg_notify "I: job_id=${job_id} orig_job_id=${orig_job_id} job_uid=${job_uid} repo_username=${repo_username}"' . "\n"
1086 . 'rg_notify "I: distro=${env_distro} major=${env_major} arch=${env_arch}"' . "\n"
1087 . 'rg_notify "I: do_sensitive_ops=$(do_sensitive_ops)"' . "\n"
1067 . 'rg_log N "I: job_id=${job_id} orig_job_id=${orig_job_id} job_uid=${job_uid} repo_username=${repo_username}"' . "\n"
1068 . 'rg_log N "I: distro=${env_distro} major=${env_major} arch=${env_arch}"' . "\n"
1069 . 'rg_log N "I: do_sensitive_ops=$(do_sensitive_ops)"' . "\n"
1088 1070 . "\n" . "\n"
1089 1071 . 'while [ 1 ]; do' . 'while [ 1 ]; do'
1090 1072 . ' echo ' . "\n" . ' echo ' . "\n"
1091 1073 . ' case "${env_distro}" in' . "\n" . ' case "${env_distro}" in' . "\n"
1092 . ' fedora|centos|rocky) sp="/mnt/rpmbuild"; name="*.rpm" ;;' . "\n"
1074 . ' fedora|centos|rocky|alma) sp="/mnt/rpmbuild"; name="*.rpm" ;;' . "\n"
1093 1075 . ' debian|ubuntu) sp="/mnt/target"; name="*.deb" ;;' . "\n" . ' debian|ubuntu) sp="/mnt/target"; name="*.deb" ;;' . "\n"
1094 1076 . ' *) break ;;' . "\n" . ' *) break ;;' . "\n"
1095 1077 . ' esac' . "\n" . ' esac' . "\n"
1096 1078 . ' rg_log_push "Locating package files in ${sp}"' . "\n" . ' rg_log_push "Locating package files in ${sp}"' . "\n"
1097 1079 . ' files=$(find ${sp} -type f -iname "${name}" 2>/dev/null | head -n1)' . "\n" . ' files=$(find ${sp} -type f -iname "${name}" 2>/dev/null | head -n1)' . "\n"
1098 1080 . ' if [ "${files}" = "" ]; then' . "\n" . ' if [ "${files}" = "" ]; then' . "\n"
1099 . ' rg_notify "Not found."' . "\n"
1081 . ' rg_log N "Not found."' . "\n"
1100 1082 . ' else' . "\n" . ' else' . "\n"
1101 . ' rg_notify "Found ["${files}"]"' . "\n"
1083 . ' rg_log N "Found ["${files}"]"' . "\n"
1102 1084 . ' fi' . "\n" . ' fi' . "\n"
1103 1085 . ' rg_log_pop' . "\n" . ' rg_log_pop' . "\n"
1104 1086 . "\n"; . "\n";
 
... ... function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1106 1088 if ($keys > 0) if ($keys > 0)
1107 1089 $s .= '' $s .= ''
1108 1090 . ' echo ' . "\n" . ' echo ' . "\n"
1109 . ' rg_notify "Importing ' . $keys . ' key(s)"' . "\n"
1091 . ' rg_log N "Importing ' . $keys . ' key(s)"' . "\n"
1110 1092 . ' rg_op_s gpg --import /mnt/build2/gpg_priv_key_*.asc || break' . "\n" . ' rg_op_s gpg --import /mnt/build2/gpg_priv_key_*.asc || break' . "\n"
1111 1093 . ' # Signing something for test' . "\n" . ' # Signing something for test' . "\n"
1112 1094 . ' rm -f /tmp/rg-sign-junk' . "\n" . ' rm -f /tmp/rg-sign-junk' . "\n"
1113 1095 . ' rg_op_s gpg --output /tmp/rg-sign-junk --no-autostart --batch \\' . "\n" . ' rg_op_s gpg --output /tmp/rg-sign-junk --no-autostart --batch \\' . "\n"
1114 1096 . ' --armour --sign /etc/hosts || break' . "\n" . ' --armour --sign /etc/hosts || break' . "\n"
1115 1097 . ' # list supported configuration' . "\n" . ' # list supported configuration' . "\n"
1116 . ' rg_notify "GPG supported configuration:"' . "\n"
1098 . ' rg_log N "GPG supported configuration:"' . "\n"
1117 1099 . ' gpg --with-colons --list-config' . "\n"; . ' gpg --with-colons --list-config' . "\n";
1118 1100
1119 1101 $s .= "\n" $s .= "\n"
 
... ... function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1124 1106 . ' echo' . "\n" . ' echo' . "\n"
1125 1107 . ' rg_log_push "== prepare pkgs phase ====="' . "\n" . ' rg_log_push "== prepare pkgs phase ====="' . "\n"
1126 1108 . ' case "${env_distro}" in' . "\n" . ' case "${env_distro}" in' . "\n"
1127 . ' fedora|centos|rocky)' . "\n"
1109 . ' fedora|centos|rocky|alma)' . "\n"
1128 1110 . ' src0="/mnt/rpmbuild"' . "\n" . ' src0="/mnt/rpmbuild"' . "\n"
1129 1111 . ' src1="${src0}/RPMS"' . "\n" . ' src1="${src0}/RPMS"' . "\n"
1130 1112 . ' src2="${src0}/SRPMS"' . "\n" . ' src2="${src0}/SRPMS"' . "\n"
1131 . "\n"
1113 . "\n"
1132 1114 . ' for i in debug src os; do' . "\n" . ' for i in debug src os; do' . "\n"
1133 1115 . ' rg_log_push "== ${i} ====="' . "\n" . ' rg_log_push "== ${i} ====="' . "\n"
1134 1116 . ' for pkg_repo_id in ${pkg_repos_list}; do' . "\n" . ' for pkg_repo_id in ${pkg_repos_list}; do' . "\n"
 
... ... function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1136 1118 . ' rg_log_push "== pkg_repo_id ${pkg_repo_id} [uid ${pkg_repo_uid}] ====="' . "\n" . ' rg_log_push "== pkg_repo_id ${pkg_repo_id} [uid ${pkg_repo_uid}] ====="' . "\n"
1137 1119 . ' dir="/mnt/build2/rpms/${pkg_repo_id}/${i}"' . "\n" . ' dir="/mnt/build2/rpms/${pkg_repo_id}/${i}"' . "\n"
1138 1120 . ' if [ "`do_sensitive_ops`" = "1" ]; then' . "\n" . ' if [ "`do_sensitive_ops`" = "1" ]; then' . "\n"
1139 . ' rg_notify "DEBUG: do_sensitive_ops==1: do not add username"' . "\n"
1121 . ' rg_log N "DEBUG: do_sensitive_ops==1: do not add username"' . "\n"
1140 1122 . ' udir="${dir}"' . "\n" . ' udir="${dir}"' . "\n"
1141 1123 . ' else' . "\n" . ' else' . "\n"
1142 . ' rg_notify "DEBUG: do_sensitive_ops!=1: add username"' . "\n"
1124 . ' rg_log N "DEBUG: do_sensitive_ops!=1: add username"' . "\n"
1143 1125 . ' udir="${dir}/${repo_username}"' . "\n" . ' udir="${dir}/${repo_username}"' . "\n"
1144 1126 . ' fi' . "\n" . ' fi' . "\n"
1145 . "\n"
1146 . ' rg_notify "Copying RPMs to ${udir}"' . "\n"
1127 . "\n"
1128 . ' rg_log N "Copying RPMs to ${udir}"' . "\n"
1147 1129 . ' rg_op_s mkdir -p "${udir}" || break' . "\n" . ' rg_op_s mkdir -p "${udir}" || break' . "\n"
1148 1130 . ' case ${i} in' . "\n" . ' case ${i} in' . "\n"
1149 1131 . ' debug)' . "\n" . ' debug)' . "\n"
 
... ... function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1170 1152 . ' done' . "\n" . ' done' . "\n"
1171 1153 . ' rg_log_pop' . "\n" . ' rg_log_pop' . "\n"
1172 1154 . ' [ "${E}" = "0" ] || break' . "\n" . ' [ "${E}" = "0" ] || break' . "\n"
1173 . "\n"
1155 . "\n"
1174 1156 . ' # We must remove debug packages after copy, else we will copy them also in "os" dir' . "\n" . ' # We must remove debug packages after copy, else we will copy them also in "os" dir' . "\n"
1175 1157 . ' if [ "${i}" = "debug" ]; then' . "\n" . ' if [ "${i}" = "debug" ]; then' . "\n"
1176 1158 . ' rm -f "${src1}"/*/*-debuginfo-*.rpm' . "\n" . ' rm -f "${src1}"/*/*-debuginfo-*.rpm' . "\n"
 
... ... function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1178 1160 . ' fi' . "\n" . ' fi' . "\n"
1179 1161 . ' done' . "\n" . ' done' . "\n"
1180 1162 . ' ;;' . "\n" . ' ;;' . "\n"
1181 . "\n"
1163 . "\n"
1182 1164 . ' debian|ubuntu)' . "\n" // TODO: we should check if no package found and return error . ' debian|ubuntu)' . "\n" // TODO: we should check if no package found and return error
1183 1165 . ' src0="/mnt/target"' . "\n" . ' src0="/mnt/target"' . "\n"
1184 1166 . ' rg_deb_extract_control "${src0}" || break' . "\n" . ' rg_deb_extract_control "${src0}" || break' . "\n"
 
... ... function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1188 1170 . ' pkg_repo_uid=${pri2uid[${pkg_repo_id}]}' . "\n" . ' pkg_repo_uid=${pri2uid[${pkg_repo_id}]}' . "\n"
1189 1171 . ' rg_log_push "== pkg_repo_id ${pkg_repo_id} [uid ${pkg_repo_uid}] ====="' . "\n" . ' rg_log_push "== pkg_repo_id ${pkg_repo_id} [uid ${pkg_repo_uid}] ====="' . "\n"
1190 1172 . ' dir="/mnt/build2/debs/${pkg_repo_id}/${i}"' . "\n" . ' dir="/mnt/build2/debs/${pkg_repo_id}/${i}"' . "\n"
1191 . "\n"
1173 . "\n"
1192 1174 . ' copy_add=""' . "\n" . ' copy_add=""' . "\n"
1193 1175 . ' case ${i} in' . "\n" . ' case ${i} in' . "\n"
1194 1176 . ' source)' . "\n" . ' source)' . "\n"
 
... ... function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1204 1186 . ' ;;' . "\n" . ' ;;' . "\n"
1205 1187 . ' esac 2>/dev/null' . "\n" . ' esac 2>/dev/null' . "\n"
1206 1188 . ' if [ "${list}" != "" ]; then' . "\n" . ' if [ "${list}" != "" ]; then' . "\n"
1207 . ' rg_notify "Copying files from ${src0} to ${dir}"' . "\n"
1208 . ' ls -l "${src0}"' . "\n"
1189 . ' rg_log N "Copying files from ${src0} to ${dir}"' . "\n"
1209 1190 . ' rg_op_s mkdir -p "${dir}" || break' . "\n" . ' rg_op_s mkdir -p "${dir}" || break' . "\n"
1210 1191 . ' for f in ${list}; do' . "\n" . ' for f in ${list}; do' . "\n"
1211 . ' rg_notify "Copying ${f}..."' . "\n"
1192 . ' rg_log N "Copying ${f}..."' . "\n"
1212 1193 . ' rg_cp -v ${copy_add} "${f}" "${dir}/"' . "\n" . ' rg_cp -v ${copy_add} "${f}" "${dir}/"' . "\n"
1213 1194 . ' done' . "\n" . ' done' . "\n"
1214 1195 . ' fi' . "\n" . ' fi' . "\n"
 
... ... function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1221 1202 . ' rg_log_pop' . "\n" . ' rg_log_pop' . "\n"
1222 1203 . ' [ "${E}" = "0" ] || break' . "\n" . ' [ "${E}" = "0" ] || break' . "\n"
1223 1204 . ' ;;' . "\n" . ' ;;' . "\n"
1224 . "\n"
1225 . ' *) rg_notify_err "Unsupported distro!"; E=0 ;;' . "\n" // TODO: should we return 0? What about supported case? We should not fail the build job! Only if user wants that!
1205 . "\n"
1206 . ' *) rg_log E "Unsupported distro!"; E=0 ;;' . "\n" // TODO: should we return 0? What about supported case? We should not fail the build job! Only if user wants that!
1226 1207 . ' esac' . "\n" . ' esac' . "\n"
1227 1208 . ' rg_log_pop' . "\n" . ' rg_log_pop' . "\n"
1228 1209 . ' [ "${E}" = "0" ] || break' . "\n\n"; . ' [ "${E}" = "0" ] || break' . "\n\n";
 
... ... function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1235 1216 . ' echo' . "\n" . ' echo' . "\n"
1236 1217 . ' rg_log_push "== create list and sign pkg_repo_id ${pkg_repo_id} ====="' . "\n" . ' rg_log_push "== create list and sign pkg_repo_id ${pkg_repo_id} ====="' . "\n"
1237 1218 . ' case "${env_distro}" in' . "\n" . ' case "${env_distro}" in' . "\n"
1238 . ' fedora|centos|rocky)' . "\n"
1219 . ' fedora|centos|rocky|alma)' . "\n"
1239 1220 . ' for i in debug src os; do' . "\n" . ' for i in debug src os; do' . "\n"
1240 1221 . ' rg_log_push "== ${i} ====="' . "\n" . ' rg_log_push "== ${i} ====="' . "\n"
1241 1222 . ' dir="/mnt/build2/rpms/${pkg_repo_id}/${i}"' . "\n" . ' dir="/mnt/build2/rpms/${pkg_repo_id}/${i}"' . "\n"
1242 1223 . ' pkg_list_file="${dir}/pkg.list"' . "\n" . ' pkg_list_file="${dir}/pkg.list"' . "\n"
1243 . "\n"
1224 . "\n"
1244 1225 . ' while [ 1 ]; do' . "\n" . ' while [ 1 ]; do' . "\n"
1245 1226 . ' rg_rpm_create_list "${dir}" "${pkg_list_file}" || break' . "\n" . ' rg_rpm_create_list "${dir}" "${pkg_list_file}" || break' . "\n"
1246 . "\n"
1227 . "\n"
1247 1228 . ' if [ "`do_sensitive_ops`" != "1" ]; then' . "\n" . ' if [ "`do_sensitive_ops`" != "1" ]; then' . "\n"
1248 . ' rg_notify "DEBUG: do_sensitive_ops!=1: skip rpm signing"' . "\n"
1229 . ' rg_log N "DEBUG: do_sensitive_ops!=1: skip rpm signing"' . "\n"
1249 1230 . ' else' . "\n" . ' else' . "\n"
1250 . ' rg_notify "DEBUG: do_sensitive_ops==1: do signing"' . "\n"
1231 . ' rg_log N "DEBUG: do_sensitive_ops==1: do signing"' . "\n"
1251 1232 . ' rg_rpm_sign ${pkg_repo_id} "${dir}" "${pkg_list_file}" || break' . "\n" . ' rg_rpm_sign ${pkg_repo_id} "${dir}" "${pkg_list_file}" || break' . "\n"
1252 1233 . ' fi' . "\n" . ' fi' . "\n"
1253 1234 . ' break' . "\n" . ' break' . "\n"
1254 1235 . ' done' . "\n" . ' done' . "\n"
1255 1236 . ' rg_log_pop' . "\n" . ' rg_log_pop' . "\n"
1256 1237 . ' done ;;' . "\n" . ' done ;;' . "\n"
1257 . "\n"
1238 . "\n"
1258 1239 . ' debian|ubuntu)' . "\n" . ' debian|ubuntu)' . "\n"
1259 1240 . ' pdir="/mnt/build2/debs/${pkg_repo_id}"' . "\n" . ' pdir="/mnt/build2/debs/${pkg_repo_id}"' . "\n"
1260 1241 . ' for i in source binary-all binary-${env_arch}; do' . "\n" . ' for i in source binary-all binary-${env_arch}; do' . "\n"
 
... ... function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1262 1243 . ' dir="${pdir}/${i}"' . "\n" . ' dir="${pdir}/${i}"' . "\n"
1263 1244 . ' pkg_list_file="${dir}.pkg.list"' . "\n" . ' pkg_list_file="${dir}.pkg.list"' . "\n"
1264 1245 . ' copy_list_file="${dir}.copy.list"' . "\n" . ' copy_list_file="${dir}.copy.list"' . "\n"
1265 . "\n"
1246 . "\n"
1266 1247 . ' while [ 1 ]; do' . "\n" . ' while [ 1 ]; do' . "\n"
1267 1248 . ' rg_deb_create_deb_list "${dir}" "${pkg_list_file}" || break' . "\n" . ' rg_deb_create_deb_list "${dir}" "${pkg_list_file}" || break' . "\n"
1268 1249 . ' rg_create_list "${dir}" "${copy_list_file}" || break' . "\n" . ' rg_create_list "${dir}" "${copy_list_file}" || break' . "\n"
1269 1250 . ' [ -s "${pkg_list_file}" ] || break' . "\n" . ' [ -s "${pkg_list_file}" ] || break' . "\n"
1270 . "\n"
1251 . "\n"
1271 1252 . ' if [ "`do_sensitive_ops`" != "1" ]; then' . "\n" . ' if [ "`do_sensitive_ops`" != "1" ]; then' . "\n"
1272 . ' rg_notify "DEBUG: do_sensitive_ops!=1: skip .deb signing"' . "\n"
1253 . ' rg_log N "DEBUG: do_sensitive_ops!=1: skip .deb signing"' . "\n"
1273 1254 . ' else' . "\n" . ' else' . "\n"
1274 . ' rg_notify "DEBUG: do_sensitive_ops==1: do signing"' . "\n"
1255 . ' rg_log N "DEBUG: do_sensitive_ops==1: do signing"' . "\n"
1275 1256 . ' rg_deb_sign ${pkg_repo_id} "${dir}" "${pkg_list_file}" || break' . "\n" . ' rg_deb_sign ${pkg_repo_id} "${dir}" "${pkg_list_file}" || break' . "\n"
1276 1257 . ' fi' . "\n" . ' fi' . "\n"
1277 1258 . ' break' . "\n" . ' break' . "\n"
1278 1259 . ' done' . "\n" . ' done' . "\n"
1279 1260 . ' rg_log_pop' . "\n" . ' rg_log_pop' . "\n"
1280 1261 . ' done ;;' . "\n" . ' done ;;' . "\n"
1281 . "\n"
1282 . ' *) rg_notify_err "Unsupported distro!"; E=0 ;;' . "\n"
1262 . "\n"
1263 . ' *) rg_log E "Unsupported distro!"; E=0 ;;' . "\n"
1283 1264 . ' esac' . "\n" . ' esac' . "\n"
1284 1265 . ' rg_log_pop' . "\n" . ' rg_log_pop' . "\n"
1285 1266 . ' [ "${E}" = "0" ] || break' . "\n\n"; . ' [ "${E}" = "0" ] || break' . "\n\n";
 
... ... function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1294 1275 . ' export pkg_repo_id=' . $pkg_repo_id . "\n" . ' export pkg_repo_id=' . $pkg_repo_id . "\n"
1295 1276 . ' export pkg_repo_uid=' . $ri['uid'] . "\n" . ' export pkg_repo_uid=' . $ri['uid'] . "\n"
1296 1277 . ' export pkg_subrepo_id=' . $pkg_subrepo_id . "\n" . ' export pkg_subrepo_id=' . $pkg_subrepo_id . "\n"
1297 . "\n"
1278 . "\n"
1298 1279 . ' echo' . "\n" . ' echo' . "\n"
1299 1280 . ' rg_log_push "== rgfs phase pkg_repo_id=${pkg_repo_id} pkg_repo_uid=${pkg_repo_uid} pkg_subrepo_id=${pkg_subrepo_id} ====="' . "\n" . ' rg_log_push "== rgfs phase pkg_repo_id=${pkg_repo_id} pkg_repo_uid=${pkg_repo_uid} pkg_subrepo_id=${pkg_subrepo_id} ====="' . "\n"
1300 1281 . ' export RGFS_LOG="/mnt/build2/rgfs/rgfs-${pkg_repo_id}-${pkg_subrepo_id}.log"' . "\n" . ' export RGFS_LOG="/mnt/build2/rgfs/rgfs-${pkg_repo_id}-${pkg_subrepo_id}.log"' . "\n"
 
... ... function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1305 1286 . ' mkdir -p "${dir}"' . "\n" . ' mkdir -p "${dir}"' . "\n"
1306 1287 . ' # -s -> disable multithreading - not supported and not needed' . "\n" . ' # -s -> disable multithreading - not supported and not needed' . "\n"
1307 1288 . ' # Please note, we cannot run strace on rgfs!' . "\n" . ' # Please note, we cannot run strace on rgfs!' . "\n"
1308 . ' rg_notify "Mounting rgfs pkg_repo_id/pkg_subrepo_id=${pkg_repo_id}/${pkg_subrepo_id}"' . "\n"
1289 . ' rg_log N "Mounting rgfs pkg_repo_id/pkg_subrepo_id=${pkg_repo_id}/${pkg_subrepo_id}"' . "\n"
1309 1290 . ' rg_op_s /usr/bin/rgfs -s -o kernel_cache -o auto_cache \\' . "\n" . ' rg_op_s /usr/bin/rgfs -s -o kernel_cache -o auto_cache \\' . "\n"
1310 1291 . ' -o noforget -o use_ino "${dir}" || break' . "\n" . ' -o noforget -o use_ino "${dir}" || break' . "\n"
1311 1292 . ' rgfs_mounted[${pkg_repo_id}-${pkg_subrepo_id}]=1' . "\n" . ' rgfs_mounted[${pkg_repo_id}-${pkg_subrepo_id}]=1' . "\n"
1312 . "\n"
1293 . "\n"
1313 1294 . ' rgfs_subrepo_root="/mnt/build2/rgfs/${pkg_repo_id}/${pkg_subrepo_id}"' . "\n" . ' rgfs_subrepo_root="/mnt/build2/rgfs/${pkg_repo_id}/${pkg_subrepo_id}"' . "\n"
1314 1295 . ' case "${env_distro}" in' . "\n" . ' case "${env_distro}" in' . "\n"
1315 . ' fedora|centos|rocky)' . "\n"
1296 . ' fedora|centos|rocky|alma)' . "\n"
1316 1297 . ' for i in os debug src; do' . "\n" . ' for i in os debug src; do' . "\n"
1317 1298 . ' rgfs="${rgfs_subrepo_root}/${env_distro}/${env_major}/${env_arch}/${i}"' . "\n" . ' rgfs="${rgfs_subrepo_root}/${env_distro}/${env_major}/${env_arch}/${i}"' . "\n"
1318 1299 . ' rpms_dir="/mnt/build2/rpms/${pkg_repo_id}/${i}"' . "\n" . ' rpms_dir="/mnt/build2/rpms/${pkg_repo_id}/${i}"' . "\n"
 
... ... function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1320 1301 . ' pkg_filter_list_file="${rpms_dir}/pkg.filter.list"' . "\n" . ' pkg_filter_list_file="${rpms_dir}/pkg.filter.list"' . "\n"
1321 1302 . ' pkg_pend_list_file="/mnt/tmp/pkg.list"' . "\n" . ' pkg_pend_list_file="/mnt/tmp/pkg.list"' . "\n"
1322 1303 . ' > "${pkg_pend_list_file}"' . "\n" . ' > "${pkg_pend_list_file}"' . "\n"
1323 . "\n"
1304 . "\n"
1324 1305 . ' rg_log_push "== ${i} ====="' . "\n" . ' rg_log_push "== ${i} ====="' . "\n"
1325 1306 . ' mkdir -p "${rpms_dir}" # when we do not build packages, we may not have this dir' . "\n" . ' mkdir -p "${rpms_dir}" # when we do not build packages, we may not have this dir' . "\n"
1326 . "\n"
1307 . "\n"
1327 1308 . ' if [ "${job_uid}" = "0" ]; then' . "\n" . ' if [ "${job_uid}" = "0" ]; then' . "\n"
1328 1309 . ' if [ -d "${rgfs}/+pending" ]; then' . "\n" . ' if [ -d "${rgfs}/+pending" ]; then' . "\n"
1329 . ' rg_notify "DEBUG: dir +pending found"' . "\n"
1310 . ' rg_log N "DEBUG: dir +pending found"' . "\n"
1330 1311 . ' rg_rpm_create_list "${rgfs}/+pending" "${pkg_pend_list_file}" || break' . "\n" . ' rg_rpm_create_list "${rgfs}/+pending" "${pkg_pend_list_file}" || break' . "\n"
1331 1312 . ' else' . "\n" . ' else' . "\n"
1332 . ' rg_notify "DEBUG: dir +pending does not exists."' . "\n"
1313 . ' rg_log N "DEBUG: dir +pending does not exists."' . "\n"
1333 1314 . ' fi' . "\n" . ' fi' . "\n"
1334 1315 . ' fi' . "\n" . ' fi' . "\n"
1335 . "\n"
1316 . "\n"
1336 1317 . ' if [ ! -s "${pkg_list_file}" ] && [ ! -s "${pkg_pend_list_file}" ]; then' . "\n" . ' if [ ! -s "${pkg_list_file}" ] && [ ! -s "${pkg_pend_list_file}" ]; then' . "\n"
1337 . ' rg_notify "pkg_list_file/pkg_pend_list_file are both empty; continue"' . "\n"
1318 . ' rg_log N "pkg_list_file/pkg_pend_list_file are both empty; continue"' . "\n"
1338 1319 . ' rg_log_pop' . "\n" . ' rg_log_pop' . "\n"
1339 1320 . ' continue' . "\n" . ' continue' . "\n"
1340 1321 . ' fi' . "\n" . ' fi' . "\n"
1341 . "\n"
1322 . "\n"
1342 1323 . ' rg_log_push "Filtering existing packages from pkg_list_file"' . "\n" . ' rg_log_push "Filtering existing packages from pkg_list_file"' . "\n"
1343 1324 . ' > "${pkg_filter_list_file}"' . "\n" . ' > "${pkg_filter_list_file}"' . "\n"
1344 1325 . ' while read file; do' . "\n" . ' while read file; do' . "\n"
1345 1326 . ' [ "${file}" = "" ] && continue' . "\n" . ' [ "${file}" = "" ] && continue' . "\n"
1346 1327 . ' if [ -r "${rgfs}/${file}" ]; then' . "\n" . ' if [ -r "${rgfs}/${file}" ]; then' . "\n"
1347 . ' rg_notify "- ${rgfs}/${file}"' . "\n"
1328 . ' rg_log N "- ${rgfs}/${file}"' . "\n"
1348 1329 . ' continue' . "\n" . ' continue' . "\n"
1349 1330 . ' fi' . "\n" . ' fi' . "\n"
1350 . ' rg_notify "+ ${rgfs}/${file}"' . "\n"
1331 . ' rg_log N "+ ${rgfs}/${file}"' . "\n"
1351 1332 . ' echo "${file}" >> ${pkg_filter_list_file}' . "\n" . ' echo "${file}" >> ${pkg_filter_list_file}' . "\n"
1352 1333 . ' done < <(cat "${pkg_list_file}"; echo; cat "${pkg_pend_list_file}"; echo)' . "\n" . ' done < <(cat "${pkg_list_file}"; echo; cat "${pkg_pend_list_file}"; echo)' . "\n"
1353 1334 . ' rg_log_pop' . "\n" . ' rg_log_pop' . "\n"
1354 . "\n"
1335 . "\n"
1355 1336 . ' if [ "${job_uid}" = "0" ]; then' . "\n" . ' if [ "${job_uid}" = "0" ]; then' . "\n"
1356 1337 . ' if [ -d "${rgfs}/+pending" ]; then' . "\n" . ' if [ -d "${rgfs}/+pending" ]; then' . "\n"
1357 . ' rg_notify "+pending dir exists"' . "\n"
1338 . ' rg_log N "+pending dir exists"' . "\n"
1358 1339 . ' rg_rpm_sign ${pkg_repo_id} "${rgfs}/+pending" "${pkg_pend_list_file}" || break' . "\n" . ' rg_rpm_sign ${pkg_repo_id} "${rgfs}/+pending" "${pkg_pend_list_file}" || break' . "\n"
1359 1340 . ' for username in `(cd "${rgfs}/+pending"; ls)`; do' . "\n" . ' for username in `(cd "${rgfs}/+pending"; ls)`; do' . "\n"
1360 1341 . ' mkdir -p "${rgfs}/${username}"' . "\n" . ' mkdir -p "${rgfs}/${username}"' . "\n"
 
... ... function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1364 1345 . ' [ "${E}" = "0" ] || break' . "\n" . ' [ "${E}" = "0" ] || break' . "\n"
1365 1346 . ' rg_op_s rmdir "${rgfs}/+pending" || break' . "\n" . ' rg_op_s rmdir "${rgfs}/+pending" || break' . "\n"
1366 1347 . ' else' . "\n" . ' else' . "\n"
1367 . ' rg_notify "+pending dir not found"' . "\n"
1348 . ' rg_log N "+pending dir not found"' . "\n"
1368 1349 . ' fi' . "\n" . ' fi' . "\n"
1369 1350 . ' fi' . "\n" . ' fi' . "\n"
1370 . "\n"
1371 . ' rg_notify "copy_to_rgfs"' . "\n"
1372 . ' if [ -s "${pkg_list_file}" ]; then' . "\n"
1373 . ' rg_log_push "Copying packages from ${rpms_dir} to ${rgfs}"' . "\n"
1351 . "\n"
1352 . ' rg_log_push "Copying packages from ${rpms_dir} to ${rgfs}"' . "\n"
1353 . ' while [ 1 ]; do' . "\n"
1354 . ' if [ ! -s "${pkg_list_file}" ]; then' . "\n"
1355 . ' rg_log N "No packages in pkg_list_file."' . "\n"
1356 . ' break' . "\n"
1357 . ' fi' . "\n"
1374 1358 . ' if [ "${job_uid}" = "${pkg_repo_uid}" ]; then' . "\n" . ' if [ "${job_uid}" = "${pkg_repo_uid}" ]; then' . "\n"
1375 1359 . ' rg_op_s mkdir -p "${rgfs}" || break' . "\n" . ' rg_op_s mkdir -p "${rgfs}" || break' . "\n"
1376 1360 . ' rg_cp -av "${rpms_dir}"/*.rpm "${rgfs}/" || break' . "\n" . ' rg_cp -av "${rpms_dir}"/*.rpm "${rgfs}/" || break' . "\n"
 
... ... function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1379 1363 . ' rg_op_s mkdir -p "${rgfs}/+pending/${repo_username}" || break' . "\n" . ' rg_op_s mkdir -p "${rgfs}/+pending/${repo_username}" || break' . "\n"
1380 1364 . ' rg_cp -av "${rpms_dir}/${repo_username}"/*.rpm "${rgfs}/+pending/${repo_username}/" || break' . "\n" . ' rg_cp -av "${rpms_dir}/${repo_username}"/*.rpm "${rgfs}/+pending/${repo_username}/" || break' . "\n"
1381 1365 . ' fi' . "\n" . ' fi' . "\n"
1382 . "\n"
1366 . "\n"
1383 1367 . ' if [ "$(do_sensitive_ops)" != "1" ]; then' . "\n" . ' if [ "$(do_sensitive_ops)" != "1" ]; then' . "\n"
1384 . ' rg_notify "Triggering the rebuild of subrepo"' . "\n"
1368 . ' rg_log N "Triggering the rebuild of subrepo"' . "\n"
1385 1369 . ' echo "${pkg_subrepo_id}" >> /mnt/build2/pkg_subrepo_dirty' . "\n" . ' echo "${pkg_subrepo_id}" >> /mnt/build2/pkg_subrepo_dirty' . "\n"
1386 1370 . ' fi' . "\n" . ' fi' . "\n"
1387 . ' rg_log_pop' . "\n"
1388 . ' else' . "\n"
1389 . ' rg_notify "No packages in pkg_list_file."' . "\n"
1390 . ' fi'. "\n"
1391 . "\n"
1392 . ' if [ "`do_sensitive_ops`" = "1" ]; then' . "\n"
1371 . ' break' . "\n"
1372 . ' done' . "\n"
1373 . ' rg_log_pop' . "\n"
1374 . ' [ "${E}" = "0" ] || break' . "\n"
1375 . "\n"
1376 . ' while [ "`do_sensitive_ops`" = "1" ]; do' . "\n" // TODO: not sure if this block can be moved in the above while
1393 1377 . ' rg_rpm_createrepo "${env_distro}" "${rgfs}" \\' . "\n" . ' rg_rpm_createrepo "${env_distro}" "${rgfs}" \\' . "\n"
1394 1378 . ' "${pkg_filter_list_file}" "${repo_username}" || break' . "\n" . ' "${pkg_filter_list_file}" "${repo_username}" || break' . "\n"
1395 . ' fi' . "\n"
1379 . ' break' . "\n"
1380 . ' done' . "\n"
1381 . ' [ "${E}" = "0" ] || break' . "\n"
1382 . "\n"
1396 1383 . ' rg_log_pop' . "\n" . ' rg_log_pop' . "\n"
1384 . ' [ "${E}" = "0" ] || break' . "\n"
1397 1385 . ' done' . "\n" . ' done' . "\n"
1398 1386 . ' ;;' . "\n" . ' ;;' . "\n"
1399 . "\n"
1387 . "\n"
1400 1388 . ' debian|ubuntu)' . "\n" . ' debian|ubuntu)' . "\n"
1401 1389 . ' for i in source binary-all binary-${env_arch}; do' . "\n" . ' for i in source binary-all binary-${env_arch}; do' . "\n"
1402 1390 . ' rgfs="${rgfs_subrepo_root}/${env_distro}/${env_codename}/main/${i}"' . "\n" . ' rgfs="${rgfs_subrepo_root}/${env_distro}/${env_codename}/main/${i}"' . "\n"
1403 1391 . ' debs_dir="/mnt/build2/debs/${pkg_repo_id}/${i}"' . "\n" . ' debs_dir="/mnt/build2/debs/${pkg_repo_id}/${i}"' . "\n"
1404 1392 . ' pkg_list_file="${debs_dir}.pkg.list"' . "\n" . ' pkg_list_file="${debs_dir}.pkg.list"' . "\n"
1405 1393 . ' list_file="${debs_dir}.copy.list"' . "\n" . ' list_file="${debs_dir}.copy.list"' . "\n"
1406 . "\n"
1394 . "\n"
1407 1395 . ' rg_log_push "== ${i} ====="' . "\n" . ' rg_log_push "== ${i} ====="' . "\n"
1396 . ' rg_log D "TODO content of list_file: "$(< "${list_file}")"."' . "\n"
1397 . ' ls -l "${debs_dir}"' . "\n"
1408 1398 . ' while [ -s "${list_file}" ]; do' . "\n" . ' while [ -s "${list_file}" ]; do' . "\n"
1409 . ' rg_notify "Copying files from ${debs_dir} to ${rgfs}"' . "\n"
1399 . ' rg_log N "Copying files from ${debs_dir} to ${rgfs}"' . "\n"
1410 1400 . ' # "+" is used because it is not allowed in username' . "\n" . ' # "+" is used because it is not allowed in username' . "\n"
1411 1401 . ' rg_op_s mkdir -p "${rgfs}/+pending/${job_id}" || break' . "\n" . ' rg_op_s mkdir -p "${rgfs}/+pending/${job_id}" || break' . "\n"
1412 1402 . ' rg_cp -av "${debs_dir}"/* "${rgfs}/+pending/${job_id}/" || break' . "\n" . ' rg_cp -av "${debs_dir}"/* "${rgfs}/+pending/${job_id}/" || break' . "\n"
1413 1403 . ' if [ "$(do_sensitive_ops)" != "1" ]; then' . "\n" . ' if [ "$(do_sensitive_ops)" != "1" ]; then' . "\n"
1414 . ' rg_notify "Triggering the rebuild of subrepo"' . "\n"
1404 . ' rg_log N "Triggering the rebuild of subrepo"' . "\n"
1415 1405 . ' echo "${pkg_subrepo_id}" >> /mnt/build2/pkg_subrepo_dirty' . "\n" . ' echo "${pkg_subrepo_id}" >> /mnt/build2/pkg_subrepo_dirty' . "\n"
1416 1406 . ' fi' . "\n" . ' fi' . "\n"
1417 1407 . ' break' . "\n" . ' break' . "\n"
1418 1408 . ' done'. "\n" . ' done'. "\n"
1409 . ' [ "${E}" = "0" ] || break' . "\n"
1419 1410 . ' while [ "${job_uid}" = "0" ]; do' . "\n" . ' while [ "${job_uid}" = "0" ]; do' . "\n"
1420 1411 . ' [ "${orig_job_id}" = "" ] && break' . "\n" . ' [ "${orig_job_id}" = "" ] && break' . "\n"
1421 1412 . ' pkg_sign_file="/tmp/pkg.sign.list"' . "\n" . ' pkg_sign_file="/tmp/pkg.sign.list"' . "\n"
 
... ... function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1424 1415 . ' rg_deb_sign ${pkg_repo_id} "${rgfs}/+pending/${orig_job_id}" "${pkg_sign_file}" || break' . "\n" . ' rg_deb_sign ${pkg_repo_id} "${rgfs}/+pending/${orig_job_id}" "${pkg_sign_file}" || break' . "\n"
1425 1416 . ' break' . "\n" . ' break' . "\n"
1426 1417 . ' done' . "\n" . ' done' . "\n"
1418 . ' [ "${E}" = "0" ] || break' . "\n"
1419 . "\n"
1427 1420 . ' rg_log_pop' . "\n" . ' rg_log_pop' . "\n"
1428 1421 . ' [ "${E}" = "0" ] || break' . "\n" . ' [ "${E}" = "0" ] || break' . "\n"
1429 1422 . ' done' . "\n" . ' done' . "\n"
1430 1423 . ' ;;' . "\n" . ' ;;' . "\n"
1431 . "\n"
1432 . ' *) rg_notify_err "Unsupported distro!"; E=0 ;;' . "\n"
1424 . "\n"
1425 . ' *) rg_log W "Unsupported distro!"; E=0 ;;' . "\n"
1433 1426 . ' esac' . "\n" . ' esac' . "\n"
1434 . "\n"
1427 . "\n"
1435 1428 . ' if [ "${rgfs_mounted[${pkg_repo_id}-${pkg_subrepo_id}]}" = "1" ]; then' . "\n" . ' if [ "${rgfs_mounted[${pkg_repo_id}-${pkg_subrepo_id}]}" = "1" ]; then' . "\n"
1436 . ' rg_notify "Unmounting rgfs"' . "\n"
1429 . ' rg_log N "Unmounting rgfs"' . "\n"
1437 1430 . ' umount "/mnt/build2/rgfs/${pkg_repo_id}/${pkg_subrepo_id}"' . "\n" . ' umount "/mnt/build2/rgfs/${pkg_repo_id}/${pkg_subrepo_id}"' . "\n"
1438 . ' rg_notify "Unmounting rgfs done"' . "\n"
1431 . ' rg_log N "Unmounting rgfs done"' . "\n"
1439 1432 . ' fi' . "\n" . ' fi' . "\n"
1440 1433 . ' rg_log_pop' . "\n" . ' rg_log_pop' . "\n"
1441 1434 . ' [ "${E}" = "0" ] || break' . "\n"; . ' [ "${E}" = "0" ] || break' . "\n";
 
... ... function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1444 1437 $s .= "\n" $s .= "\n"
1445 1438 . ' break' . "\n" . ' break' . "\n"
1446 1439 . 'done' . "\n" . 'done' . "\n"
1447 . 'rg_notify "Done [E=${E}]"' . "\n"
1440 . 'rg_log N "Done [E=${E}] [$(date)]"' . "\n"
1448 1441 . 'exit ${E}' . "\n"; . 'exit ${E}' . "\n";
1449 1442
1450 1443 $f = $job['main'] . '/root/build_repo.sh'; $f = $job['main'] . '/root/build_repo.sh';
 
... ... function start_worker($job)
1715 1708 . ' >>/mnt/status/RG_LABELS' . "\n"; . ' >>/mnt/status/RG_LABELS' . "\n";
1716 1709
1717 1710 $s .= 'date +%s > ' . $prefix . '.start' . "\n" $s .= 'date +%s > ' . $prefix . '.start' . "\n"
1718 . 'rg_notify "Executing [' . $i['cmd'] . ']"' . "\n"
1711 . 'rg_log N "Executing [' . $i['cmd'] . ']"' . "\n"
1719 1712 . '(' . $i['cmd'] . ') &>' . $prefix . '.log' . "\n" . '(' . $i['cmd'] . ') &>' . $prefix . '.log' . "\n"
1720 1713 . 'E=${?}' . "\n" . 'E=${?}' . "\n"
1721 1714 . 'echo ${E} > ' . $prefix . ".status\n" . 'echo ${E} > ' . $prefix . ".status\n"
1722 1715 . 'date +%s > ' . $prefix . '.done' . "\n" . 'date +%s > ' . $prefix . '.done' . "\n"
1723 1716 . 'if [ "${E}" != "0" ]; then' . "\n" . 'if [ "${E}" != "0" ]; then' . "\n"
1724 . ' rg_notify "Command failed with code ${E}"' . "\n"
1717 . ' rg_log N "Command failed with code ${E}"' . "\n"
1725 1718 . $lnok . $lnok
1726 1719 . ($i['abort'] ? ' exit ${E}' . "\n" : '') . ($i['abort'] ? ' exit ${E}' . "\n" : '')
1727 1720 . 'else' . "\n" . 'else' . "\n"
 
... ... function start_worker($job)
1747 1740 $p_i_cmd = ''; $p_i_cmd = '';
1748 1741 if (!empty($job['packages'])) { if (!empty($job['packages'])) {
1749 1742 rg_log_debug('packages: ' . $job['packages'] . '.'); rg_log_debug('packages: ' . $job['packages'] . '.');
1750 $p_i_cmd .= 'rg_notify "Installing webhook specified packages"' . "\n";
1743 $p_i_cmd .= 'rg_log N "Installing webhook specified packages"' . "\n";
1751 1744
1752 1745 // In some versions, we did not replaced \n\r\t correctly. Do it here // In some versions, we did not replaced \n\r\t correctly. Do it here
1753 1746 $job['packages'] = preg_replace('/\s/', ' ', $job['packages']); $job['packages'] = preg_replace('/\s/', ' ', $job['packages']);
 
... ... function start_worker($job)
1759 1752 $p_i_cmd .= '(' . "\n" $p_i_cmd .= '(' . "\n"
1760 1753 . 'p=' . escapeshellarg($p) . "\n" . 'p=' . escapeshellarg($p) . "\n"
1761 1754 . 'if [[ ${p} =~ :// ]]; then' . "\n" . 'if [[ ${p} =~ :// ]]; then' . "\n"
1762 . ' rg_notify_warn "We do not allow remote packages for security reasons"' . "\n"
1755 . ' rg_log W "We do not allow remote packages for security reasons"' . "\n"
1763 1756 . 'elif [ -r "/etc/rgw/pkg/${p}" ]; then' . "\n" . 'elif [ -r "/etc/rgw/pkg/${p}" ]; then' . "\n"
1764 . ' rg_notify "Package ${p} was already installed by rgw"' . "\n"
1757 . ' rg_log N "Package ${p} was already installed by rgw"' . "\n"
1765 1758 . 'else' . "\n" . 'else' . "\n"
1766 . ' rg_notify "Installing package ${p}"' . "\n"
1759 . ' rg_log N "Installing package ${p}"' . "\n"
1767 1760 . ' ${rg_pkg_cmd} "${p}"' . "\n" . ' ${rg_pkg_cmd} "${p}"' . "\n"
1768 1761 . 'fi' . "\n" . 'fi' . "\n"
1769 1762 . 'echo; echo' . "\n" . 'echo; echo' . "\n"
 
... ... function start_worker($job)
1801 1794 . 'source /mnt/build_tools.sh' . "\n" . 'source /mnt/build_tools.sh' . "\n"
1802 1795 . "\n" . "\n"
1803 1796 . 'chmod a+rw /dev/virtio-ports/rgw # Else, we cannot signal from inside the build' . "\n" . 'chmod a+rw /dev/virtio-ports/rgw # Else, we cannot signal from inside the build' . "\n"
1804 . 'rg_notify "rg.sh started"' . "\n"
1797 . 'rg_log N "rg.sh started"' . "\n"
1805 1798 . "\n" . "\n"
1806 1799 . 'mkdir /mnt/tmp && chmod 1777 /mnt/tmp' . "\n" . 'mkdir /mnt/tmp && chmod 1777 /mnt/tmp' . "\n"
1807 1800 . 'mount --bind /mnt/tmp /tmp' . "\n" . 'mount --bind /mnt/tmp /tmp' . "\n"
 
... ... function start_worker($job)
1817 1810 . 'id' . "\n" . 'id' . "\n"
1818 1811 . 'date +%s > /mnt/T_START' . "\n" . 'date +%s > /mnt/T_START' . "\n"
1819 1812 . "\n" . "\n"
1820 . 'rg_notify "Waiting for net"' . "\n"
1813 . 'rg_log N "Waiting for net"' . "\n"
1821 1814 . '# Waiting for net' . "\n" . '# Waiting for net' . "\n"
1822 1815 . 'while [ "`ip ro li | grep ^default`" = "" ]; do' . "\n" . 'while [ "`ip ro li | grep ^default`" = "" ]; do' . "\n"
1823 1816 . ' sleep 1' . "\n" . ' sleep 1' . "\n"
1824 1817 . 'done' . "\n" . 'done' . "\n"
1825 1818 . 'date +%s > /mnt/T_NET_OK' . "\n\n" . 'date +%s > /mnt/T_NET_OK' . "\n\n"
1826 . 'rg_notify "Net OK"' . "\n"
1819 . 'rg_log N "Net OK"' . "\n"
1827 1820 . "\n" . "\n"
1828 1821 . $p_i_cmd . $p_i_cmd
1829 1822 . 'date +%s > /mnt/T_PKGS_OK' . "\n\n" . 'date +%s > /mnt/T_PKGS_OK' . "\n\n"
 
... ... function start_worker($job)
1836 1829 . '# Restricting dmesg' . "\n" . '# Restricting dmesg' . "\n"
1837 1830 . 'sysctl kernel.dmesg_restrict=1' . "\n" . 'sysctl kernel.dmesg_restrict=1' . "\n"
1838 1831 . "\n" . "\n"
1839 . '# Disabling root login' . "\n"
1840 . 'chage -E 0 root' . "\n"
1841 . 'if [ "${?}" != "0" ]; then' . "\n"
1842 . ' ERR="cannot disable root account"' . "\n"
1843 . 'fi' . "\n"
1844 . "\n"
1845 . '# Disable network access if needed' . "\n"
1846 . 'if [ "' . $conf['net'] . '" = "1" ]; then' . "\n"
1847 . ' echo -n # we allow network access' . "\n"
1848 . 'else' . "\n"
1849 . ' iptables -I OUTPUT -m owner --uid-owner build -j REJECT' . "\n"
1832 . 'while [ "${ERR}" = "" ]; do' . "\n"
1833 . ' # Disabling root login' . "\n"
1834 . ' chage -E 0 root' . "\n"
1850 1835 . ' if [ "${?}" != "0" ]; then' . "\n" . ' if [ "${?}" != "0" ]; then' . "\n"
1851 . ' ERR="Cannot disable network access for build user"' . "\n"
1836 . ' ERR="cannot disable root account"' . "\n"
1837 . ' break' . "\n"
1838 . ' fi' . "\n"
1839 . "\n"
1840 . ' # Disabling network access if needed' . "\n"
1841 . ' if [ "' . $conf['net'] . '" = "1" ]; then' . "\n"
1842 . ' rg_log N "we allow network access"' . "\n"
1843 . ' else' . "\n"
1844 . ' iptables -I OUTPUT -m owner --uid-owner build -j REJECT' . "\n"
1845 . ' if [ "${?}" != "0" ]; then' . "\n"
1846 . ' ERR="Cannot disable network access for build user"' . "\n"
1847 . ' break' . "\n"
1848 . ' fi' . "\n"
1852 1849 . ' fi' . "\n" . ' fi' . "\n"
1853 . 'fi' . "\n"
1854 1850 . "\n" . "\n"
1855 . 'while [ "${ERR}" = "" ]; do' . "\n"
1856 1851 . ' if [ "${job_url}" != "" ]; then' . "\n" . ' if [ "${job_url}" != "" ]; then' . "\n"
1857 . ' rg_notify "Checkout..."' . "\n"
1852 . ' rg_log N "Checkout..."' . "\n"
1858 1853 . ' su - build -c "cd /mnt/target/source && git branch -f rgw ${job_head} && git checkout --force rgw" >/mnt/status/checkout 2>&1' . "\n" . ' su - build -c "cd /mnt/target/source && git branch -f rgw ${job_head} && git checkout --force rgw" >/mnt/status/checkout 2>&1' . "\n"
1859 1854 . ' if [ "${?}" != "0" ]; then' . "\n" . ' if [ "${?}" != "0" ]; then' . "\n"
1860 1855 . ' ERR="cannot checkout: $(< /mnt/status/checkout)"' . "\n" . ' ERR="cannot checkout: $(< /mnt/status/checkout)"' . "\n"
1861 1856 . ' break' . "\n" . ' break' . "\n"
1862 1857 . ' fi' . "\n" . ' fi' . "\n"
1863 1858 . ' fi' . "\n" . ' fi' . "\n"
1864 . "\n"
1859 . "\n"
1865 1860 . ' case ${env_distro} in' . "\n" . ' case ${env_distro} in' . "\n"
1866 . ' fedora|centos|rocky)' . "\n"
1861 . ' fedora|centos|rocky|alma)' . "\n"
1867 1862 . ' rg_rpm_extract_info /mnt/target/source ;;' . "\n" . ' rg_rpm_extract_info /mnt/target/source ;;' . "\n"
1868 1863 . ' debian|ubuntu)' . "\n" . ' debian|ubuntu)' . "\n"
1869 1864 . ' rg_deb_extract_info /mnt/target/source ;;' . "\n" . ' rg_deb_extract_info /mnt/target/source ;;' . "\n"
1870 1865 . ' esac' . "\n" . ' esac' . "\n"
1871 . "\n"
1866 . "\n"
1872 1867 . ' if [ "${pkg_deps}" != "" ]; then' . "\n" . ' if [ "${pkg_deps}" != "" ]; then' . "\n"
1873 . ' rg_notify "Installing dependencies [${pkg_deps}]"' . "\n"
1868 . ' rg_log N "Installing dependencies [${pkg_deps}]"' . "\n"
1874 1869 . ' final=""' . "\n" . ' final=""' . "\n"
1875 1870 . ' for p in ${pkg_deps}; do' . "\n" . ' for p in ${pkg_deps}; do' . "\n"
1876 1871 . ' if [[ ${p} =~ :// ]]; then' . "\n" . ' if [[ ${p} =~ :// ]]; then' . "\n"
1877 . ' rg_notify_warn "We do not allow remote packages for security reasons"' . "\n" // TODO: the net is already disabled; only for user!
1872 . ' rg_log W "We do not allow remote packages for security reasons"' . "\n" // TODO: the net is already disabled; only for user!
1878 1873 . ' continue' . "\n" . ' continue' . "\n"
1879 1874 . ' fi' . "\n" . ' fi' . "\n"
1880 1875 . ' if [ -r "/etc/rgw/pkg/${p}" ]; then' . "\n" . ' if [ -r "/etc/rgw/pkg/${p}" ]; then' . "\n"
1881 . ' rg_notify "Package ${p} was already installed by rgw"' . "\n"
1876 . ' rg_log N "Package ${p} was already installed by rgw"' . "\n"
1882 1877 . ' continue' . "\n" . ' continue' . "\n"
1883 1878 . ' fi' . "\n" . ' fi' . "\n"
1884 1879 . ' final+=" ${p}"' . "\n" . ' final+=" ${p}"' . "\n"
1885 1880 . ' done' . "\n" . ' done' . "\n"
1886 1881 . ' if [ "${final}" != "" ]; then' . "\n" . ' if [ "${final}" != "" ]; then' . "\n"
1887 . ' rg_notify "Installing package(s) ${final}"' . "\n"
1882 . ' rg_log N "Installing package(s) ${final}"' . "\n"
1888 1883 . ' ${rg_pkg_cmd} ${final} 2>/tmp/install_pkg.err' . "\n" . ' ${rg_pkg_cmd} ${final} 2>/tmp/install_pkg.err' . "\n"
1889 . ' [ "${?}" = "0" ] || rg_notify_warn "Cannot install ${final}: "$(< /tmp/install_pkg.err)' . "\n"
1884 . ' [ "${?}" = "0" ] || rg_log W "Cannot install ${final}: "$(< /tmp/install_pkg.err)' . "\n"
1890 1885 . ' # we ignore errors here' . "\n" . ' # we ignore errors here' . "\n"
1891 1886 . ' fi' . "\n" . ' fi' . "\n"
1892 1887 . ' fi' . "\n" . ' fi' . "\n"
1893 . "\n"
1894 . ' rg_notify "Starting build.sh script"' . "\n"
1888 . "\n"
1889 . ' rg_log N "Starting build.sh script"' . "\n"
1895 1890 . ' date +%s > /mnt/status/build.sh.start' . "\n" // TODO: not read and not exposed? . ' date +%s > /mnt/status/build.sh.start' . "\n" // TODO: not read and not exposed?
1896 1891 . ' su - build -c "bash /mnt/build.sh" &>/mnt/status/build.log' . "\n" . ' su - build -c "bash /mnt/build.sh" &>/mnt/status/build.log' . "\n"
1897 1892 . ' E=${?}' . "\n" . ' E=${?}' . "\n"
1898 1893 . ' date +%s > /mnt/status/build.sh.stop' . "\n" // TODO: not read and not exposed? . ' date +%s > /mnt/status/build.sh.stop' . "\n" // TODO: not read and not exposed?
1899 . ' rg_notify "build.sh finished with code ${E}"' . "\n"
1894 . ' rg_log N "build.sh finished with code ${E}"' . "\n"
1900 1895 . ' if [ "${E}" != "0" ]; then' . "\n" . ' if [ "${E}" != "0" ]; then' . "\n"
1901 . ' rg_notify_warn "DEBUG: build.sh exited with code ${E}"' . "\n"
1896 . ' rg_log W "DEBUG: build.sh exited with code ${E}"' . "\n"
1902 1897 . ' ERR="build.sh returned ${E}"' . "\n" . ' ERR="build.sh returned ${E}"' . "\n"
1903 1898 . ' break' . "\n" . ' break' . "\n"
1904 1899 . ' fi' . "\n" . ' fi' . "\n"
1905 . "\n"
1906 . ' rg_notify "Starting build_rpms/debs.sh script"' . "\n"
1900 . "\n"
1901 . ' rg_log N "Starting build_rpms/debs.sh script"' . "\n"
1907 1902 . ' date +%s > /mnt/T_MKPKGS_START' . "\n" // TODO: not exposed . ' date +%s > /mnt/T_MKPKGS_START' . "\n" // TODO: not exposed
1908 1903 . ' case ${env_distro} in' . "\n" . ' case ${env_distro} in' . "\n"
1909 . ' fedora|centos|rocky)' . "\n" // TODO: pass 'rpm' not the distro? Same for 'deb'.
1904 . ' fedora|centos|rocky|alma)' . "\n" // TODO: pass 'rpm' not the distro? Same for 'deb'.
1910 1905 . ' chown -Rv build:build /mnt/target-dotrepo-*' . "\n" . ' chown -Rv build:build /mnt/target-dotrepo-*' . "\n"
1911 1906 . ' su - build -c "bash /mnt/build_rpms.sh" &>/mnt/status/build_pkgs.log ;;' . "\n" // TODO replace everywhere build_rpms.log with build_pkgs.log . ' su - build -c "bash /mnt/build_rpms.sh" &>/mnt/status/build_pkgs.log ;;' . "\n" // TODO replace everywhere build_rpms.log with build_pkgs.log
1912 1907 . ' debian|ubuntu)' . "\n" . ' debian|ubuntu)' . "\n"
1913 1908 . ' su - build -c "bash /mnt/build_debs.sh" &>/mnt/status/build_pkgs.log ;;' . "\n" . ' su - build -c "bash /mnt/build_debs.sh" &>/mnt/status/build_pkgs.log ;;' . "\n"
1914 . ' *) rg_notify "Unsupported distro" ;;' . "\n"
1909 . ' *) rg_log N "Unsupported distro" ;;' . "\n"
1915 1910 . ' esac' . "\n" . ' esac' . "\n"
1916 1911 . ' E=${?}' . "\n" . ' E=${?}' . "\n"
1917 . ' E=0' . "\n" // TODO: for now, we do not error on this
1918 1912 . ' date +%s > /mnt/T_MKPKGS_END' . "\n" . ' date +%s > /mnt/T_MKPKGS_END' . "\n"
1919 . ' rg_notify "finished with code ${E}"' . "\n"
1913 . ' rg_log N "finished with code ${E}"' . "\n"
1914 . ' E=0; rg_log W "TODO For now we ignore the error code!"' . "\n"
1920 1915 . ' if [ "${E}" != "0" ]; then' . "\n" . ' if [ "${E}" != "0" ]; then' . "\n"
1921 1916 . ' ERR="build_rpms/debs.sh returned ${E}"' . "\n" . ' ERR="build_rpms/debs.sh returned ${E}"' . "\n"
1922 1917 . ' break' . "\n" . ' break' . "\n"
1923 1918 . ' fi' . "\n" . ' fi' . "\n"
1924 . "\n"
1919 . "\n"
1925 1920 . ' # Moving everything to build2 dir. TODO: Why?' . "\n" . ' # Moving everything to build2 dir. TODO: Why?' . "\n"
1926 1921 . ' [ -d /mnt/target/source/rocketgit ] && cp -rf /mnt/target/source/rocketgit /mnt/build2' . "\n" . ' [ -d /mnt/target/source/rocketgit ] && cp -rf /mnt/target/source/rocketgit /mnt/build2' . "\n"
1927 1922 . ' # /mnt/target/* because we need the deb programs to access packages' . "\n" . ' # /mnt/target/* because we need the deb programs to access packages' . "\n"
 
... ... function start_worker($job)
1931 1926 . "\n" . "\n"
1932 1927 . ' bash /mnt/build_rgfs.sh &>/mnt/status/build_rgfs.log' . "\n" . ' bash /mnt/build_rgfs.sh &>/mnt/status/build_rgfs.log' . "\n"
1933 1928 . ' if [ "${?}" != "0" ]; then' . "\n" . ' if [ "${?}" != "0" ]; then' . "\n"
1934 . ' rg_notify "error building rgfs; trying to continue"' . "\n"
1929 . ' rg_log N "error building rgfs; trying to continue"' . "\n"
1935 1930 . ' fi' . "\n" . ' fi' . "\n"
1936 1931 . "\n" . "\n"
1937 1932 . ' if [ -x /mnt/build_repo.sh ]; then' . "\n" . ' if [ -x /mnt/build_repo.sh ]; then' . "\n"
1938 . ' rg_notify "Starting build_repo.sh script"' . "\n"
1933 . ' rg_log N "Starting build_repo.sh script"' . "\n"
1939 1934 . ' date +%s > /mnt/T_REPO_START' . "\n" . ' date +%s > /mnt/T_REPO_START' . "\n"
1940 1935 . ' ls -l /home; mkdir -p /home/build2; chown build2:build2 /home/build2' . "\n" . ' ls -l /home; mkdir -p /home/build2; chown build2:build2 /home/build2' . "\n"
1941 1936 . ' su - build2 -c "bash /mnt/build_repo.sh" &>/mnt/status/build_repo.log' . "\n" . ' su - build2 -c "bash /mnt/build_repo.sh" &>/mnt/status/build_repo.log' . "\n"
1942 1937 . ' E=${?}' . "\n" . ' E=${?}' . "\n"
1943 1938 . ' date +%s > /mnt/T_REPO_END' . "\n" . ' date +%s > /mnt/T_REPO_END' . "\n"
1944 . ' rg_notify "build_repo.sh finished with code ${E}"' . "\n"
1945 1939 . ' if [ "${E}" != "0" ]; then' . "\n" . ' if [ "${E}" != "0" ]; then' . "\n"
1940 . ' rg_log E "build_repo.sh finished with code ${E}"' . "\n"
1946 1941 . ' ERR="build_repo.sh returned ${E}"' . "\n" . ' ERR="build_repo.sh returned ${E}"' . "\n"
1947 1942 . ' break' . "\n" . ' break' . "\n"
1948 1943 . ' fi' . "\n" . ' fi' . "\n"
1944 . ' rg_log N "build_repo.sh finished with code ${E}"' . "\n"
1949 1945 . ' fi' . "\n" . ' fi' . "\n"
1950 1946 . "\n" . "\n"
1951 1947 . ' break' . "\n" . ' break' . "\n"
1952 1948 . 'done' . "\n" . 'done' . "\n"
1953 1949 . 'if [ "${ERR}" != "" ]; then' . "\n" . 'if [ "${ERR}" != "" ]; then' . "\n"
1954 1950 . ' echo "${ERR}" > /mnt/status/err' . "\n" . ' echo "${ERR}" > /mnt/status/err' . "\n"
1955 . ' rg_notify_err "ERROR: ${ERR}"' . "\n"
1951 . ' rg_log E "ERROR: ${ERR}"' . "\n"
1956 1952 . 'fi' . "\n" . 'fi' . "\n"
1957 1953 . 'date +%s > /mnt/T_DONE' . "\n\n" . 'date +%s > /mnt/T_DONE' . "\n\n"
1958 . 'rg_notify "DONE"' . "\n"
1954 . 'rg_log N "DONE"' . "\n"
1959 1955 . "\n" . "\n"
1960 1956 . 'dmesg > /mnt/status/dmesg' . "\n" . 'dmesg > /mnt/status/dmesg' . "\n"
1961 1957 . "\n" . "\n"
1962 . 'rg_notify "Syncing disks"' . "\n"
1958 . 'rg_log N "Syncing disks"' . "\n"
1963 1959 . 'sync' . "\n" . 'sync' . "\n"
1964 1960 . "\n" . "\n"
1965 . 'rg_notify "Shutting down"' . "\n"
1961 . 'rg_log N "Shutting down"' . "\n"
1966 1962 . 'while [ -r /root/stop ]; do' . "\n" . 'while [ -r /root/stop ]; do' . "\n"
1967 1963 . ' sleep 1' . "\n" . ' sleep 1' . "\n"
1968 1964 . 'done' . "\n" . 'done' . "\n"
 
... ... function xhandle_one($key, $data)
2073 2069 } }
2074 2070
2075 2071 if (strcmp($op, 'FEATURES') == 0) { // what master suports if (strcmp($op, 'FEATURES') == 0) { // what master suports
2076 rg_log_ml($key . ': FEATURES command: ' . rg_array2string($u));
2077 2072 $features = $u['features']; $features = $u['features'];
2073 rg_log_ml($key . ': FEATURES command: ' . rg_array2string($features));
2078 2074 return; return;
2079 2075 } }
2080 2076
 
... ... function xchan($key, $data)
2220 2216 return $ret; return $ret;
2221 2217
2222 2218 $one = substr($data, 0, $pos); $one = substr($data, 0, $pos);
2223 rg_log($key . ': DEBUG: received data from VM: ' . $one);
2219 rg_log($key . ': DEBUG: VM: ' . $one);
2224 2220 $data = substr($data, $pos + 1); $data = substr($data, $pos + 1);
2225 2221 $ret += $pos + 1; $ret += $pos + 1;
2226 2222
 
... ... function vm_do_stats(&$job, $cd)
2869 2865 global $features; global $features;
2870 2866
2871 2867 do { do {
2872 if (!isset($features['stats'])) {
2873 rg_log_debug('stats feature is not supported by the server');
2868 if (!isset($features['vm_stats'])) {
2869 rg_log_debug('vm_stats feature is not supported by the server');
2870 rg_log_debug('supported: ' . rg_array2string($features));
2874 2871 break; break;
2875 2872 } }
2876 2873
 
... ... function vm_do_stats(&$job, $cd)
2898 2895 //rg_log_debug('d: ' . print_r($d, TRUE)); //rg_log_debug('d: ' . print_r($d, TRUE));
2899 2896
2900 2897 $j = array( $j = array(
2901 'op' => 'stats',
2898 'op' => 'vm_stats',
2902 2899 'id' => $job['id'], 'id' => $job['id'],
2903 2900 'stats' => $d); 'stats' => $d);
2904 2901 rg_conn_enq('master', @json_encode($j) . "\n"); rg_conn_enq('master', @json_encode($j) . "\n");
 
... ... function send_worker_stats()
2918 2915 global $features; global $features;
2919 2916 global $stats; global $stats;
2920 2917
2921 if (!isset($features['allow_stats']))
2918 if (!isset($features['worker_stats']))
2922 2919 return; return;
2923 2920
2924 2921 $load = rg_load(); $load = rg_load();
File tests/http_settings.php changed (mode: 100644) (index 35ae6ff..5be1a3a)
... ... rg_log_exit();
137 137
138 138 rg_log(''); rg_log('');
139 139 rg_log_enter('Create some fake plans'); rg_log_enter('Create some fake plans');
140 $sql = 'DELETE FROM plans WHERE id = 1000';
141 $res = rg_sql_query($db, $sql);
140 142 $sql = 'INSERT INTO plans (id) VALUES (1000)'; $sql = 'INSERT INTO plans (id) VALUES (1000)';
141 143 $res = rg_sql_query($db, $sql); $res = rg_sql_query($db, $sql);
142 144 rg_cache_unset('plan::list', 0); rg_cache_unset('plan::list', 0);
File tests/prof.php changed (mode: 100644) (index 41531ff..f9cdc1c)
... ... $rg_no_db = TRUE;
15 15 require_once("common.php"); require_once("common.php");
16 16
17 17 rg_prof_start("label1"); rg_prof_start("label1");
18 sleep(.1);
18 usleep(100000);
19 19 rg_prof_set(array("xxx" => 1)); rg_prof_set(array("xxx" => 1));
20 20 rg_prof_end("label1"); rg_prof_end("label1");
21 21 $t = rg_prof_get(); $t = rg_prof_get();
File tests/wh_build.php changed (mode: 100644) (index 6498dc9..b3da30d)
... ... rg_log_exit();
321 321
322 322 rg_log(''); rg_log('');
323 323 rg_log_enter('Downloading package as good (owner) user'); rg_log_enter('Downloading package as good (owner) user');
324 $u = rg_pkg_transform_user('deb', $rg_ui['username']);
324 $u = rg_pkg_transform_user('deb', $rg_ui['uid'], $rg_ui['username']);
325 325 $u1 = substr($u, 0, 1); $u2 = substr($u, 1, 1); $u1 = substr($u, 0, 1); $u2 = substr($u, 1, 1);
326 326 $r = rg_test_packages_file($rg_ui, $rg_ui, 'user', $r = rg_test_packages_file($rg_ui, $rg_ui, 'user',
327 327 $pkg_repo_priv_name, $pkg_subrepo_priv['sr::name'], $pkg_repo_priv_name, $pkg_subrepo_priv['sr::name'],
 
... ... rg_log_exit();
354 354
355 355 rg_log(''); rg_log('');
356 356 rg_log_enter('Downloading package as bad (not owner) user'); rg_log_enter('Downloading package as bad (not owner) user');
357 $u = rg_pkg_transform_user('deb', $rg_ui['username']);
357 $u = rg_pkg_transform_user('deb', $rg_ui['uid'], $rg_ui['username']);
358 358 $u1 = substr($u, 0, 1); $u2 = substr($u, 1, 1); $u1 = substr($u, 0, 1); $u2 = substr($u, 1, 1);
359 359 $r = rg_test_packages_file($rg_bad, $rg_ui, 'user', $r = rg_test_packages_file($rg_bad, $rg_ui, 'user',
360 360 $pkg_repo_priv_name, $pkg_subrepo_priv['sr::name'], $pkg_repo_priv_name, $pkg_subrepo_priv['sr::name'],
File tests/wh_lambda.php changed (mode: 100644) (index ef90f3a..ddcf21f)
... ... if ($a === NULL) {
110 110 exit(1); exit(1);
111 111 } }
112 112 $e = 'Hello from Lambda; key: aaa branch=main repo=wh-cloud'; $e = 'Hello from Lambda; key: aaa branch=main repo=wh-cloud';
113 if (strcmp($a, $e) != 0) {
113 if (is_array($a) || (strcmp($a, $e) != 0)) {
114 114 rg_log_ml('r: ' . print_r($r, TRUE)); rg_log_ml('r: ' . print_r($r, TRUE));
115 rg_log('Seems hook did not executed correctly [' . $a . '] != [' . $e . ']!');
115 rg_log('Seems hook did not executed correctly [' . json_encode($a) . '] != [' . json_encode($e) . ']!');
116 116 exit(1); exit(1);
117 117 } }
118 118 rg_log_exit(); rg_log_exit();
Hints:
Before first commit, do not forget to setup your git environment:
git config --global user.name "your_name_here"
git config --global user.email "your@email_here"

Clone this repository using HTTP(S):
git clone https://rocketgit.com/user/catalinux/rocketgit

Clone this repository using ssh (do not forget to upload a key first):
git clone ssh://rocketgit@ssh.rocketgit.com/user/catalinux/rocketgit

Clone this repository using git:
git clone git://git.rocketgit.com/user/catalinux/rocketgit

You are allowed to anonymously push to this repository.
This means that your pushed commits will automatically be transformed into a merge request:
... clone the repository ...
... make some changes and some commits ...
git push origin main