RocketGit

catalinux / rocketgit (public) (License: Affero GPLv3) (since 2015-03-19) (hash sha1)

Main RocketGit repo

Clone URLs: https://rocketgit.com/user/catalinux/rocketgit ssh://rocketgit@ssh.rocketgit.com/user/catalinux/rocketgit git://git.rocketgit.com/user/catalinux/rocketgit

List of commits:

Subject	Hash	Author	Date (UTC)
Cosmetic	a4543d8257235dc5c93dd5eecdfb9caa3c2acafa	Catalin(ux) M. BOIE	2021-11-02 07:04:36
Fixed partion create logic	c8897c0e6f31a0ab917d2320018c5920e0abd36b	Catalin(ux) M. BOIE	2021-11-02 06:01:58
Typos corrected	c1a5002a1763327f4df8d5d0b419fcc2604f4698	Catalin(ux) M. BOIE	2021-10-30 11:08:34
compare: Updated number of lines	d1052eb4b72309dd216e3fd0da2dd311bbeb1383	Catalin(ux) M. BOIE	2021-10-30 10:56:39
Added docs for distro pkgs	401c582772303eff8ba7fde3fcf09064d11bf828	Catalin(ux) M. BOIE	2021-10-30 10:47:43
Added a feature island for packages	115412032fca7ad2166ffbc0a526b358ee1006c4	Catalin(ux) M. BOIE	2021-10-28 07:00:14
Fixed spec file	a4c1a8218cfc9cc5b822c587fa3fc709e5d2bf6b	Catalin(ux) M. BOIE	2021-10-27 19:37:23
Get rid of xinetd	ebc765c9a21a9803b67484795625f7eb46642ae9	Catalin(ux) M. BOIE	2021-10-27 14:18:45
Added /etc/ssh/sshd_config.d in .spec file (again)	f225de7ddc3d78d4ec3699c63df6fd754500a9e3	Catalin(ux) M. BOIE	2021-10-26 07:05:26
Compare updated to add .rpm info	0d0a890fe98689b7b70987824e98d34ca5204d45	Catalin(ux) M. BOIE	2021-10-26 06:32:39
Fix rg_str_replace regarding ints; .spec small changes	7242f96f08983a70345e2665cfeb1ea592e3f3f7	Catalin(ux) M. BOIE	2021-10-26 06:17:39
builder: fixed immediate exit	27d21c9ef6bb10beaf1d4c8da29a90d649e68b60	Catalin(ux) M. BOIE	2021-10-24 14:41:29
We need .spec file before configure	840c7c3c0dd68c8ff6796b6494ca38afe8f4a5e6	Catalin(ux) M. BOIE	2021-10-24 14:28:59
rgfs: respect the requested size asked to be read	f6f9e3f26e9c2bc98c38ded6a732cf53630d5688	Catalin(ux) M. BOIE	2021-10-24 09:25:51
builder: delay 5s if cannot contact the database	b672b80542e80439243127aef2de4bda7d884e8a	Catalin(ux) M. BOIE	2021-10-24 08:26:35
Cosmetic	ea40e49373739590d9794d203ddd7e2b18464e29	Catalin(ux) M. BOIE	2021-10-23 15:35:34
Typo	7d156cc190c150239f16ea0b9807eb027f8cd79f	Catalin(ux) M. BOIE	2021-10-22 17:41:34
worker: be more verbose	0f0e6216ad6f5923d08c10d7461d2259e92f69cd	Catalin(ux) M. BOIE	2021-10-22 17:32:01
Add /etc/ssh files to spec	91c7c4f0bd30f01abe48e5c4f4c75857982a8f21	Catalin(ux) M. BOIE	2021-10-22 06:30:23
Removed Changelog file	8e42a6cb730000a104956bc7d265321d67d49bfe	Catalin(ux) M. BOIE	2021-10-22 06:22:24

Commit a4543d8257235dc5c93dd5eecdfb9caa3c2acafa - Cosmetic
Author: Catalin(ux) M. BOIE
Author date (UTC): 2021-11-02 07:04
Committer name: Catalin(ux) M. BOIE
Committer date (UTC): 2021-11-06 14:37
Parent(s): c8897c0e6f31a0ab917d2320018c5920e0abd36b
Signer:
Signing key:
Signing status: N
Tree: 997b5a8420c22186d8229dd72337da53b6ebda24

File	Lines added	Lines deleted
TODO	21	3
docker/TODO	1	0
docker/rg.sh	1	0
duilder.conf	1	1
inc/events.inc.php	3	2
inc/user/packages_rpm.inc.php	2	2
inc/user/pkg_subrepo.inc.php	2	1
inc/util.inc.php	1	0
inc/wh/build.inc.php	3	3
inc/wh/cloud.inc.php	1	1
inc/wh/http.inc.php	1	1
inc/wh/lambda.inc.php	1	1
rocketgit.spec	2	0
scripts/rgfs.php	0	6
scripts/worker.php	45	33
techdocs/rg_build.drawio	1	0
techdocs/rpm.txt	110	0

File TODO changed (mode: 100644) (index 4c488b9..6a3822c)
1	1	== Always ==	== Always ==
2	2	[ ] Always make sure we link the docs from the Features page.	[ ] Always make sure we link the docs from the Features page.
	3		[ ] Upgrade rocketgit.com to the new version.
3	4	[ ] Announce internal users.	[ ] Announce internal users.
4	5
5	6	== After deploy on rocketgit.com ==	== After deploy on rocketgit.com ==
6		[ ] Add to "Corporate friendly": a link to a frustrations page about current software.
7	7	[ ] Resync nginx conf on rg2 (because ws builder?).	[ ] Resync nginx conf on rg2 (because ws builder?).
8	8	[ ] Activate rocketgit-git.socket.	[ ] Activate rocketgit-git.socket.
9		[ ] Switch worker@com to websocket.
10		[ ] docker
	9		[ ] Switch worker@com to websocket (on r1).
	10		[ ] docker + VM image
11	11	[ ] old environments are present in the list! At least on dev1.	[ ] old environments are present in the list! At least on dev1.
12	12	Store the last update date (or the worker id) and do not show the	Store the last update date (or the worker id) and do not show the
13	13	old ones.	old ones.
14	14	[ ] Check if SELinux module is inserted.	[ ] Check if SELinux module is inserted.
15	15	[ ] Announce RLUG	[ ] Announce RLUG
16	16	[ ] Announce LWN	[ ] Announce LWN
	17		[ ] Announce DevClass
17	18	[ ] Announce build jobs users that now is working.	[ ] Announce build jobs users that now is working.
18	19	[ ] Partner with Solo Keys?	[ ] Partner with Solo Keys?
19	20	[ ] Enable disabled environments.	[ ] Enable disabled environments.
20	21	[ ]	[ ]
21	22
22	23	== Now ==	== Now ==
	24		[ ] Release message:
	25		ssh sk support
	26		distro packages
	27		More distributions and architectures added.
	28		[ ] I must have rocketgit-global-stable 2.89 and for testing 2.33 and I do not! Seems the trigger is not done!
	29		Seems pkg_generate_dotrepo_rpm_callback, which updates the db version
	30		(I think - check) was called, but the packages were not prepared.
	31		Seems rg_pkg_prepare_ev_for_dotrepo is skipping some repos because
	32		uid does not match.
23	33	[ ]	[ ]
24	34
25	35	== hook update	== hook update

54	64	Seems we are cloning the namespace, before knowing if the user can push!	Seems we are cloning the namespace, before knowing if the user can push!
55	65
56	66	== FOR OTHER RELEASE ==	== FOR OTHER RELEASE ==
	67		[ ] Describe how to remove RocketGit repositories.
	68		[ ] Explain why we have USERNAME+PROJECT packages names also in private repos.
	69		[ ] pkg_subrepo:distro_info should be exposed on web.
	70		[ ] Add a "Test!" button for webhooks!
	71		[ ] When I list user repositories, I have "Repo: global - public" under "Type: User"
	72		which seems wrong!
	73		[ ] Add to "Corporate friendly": a link to a frustrations page about current software.
	74		[ ] We can enforce per user the ssh sk security?
57	75	[ ] Have a repo a mirror of an official one, in a branch add custom changes,	[ ] Have a repo a mirror of an official one, in a branch add custom changes,
58	76	auto pull and build packages.	auto pull and build packages.
59	77	[ ] Promise to never relase version 2.0 - to avoid setbacks for upgrades.	[ ] Promise to never relase version 2.0 - to avoid setbacks for upgrades.

File docker/TODO changed (mode: 100644) (index c8dca5b..1db7147)
	1		[ ] Add global rocketgit distro repo when building the container.
1	2	[ ] Fix rocketgit link	[ ] Fix rocketgit link
2	3	[ ] Specify docker build http://...	[ ] Specify docker build http://...
3	4	[ ] What about the composer file?	[ ] What about the composer file?

File docker/rg.sh changed (mode: 100755) (index 079ca37..6385ff2)
...	...	while [ 1 ]; do
67	67	#su - rocketgit -c /usr/share/rocketgit/scripts/worker.sh &	#su - rocketgit -c /usr/share/rocketgit/scripts/worker.sh &
68	68
69	69	# More work to do here	# More work to do here
	70		# It is a solution to tunnel the traffic using nginx?
70	71	#if [ "`ps ax\|grep -v grep\|grep git-daemon`" = "" ]; then	#if [ "`ps ax\|grep -v grep\|grep git-daemon`" = "" ]; then
71	72	# echo "Running git-daemon..."	# echo "Running git-daemon..."
72	73	# /usr/libexec/git-core/git-daemon --base-path=/var/lib/git \	# /usr/libexec/git-core/git-daemon --base-path=/var/lib/git \

File duilder.conf changed (mode: 100644) (index d5789e7..e31b63f)
1	1	PRJ="rocketgit"	PRJ="rocketgit"
2		VER="0.73"
	2		VER="0.74"
3	3	REV="1"	REV="1"
4	4	SHORT_DESCRIPTION="Light and fast Git hosting solution"	SHORT_DESCRIPTION="Light and fast Git hosting solution"
5	5	EXCLUDE=".exclude"	EXCLUDE=".exclude"

File inc/events.inc.php changed (mode: 100644) (index 80e12db..8672021)
...	...	function rg_event_process($db, $ev_id, $event)
142	142	global $rg_event_split_functions;	global $rg_event_split_functions;
143	143
144	144	rg_prof_start("event_process");	rg_prof_start("event_process");
145		rg_log_enter("event_process: ev_id=$ev_id"
146		. " event=" . rg_array2string($event));
	145		rg_log_enter('event_process: ev_id=' . $ev_id
	146		. ' cat=' . $event['category']);
147	147
148	148	$ret = FALSE;	$ret = FALSE;
149	149	$rollback = FALSE;	$rollback = FALSE;
150	150	do {	do {
151	151	rg_debug_set($event['debug']);	rg_debug_set($event['debug']);
	152		rg_log_debug('ev: ' . print_r($event, TRUE));
152	153
153	154	if (isset($event['ui_login']))	if (isset($event['ui_login']))
154	155	rg_ui_login_set($event['ui_login']);	rg_ui_login_set($event['ui_login']);

File inc/user/packages_rpm.inc.php changed (mode: 100644) (index 7994b28..0733757)
...	...	function rg_pkg_prepare_ev_for_dotrepo($db, &$ev)
107	107	//rg_log_debug('ri: ' . rg_array2string($ri));	//rg_log_debug('ri: ' . rg_array2string($ri));
108	108	if ($ev['uid'] != $ri['uid']) {	if ($ev['uid'] != $ri['uid']) {
109	109	rg_log_debug('we cannot build this'	rg_log_debug('we cannot build this'
110		. ' dotrepo because ri[uid](' . $ri['uid']
111		. ') != ev[uid](' . $ev['uid'] . ')');
	110		. ' dotrepo because pkg_repo_uid(' . $ri['uid']
	111		. ') != event uid(' . $ev['uid'] . ')');
112	112	continue;	continue;
113	113	}	}
114	114

File inc/user/pkg_subrepo.inc.php changed (mode: 100644) (index 69744c8..ee405d8)
...	...	function rg_pkg_subrepo_event_edit($db, $ev)
59	59	continue;	continue;
60	60	}	}
61	61
62		rg_log_debug('env ' . $env . ' needs rebuilding of dotrepo');
	62		rg_log_debug('env ' . $env . ' needs rebuilding of'
	63		. ' dotrepo because ver was incremented');
63	64	$a['env'] = $env;	$a['env'] = $env;
64	65	$ret[] = $a;	$ret[] = $a;
65	66	}	}

File inc/util.inc.php changed (mode: 100644) (index 4aa510a..dcc8bac)
...	...	function rg_is_bot($ua)
2946	2946	if (strstr($ua, 'clark-Crowler2/')) return TRUE;	if (strstr($ua, 'clark-Crowler2/')) return TRUE;
2947	2947	if (strstr($ua, ' Cliqzbot/')) return TRUE;	if (strstr($ua, ' Cliqzbot/')) return TRUE;
2948	2948	if (strstr($ua, ' coccocbot-web')) return TRUE;	if (strstr($ua, ' coccocbot-web')) return TRUE;
	2949		if (strstr($ua, ' DataForSeoBot/')) return TRUE;
2949	2950	if (strstr($ua, 'dcrawl/')) return TRUE;	if (strstr($ua, 'dcrawl/')) return TRUE;
2950	2951	if (strstr($ua, ' DNSResearchBot')) return TRUE;	if (strstr($ua, ' DNSResearchBot')) return TRUE;
2951	2952	if (strstr($ua, 'DomainStatsBot/')) return TRUE;	if (strstr($ua, 'DomainStatsBot/')) return TRUE;

File inc/wh/build.inc.php changed (mode: 100644) (index 08e8116..a505bfb)
...	...	function rg_wh_build_send($db, $ev)
182	182
183	183	// If the web hook does not contain our type, skip it	// If the web hook does not contain our type, skip it
184	184	if (!strchr($wh['idata']['events'], $ev['wh_event'])) {	if (!strchr($wh['idata']['events'], $ev['wh_event'])) {
185		rg_log_debug('' . $ev['wh_event']
	185		rg_log_debug($ev['wh_event']
186	186	. ' is not present in ['	. ' is not present in ['
187	187	. $wh['idata']['events'] . ']');	. $wh['idata']['events'] . ']');
188	188	continue;	continue;

...	...	function rg_wh_build_send($db, $ev)
190	190
191	191	if (isset($ev['ri']['name'])	if (isset($ev['ri']['name'])
192	192	&& !rg_repo_compare_refs($wh['repo'], $ev['ri']['name'])) {	&& !rg_repo_compare_refs($wh['repo'], $ev['ri']['name'])) {
193		rg_log('hook is not for this repo');
	193		rg_log_debug('hook is not for this repo');
194	194	continue;	continue;
195	195	}	}
196	196
197	197	if (isset($ev['refname'])	if (isset($ev['refname'])
198	198	&& !rg_repo_compare_refs($wh['refname'], $ev['refname'])) {	&& !rg_repo_compare_refs($wh['refname'], $ev['refname'])) {
199		rg_log('hook is not for this ref');
	199		rg_log_debug('hook is not for this ref');
200	200	continue;	continue;
201	201	}	}
202	202

File inc/wh/cloud.inc.php changed (mode: 100644) (index 03fa943..8ad7324)
...	...	function rg_wh_cloud_send($db, $ev)
118	118
119	119	// If the web hook does not contain our type, skip it	// If the web hook does not contain our type, skip it
120	120	if (!strchr($wh['idata']['events'], $ev['wh_event'])) {	if (!strchr($wh['idata']['events'], $ev['wh_event'])) {
121		rg_log_debug('' . $ev['wh_event']
	121		rg_log_debug($ev['wh_event']
122	122	. ' is not present in ['	. ' is not present in ['
123	123	. $wh['idata']['events'] . ']');	. $wh['idata']['events'] . ']');
124	124	continue;	continue;

File inc/wh/http.inc.php changed (mode: 100644) (index 3ab9d4c..a1e035d)
...	...	function rg_wh_http_send($db, $ev)
202	202
203	203	// If the web hook does not contain our type, skip it	// If the web hook does not contain our type, skip it
204	204	if (!strchr($wh['idata']['events'], $ev['wh_event'])) {	if (!strchr($wh['idata']['events'], $ev['wh_event'])) {
205		rg_log_debug('' . $ev['wh_event']
	205		rg_log_debug($ev['wh_event']
206	206	. ' is not present in ['	. ' is not present in ['
207	207	. $wh['idata']['events'] . ']');	. $wh['idata']['events'] . ']');
208	208	continue;	continue;

File inc/wh/lambda.inc.php changed (mode: 100644) (index df00b1f..27d451e)
...	...	function rg_wh_lambda_send($db, $ev)
82	82
83	83	// If the web hook does not contain our type, skip it	// If the web hook does not contain our type, skip it
84	84	if (!strchr($wh['idata']['events'], $ev['wh_event'])) {	if (!strchr($wh['idata']['events'], $ev['wh_event'])) {
85		rg_log_debug('' . $ev['wh_event']
	85		rg_log_debug($ev['wh_event']
86	86	. ' is not present in ['	. ' is not present in ['
87	87	. $wh['idata']['events'] . ']');	. $wh['idata']['events'] . ']');
88	88	continue;	continue;

File rocketgit.spec changed (mode: 100644) (index 8f47ba5..27953ef)
...	...	URL: https://rocketgit.com/
15	15	BuildRoot: %{_tmppath}/%{name}-%{version}-buildroot	BuildRoot: %{_tmppath}/%{name}-%{version}-buildroot
16	16	BuildArch: noarch	BuildArch: noarch
17	17
	18		Provides: rocketgit
	19
18	20	# fuse is required for rgfs	# fuse is required for rgfs
19	21	Requires: httpd-filesystem, nginx-filesystem, php-fpm, php-cli, php-pgsql, php-mbstring	Requires: httpd-filesystem, nginx-filesystem, php-fpm, php-cli, php-pgsql, php-mbstring
20	22	Requires: git-core, cronie, php-json, php-ldap	Requires: git-core, cronie, php-json, php-ldap

 error_reporting(E_ALL);
 set_time_limit(0);
-$_s = microtime(TRUE);
 require_once('/etc/rocketgit/config.php');
 require_once(__DIR__ . '/../inc/init.inc.php');
 require_once(__DIR__ . '/../inc/log.inc.php');
  */
 function xdestroy($key)
 {
-	global $workers;
-	$workers--;
 }
 /*
 // What features the rgfs supports
 $features = array();
-$workers = 0;
 $original_mtime = @filemtime(__FILE__);
 do {
 	// Check our mtime so we can upgrade the software and this script

File scripts/worker.php changed (mode: 100644) (index 429c2b9..fa71818)
...	...	function start_worker_build_tools($job, &$reason, &$reason2)
416	416	. ' echo "M${@}" > /dev/virtio-ports/rgw' . "\n"	. ' echo "M${@}" > /dev/virtio-ports/rgw' . "\n"
417	417	. '}' . "\n"	. '}' . "\n"
418	418	. "\n"	. "\n"
	419		. 'function rg_notify_err()' . "\n"
	420		. '{' . "\n"
	421		. ' echo "E${@}" > /dev/virtio-ports/rgw' . "\n"
	422		. '}' . "\n"
	423		. "\n"
419	424	. 'rg_log_prefix=""' . "\n"	. 'rg_log_prefix=""' . "\n"
420	425	. 'function xecho()' . "\n"	. 'function xecho()' . "\n"
421	426	. '{' . "\n"	. '{' . "\n"

...	...	function start_worker_build_tools($job, &$reason, &$reason2)
558	563	. ' local dir=${1}; shift' . "\n"	. ' local dir=${1}; shift' . "\n"
559	564	. "\n"	. "\n"
560	565	. ' xecho "rg_extract_rpm_info: dir=${dir} [`date`]"' . "\n"	. ' xecho "rg_extract_rpm_info: dir=${dir} [`date`]"' . "\n"
561		. ' spec_file=`find "${dir}" -type f -iname \'*.spec\' \| head -n1`' . "\n"
	566		. ' spec_file=`find "${dir}" -iname \'*.spec\' -type f -printf "%d %p\n" \| sort -n \| head -n1 \| awk \'{print $2}\'`' . "\n"
562	567	. ' if [ "${spec_file}" = "" ]; then' . "\n"	. ' if [ "${spec_file}" = "" ]; then' . "\n"
563	568	. ' xecho " Cannot find any .spec file in ${dir}!"' . "\n"	. ' xecho " Cannot find any .spec file in ${dir}!"' . "\n"
564	569	. ' ls -l "${dir}"' . "\n"	. ' ls -l "${dir}"' . "\n"

...	...	function start_worker_build_repo($job, $conf, &$reason, &$reason2)
906	911	. ' fi' . "\n"	. ' fi' . "\n"
907	912	. ' done' . "\n"	. ' done' . "\n"
908	913	. ' ;;' . "\n"	. ' ;;' . "\n"
909		. ' *) xecho "Unknown distro!"; E=1 ;;' . "\n"
	914		. ' *) xecho "Unsupported distro!"; E=1 ;;' . "\n"
910	915	. ' esac' . "\n"	. ' esac' . "\n"
911	916	. ' rg_log_pop' . "\n"	. ' rg_log_pop' . "\n"
912	917	. ' [ "${E}" = "0" ] \|\| break' . "\n\n";	. ' [ "${E}" = "0" ] \|\| break' . "\n\n";

...	...	function start_worker_build_repo($job, $conf, &$reason, &$reason2)
936	941	. ' rg_log_pop' . "\n"	. ' rg_log_pop' . "\n"
937	942	. ' done' . "\n"	. ' done' . "\n"
938	943	. ' ;;' . "\n"	. ' ;;' . "\n"
939		. ' *) xecho "Unknown distro!"; E=1 ;;' . "\n"
	944		. ' *) xecho "Unsupported distro!"; E=1 ;;' . "\n"
940	945	. ' esac' . "\n"	. ' esac' . "\n"
941	946	. ' rg_log_pop' . "\n"	. ' rg_log_pop' . "\n"
942	947	. ' [ "${E}" = "0" ] \|\| break' . "\n\n";	. ' [ "${E}" = "0" ] \|\| break' . "\n\n";
943	948
944		// TODO: we have a race for 'rg_op cat a > b': writing to b may fail => break is executed, but E is not set!
945	949	foreach ($job['pkg_maps'] as $junk => $info) {	foreach ($job['pkg_maps'] as $junk => $info) {
946	950	$pkg_subrepo_id = $info['pkg_subrepo_id'];	$pkg_subrepo_id = $info['pkg_subrepo_id'];
947	951	$si = $job['pkg_subrepos'][$pkg_subrepo_id];	$si = $job['pkg_subrepos'][$pkg_subrepo_id];

...	...	function start_worker_build_repo($job, $conf, &$reason, &$reason2)
951	955	. ' export pkg_repo_id=' . $pkg_repo_id . "\n"	. ' export pkg_repo_id=' . $pkg_repo_id . "\n"
952	956	. ' export pkg_repo_uid=' . $ri['uid'] . "\n"	. ' export pkg_repo_uid=' . $ri['uid'] . "\n"
953	957	. ' export pkg_subrepo_id=' . $pkg_subrepo_id . "\n"	. ' export pkg_subrepo_id=' . $pkg_subrepo_id . "\n"
954		. "\n"
	958		. "\n"
955	959	. ' echo' . "\n"	. ' echo' . "\n"
956	960	. ' rg_log_push "== rgfs phase pkg_repo_id=${pkg_repo_id} pkg_subrepo_id=${pkg_subrepo_id} ====="' . "\n"	. ' rg_log_push "== rgfs phase pkg_repo_id=${pkg_repo_id} pkg_subrepo_id=${pkg_subrepo_id} ====="' . "\n"
957	961	. ' xecho "Mounting rgfs pkg_repo_id/pkg_subrepo_id=${pkg_repo_id}/${pkg_subrepo_id}"' . "\n"	. ' xecho "Mounting rgfs pkg_repo_id/pkg_subrepo_id=${pkg_repo_id}/${pkg_subrepo_id}"' . "\n"

...	...	function start_worker_build_repo($job, $conf, &$reason, &$reason2)
966	970	. ' rg_op_s /usr/bin/rgfs -s -o kernel_cache -o auto_cache \\' . "\n"	. ' rg_op_s /usr/bin/rgfs -s -o kernel_cache -o auto_cache \\' . "\n"
967	971	. ' -o noforget -o use_ino "${dir}" \|\| break' . "\n"	. ' -o noforget -o use_ino "${dir}" \|\| break' . "\n"
968	972	. ' rgfs_mounted[${pkg_repo_id}-${pkg_subrepo_id}]=1' . "\n"	. ' rgfs_mounted[${pkg_repo_id}-${pkg_subrepo_id}]=1' . "\n"
969		. "\n"
	973		. "\n"
970	974	. ' rgfs_subrepo_root="/mnt/build2/rgfs/${pkg_repo_id}/${pkg_subrepo_id}"' . "\n"	. ' rgfs_subrepo_root="/mnt/build2/rgfs/${pkg_repo_id}/${pkg_subrepo_id}"' . "\n"
971	975	. ' xecho "DEBUG: env_distro=${env_distro}"' . "\n"	. ' xecho "DEBUG: env_distro=${env_distro}"' . "\n"
972	976	. ' case "${env_distro}" in' . "\n"	. ' case "${env_distro}" in' . "\n"

...	...	function start_worker_build_repo($job, $conf, &$reason, &$reason2)
975	979	. ' rgfs="${rgfs_subrepo_root}/${env_distro}/${env_major}/${env_arch}/${i}"' . "\n"	. ' rgfs="${rgfs_subrepo_root}/${env_distro}/${env_major}/${env_arch}/${i}"' . "\n"
976	980	. ' rpms_dir="/mnt/build2/rpms/${pkg_repo_id}/${i}"' . "\n"	. ' rpms_dir="/mnt/build2/rpms/${pkg_repo_id}/${i}"' . "\n"
977	981	. ' pkg_list_file="${rpms_dir}/pkg.list"' . "\n"	. ' pkg_list_file="${rpms_dir}/pkg.list"' . "\n"
978		. ' pkg_common_list_file="${rpms_dir}/pkg.common.list"' . "\n"
979	982	. ' pkg_filter_list_file="${rpms_dir}/pkg.filter.list"' . "\n"	. ' pkg_filter_list_file="${rpms_dir}/pkg.filter.list"' . "\n"
980	983	. ' pkg_pend_list_file="/mnt/tmp/pkg.list"' . "\n"	. ' pkg_pend_list_file="/mnt/tmp/pkg.list"' . "\n"
981		. ' > "${pkg_common_list_file}"' . "\n"
982	984	. ' > "${pkg_pend_list_file}"' . "\n"	. ' > "${pkg_pend_list_file}"' . "\n"
983		. "\n"
	985		. "\n"
984	986	. ' rg_log_push "== ${i} ====="' . "\n"	. ' rg_log_push "== ${i} ====="' . "\n"
985	987	. ' mkdir -p "${rpms_dir}" # when we do not build packages, we may not have this dir' . "\n"	. ' mkdir -p "${rpms_dir}" # when we do not build packages, we may not have this dir' . "\n"
986		. "\n"
987		. ' rg_cp "${pkg_list_file}" "${pkg_common_list_file}" \|\| break' . "\n"
	988		. "\n"
988	989	. ' if [ "${job_uid}" = "0" ]; then' . "\n"	. ' if [ "${job_uid}" = "0" ]; then' . "\n"
989	990	. ' if [ -d "${rgfs}/+pending" ]; then' . "\n"	. ' if [ -d "${rgfs}/+pending" ]; then' . "\n"
990	991	. ' xecho "DEBUG: dir +pending found"' . "\n"	. ' xecho "DEBUG: dir +pending found"' . "\n"
991	992	. ' rg_create_list "${rgfs}/+pending" "${pkg_pend_list_file}" \|\| break' . "\n"	. ' rg_create_list "${rgfs}/+pending" "${pkg_pend_list_file}" \|\| break' . "\n"
992		. ' rg_op cat "${pkg_pend_list_file}" >> "${pkg_common_list_file}"' . "\n"
993	993	. ' else' . "\n"	. ' else' . "\n"
994	994	. ' xecho "DEBUG: dir +pending does not exists."' . "\n"	. ' xecho "DEBUG: dir +pending does not exists."' . "\n"
995	995	. ' fi' . "\n"	. ' fi' . "\n"
996	996	. ' fi' . "\n"	. ' fi' . "\n"
997		. ' xecho "DEBUG pkg_common_list_file: ["`cat "${pkg_common_list_file}"`"]"' . "\n"
998		. "\n"
999		. ' if [ ! -s "${pkg_common_list_file}" ]; then' . "\n"
1000		. ' xecho "pkg_common_list_files is empty"' . "\n"
	997		. "\n"
	998		. ' if [ ! -s "${pkg_list_file}" ] && [ ! -s "${pkg_pend_list_file}" ]; then' . "\n"
	999		. ' xecho "pkg_list_file/pkg_pend_list_file are both empty; continue"' . "\n"
1001	1000	. ' rg_log_pop' . "\n"	. ' rg_log_pop' . "\n"
1002	1001	. ' continue' . "\n"	. ' continue' . "\n"
1003	1002	. ' fi' . "\n"	. ' fi' . "\n"
1004		. "\n"
	1003		. "\n"
1005	1004	. ' rg_log_push "Filtering existing packages from pkg_list_file"' . "\n"	. ' rg_log_push "Filtering existing packages from pkg_list_file"' . "\n"
1006	1005	. ' > "${pkg_filter_list_file}"' . "\n"	. ' > "${pkg_filter_list_file}"' . "\n"
1007	1006	. ' while read file; do' . "\n"	. ' while read file; do' . "\n"
	1007		. ' [ "${file}" = "" ] && continue' . "\n"
1008	1008	. ' if [ -r "${rgfs}/${file}" ]; then' . "\n"	. ' if [ -r "${rgfs}/${file}" ]; then' . "\n"
1009	1009	. ' xecho "- ${rgfs}/${file}"' . "\n"	. ' xecho "- ${rgfs}/${file}"' . "\n"
1010	1010	. ' continue' . "\n"	. ' continue' . "\n"
1011	1011	. ' fi' . "\n"	. ' fi' . "\n"
1012	1012	. ' xecho "+ ${rgfs}/${file}"' . "\n"	. ' xecho "+ ${rgfs}/${file}"' . "\n"
1013	1013	. ' echo "${file}" >> ${pkg_filter_list_file}' . "\n"	. ' echo "${file}" >> ${pkg_filter_list_file}' . "\n"
1014		. ' done < "${pkg_common_list_file}"' . "\n"
	1014		. ' done < <(cat "${pkg_list_file}"; echo; cat "${pkg_pend_list_file}"; echo' . "\n"
1015	1015	. ' rg_log_pop' . "\n"	. ' rg_log_pop' . "\n"
1016		. "\n"
	1016		. "\n"
1017	1017	. ' if [ "${job_uid}" = "0" ]; then' . "\n"	. ' if [ "${job_uid}" = "0" ]; then' . "\n"
1018	1018	. ' if [ -d "${rgfs}/+pending" ]; then' . "\n"	. ' if [ -d "${rgfs}/+pending" ]; then' . "\n"
1019	1019	. ' xecho "+pending dir exists"' . "\n"	. ' xecho "+pending dir exists"' . "\n"

...	...	function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1040	1040	. ' rg_op_s mkdir -p "${rgfs}" \|\| break' . "\n"	. ' rg_op_s mkdir -p "${rgfs}" \|\| break' . "\n"
1041	1041	. ' if [ "${job_uid}" = "${pkg_repo_uid}" ]; then' . "\n"	. ' if [ "${job_uid}" = "${pkg_repo_uid}" ]; then' . "\n"
1042	1042	. ' rg_cp -av "${rpms_dir}"/*.rpm "${rgfs}/" \|\| break' . "\n"	. ' rg_cp -av "${rpms_dir}"/*.rpm "${rgfs}/" \|\| break' . "\n"
1043		. ' else' . "\n"
	1043		. ' else' . "\n" // TODO: shouldn't I copy directly to ${rgfs}/${repo_username} if do_sensitive_ops==1? Seems it cannot happen! when we are uid 0, we do not need to copy!
1044	1044	. ' # "+" is used because it is not allowed in username' . "\n"	. ' # "+" is used because it is not allowed in username' . "\n"
1045	1045	. ' rg_op_s mkdir -p "${rgfs}/+pending/${repo_username}" \|\| break' . "\n"	. ' rg_op_s mkdir -p "${rgfs}/+pending/${repo_username}" \|\| break' . "\n"
1046	1046	. ' rg_cp -av "${rpms_dir}/${repo_username}"/*.rpm "${rgfs}/+pending/${repo_username}/" \|\| break' . "\n"	. ' rg_cp -av "${rpms_dir}/${repo_username}"/*.rpm "${rgfs}/+pending/${repo_username}/" \|\| break' . "\n"
1047	1047	. ' fi' . "\n"	. ' fi' . "\n"
	1048		. "\n"
	1049		. ' if [ "`do_sensitive_ops`" != "1" ]; then' . "\n"
	1050		. ' xecho "Triggering the rebuild of subrepo"' . "\n"
	1051		. ' echo "${pkg_subrepo_id}" >> /mnt/build2/pkg_subrepo_dirty' . "\n" // TODO: Shouldn't be done by rgfs?
	1052		. ' fi' . "\n"
1048	1053	. ' rg_log_pop' . "\n"	. ' rg_log_pop' . "\n"
1049	1054	. ' else' . "\n"	. ' else' . "\n"
1050	1055	. ' xecho "No packages in pkg_list_file."' . "\n"	. ' xecho "No packages in pkg_list_file."' . "\n"
1051	1056	. ' fi'. "\n"	. ' fi'. "\n"
1052		. "\n"
1053		. ' if [ "`do_sensitive_ops`" != "1" ]; then' . "\n"
1054		. ' xecho "Triggering the rebuild of subrepo"' . "\n"
1055		. ' echo "${pkg_subrepo_id}" >> /mnt/build2/pkg_subrepo_dirty' . "\n"
1056		. ' else' . "\n"
	1057		. "\n"
	1058		. ' if [ "`do_sensitive_ops`" = "1" ]; then' . "\n"
1057	1059	. ' rg_rpm_createrepo "${env_distro}" "${rgfs}" \\' . "\n"	. ' rg_rpm_createrepo "${env_distro}" "${rgfs}" \\' . "\n"
1058	1060	. ' "${pkg_filter_list_file}" "${repo_username}" \|\| break' . "\n"	. ' "${pkg_filter_list_file}" "${repo_username}" \|\| break' . "\n"
1059	1061	. ' fi' . "\n";	. ' fi' . "\n";

...	...	function start_worker_build_repo($job, $conf, &$reason, &$reason2)
1061	1063	$s .= "\n"	$s .= "\n"
1062	1064	. ' rg_log_pop' . "\n"	. ' rg_log_pop' . "\n"
1063	1065	. ' done ;;' . "\n"	. ' done ;;' . "\n"
1064		. ' *) xecho "Unknown distro!"; E=1 ;;' . "\n"
	1066		. ' *) xecho "Unsupported distro!"; E=1 ;;' . "\n"
1065	1067	. ' esac' . "\n"	. ' esac' . "\n"
1066	1068	. ' [ "${E}" = "0" ] \|\| break' . "\n"	. ' [ "${E}" = "0" ] \|\| break' . "\n"
1067		. "\n"
	1069		. "\n"
1068	1070	. ' if [ "${rgfs_mounted[${pkg_repo_id}-${pkg_subrepo_id}]}" = "1" ]; then' . "\n"	. ' if [ "${rgfs_mounted[${pkg_repo_id}-${pkg_subrepo_id}]}" = "1" ]; then' . "\n"
1069	1071	. ' xecho "Unmounting rgfs"' . "\n"	. ' xecho "Unmounting rgfs"' . "\n"
1070	1072	. ' umount "/mnt/build2/rgfs/${pkg_repo_id}/${pkg_subrepo_id}"' . "\n"	. ' umount "/mnt/build2/rgfs/${pkg_repo_id}/${pkg_subrepo_id}"' . "\n"

...	...	function start_worker($job)
1367	1369	$p_i_cmd = '';	$p_i_cmd = '';
1368	1370	if (!empty($job['packages'])) {	if (!empty($job['packages'])) {
1369	1371	rg_log_debug('packages: ' . $job['packages'] . '.');	rg_log_debug('packages: ' . $job['packages'] . '.');
1370		$p_i_cmd .= 'echo "MInstalling webhook specified packages" > /dev/virtio-ports/rgw' . "\n";
	1372		$p_i_cmd .= 'rg_notify "Installing webhook specified packages"' . "\n";
1371	1373
1372	1374	$pkgs = explode(' ', $job['packages']);	$pkgs = explode(' ', $job['packages']);
1373	1375	$p_i_cmd .= '> /mnt/packages.log' . "\n";	$p_i_cmd .= '> /mnt/packages.log' . "\n";

...	...	function start_worker($job)
1411	1413	. "\n"	. "\n"
1412	1414	. 'source /mnt/build_tools.sh' . "\n"	. 'source /mnt/build_tools.sh' . "\n"
1413	1415	. "\n"	. "\n"
	1416		. 'chmod a+rw /dev/virtio-ports/rgw # Else, we cannot signal from inside the build' . "\n"
	1417		. 'rg_notify "rg.sh started"' . "\n"
	1418		. "\n"
1414	1419	. 'mkdir /mnt/tmp && chmod 1777 /mnt/tmp' . "\n"	. 'mkdir /mnt/tmp && chmod 1777 /mnt/tmp' . "\n"
1415	1420	. 'mount --bind /mnt/tmp /tmp' . "\n"	. 'mount --bind /mnt/tmp /tmp' . "\n"
1416	1421	. 'mkdir -p /mnt/var/tmp && chmod 1777 /mnt/var/tmp' . "\n"	. 'mkdir -p /mnt/var/tmp && chmod 1777 /mnt/var/tmp' . "\n"

...	...	function start_worker($job)
1424	1429	. 'ERR=""' . "\n"	. 'ERR=""' . "\n"
1425	1430	. 'id' . "\n"	. 'id' . "\n"
1426	1431	. 'date +%s > /mnt/T_START' . "\n"	. 'date +%s > /mnt/T_START' . "\n"
1427		. 'chmod a+rw /dev/virtio-ports/rgw # Else, we cannot signal from inside the build' . "\n"
1428		. 'echo "MVM started" > /dev/virtio-ports/rgw' . "\n"
1429	1432	. "\n"	. "\n"
1430		. 'echo "MWaiting for net" > /dev/virtio-ports/rgw' . "\n"
	1433		. 'rg_notify "Waiting for net"' . "\n"
1431	1434	. '# Waiting for net' . "\n"	. '# Waiting for net' . "\n"
1432	1435	. 'while [ "`ip ro li \| grep ^default`" = "" ]; do' . "\n"	. 'while [ "`ip ro li \| grep ^default`" = "" ]; do' . "\n"
1433	1436	. ' sleep 1' . "\n"	. ' sleep 1' . "\n"
1434	1437	. 'done' . "\n"	. 'done' . "\n"
1435	1438	. 'date +%s > /mnt/T_NET_OK' . "\n\n"	. 'date +%s > /mnt/T_NET_OK' . "\n\n"
	1439		. 'rg_notify "Net OK"' . "\n"
1436	1440	. "\n"	. "\n"
1437	1441	. $p_i_cmd	. $p_i_cmd
1438	1442	. 'date +%s > /mnt/T_PKGS_OK' . "\n\n"	. 'date +%s > /mnt/T_PKGS_OK' . "\n\n"

...	...	function start_worker($job)
1484	1488	. ' fi' . "\n"	. ' fi' . "\n"
1485	1489	. ' fi' . "\n"	. ' fi' . "\n"
1486	1490	. "\n"	. "\n"
	1491		. ' rg_notify "Stating build.sh script"' . "\n"
1487	1492	. ' echo; echo "`date`: Running build.sh script"' . "\n"	. ' echo; echo "`date`: Running build.sh script"' . "\n"
1488	1493	. ' date +%s > /mnt/status/build.sh.start' . "\n" // TODO: not read and not exposed?	. ' date +%s > /mnt/status/build.sh.start' . "\n" // TODO: not read and not exposed?
1489	1494	. ' su - build -c "bash /mnt/build.sh" &>/mnt/status/build.log' . "\n"	. ' su - build -c "bash /mnt/build.sh" &>/mnt/status/build.log' . "\n"
1490	1495	. ' E=${?}' . "\n"	. ' E=${?}' . "\n"
1491	1496	. ' date +%s > /mnt/status/build.sh.stop' . "\n" // TODO: not read and not exposed?	. ' date +%s > /mnt/status/build.sh.stop' . "\n" // TODO: not read and not exposed?
	1497		. ' rg_notify "build.sh finished with code ${E}"' . "\n"
1492	1498	. ' if [ "${E}" != "0" ]; then' . "\n"	. ' if [ "${E}" != "0" ]; then' . "\n"
1493	1499	. ' echo "`date`: DEBUG: build.sh exited with code ${E}"' . "\n"	. ' echo "`date`: DEBUG: build.sh exited with code ${E}"' . "\n"
1494	1500	. ' ERR="build.sh returned ${E}"' . "\n"	. ' ERR="build.sh returned ${E}"' . "\n"
1495	1501	. ' break' . "\n"	. ' break' . "\n"
1496	1502	. ' fi' . "\n"	. ' fi' . "\n"
1497	1503	. "\n"	. "\n"
	1504		. ' rg_notify "Stating build_rpms.sh script"' . "\n"
1498	1505	. ' echo; echo "`date`: Running build_rpms.sh script"' . "\n"	. ' echo; echo "`date`: Running build_rpms.sh script"' . "\n"
1499	1506	. ' date +%s > /mnt/T_RPMS_START' . "\n"	. ' date +%s > /mnt/T_RPMS_START' . "\n"
1500	1507	. ' su - build -c "bash /mnt/build_rpms.sh" &>/mnt/status/build_rpms.log' . "\n"	. ' su - build -c "bash /mnt/build_rpms.sh" &>/mnt/status/build_rpms.log' . "\n"
1501	1508	. ' E=${?}' . "\n"	. ' E=${?}' . "\n"
1502	1509	. ' date +%s > /mnt/T_RPMS_END' . "\n"	. ' date +%s > /mnt/T_RPMS_END' . "\n"
	1510		. ' rg_notify "build_rpms.sh finished with code ${E}"' . "\n"
1503	1511	. ' if [ "${E}" != "0" ]; then' . "\n"	. ' if [ "${E}" != "0" ]; then' . "\n"
1504	1512	. ' ERR="build_rpms.sh returned ${E}"' . "\n"	. ' ERR="build_rpms.sh returned ${E}"' . "\n"
1505	1513	. ' break' . "\n"	. ' break' . "\n"

...	...	function start_worker($job)
1510	1518	. ' [ -d /mnt/target/rocketgit ] && mv -f /mnt/target/rocketgit /mnt/build2' . "\n"	. ' [ -d /mnt/target/rocketgit ] && mv -f /mnt/target/rocketgit /mnt/build2' . "\n"
1511	1519	. ' chown -R build2:build2 /mnt/build2 /mnt/rpmbuild' . "\n"	. ' chown -R build2:build2 /mnt/build2 /mnt/rpmbuild' . "\n"
1512	1520	. "\n"	. "\n"
	1521		. ' rg_notify "Stating build_rgfs.sh script"' . "\n"
1513	1522	. ' echo; echo "`date`: Running build_rgfs.sh script"' . "\n"	. ' echo; echo "`date`: Running build_rgfs.sh script"' . "\n"
1514	1523	. ' bash /mnt/build_rgfs.sh &>/mnt/status/build_rgfs.log' . "\n"	. ' bash /mnt/build_rgfs.sh &>/mnt/status/build_rgfs.log' . "\n"
1515	1524	. ' if [ "${?}" != "0" ]; then' . "\n"	. ' if [ "${?}" != "0" ]; then' . "\n"

...	...	function start_worker($job)
1517	1526	. ' fi' . "\n"	. ' fi' . "\n"
1518	1527	. "\n"	. "\n"
1519	1528	. ' if [ -x /mnt/build_repo.sh ]; then' . "\n"	. ' if [ -x /mnt/build_repo.sh ]; then' . "\n"
	1529		. ' rg_notify "Stating build_repo.sh script"' . "\n"
1520	1530	. ' echo; echo "`date`: Running build_repo.sh script"' . "\n"	. ' echo; echo "`date`: Running build_repo.sh script"' . "\n"
1521	1531	. ' date +%s > /mnt/T_REPO_START' . "\n"	. ' date +%s > /mnt/T_REPO_START' . "\n"
1522	1532	. ' su - build2 -c "bash /mnt/build_repo.sh" &>/mnt/status/build_repo.log' . "\n"	. ' su - build2 -c "bash /mnt/build_repo.sh" &>/mnt/status/build_repo.log' . "\n"
1523	1533	. ' E=${?}' . "\n"	. ' E=${?}' . "\n"
1524	1534	. ' date +%s > /mnt/T_REPO_END' . "\n"	. ' date +%s > /mnt/T_REPO_END' . "\n"
	1535		. ' rg_notify "build_repo.sh finished with code ${E}"' . "\n"
1525	1536	. ' if [ "${E}" != "0" ]; then' . "\n"	. ' if [ "${E}" != "0" ]; then' . "\n"
1526	1537	. ' ERR="build_repo.sh returned ${E}"' . "\n"	. ' ERR="build_repo.sh returned ${E}"' . "\n"
1527	1538	. ' break' . "\n"	. ' break' . "\n"

...	...	function start_worker($job)
1532	1543	. 'done' . "\n"	. 'done' . "\n"
1533	1544	. 'if [ "${ERR}" != "" ]; then' . "\n"	. 'if [ "${ERR}" != "" ]; then' . "\n"
1534	1545	. ' echo "${ERR}" > /mnt/status/err' . "\n"	. ' echo "${ERR}" > /mnt/status/err' . "\n"
1535		. ' echo "EERROR: ${ERR}" > /dev/virtio-ports/rgw' . "\n"
	1546		. ' rg_notify_err "ERROR: ${ERR}"' . "\n"
1536	1547	. 'fi' . "\n"	. 'fi' . "\n"
1537	1548	. 'date +%s > /mnt/T_DONE' . "\n\n"	. 'date +%s > /mnt/T_DONE' . "\n\n"
1538		. 'echo "MDONE" > /dev/virtio-ports/rgw' . "\n"
	1549		. 'rg_notify "DONE"' . "\n"
1539	1550	. "\n"	. "\n"
1540	1551	. 'echo; echo "`date`: Syncing"' . "\n"	. 'echo; echo "`date`: Syncing"' . "\n"
1541	1552	. 'sync' . "\n"	. 'sync' . "\n"
1542	1553	. "\n"	. "\n"
	1554		. 'rg_notify "Shutting down"' . "\n"
1543	1555	. 'echo; echo "`date`: Shutting down"' . "\n"	. 'echo; echo "`date`: Shutting down"' . "\n"
1544	1556	. 'while [ -r /root/stop ]; do' . "\n"	. 'while [ -r /root/stop ]; do' . "\n"
1545	1557	. ' sleep 1' . "\n"	. ' sleep 1' . "\n"

File techdocs/rg_build.drawio added (mode: 100644) (index 0000000..d52605d)
	1		<mxfile host="Electron" modified="2021-11-06T12:59:58.633Z" agent="5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/15.4.0 Chrome/91.0.4472.164 Electron/13.5.0 Safari/537.36" etag="TIu6mqgRvJ1CdEEtiucO" version="15.4.0" type="device"><diagram id="C5RBs43oDa-KdzZeNtuy" name="Page-1">7Zxbd5s4EMc/TU6fkgPCxu5jnKRpd9vtJdkm2RcfGWRQjJEr5Fs/fSUQFyNiUxdwl5KXICGEmfnzGzGMfWZczTe3FC7cD8RG3hnQ7M2ZcX0GANAMnf8TPduoR9cHr6Meh2Jb9qUdd/g7kp2a7F1iGwU7AxkhHsOL3U6L+D6y2E4fpJSsd4dNibd71gV0kNJxZ0FP7X3ANnOj3iEYpP1vEXbc+My6Ka9vDuPB8koCF9pknekybs6MK0oIi7bmmyvkCevFdnl4t33w3s/M278+B9/gv6O/7//5eh5N9uZnDkkugSKfVTu1EU29gt5S2muNJi4hszNgevxcownlW47YQhtkLRl3ZWQMto0tTMnSt5E4i86HrV3M0N0CWmLvmouK97ls7sndU+IzKRKdW3DkeDAQHtb4dsAomSVeEqMTk4vdJa0grbVClKFNRgPSKreIzBGjWz5E7u1L/0qFm7K5TtWixypyM0rpyXFQCtRJJk6dwDekH4p98h2aTwb48mn19tYEj+OPm/f/DaQbsz5xMBNXvgzcthtfL2l8UJfxdcX4e0yu7Zr3GTG2lfaFS0Z4F6HMJQ7xofeekEWBG0Qb+falIB1vTzxizaKuN1h88vAcvCXHDw97yoMT5I2gNXPCD3pFPEL5Lp/4SExlczrKa0k/3E3a+5OeDsiSWuiwmhmkDto3nySR+Hx7dUORBxle7dK9SAby0E8E+yzVmw5ygtNzSoouSB6VxemhiczcRNEVKxOFqkyu53ihAkWojGLHQVQYcSV8pxB87Y4nS+zZ44ALqvUkMU7NcTW2digpi5J9iMiiZN+t0TRK8kI6liSDZkHS+0WQjIUalCFTIo6H4emRDyceV7nYWmFK/DmftfXrSKCVBFDe35UBqN8BqNq1DCgJoN7/fS2TR1nNCDIVoY4EXkIEUWQhbiKOC+2ZTNoODaMsNGpbtQw6aFQLjV5JaJgngsawImiAPH1qhsZQEeoDobNCZmhQPO9wCUMq1h3a1w+t40iy2vhtQPK6A0m1IDFLgmR4GpCAqkBi5JN7NYMk5lZGqdS5ELlWDQtayMR3+zImxuB3Y4bepV8rhsawJDTim6BuaoCc5pSn37LUyE9kNLz80NUEbPSaxnKRNSNL1n5cDE6Oiy7FWjEu4pvoMC8aSrIaRk50xpG8MIbahZb50/dPWzc91KzrRbBAFu+aYu5YufQIEKScJ+1fevR6J2dJly2tmiVl06V6Q/nS9qw91IRp+D4m89AS5jrCdy5QNJdBmBh5FQ571Xqc9MvWkdSHky6PWjVOyiZS9YYyqUnMimNYPm9RFifKRP2GcaKmUqPXu3QxDzqmxEwZnpwpXUq1aqaUzanqDSVVFRS8rogp/YaTqvHpVKagBSnJFNB+qJj9klDJh4TKoAK6lGvVUCmbc43vkaah0s+Xqh4NlTyd6oaKmnMVr3JjkpDFooWpEuXN7snTrqBLu3If0+2jnDlsPImZL/px83qTfCzR2satDWaPme3MUbyVHiQa26OEdbiIrWyKN/Zql5Ype1+oSd2kJMUnDE9xWJLCXJHfncQVbi3H1ekzu6AosxvVK9t4FRcspxWHFvQ84aekqDkeIl02tjMVz/wTJfszfZmJW+fhfMni6eORmg790+JR1TEClI0Rp0rdH/tcrMSIhl/8gT8pL4y8CVlXsWyqWt5lU8lx6Gha3koG+Fh5K/mjuuWtppILYu199AWjsLiu+BtGL4VSZSolSCffTXomky5S/25LsS6DXTXK+mVRdqKy4KSo9JfLghuO1PHpClAmFLYjWvPbksQ7zoNQS5d8APfzJt0ZY+YL1yBit2GxYIgqcfcV/uRGBlPRKV/gFMcB271XIhHnxDnlks91QQ87Pm9aXI384ccYCbhg/gR0KXfMsW2Hui2i3+6tWhfFQP4bKyrFQK+AYnm9HEOx68Ho1rx7/jieA2O4CD5bT48fCn6g491UPjdOOBhEWONGELagKAjD25XcHT39z08caBTTl/Va2UDTaPlFoYu6lxqFtK8y8BS85Xj5bqk9zBxdfKFMVF/xRaF5gKLUK8mRl+NBso5dzJyxg3xEIUN8scvEy1VRtTG2cnPsLn7zd4JL5pNlcJg8OwIV8n8D59gThnuLvBUSgaOAT0qMKVA+PyX2Hd4y09Z9eKed92rklpnjVlGJh1bArfxXnSrjVvdmpUJu7QveB7EFmsHWsXms/ETV1XfwZvqbdtHw9KcBjZsf</diagram></mxfile>

File techdocs/rpm.txt added (mode: 100644) (index 0000000..db1d81c)
	1	Try to describe what happens when a build hook is present,
	2	a .spec file is present in the repo and the user pushes.
	3
	4	== Scenario 1 ==
	5	- User pushes
	6	- Web hook will trigger event 'rg_wh_build_send'
	7	- 'rg_wh_build_send' will trigger event 'rg_wh_build_send_one' for each enabled environments
	8	- 'rg_wh_build_send_one' will
	9	- get the pkg_maps, repo and subrepo info
	10	- sets exec:prepare_rpms and exec:copy_to_rgfs flags
	11	- will call 'rg_pkg_prepare_ev_for_dotrepo' to create user repositories
	12	- will call 'rg_builder_add'
	13	- Builer will send the job to a worker
	14	- Worker will start the VM
	15	- Inside VM, script rg.sh is started
	16	- Checkout correct branch
	17	- .spec file is searched are BuildRequires are installed
	18	- 'build.sh' starts [as user 'build']
	19	- Secrets are exported
	20	- The commands defined in the webhook are run
	21	- 'build_rpms.sh' starts [as user 'build']
	22	- 'rg_build_rpm' shell function is called for '/mnt/target'
	23	- for each job[rpm_repo_files] 'rg_build_dotrepo /mnt/target-dotrepo-<subrepo_id>' is called
	24	- 'build_rgfs.sh' starts [as user 'root']
	25	- 'build_repo.sh' starts [as user 'build2']
	26	- If no pkg mapping is defined, exit
	27	- gpg keys are stored in /mnt/build2
	28	- Search for .rpm packages in /mnt/rpmbuild
	29	- Imports gpg keys
	30	- If flag exec:prepare_rpms is set:
	31	- .rpms are copied into /mnt/build2/rpms/<pkg_repo_id}/{debug,os,src}[/username]
	32	- A file A, containing the rpms, is created and rpms are signed
	33	- For each mapping
	34	- a rgfs is mounted
	35	- if '+pending' dir is found, we add the rpms to the file A
	36	- if file A is empty, continue with next mapping
	37	- If a rpm file is already in the repo, remove the name from file A
	38	- if the build is done as uid 0, sign the '+pending' rpms
	39	- if exec:copy_to_rgfs flag is present, copy the signed rpms to rgfs dir
	40	- if uid is not 0 and pkg_repo uid != job_uid, save the pkg_subrepo_id into /mnt/build2/pkg_subrepo_dirty
	41	- If uid is 0 or pkg_repo_uid == job_uid, call rg_rpm_createrepo
	42	- Unmount all rgfs dirs
	43
	44	- VM is shutdown and logs and timings are collected
	45	- Worker notifies the Builder
	46	- Builder calls 'builder_done' function which will trigger 'wh_build_job_done'
	47
	48
	49	== Scenario 2 ==
	50	- User, from the web interface, edits a pkg_repo/subrepo:
	51	- 'pkg_subrepo_edit' event is triggered (user edited a pkg subrepo)
	52	triggers 'rg_pkg_subrepo_event_edit' event
	53	- or 'rg_pkg_subrepo_event_pkg_repo_changed' event is triggered (user edited a pkg repo)
	54	triggers 'rg_pkg_subrepo_event_edit' event
	55
	56
	57	== Events called from the scenarios ==
	58	- Event 'wh_build_job_done' [function rg_wh_build_job_done]
	59	If callbacks are present, add events for all of them
	60
	61	- Event 'rg_pkg_subrepo_event_edit':
	62	- flag force_regenerate_dotrepo is set to 1 because version was incremented
	63	- only for environments which already have a dotrepo (no build for empty pkg repos)
	64	- calls 'pkg_generate_dotrepo_rpm' event for all enabled envs
	65
	66	- Event 'pkg_generate_dotrepo_rpm' [function rg_pkg_event_generate_dotrepo_rpm]:
	67	- sets 'exec:worker_must_be_global' flag - we do not trust user workers
	68	- prepares a fake pkg_maps from the pkg_repo/subrepo passed in the event
	69	- calls 'rg_pkg_prepare_ev_for_dotrepo' function
	70	- calls 'rg_builder_add'
	71
	72	- Event 'pkg_subrepo_regenerate' [function rg_pkg_subrepo_event_regenerate]:
	73	- 'uid' is set to 0
	74	- for each subrepo passed in the event a fake pkg_maps is created
	75	- calls 'rg_pkg_prepare_ev_for_dotrepo' function
	76	- calls 'rg_builder_add'
	77
	78	- Function 'rg_pkg_prepare_ev_for_dotrepo':
	79	- it avoids not enabled environments (state::distros)
	80	- for each pkg mapping:
	81	- if (event uid != pkg_repo uid) skip (we do not allow a user to influence other repos)
	82	TODO: what if event uid is 0?!
	83	- if (!force_regenerate_dotrepo && we already generated the last version of dotrepo) skip
	84	- adds files which will go in /etc/yum.repos.d and gpg keys
	85	- it will add an event callback: 'pkg_generate_dotrepo_rpm_callback'
	86
	87	- Event 'rg_pkg_subrepo_update_distro_info' [function rg_pkg_event_generate_dotrepo_rpm_callback function]
	88	it calls: rg_pkg_subrepo_update_distro_info and
	89	triggers pkg_subrepo_regenerate if ev[status][pkg_subrepo_dirty'] is present
	90	this flag is set when build_repo.sh is copying something to +pending
	91
	92	- Event 'pkg_generate_dotrepo_rpm_callback' [function rg_pkg_event_generate_dotrepo_rpm_callback]:
	93	- called when a dotrepo was done
	94	- for each (subrepo, ver), call 'rg_pkg_subrepo_update_distro_info'
	95	- if 'status:pkg_subrepo_dirty' is present, trigger 'pkg_subrepo_regenerate' event
	96	Set by the 'worker', based on /root/build2/pkg_subrepo_dirty file
	97	Set by exec:copy_to_rgfs phase of build_repo.sh when some .rpm are copied on rgfs
	98
	99
	100	== old ==
	101	- The .spec file is found and .rpm files are generated.
	102	- The .rpm files are moved to '+pending folder'.
	103	We do not create directly the repo because we do not trust the
	104	environment anymore.
	105	- The finish of the job will trigger the rebuild of the repo.
	106	- If a "dotrepo" .rpm file (containing files which will be put in
	107	/etc/yum.repos.d) is not built for that repo/distro/major/arch
	108	the build is triggered.
	109	== old end ==
	110

Hints:
Before first commit, do not forget to setup your git environment:

git config --global user.name "your_name_here"
git config --global user.email "your@email_here"

Clone this repository using HTTP(S):

git clone https://rocketgit.com/user/catalinux/rocketgit

Clone this repository using ssh (do not forget to upload a key first):

git clone ssh://rocketgit@ssh.rocketgit.com/user/catalinux/rocketgit

Clone this repository using git:

git clone git://git.rocketgit.com/user/catalinux/rocketgit

You are allowed to anonymously push to this repository.
This means that your pushed commits will automatically be transformed into a merge request:

... clone the repository ...
... make some changes and some commits ...
git push origin main