RocketGit

catalinux / rocketgit (public) (License: Affero GPLv3) (since 2015-03-19) (hash sha1)

Main RocketGit repo

Clone URLs: https://rocketgit.com/user/catalinux/rocketgit ssh://rocketgit@ssh.rocketgit.com/user/catalinux/rocketgit git://git.rocketgit.com/user/catalinux/rocketgit

List of commits:

Subject	Hash	Author	Date (UTC)
Lots of things, mostly worker fixes	8768227306ef89ecc51d392cf986e75a7044dde4	Catalin(ux) M. BOIE	2020-04-10 15:27:11
Added functional test for HTTP 304 code	93205ec14d1073c434360abf7ea525a75407b1ab	Catalin(ux) M. BOIE	2020-04-02 14:05:49
Lots of changes all over the place	3ddbd9e686f514eed1e715d7702a910ca3f8b22c	Catalin(ux) M. BOIE	2020-04-01 07:17:57
More tests updates	d5d45ace725f6ab336998a328136a2f7f9474df1	Catalin(ux) M. BOIE	2020-04-01 05:30:54
Lots of tests updates	183c4af52e56ae4e4fd74a35e2b17ac79952aec6	Catalin(ux) M. BOIE	2020-03-20 16:10:33
Improved a little bit the mail reporting for admin	576f96d554b1e273391ac54404bf5f885325e5e8	Catalin(ux) M. BOIE	2020-03-03 19:25:17
Worker and builder big changes	90f596e4fd8fa291b1da831bce609b486c7a5875	Catalin(ux) M. BOIE	2020-01-06 13:31:20
Samples update (switch to systemd services for builder and worker)	8bfde1fb79778d024350bf5d75f076fb5e178512	Catalin(ux) M. BOIE	2020-01-06 12:50:21
builder: show queue in the web page	6cab7820b0bd2622afb77391a9e9515e0e837eef	Catalin(ux) M. BOIE	2020-01-06 12:49:26
totp: Allow also years for 'val' command	d666df0cec948f938469b9eb12a1b981cb41cfa1	Catalin(ux) M. BOIE	2020-01-04 14:39:33
worker: change git clone parameters because we could not clone	8eb2727ac17e12e148f39c72e1f4bd1e31319b33	Catalin(ux) M. BOIE	2019-12-19 03:34:26
Added a TODO for worker	492fb4546829f53adacd74c4647c3634f7c37065	Catalin(ux) M. BOIE	2019-12-18 22:23:12
worker improvements	4387b00291b5848aa08c303f9d62a126a6806a35	Catalin(ux) M. BOIE	2019-12-18 22:17:23
.spec: require tidy for building the tests	c0deab2ce24388c91d2bc2e96338a260c9657dd6	Catalin(ux) M. BOIE	2019-12-18 22:15:29
nginx configuration update	015c9404229f2ff7fb91cc57ab21289f01878334	Catalin(ux) M. BOIE	2019-12-18 21:55:13
Added Business model to comparison	c62d466580aca0ff3371f3ba369959c7ef7631d0	Catalin(ux) M. BOIE	2019-11-21 20:44:16
CSS small correction	ded9ef6bce5bebfa815bc7ae5733bc8e0810af3b	Catalin(ux) M. BOIE	2019-11-21 18:20:41
Added LDAP/qrencode to features/thanks pages	d269d59787f61045f5831c6985899c28da08ebeb	Catalin(ux) M. BOIE	2019-11-21 18:20:22
Invalidate slave table cache when updating the db structure, else new slaves will not be created	8e1bf7feeb87818179e7152a356e0f408da706f2	Catalin(ux) M. BOIE	2019-11-19 19:33:15
docker: push also the latest label	a72e2e74355d772ae45bb99798ad47082390943c	Catalin(ux) M. BOIE	2019-11-19 19:32:11

Commit 8768227306ef89ecc51d392cf986e75a7044dde4 - Lots of things, mostly worker fixes
Author: Catalin(ux) M. BOIE
Author date (UTC): 2020-04-10 15:27
Committer name: Catalin(ux) M. BOIE
Committer date (UTC): 2020-04-10 15:27
Parent(s): 93205ec14d1073c434360abf7ea525a75407b1ab
Signing key:
Tree: bee60eb55b40b360b7e3750ad0e99ec73f7315c5

File	Lines added	Lines deleted
History.txt	1	0
TODO	28	10
inc/admin.inc.php	1	1
inc/apikeys.inc.php	2	2
inc/builder.inc.php	35	8
inc/conn.inc.php	1	1
inc/fixes.inc.php	3	12
inc/keys.inc.php	20	17
inc/login/login.php	4	4
inc/mr.inc.php	2	5
inc/prof.inc.php	4	1
inc/repo.inc.php	50	50
inc/totp.inc.php	1	1
inc/user.inc.php	12	15
inc/user/forgot_send.php	1	1
inc/util.inc.php	18	34
inc/wh/build.inc.php	17	11
inc/wh/core.inc.php	1	0
inc/workers.inc.php	38	2
root/themes/default/builder/queue/line.html	14	8
root/themes/default/doc/worker.html	0	1
root/themes/default/repo/log/header.html	1	1
root/themes/default/user/settings/workers/add.html	1	1
root/themes/default/user/settings/workers/hints.html	4	2
samples/worker-main.conf.sample	21	7
scripts/builder.TODO	8	0
scripts/builder.php	62	31
scripts/cachec.php	1	1
scripts/remote.php	26	9
scripts/worker.TODO	10	3
scripts/worker.php	273	61
tests/by_http.php	11	7
tests/config.php	4	0
tests/git_big_push.php	0	2
tests/keys.php	13	2
tests/wh_cloud.php	8	2
tests/wh_lambda.php	8	2

File History.txt changed (mode: 100644) (index f573255..09165f1)
21	21	2019-09-01 - rocketgit.com achieved 1000 repositories.	2019-09-01 - rocketgit.com achieved 1000 repositories.
22	22	2019-09-18 - GitLab raised $278m in funding.	2019-09-18 - GitLab raised $278m in funding.
23	23	2019-11-10 - RocketGit 0.71 released.	2019-11-10 - RocketGit 0.71 released.
	24		2020-04-10 - Fedora has chosen GitLab EE as the Git Forge

File TODO changed (mode: 100644) (index ea8fe66..5cec33b)
1	1	== Where I stopped last time ==	== Where I stopped last time ==
2	2
3	3	== Now ==	== Now ==
4		[ ] Functional test for 'rg_process_input'! No release without this!
5		We will need to fake some requests, even if no git data will be passed.
6	4	[ ] docker	[ ] docker
7		[ ] Resync nginx conf on rg2.
	5		[ ] Resync nginx conf on rg2 (because ws builder?).
	6		[ ]
	7
	8		== BEFORE NEXT RELEASE ==
	9		[ ] worker: functional tests for cloning.
	10		[ ] worker: allow user to forbid global workers.
	11		[ ] worker: trigger for push: trigger a git mirror to other (specified) repo(s)
	12		[ ] worker: Let the user choose how to clone a repo (ssh/https/git).
	13		[ ] worker: Make error2 visible in the interface.
	14		[ ] worker: test what happens if the worker process restarts. We should be able
	15		to survive.
	16		[ ] workers+webhooks: seems there is a mess showing the last status.
	17		We should unify the status. update_last_output must go away; we
	18		need to keep the serialized data and generate it when needed.
	19		[ ] workers: Do not show the 'user' column if I am looking at that table
	20		as a normal user (all of the rows are mine).
	21		[ ] wh: seems an old env list is kept when we edit the hook. It should not.
	22		[ ] workers: use transactions for rg_worker_update and rg_worker_insert.
	23		[ ] worker: some stuff is stored in 'root/status', others in 'root'.
	24		Store everything in 'root/status'?
	25		[ ] worker: hook_id must link to the webhook.
	26		[ ] We do a namespace copy (copy_tree) before calling 'repo_fetch_push_helper',
	27		When we may return 401, so the copy is useless!
8	28	[ ] worker: have a flag which allows internet access?	[ ] worker: have a flag which allows internet access?
9	29	Better to let the owner provide a template which may have or	Better to let the owner provide a template which may have or
10	30	may have not Internet access. Think about private networks! Better	may have not Internet access. Think about private networks! Better
11	31	with template.	with template.
12		[ ] worker: allow ssh (root) login only by key, to not be able to become root.
13		[ ]
14
15		== BEFORE NEXT RELEASE ==
16	32	[ ] docker: ssh and git clone URLs are using docker hostname which probably	[ ] docker: ssh and git clone URLs are using docker hostname which probably
17	33	is not working. Use some environment variables?	is not working. Use some environment variables?
18	34	Or disable them if admin did not set the correct URL?	Or disable them if admin did not set the correct URL?

33	49	[ ] user_login_by_user_pass_helper: when splitting login_token, verify to have	[ ] user_login_by_user_pass_helper: when splitting login_token, verify to have
34	50	only digits, else, just abort.	only digits, else, just abort.
35	51	[ ] Investigate --push-option for git-push: we may instruct the server to do	[ ] Investigate --push-option for git-push: we may instruct the server to do
36		stuff.
	52		stuff. Seems the only way is to store receive.advertisePushOptions=true
	53		in ${HOME/.gitconfig. Probably other options (including custom ones)
	54		may be added there. Build something generic.
	55		Then, activate it for tests.
37	56	[ ] Should I add the event string to table 'events'? Maybe we want to filter	[ ] Should I add the event string to table 'events'? Maybe we want to filter
38	57	by it in the future? Or make graphs?	by it in the future? Or make graphs?
39	58	[ ] stats: add a graph with the events queue (not processed yet events)?	[ ] stats: add a graph with the events queue (not processed yet events)?

116	135	This is fixed, but the consumers of rg_git_log_simple	This is fixed, but the consumers of rg_git_log_simple
117	136	must check if the array is empty and show a message.	must check if the array is empty and show a message.
118	137	[ ] git-receive-pack - it seems stateless-rpc and other thigs are gone!	[ ] git-receive-pack - it seems stateless-rpc and other thigs are gone!
119		[ ] php_errormsg -> rg_php_err().
120		[ ] Log also the time for push/fetch.
	138		[ ] Log also the time for push/fetch. Where?
121	139	[ ] In the reports, report also the size of the database? Tables? Indexes?	[ ] In the reports, report also the size of the database? Tables? Indexes?
122	140	[ ] Size of 'rocketgit' repo is not correct! Seems correct.	[ ] Size of 'rocketgit' repo is not correct! Seems correct.
123	141	[ ] When we try to send the report, check last success date and generate from	[ ] When we try to send the report, check last success date and generate from

File inc/admin.inc.php changed (mode: 100644) (index 35df737..df28ff3)
...	...	function rg_admin_invites_high_level($db, $rg)
92	92	break;	break;
93	93
94	94	$inv['list'] = rg_var_str('inv::list');	$inv['list'] = rg_var_str('inv::list');
95		$inv['subject'] = rg_var_str('inv::subject');
	95		$inv['subject'] = trim(rg_var_str('inv::subject'));
96	96	$inv['body'] = rg_var_str('inv::body');	$inv['body'] = rg_var_str('inv::body');
97	97
98	98	while (isset($_FILES['inv::file'])) {	while (isset($_FILES['inv::file'])) {

File inc/apikeys.inc.php changed (mode: 100644) (index 8897220..a1b1fc9)
...	...	function rg_ak_add_high_level($db, $rg, $paras)
501	501	$doit = rg_var_uint('doit');	$doit = rg_var_uint('doit');
502	502	rg_log('DEBUG: doit=' . $doit);	rg_log('DEBUG: doit=' . $doit);
503	503	while ($doit == 1) {	while ($doit == 1) {
504		$rg['ak']['name'] = rg_var_str('ak::name');
505		$rg['ak']['key'] = rg_var_str('ak::key');
	504		$rg['ak']['name'] = trim(rg_var_str('ak::name'));
	505		$rg['ak']['key'] = trim(rg_var_str('ak::key'));
506	506
507	507	if (strlen($rg['ak']['name']) == 0) {	if (strlen($rg['ak']['name']) == 0) {
508	508	$errmsg[] = 'invalid name';	$errmsg[] = 'invalid name';

File inc/builder.inc.php changed (mode: 100644) (index a7d6fd0..b453bbc)
...	...	function rg_builder_done($db, $job, $s)
109	109	rg_log_ml('DEBUG: builder_done: job: ' . print_r($job, TRUE));	rg_log_ml('DEBUG: builder_done: job: ' . print_r($job, TRUE));
110	110	rg_log_ml('DEBUG: builder_done: status: ' . print_r($s, TRUE));	rg_log_ml('DEBUG: builder_done: status: ' . print_r($s, TRUE));
111	111
	112		// old way
	113		$req = isset($job['request']) ? $job['request'] : $job;
	114
112	115	$job['done'] = time();	$job['done'] = time();
113	116
114	117	$ret = FALSE;	$ret = FALSE;

...	...	function rg_builder_done($db, $job, $s)
123	126	$labels = $s['labels'];	$labels = $s['labels'];
124	127
125	128	// Some cosmetic stuff	// Some cosmetic stuff
126		$env = $job['env'];
	129		$env = $req['env'];
127	130	$labels[] = 'worker_elap/' . ($s['done'] - $s['start']) . 's';	$labels[] = 'worker_elap/' . ($s['done'] - $s['start']) . 's';
128	131	$labels[] = 'wait_time/' . ($job['worker_sent'] - $job['itime']) . 's';	$labels[] = 'wait_time/' . ($job['worker_sent'] - $job['itime']) . 's';
129	132	$labels[] = 'date/' . gmdate('Y-m-d', $job['itime']);	$labels[] = 'date/' . gmdate('Y-m-d', $job['itime']);

...	...	function rg_builder_done($db, $job, $s)
132	135	// add labels to the commit	// add labels to the commit
133	136	$params = array(	$params = array(
134	137	'repo_id' => $job['repo_id'],	'repo_id' => $job['repo_id'],
135		'head' => $job['head'],
	138		'head' => $req['head'],
136	139	'type' => 'build',	'type' => 'build',
137	140	'misc' => $env,	'misc' => $env,
138	141	'labels' => rg_serialize($labels),	'labels' => rg_serialize($labels),

...	...	function rg_builder_done($db, $job, $s)
157	160	break;	break;
158	161	rg_sql_free_result($res);	rg_sql_free_result($res);
159	162	rg_cache_unset('repo_commit_labels' . '::' . $job['repo_id']	rg_cache_unset('repo_commit_labels' . '::' . $job['repo_id']
160		. '::' . $job['head'], RG_SOCKET_NO_WAIT);
	163		. '::' . $req['head'], RG_SOCKET_NO_WAIT);
161	164
162	165	$params = array(	$params = array(
163	166	'id' => $job['id'],	'id' => $job['id'],

...	...	function rg_builder_done($db, $job, $s)
175	178	$ev = array(	$ev = array(
176	179	'category' => 'wh_build_job_done',	'category' => 'wh_build_job_done',
177	180	'prio' => 100,	'prio' => 100,
178		'ui' => array('uid' => $job['uid']),
	181		'ui' => array('uid' => $req['uid']),
179	182	'job' => $job,	'job' => $job,
180	183	'status' => $s	'status' => $s
181	184	);	);

...	...	function rg_builder_done($db, $job, $s)
192	195
193	196	$rollback = FALSE;	$rollback = FALSE;
194	197
	198		$key = 'wh' . '::' . $req['uid'] . '::' . 'list'
	199		. '::' . $req['hook_id'];
	200		unset($params['id']);
	201		rg_cache_merge($key, $params, RG_SOCKET_NO_WAIT);
	202
195	203	rg_event_signal_daemon('', 0);	rg_event_signal_daemon('', 0);
196	204
197	205	$ret = TRUE;	$ret = TRUE;

...	...	function rg_builder_done($db, $job, $s)
209	217	*/	*/
210	218	function rg_builder_nice_status(&$a)	function rg_builder_nice_status(&$a)
211	219	{	{
212		//rg_log_ml('nice_status: a: ' . print_r($a, TRUE));
	220		rg_log_ml('DEBUG: nice_status: a: ' . print_r($a, TRUE));
213	221	$a['start_nice'] = gmdate('Y-m-d H:i', intval($a['start']));	$a['start_nice'] = gmdate('Y-m-d H:i', intval($a['start']));
214	222	$a['net_ok_nice'] = gmdate('Y-m-d H:i', intval($a['net_ok']));	$a['net_ok_nice'] = gmdate('Y-m-d H:i', intval($a['net_ok']));
215	223	$a['pkgs_ok_nice'] = gmdate('Y-m-d H:i', intval($a['pkgs_ok']));	$a['pkgs_ok_nice'] = gmdate('Y-m-d H:i', intval($a['pkgs_ok']));
216	224	$a['done_nice'] = gmdate('Y-m-d H:i', intval($a['done']));	$a['done_nice'] = gmdate('Y-m-d H:i', intval($a['done']));
217	225
	226		if (isset($a['vm_start']) && ($a['vm_start'] > 0))
	227		$a['vm_start_nice'] = gmdate('Y-m-d H:i', $a['vm_start']);
	228		else
	229		$a['vm_start_nice'] = 'n/a';
	230
	231		if (isset($a['build_sh_start']) && ($a['build_sh_start'] > 0))
	232		$a['build_sh_start_nice'] = gmdate('Y-m-d H:i', $a['build_sh_start']);
	233		else
	234		$a['build_sh_start_nice'] = 'n/a';
	235
	236		if (!isset($a['clone_elap']))
	237		$a['clone_elap_nice'] = 'n/a';
	238		else
	239		$a['clone_elap_nice'] = $a['clone_elap'] . 's';
	240
218	241	foreach ($a['cmds'] as &$i) {	foreach ($a['cmds'] as &$i) {
219	242	if (empty($i['start']))	if (empty($i['start']))
220	243	continue;	continue;

...	...	function rg_builder_nice_status(&$a)
233	256	*/	*/
234	257	function rg_builder_cosmetic($db, &$row)	function rg_builder_cosmetic($db, &$row)
235	258	{	{
	259		rg_log_ml('DEBUG: builder_cosmetic: ' . print_r($row, TRUE));
	260
236	261	if (isset($row['itime']))	if (isset($row['itime']))
237	262	$row['itime_nice'] = gmdate('Y-m-d H:i', $row['itime']);	$row['itime_nice'] = gmdate('Y-m-d H:i', $row['itime']);
	263		else
	264		$row['itime_nice'] = 'n/a';
238	265
239	266	if (!isset($row['done']))	if (!isset($row['done']))
240	267	$row['done'] = 0;	$row['done'] = 0;
241
242	268	if ($row['done'] > 0)	if ($row['done'] > 0)
243	269	$row['done_nice'] = gmdate('Y-m-d H:i', $row['done']);	$row['done_nice'] = gmdate('Y-m-d H:i', $row['done']);
244	270	else	else

...	...	function rg_builder_list_high_level($db, $rg, $op, $paras)
291	317	if (!empty($i['status']['packages'])) {	if (!empty($i['status']['packages'])) {
292	318	$_x = array();	$_x = array();
293	319	$_x['packages'] = rg_xss_safe($i['status']['packages']);	$_x['packages'] = rg_xss_safe($i['status']['packages']);
294		$i['HTML:status_packages'] = rg_template('builder/packages.html', $_x, TRUE/xss/);
	320		$i['HTML:status_packages'] =
	321		rg_template('builder/packages.html', $_x, TRUE/xss/);
295	322	} else {	} else {
296	323	$i['HTML:status_packages'] = '';	$i['HTML:status_packages'] = '';
297	324	}	}
298	325
299		$s = rg_builder_nice_status($i['status']);
	326		rg_builder_nice_status($i['status']);
300	327	$i['HTML:status_list'] = rg_template_table('builder/cmds',	$i['HTML:status_list'] = rg_template_table('builder/cmds',
301	328	$i['status']['cmds'], $rg);	$i['status']['cmds'], $rg);
302	329	}	}

File inc/conn.inc.php changed (mode: 100644) (index 4c52ed1..88cc895)
...	...	function rg_conn_recv($key)
165	165	return FALSE;	return FALSE;
166	166	}	}
167	167	if ($r === 0) {	if ($r === 0) {
168		rg_log($key . ': cannot receive: remove closed');
	168		rg_log($key . ': cannot receive: remote closed normally');
169	169	$s['func_close']($key);	$s['func_close']($key);
170	170	rg_conn_destroy($key);	rg_conn_destroy($key);
171	171	return FALSE;	return FALSE;

File inc/fixes.inc.php changed (mode: 100644) (index 24f991a..000631c)
...	...	function rg_fixes_keys_regen($db)
116	116	*/	*/
117	117	function rg_fixes_repo_index_by_id_one($uid, $repo_id, $repo_name)	function rg_fixes_repo_index_by_id_one($uid, $repo_id, $repo_name)
118	118	{	{
119		global $php_errormsg;
120
121	119	rg_log_enter("fixes_repo_index_by_id_one:"	rg_log_enter("fixes_repo_index_by_id_one:"
122	120	. " uid=$uid repo_id=$repo_id repo_name=$repo_name");	. " uid=$uid repo_id=$repo_id repo_name=$repo_name");
123	121

...	...	function rg_fixes_repo_index_by_id_one($uid, $repo_id, $repo_name)
159	157	$by_id_rel = rg_repo_path_by_id_rel($uid, $repo_id);	$by_id_rel = rg_repo_path_by_id_rel($uid, $repo_id);
160	158	$r = symlink($by_id_rel, $by_name);	$r = symlink($by_id_rel, $by_name);
161	159	if ($r !== TRUE) {	if ($r !== TRUE) {
162		rg_log("Cannot symlink $by_id_rel <- $by_name ($php_errormsg)!");
	160		rg_log("Cannot symlink $by_id_rel <- $by_name (" . rg_php_err() . ")!");
163	161	break;	break;
164	162	}	}
165	163	}	}

...	...	function rg_fixes_repo_index_by_id($db)
217	215	*/	*/
218	216	function rg_fixes_user_index_by_id_one($uid, $username)	function rg_fixes_user_index_by_id_one($uid, $username)
219	217	{	{
220		global $php_errormsg;
221	218	global $rg_repos;	global $rg_repos;
222	219
223	220	rg_log_enter("fixes_user_index_by_id_one: uid=$uid username=$username");	rg_log_enter("fixes_user_index_by_id_one: uid=$uid username=$username");

...	...	function rg_fixes_user_index_by_id_one($uid, $username)
264	261	$by_id_rel = rg_user_path_by_id_rel($uid);	$by_id_rel = rg_user_path_by_id_rel($uid);
265	262	$r = symlink($by_id_rel, $user_path_name);	$r = symlink($by_id_rel, $user_path_name);
266	263	if ($r !== TRUE) {	if ($r !== TRUE) {
267		rg_log("Cannot symlink $by_id_rel <- $user_path_name ($php_errormsg)!");
	264		rg_log("Cannot symlink $by_id_rel <- $user_path_name (" . rg_php_err() . ")!");
268	265	break;	break;
269	266	}	}
270	267	}	}

...	...	function rg_fixes_wh_ver5($db)
572	569	*/	*/
573	570	function rg_fixes_repos_last_mr_id($db)	function rg_fixes_repos_last_mr_id($db)
574	571	{	{
575		global $php_errormsg;
576
577	572	rg_log_enter('rg_fixes_repos_last_mr_id');	rg_log_enter('rg_fixes_repos_last_mr_id');
578	573
579	574	$error = FALSE;	$error = FALSE;

...	...	function rg_fixes_repos_last_mr_id($db)
634	629	$to = $path . '/refs/mr/' . $id;	$to = $path . '/refs/mr/' . $id;
635	630	$r = @rename($from, $to);	$r = @rename($from, $to);
636	631	if ($r === FALSE) {	if ($r === FALSE) {
637		rg_log('cannot rename [' . $from . '] -> [' . $to . ']: ' . $php_errormsg . '!');
	632		rg_log('cannot rename [' . $from . '] -> [' . $to . ']: ' . rg_php_err() . '!');
638	633	$error = TRUE;	$error = TRUE;
639	634	break;	break;
640	635	}	}

...	...	function rg_fixes_repos_last_mr_id($db)
660	655	*/	*/
661	656	function rg_fixes_drop_if_exists($db)	function rg_fixes_drop_if_exists($db)
662	657	{	{
663		global $php_errormsg;
664
665	658	rg_log_enter('rg_fixes_drop_if_exists');	rg_log_enter('rg_fixes_drop_if_exists');
666	659
667	660	$ret = TRUE;	$ret = TRUE;

...	...	function rg_fixes_drop_if_exists($db)
692	685	*/	*/
693	686	function rg_fixes_fingerprint_sha256($db)	function rg_fixes_fingerprint_sha256($db)
694	687	{	{
695		global $php_errormsg;
696
697	688	rg_log_enter('rg_fixes_fingerprint_sha256');	rg_log_enter('rg_fixes_fingerprint_sha256');
698	689
699	690	$ret = TRUE;	$ret = TRUE;

File inc/keys.inc.php changed (mode: 100644) (index 58918a0..ddaba79)
...	...	function rg_keys_info($key)
199	199	$ret = array();	$ret = array();
200	200	$ret['ok'] = 0;	$ret['ok'] = 0;
201	201	while(1) {	while(1) {
	202		$key = trim($key);
	203		$key = str_replace("\n", ' ', $key);
	204
202	205	if (empty($key)) {	if (empty($key)) {
203	206	rg_keys_set_error('you did not uploaded the key');	rg_keys_set_error('you did not uploaded the key');
204	207	break;	break;

...	...	function rg_keys_info($key)
340	343	continue;	continue;
341	344
342	345	$ret['comment'] = trim(substr($t[1], $off));	$ret['comment'] = trim(substr($t[1], $off));
	346		$ret['key'] = trim($ret['key']);
343	347
344	348	$error = FALSE;	$error = FALSE;
345	349	break;	break;

...	...	function rg_keys_output_line($i)
667	671	. ' ' . $i['flags']	. ' ' . $i['flags']
668	672	. '"'	. '"'
669	673	. ',' . $rg_ssh_paras	. ',' . $rg_ssh_paras
670		. ' ' . trim($i['key']) . "\n";
	674		. ' ' . $i['key'] . "\n";
671	675	}	}
672	676
673	677	/*	/*

...	...	function rg_keys_output_line($i)
675	679	*/	*/
676	680	function rg_keys_regen($db)	function rg_keys_regen($db)
677	681	{	{
678		global $php_errormsg;
679	682	global $rg_keys_file;	global $rg_keys_file;
680	683	global $rg_scripts;	global $rg_scripts;
681	684

...	...	function rg_keys_regen($db)
684	687	$now = time();	$now = time();
685	688	$ret = FALSE;	$ret = FALSE;
686	689	while (1) {	while (1) {
687		// create .ssh folder if does not exists
688		$dir = dirname($rg_keys_file);
689		if (!file_exists($dir)) {
690		if (!@mkdir($dir, 0700, TRUE)) {
691		rg_keys_set_error("cannot create dir [$dir] ($php_errormsg)");
692		break;
693		}
694		chown($dir, "rocketgit");
695		chgrp($dir, "rocketgit");
696		}
697
698	690	$akp = rg_state_get($db, 'AuthorizedKeysCommand');	$akp = rg_state_get($db, 'AuthorizedKeysCommand');
699	691	if ($akp === FALSE) {	if ($akp === FALSE) {
700	692	rg_keys_set_error('cannot get state of AuthorizedKeysCommand');	rg_keys_set_error('cannot get state of AuthorizedKeysCommand');

...	...	function rg_keys_regen($db)
708	700	break;	break;
709	701	}	}
710	702
	703		// create .ssh folder if does not exists
	704		$dir = dirname($rg_keys_file);
	705		if (!file_exists($dir)) {
	706		if (!@mkdir($dir, 0700, TRUE)) {
	707		rg_keys_set_error("cannot create dir [$dir] (" . rg_php_err() . ")");
	708		break;
	709		}
	710		chown($dir, "rocketgit");
	711		chgrp($dir, "rocketgit");
	712		}
	713
711	714	$tmp = $rg_keys_file . ".tmp";	$tmp = $rg_keys_file . ".tmp";
712	715	$f = @fopen($tmp, "w");	$f = @fopen($tmp, "w");
713	716	if ($f === FALSE) {	if ($f === FALSE) {
714		rg_keys_set_error("cannot open file $tmp ($php_errormsg)");
	717		rg_keys_set_error("cannot open file $tmp (" . rg_php_err() . ")");
715	718	break;	break;
716	719	}	}
717	720
718	721	if (chmod($tmp, 0600) === FALSE) {	if (chmod($tmp, 0600) === FALSE) {
719		rg_keys_set_error("cannot chmod tmp file ($php_errmsg)");
	722		rg_keys_set_error("cannot chmod tmp file (" . rg_php_err() . ")");
720	723	fclose($f);	fclose($f);
721	724	break;	break;
722	725	}	}

...	...	function rg_keys_regen($db)
772	775	$buf = rg_keys_output_line($row);	$buf = rg_keys_output_line($row);
773	776
774	777	if (@fwrite($f, $buf) === FALSE) {	if (@fwrite($f, $buf) === FALSE) {
775		rg_keys_set_error("cannot write; disk space problems? ($php_errormsg)");
	778		rg_keys_set_error("cannot write; disk space problems? (" . rg_php_err() . ")");
776	779	$errors = 1;	$errors = 1;
777	780	break;	break;
778	781	}	}

...	...	function rg_keys_regen($db)
785	788	}	}
786	789
787	790	if (@rename($tmp, $rg_keys_file) === FALSE) {	if (@rename($tmp, $rg_keys_file) === FALSE) {
788		rg_keys_set_error("cannot rename $tmp to $rg_keys_file ($php_errormsg)");
	791		rg_keys_set_error("cannot rename $tmp to $rg_keys_file (" . rg_php_err() . ")");
789	792	unlink($tmp);	unlink($tmp);
790	793	break;	break;
791	794	}	}

File inc/login/login.php changed (mode: 100644) (index 6509f76..a507408)
1	1	<?php	<?php
2	2	rg_log("FILE: /inc/login/login");	rg_log("FILE: /inc/login/login");
3	3
4		$user = rg_var_str("user");
5		$pass = rg_var_str("pass");
6		$login_token = rg_var_str("login_token");
7		$lock_ip = rg_var_uint("lock_ip");
	4		$user = trim(rg_var_str('user'));
	5		$pass = trim(rg_var_str('pass'));
	6		$login_token = trim(rg_var_str('login_token'));
	7		$lock_ip = rg_var_uint('lock_ip');
8	8
9	9	$_login = "";	$_login = "";
10	10

File inc/mr.inc.php changed (mode: 100644) (index 8672fbb..4451835)
...	...	function rg_mr_event_add_to_db($db, $a)
169	169	*/	*/
170	170	function rg_mr_queue_load_file($file)	function rg_mr_queue_load_file($file)
171	171	{	{
172		global $php_errormsg;
173
174	172	$ret = array();	$ret = array();
175	173	$ret['ok'] = 0;	$ret['ok'] = 0;
176	174
177	175	$c = @file_get_contents($file);	$c = @file_get_contents($file);
178	176	if ($c === FALSE) {	if ($c === FALSE) {
179		rg_mr_set_error("cannot load a merge request from $file ($php_errormsg)");
	177		rg_mr_set_error("cannot load a merge request from $file (" . rg_php_err() . ")");
180	178	return $ret;	return $ret;
181	179	}	}
182	180

...	...	function rg_mr_queue_load_file($file)
195	193	*/	*/
196	194	function rg_mr_queue_process($db)	function rg_mr_queue_process($db)
197	195	{	{
198		global $php_errormsg;
199	196	global $rg_mr_queue;	global $rg_mr_queue;
200	197
201	198	rg_prof_start("mr_queue_process");	rg_prof_start("mr_queue_process");

...	...	function rg_mr_queue_process($db)
203	200	$ret = TRUE;	$ret = TRUE;
204	201	$dir = @opendir($rg_mr_queue);	$dir = @opendir($rg_mr_queue);
205	202	if ($dir === FALSE) {	if ($dir === FALSE) {
206		rg_mr_set_error("cannot open dir $rg_mr_queue ($php_errormsg)!");
	203		rg_mr_set_error("cannot open dir $rg_mr_queue (" . rg_php_err() . ")!");
207	204	return FALSE;	return FALSE;
208	205	}	}
209	206

File inc/prof.inc.php changed (mode: 100644) (index 65fdff0..7a5d334)
...	...	function rg_prof_text()
173	173	$ret .= $add;	$ret .= $add;
174	174	$add = "\n";	$add = "\n";
175	175
176		$ret .= str_pad($label, 22, ' ');
	176		if (strlen($label) >= 21)
	177		$ret .= $label . "\n" . str_pad('', 22, ' ');
	178		else
	179		$ret .= str_pad($label, 22, ' ');
177	180
178	181	foreach ($vars as $k => $junk) {	foreach ($vars as $k => $junk) {
179	182	if (!isset($per_label[$k]))	if (!isset($per_label[$k]))

File inc/repo.inc.php changed (mode: 100644) (index 9f2f761..081840e)
...	...	function rg_repo_event_del($db, $event)
542	542	*/	*/
543	543	function rg_repo_event_symlink_by_name($db, $e)	function rg_repo_event_symlink_by_name($db, $e)
544	544	{	{
545		global $php_errormsg;
546
547	545	rg_prof_start("repo_event_symlink_by_name");	rg_prof_start("repo_event_symlink_by_name");
548	546
549	547	$id_path = rg_repo_path_by_id($e['ui']['uid'], $e['ri']['repo_id']);	$id_path = rg_repo_path_by_id($e['ui']['uid'], $e['ri']['repo_id']);

...	...	function rg_repo_event_symlink_by_name($db, $e)
593	591	// Now, the new name is free, do the link	// Now, the new name is free, do the link
594	592	$r = symlink($id_path_rel, $new_path);	$r = symlink($id_path_rel, $new_path);
595	593	if ($r !== TRUE) {	if ($r !== TRUE) {
596		rg_internal_error("cannot symlink $id_path -> $new_path ($php_errormsg)!");
	594		rg_internal_error("cannot symlink $id_path -> $new_path (" . rg_php_err() . ")!");
597	595	break;	break;
598	596	}	}
599	597

...	...	function rg_repo_admin_rights($db, $rg, $type)
1622	1620	$a['type'] = $type;	$a['type'] = $type;
1623	1621	$a['right_id'] = rg_var_uint("right_id");	$a['right_id'] = rg_var_uint("right_id");
1624	1622	$a['edit_id'] = rg_var_uint("edit_id");	$a['edit_id'] = rg_var_uint("edit_id");
1625		$a['username'] = rg_var_str("username");
	1623		$a['username'] = trim(rg_var_str("username"));
1626	1624	$a['rights'] = rg_rights_a2s(rg_var_str("rights"));	$a['rights'] = rg_rights_a2s(rg_var_str("rights"));
1627		$a['misc'] = rg_var_str("misc");
1628		$a['ip'] = rg_var_str("ip");
	1625		$a['misc'] = trim(rg_var_str("misc"));
	1626		$a['ip'] = trim(rg_var_str("ip"));
1629	1627	$a['prio'] = rg_var_uint("prio");	$a['prio'] = rg_var_uint("prio");
1630	1628	$a['description'] = trim(rg_var_str("description"));	$a['description'] = trim(rg_var_str("description"));
1631	1629	//rg_log_ml("CHECK: a(POST)=" . print_r($a, TRUE));	//rg_log_ml("CHECK: a(POST)=" . print_r($a, TRUE));

...	...	function rg_repo_admin($db, &$rg, $paras)
2124	2122	*/	*/
2125	2123	function rg_repo_search_high_level($db, $rg, $ui, $url)	function rg_repo_search_high_level($db, $rg, $ui, $url)
2126	2124	{	{
2127		$q = rg_var_str('q');
	2125		$q = trim(rg_var_str('q'));
2128	2126
2129	2127	$ret = '';	$ret = '';
2130	2128

...	...	function rg_repo_api($db, $a)
2466	2464	* @host - the host name accessed - may be different than the good one	* @host - the host name accessed - may be different than the good one
2467	2465	* @ip - from where the connections come	* @ip - from where the connections come
2468	2466	* @cmd: 'git-upload-pack' or 'git-receive-pack'	* @cmd: 'git-upload-pack' or 'git-receive-pack'
	2467		* @flags: B = bypass rights check (for example, for a global worker)
2469	2468	* TODO: move it to user.inc.php?!	* TODO: move it to user.inc.php?!
2470	2469	*/	*/
2471	2470	function rg_repo_fetch_push_helper($db, $host, $ip, $login_ui, $prefix, $user,	function rg_repo_fetch_push_helper($db, $host, $ip, $login_ui, $prefix, $user,
2472		$repo, $cmd)
	2471		$repo, $cmd, $flags)
2473	2472	{	{
2474	2473	global $rg_log_sid;	global $rg_log_sid;
2475	2474
2476	2475	rg_prof_start('repo_fetch_push_helper');	rg_prof_start('repo_fetch_push_helper');
2477	2476	rg_log_enter('repo_fetch_push_helper: host=' . $host	rg_log_enter('repo_fetch_push_helper: host=' . $host
2478	2477	. ' ip=' . $ip . ' prefix=' . $prefix . ' user=[' . $user . ']'	. ' ip=' . $ip . ' prefix=' . $prefix . ' user=[' . $user . ']'
2479		. ' repo=[' . $repo . '] cmd=[' . $cmd . ']');
	2478		. ' repo=[' . $repo . '] cmd=[' . $cmd . '] flags=' . $flags);
2480	2479
2481	2480	$ret = array('ok' => 0, 'allow' => 0, 'push_allowed' => 0);	$ret = array('ok' => 0, 'allow' => 0, 'push_allowed' => 0);
2482	2481	while (1) {	while (1) {

...	...	function rg_repo_fetch_push_helper($db, $host, $ip, $login_ui, $prefix, $user,
2557	2556
2558	2557	$ret['ok'] = 1;	$ret['ok'] = 1;
2559	2558
2560		$x = array();
2561		$x['obj_id'] = $ret['ri']['repo_id'];
2562		$x['type'] = 'repo_refs';
2563		$x['owner'] = $ret['ri']['uid'];
2564		$x['uid'] = $login_ui['uid'];
2565		$x['username'] = $login_ui['uid'] == 0 ? '' : $login_ui['username'];
2566		$x['needed_rights'] = $needed_rights;
2567		$x['ip'] = $ip;
2568		$x['misc'] = '';
2569		$r = rg_rights_allow($db, $x);
2570		// TODO: what if an error occured? How do we signal this?! $r must have ok key
2571		if ($r !== TRUE) {
2572		$ret['errmsg'] = 'non existing repo or you are not allowed to push';
2573		break;
2574		}
2575
2576		// We need to know if Push (and not anon) is allowed, so we can
2577		// give the user a chance to authenticate.
2578		// TODO: change rg_rights_allow to return what rights are
2579		// allowed and use it.
2580		if ($ret['push'] == 1) {
2581		$x['needed_rights'] = 'P';
	2559		if (!strstr($flags, 'B')) {
	2560		$x = array();
	2561		$x['obj_id'] = $ret['ri']['repo_id'];
	2562		$x['type'] = 'repo_refs';
	2563		$x['owner'] = $ret['ri']['uid'];
	2564		$x['uid'] = $login_ui['uid'];
	2565		$x['username'] = $login_ui['uid'] == 0 ? '' : $login_ui['username'];
	2566		$x['needed_rights'] = $needed_rights;
	2567		$x['ip'] = $ip;
	2568		$x['misc'] = '';
2582	2569	$r = rg_rights_allow($db, $x);	$r = rg_rights_allow($db, $x);
2583		if ($r === TRUE)
2584		$ret['push_allowed'] = 1;
2585		}
2586
2587		// If we are enrolled, ask for login token
2588		// For push we always ask for it, for fetch only if repo is
2589		// NOT public. And we can ask only if we have a ssh connection.
2590		// For git protocol we cannot because we do not have the
2591		// username/uid of the connecting user.
2592		if (isset($_SERVER['SSH_CONNECTION'])) {
2593		if (($ret['ri']['public'] == 0) \|\| ($ret['push'] == 1)) {
2594		$r = rg_totp_verify_ip($db, $login_ui['uid'],
2595		$ip);
2596		if (($r['ok'] !== 1)
2597		\|\| ($r['enrolled'] && empty($r['ip_list']))) {
2598		$ret['errmsg'] = rg_totp_error();
2599		$ret['ok'] = 0;
2600		break;
	2570		// TODO: what if an error occured? How do we signal this?! $r must have ok key
	2571		if ($r !== TRUE) {
	2572		$ret['errmsg'] = 'non existing repo or you are not allowed to push';
	2573		break;
	2574		}
	2575
	2576		// We need to know if Push (and not anon) is allowed, so we can
	2577		// give the user a chance to authenticate.
	2578		// TODO: change rg_rights_allow to return what rights are
	2579		// allowed and use it.
	2580		if ($ret['push'] == 1) {
	2581		$x['needed_rights'] = 'P';
	2582		$r = rg_rights_allow($db, $x);
	2583		if ($r === TRUE)
	2584		$ret['push_allowed'] = 1;
	2585		}
	2586
	2587		// If we are enrolled, ask for login token
	2588		// For push we always ask for it, for fetch only if repo is
	2589		// NOT public. And we can ask only if we have a ssh connection.
	2590		// For git protocol we cannot because we do not have the
	2591		// username/uid of the connecting user.
	2592		if (isset($_SERVER['SSH_CONNECTION'])) {
	2593		if (($ret['ri']['public'] == 0) \|\| ($ret['push'] == 1)) {
	2594		$r = rg_totp_verify_ip($db, $login_ui['uid'],
	2595		$ip);
	2596		if (($r['ok'] !== 1)
	2597		\|\| ($r['enrolled'] && empty($r['ip_list']))) {
	2598		$ret['errmsg'] = rg_totp_error();
	2599		$ret['ok'] = 0;
	2600		break;
	2601		}
2601	2602	}	}
2602	2603	}	}
2603	2604	}	}

...	...	function rg_repo_fetch_push_helper($db, $host, $ip, $login_ui, $prefix, $user,
2672	2673	return $ret;	return $ret;
2673	2674	}	}
2674	2675
2675		?>

File inc/totp.inc.php changed (mode: 100644) (index e3d9e15..00dfbcf)
...	...	function rg_totp_enroll_high_level($db, $rg, $paras)
1299	1299
1300	1300	$enroll = rg_var_uint('enroll');	$enroll = rg_var_uint('enroll');
1301	1301	while ($enroll == 1) {	while ($enroll == 1) {
1302		$name = rg_var_str('totp::name');
	1302		$name = trim(rg_var_str('totp::name'));
1303	1303	$ver = rg_var_str('totp::ver');	$ver = rg_var_str('totp::ver');
1304	1304	$secret = rg_var_str('totp::secret');	$secret = rg_var_str('totp::secret');
1305	1305

File inc/user.inc.php changed (mode: 100644) (index 2bf94e1..c510e64)
...	...	function rg_user_cosmetic(&$row)
193	193	*/	*/
194	194	function rg_user_link_by_name($db, $event)	function rg_user_link_by_name($db, $event)
195	195	{	{
196		global $php_errormsg;
197
198	196	rg_log("user_link_by_name: event=" . rg_array2string($event));	rg_log("user_link_by_name: event=" . rg_array2string($event));
199	197
200	198	$by_id = rg_user_path_by_id($event['ui']['uid']);	$by_id = rg_user_path_by_id($event['ui']['uid']);
201	199	if (!is_dir($by_id) && (mkdir($by_id, 0700, TRUE) === FALSE)) {	if (!is_dir($by_id) && (mkdir($by_id, 0700, TRUE) === FALSE)) {
202		rg_user_set_error("cannot mkdir by_id=$by_id ($php_errormsg)");
	200		rg_user_set_error("cannot mkdir by_id=$by_id (" . rg_php_err() . ")");
203	201	return FALSE;	return FALSE;
204	202	}	}
205	203

...	...	function rg_user_link_by_name($db, $event)
207	205	$by_name_parent = dirname($by_name);	$by_name_parent = dirname($by_name);
208	206	if (!is_dir($by_name_parent)	if (!is_dir($by_name_parent)
209	207	&& (mkdir($by_name_parent, 0700, TRUE) === FALSE)) {	&& (mkdir($by_name_parent, 0700, TRUE) === FALSE)) {
210		rg_user_set_error("cannot mkdir by_name_parent=$by_name_parent ($php_errmsg)");
	208		rg_user_set_error("cannot mkdir by_name_parent=$by_name_parent (" . rg_php_err() . ")");
211	209	return FALSE;	return FALSE;
212	210	}	}
213	211

...	...	function rg_user_link_by_name($db, $event)
215	213	if (is_link($by_name))	if (is_link($by_name))
216	214	unlink($by_name);	unlink($by_name);
217	215	if (symlink($by_id_rel, $by_name) === FALSE) {	if (symlink($by_id_rel, $by_name) === FALSE) {
218		rg_user_set_error("cannot symlink $by_id_rel <- $by_name ($php_errormsg)!");
	216		rg_user_set_error("cannot symlink $by_id_rel <- $by_name (" . rg_php_err() . ")!");
219	217	return FALSE;	return FALSE;
220	218	}	}
221	219

...	...	function rg_user_insert_rename($db, $uid, $old_name)
435	433	*/	*/
436	434	function rg_user_rename($db, $ui, $new_name)	function rg_user_rename($db, $ui, $new_name)
437	435	{	{
438		global $php_errormsg;
439
440	436	rg_prof_start("user_rename");	rg_prof_start("user_rename");
441	437	rg_log_enter("user_rename: from=[" . $ui['username'] . "]"	rg_log_enter("user_rename: from=[" . $ui['username'] . "]"
442	438	. " to=[" . $new_name . "]...");	. " to=[" . $new_name . "]...");

...	...	function rg_user_rename($db, $ui, $new_name)
481	477	// Now, the new name is free, do the link	// Now, the new name is free, do the link
482	478	$r = symlink($old_path, $new_path);	$r = symlink($old_path, $new_path);
483	479	if ($r !== TRUE) {	if ($r !== TRUE) {
484		rg_internal_error("Cannot symlink $old_path -> $new_path ($php_errormsg)!");
	480		rg_internal_error("Cannot symlink $old_path -> $new_path (" . rg_php_err() . ")!");
485	481	break;	break;
486	482	}	}
487	483	}	}

...	...	function rg_user_forgot_pass_mail_prepare($db, $email)
1625	1621	*/	*/
1626	1622	function rg_user_forgot_pass_mail($db, $rg, $email)	function rg_user_forgot_pass_mail($db, $rg, $email)
1627	1623	{	{
1628		global $php_errormsg;
1629	1624	global $rg_admin_name;	global $rg_admin_name;
1630	1625
1631	1626	rg_prof_start('user_forgot_pass_mail');	rg_prof_start('user_forgot_pass_mail');

...	...	function rg_user_edit_high_level($db, &$rg)
1962	1957
1963	1958	$ui = array();	$ui = array();
1964	1959	$ui['uid'] = $rg['target_ui']['uid'];	$ui['uid'] = $rg['target_ui']['uid'];
1965		$ui['username'] = rg_var_str("username");
1966		$ui['realname'] = rg_var_str("realname");
1967		$ui['email'] = rg_var_str("email");
	1960		$ui['username'] = trim(rg_var_str("username"));
	1961		$ui['realname'] = trim(rg_var_str("realname"));
	1962		$ui['email'] = trim(rg_var_str("email"));
1968	1963	$ui['pass'] = rg_var_str("pass");	$ui['pass'] = rg_var_str("pass");
1969	1964	$ui['pass2'] = rg_var_str("pass2");	$ui['pass2'] = rg_var_str("pass2");
1970	1965	$ui['is_admin'] = rg_var_bool("is_admin");	$ui['is_admin'] = rg_var_bool("is_admin");

...	...	function rg_process_input($content_length, $content_encoding, &$err)
2211	2206	. ' is bigger than the max allowed'	. ' is bigger than the max allowed'
2212	2207	. ' (' . $max_nice . ').' . "\n"	. ' (' . $max_nice . ').' . "\n"
2213	2208	. 'You may want to ask the admin to raise'	. 'You may want to ask the admin to raise'
2214		. ' the limits (post_max_size).';
	2209		. ' the PHP limit (post_max_size).';
2215	2210	rg_internal_error($err);	rg_internal_error($err);
2216	2211	break;	break;
2217	2212	}	}

...	...	function rg_user_http_git($db, $rg, $paras)
2372	2367
2373	2368	$host = isset($_SERVER['HTTP_HOST']) ? $_SERVER['HTTP_HOST'] : '';	$host = isset($_SERVER['HTTP_HOST']) ? $_SERVER['HTTP_HOST'] : '';
2374	2369	// TODO: not clear if passing here login_ui is correct	// TODO: not clear if passing here login_ui is correct
	2370		$helper_flags = '';
2375	2371	$r = rg_repo_fetch_push_helper($db, $host, $rg['ip'],	$r = rg_repo_fetch_push_helper($db, $host, $rg['ip'],
2376		$rg['login_ui'], $prefix, $user, $repo, $service);
	2372		$rg['login_ui'], $prefix, $user, $repo, $service,
	2373		$helper_flags);
2377	2374	rg_log_ml('DEBUG: repo_fetch_push_helper returns: ' . print_r($r, TRUE));	rg_log_ml('DEBUG: repo_fetch_push_helper returns: ' . print_r($r, TRUE));
2378	2375	if ($r['ok'] !== 1) {	if ($r['ok'] !== 1) {
2379	2376	rg_log('DEBUG: set errror: ' . $r['errmsg']);	rg_log('DEBUG: set errror: ' . $r['errmsg']);

...	...	function rg_user_http_git($db, $rg, $paras)
2504	2501	. ' ' . escapeshellarg($repo_path);	. ' ' . escapeshellarg($repo_path);
2505	2502	$cmd['cmds']['cmd1']['cb_input_stderr_func'] = 'rg_git_band_2';	$cmd['cmds']['cmd1']['cb_input_stderr_func'] = 'rg_git_band_2';
2506	2503	$e = rg_exec2($cmd);	$e = rg_exec2($cmd);
2507		rg_log_ml('XDEBUG: e: ' . print_r($e, TRUE));
	2504		//rg_log_ml('XDEBUG: e: ' . print_r($e, TRUE));
2508	2505	if (!isset($e['code']))	if (!isset($e['code']))
2509	2506	exit(1);	exit(1);
2510	2507	if ($e['ok'] != 1) {	if ($e['ok'] != 1) {

File inc/user/forgot_send.php changed (mode: 100644) (index c31ffd5..1d85b0a)
1	1	<?php	<?php
2	2	rg_log("FILE: /inc/user/forgot_send");	rg_log("FILE: /inc/user/forgot_send");
3	3
4		$email = rg_var_str("email");
	4		$email = trim(rg_var_str('email'));
5	5
6	6	$_forgot = "";	$_forgot = "";
7	7

File inc/util.inc.php changed (mode: 100644) (index 91c9689..77ed7d0)
...	...	function rg_id($len)
160	160	$_lock = array();	$_lock = array();
161	161	function rg_lock($file)	function rg_lock($file)
162	162	{	{
163		global $php_errormsg;
164
165	163	global $_lock;	global $_lock;
166	164	global $rg_lock_dir;	global $rg_lock_dir;
167	165

...	...	function rg_lock($file)
178	176	$f = @fopen($rg_lock_dir . "/" . $file, "w");	$f = @fopen($rg_lock_dir . "/" . $file, "w");
179	177	if ($f === FALSE) {	if ($f === FALSE) {
180	178	rg_internal_error('Cannot open lock [' . $rg_lock_dir	rg_internal_error('Cannot open lock [' . $rg_lock_dir
181		. '/' . $file . '] (' . $php_errormsg . ').');
	179		. '/' . $file . '] (' . rg_php_err() . ').');
182	180	return FALSE;	return FALSE;
183	181	}	}
184	182

...	...	function rg_chars_allow($name, $allowed_regexp, &$invalid)
557	555	*/	*/
558	556	function rg_rmdir($dir)	function rg_rmdir($dir)
559	557	{	{
560		global $php_errormsg;
561
562	558	if (!is_dir($dir))	if (!is_dir($dir))
563	559	return TRUE;	return TRUE;
564	560

...	...	function rg_rmdir($dir)
580	576	}	}
581	577
582	578	if (!unlink($path)) {	if (!unlink($path)) {
583		rg_util_set_error("cannot remove [$path] ($php_errormsg)");
	579		rg_util_set_error("cannot remove [$path] (" . rg_php_err() . ")");
584	580	return FALSE;	return FALSE;
585	581	}	}
586	582	}	}
587	583	}	}
588	584
589	585	if (!rmdir($dir)) {	if (!rmdir($dir)) {
590		rg_util_set_error("cannot remove main dir ($php_errormsg)");
	586		rg_util_set_error("cannot remove main dir (" . rg_php_err() . ")");
591	587	return FALSE;	return FALSE;
592	588	}	}
593	589

...	...	function rg_theme_resolve_path($path)
615	611	*/	*/
616	612	function rg_file_get_contents($f)	function rg_file_get_contents($f)
617	613	{	{
618		global $php_errormsg;
619
620	614	if (!file_exists($f))	if (!file_exists($f))
621	615	return "";	return "";
622	616
623	617	$c = @file_get_contents($f);	$c = @file_get_contents($f);
624	618	if ($c === FALSE) {	if ($c === FALSE) {
625		rg_internal_error("Could not load file [$f] ($php_errormsg).");
	619		rg_internal_error("Could not load file [$f] (" . rg_php_err() . ").");
626	620	return "";	return "";
627	621	}	}
628	622

...	...	function rg_template_string(&$s, $off, &$data, $xss_protection)
962	956	if (isset($_t[1]))	if (isset($_t[1]))
963	957	$data[$_t[0]] = $_t[1];	$data[$_t[0]] = $_t[1];
964	958	$value = '';	$value = '';
	959		} else if (strcmp($var, 'DUMP') == 0) {
	960		$value = rg_xss_safe(print_r($data, TRUE));
965	961	} else {	} else {
966	962	//rg_log("DEBUG: VAR [$var] NOT FOUND!");	//rg_log("DEBUG: VAR [$var] NOT FOUND!");
967	963	}	}

...	...	function rg_exec2_helper_populate_out_buf($index, &$info)
1350	1346	$info['out_buf_helper']($info);	$info['out_buf_helper']($info);
1351	1347	}	}
1352	1348
	1349		$rg_util_debug &&
1353	1350	rg_log_exit();	rg_log_exit();
1354	1351	}	}
1355	1352

...	...	function rg_callback_hexa($matches)
1803	1800	}	}
1804	1801
1805	1802	/*	/*
1806		* Transforms an array in a string
	1803		* Transforms an array into a string
1807	1804	*/	*/
1808	1805	function rg_array2string($a)	function rg_array2string($a)
1809	1806	{	{

...	...	function rg_array2string($a)
1834	1831	*/	*/
1835	1832	function rg_dir_load_pattern($dir, $pattern)	function rg_dir_load_pattern($dir, $pattern)
1836	1833	{	{
1837		global $php_errormsg;
1838
1839	1834	$ret = FALSE;	$ret = FALSE;
1840	1835	if (!file_exists($dir)) {	if (!file_exists($dir)) {
1841	1836	rg_util_set_error("$dir does not exists");	rg_util_set_error("$dir does not exists");

...	...	function rg_dir_load_pattern($dir, $pattern)
1844	1839
1845	1840	$d = @scandir($dir);	$d = @scandir($dir);
1846	1841	if ($d === FALSE) {	if ($d === FALSE) {
1847		rg_util_set_error("cannot scan dir $dir ($php_errormsg)");
	1842		rg_util_set_error("cannot scan dir $dir (" . rg_php_err() . ")");
1848	1843	return $ret;	return $ret;
1849	1844	}	}
1850	1845

...	...	function rg_dir_load_deep($dir)
1902	1897	*/	*/
1903	1898	function rg_copy_tree($src, $dst, $mask)	function rg_copy_tree($src, $dst, $mask)
1904	1899	{	{
1905		global $php_errormsg;
1906
1907	1900	rg_prof_start("copy_tree");	rg_prof_start("copy_tree");
1908	1901	rg_log_enter("copy_tree($src, $dst, mask=$mask)");	rg_log_enter("copy_tree($src, $dst, mask=$mask)");
1909	1902

...	...	function rg_copy_tree($src, $dst, $mask)
1912	1905	if (!is_dir($dst)) {	if (!is_dir($dst)) {
1913	1906	$r = @mkdir($dst, $mask, TRUE);	$r = @mkdir($dst, $mask, TRUE);
1914	1907	if ($r !== TRUE) {	if ($r !== TRUE) {
1915		rg_log("ERROR: Cannot mkdir [$dst] ($php_errormsg).");
	1908		rg_log("ERROR: Cannot mkdir [$dst] (" . rg_php_err() . ").");
1916	1909	break;	break;
1917	1910	}	}
1918	1911	}	}

...	...	function rg_copy_tree($src, $dst, $mask)
1928	1921	$r = @mkdir($dst . "/" . $obj, $mask);	$r = @mkdir($dst . "/" . $obj, $mask);
1929	1922	if ($r !== TRUE) {	if ($r !== TRUE) {
1930	1923	rg_log("ERROR: Cannot mkdir [$dst/$obj]"	rg_log("ERROR: Cannot mkdir [$dst/$obj]"
1931		. " ($php_errormsg).");
	1924		. " (" . rg_php_err() . ").");
1932	1925	$err = TRUE;	$err = TRUE;
1933	1926	break;	break;
1934	1927	}	}

...	...	function rg_copy_tree($src, $dst, $mask)
1942	1935	} else {	} else {
1943	1936	$r = @copy($src . "/" . $obj, $dst . "/" . $obj);	$r = @copy($src . "/" . $obj, $dst . "/" . $obj);
1944	1937	if ($r !== TRUE) {	if ($r !== TRUE) {
1945		rg_log("ERROR: Cannot copy file ($php_errormsg).");
	1938		rg_log("ERROR: Cannot copy file (" . rg_php_err() . ").");
1946	1939	$err = TRUE;	$err = TRUE;
1947	1940	break;	break;
1948	1941	}	}

...	...	function rg_copy_tree($src, $dst, $mask)
1964	1957	*/	*/
1965	1958	function rg_del_tree($dst)	function rg_del_tree($dst)
1966	1959	{	{
1967		global $php_errormsg;
1968
1969	1960	rg_prof_start('del_tree');	rg_prof_start('del_tree');
1970	1961
1971	1962	$ret = FALSE;	$ret = FALSE;

...	...	function rg_del_tree($dst)
1990	1981	} else {	} else {
1991	1982	$r = @unlink($dst . '/' . $obj);	$r = @unlink($dst . '/' . $obj);
1992	1983	if ($r !== TRUE) {	if ($r !== TRUE) {
1993		rg_log("ERROR: Cannot del file [" . $dst . '/' . $obj . "] ($php_errormsg).");
	1984		rg_log("ERROR: Cannot del file [" . $dst . '/' . $obj . "] (" . rg_php_err() . ").");
1994	1985	$err = TRUE;	$err = TRUE;
1995	1986	break;	break;
1996	1987	}	}

...	...	function rg_del_tree($dst)
1999	1990
2000	1991	$r = @rmdir($dst);	$r = @rmdir($dst);
2001	1992	if ($r !== TRUE) {	if ($r !== TRUE) {
2002		rg_log("ERROR: Cannot del dir [" . $dst . "] ($php_errormsg).");
	1993		rg_log("ERROR: Cannot del dir [" . $dst . "] (" . rg_php_err() . ").");
2003	1994	$err = TRUE;	$err = TRUE;
2004	1995	break;	break;
2005	1996	}	}

...	...	function rg_file_get_tail($f, $bytes)
2612	2603	*/	*/
2613	2604	function rg_unserialize($s)	function rg_unserialize($s)
2614	2605	{	{
2615		if (strncmp($s, '{', 1) == 0) {
	2606		$c = substr($s, 0, 1);
	2607		if ((strcmp($c, '{') == 0) \|\| (strcmp($c, '"') == 0)
	2608		\|\| (strcmp($c, '[') == 0)) {
2616	2609	$r = @json_decode($s, TRUE);	$r = @json_decode($s, TRUE);
2617	2610	if ($r !== NULL)	if ($r !== NULL)
2618	2611	return $r;	return $r;

...	...	function rg_unserialize($s)
2630	2623	return $r;	return $r;
2631	2624	}	}
2632	2625
2633		if (strcmp($s, '[]') == 0)
2634		return array();
2635
2636		if (strncmp($s, '"', 1) == 0)
2637		return substr($s, 1, -1);
2638
2639	2626	return $s;	return $s;
2640	2627	}	}
2641	2628

...	...	function rg_unserialize($s)
2644	2631	*/	*/
2645	2632	function rg_serialize($a)	function rg_serialize($a)
2646	2633	{	{
2647		if (!is_array($a))
2648		return $a;
2649
2650		$r = @json_encode($a, TRUE);
	2634		$r = @json_encode($a);
2651	2635	if ($r === FALSE) {	if ($r === FALSE) {
2652	2636	$m = 'error encoding json: ' . json_last_error_msg();	$m = 'error encoding json: ' . json_last_error_msg();
2653	2637	rg_internal_error($m);	rg_internal_error($m);

File inc/wh/build.inc.php changed (mode: 100644) (index efe0a95..2798151)
...	...	function rg_wh_build_job_done($db, $ev)
23	23	rg_prof_start('wh_build_job_done');	rg_prof_start('wh_build_job_done');
24	24	rg_log_ml('wh_build_job_done: ev: ' . print_r($ev, TRUE));	rg_log_ml('wh_build_job_done: ev: ' . print_r($ev, TRUE));
25	25
	26		$job = $ev['job'];
	27		$req = isset($job['request']) ? $job['request'] : $job;
	28
26	29	$ret = FALSE;	$ret = FALSE;
27	30	while (1) {	while (1) {
28	31	$s = $ev['status'];	$s = $ev['status'];
29		$out = 'Worker: ' . $ev['job']['worker_name'] . "\n";
	32		$out = 'Worker: ' . $job['worker_name'] . "\n";
30	33	$out .= 'Date (UTC): '	$out .= 'Date (UTC): '
31		. gmdate('Y-m-d H:i', $ev['job']['done']) . "\n";
	34		. gmdate('Y-m-d H:i', $job['done']) . "\n";
32	35	$out .= 'Elapsed time: '	$out .= 'Elapsed time: '
33		. ($ev['job']['done'] - $ev['job']['worker_started'])
34		. 's' . "\n";
35		$out .= "\n";
36		$out .= 'Packages:' . "\n" . $s['packages'];
	36		. ($job['done'] - $job['worker_started'])
	37		. 's' . "\n\n";
	38
	39		$out .= 'Packages installation:' . "\n";
	40		$out .= $s['packages'] . "\n\n";
	41
37	42	foreach ($s['cmds'] as $index => $i) {	foreach ($s['cmds'] as $index => $i) {
38		if (empty($ev['job']['cmds'][$index]['cmd']))
	43		if (empty($req['cmds'][$index]['cmd']))
39	44	continue;	continue;
40	45
41		$out .= "\n\n" . 'Command['
42		. $ev['job']['cmds'][$index]['cmd'] . ']: '
43		. "\n" . trim($i['log']);
	46		$out .= 'Command['
	47		. $req['cmds'][$index]['cmd'] . ']:' . "\n"
	48		. trim($i['log']) . "\n\n";
44	49	}	}
45	50
	51		// TODO: should we check error code?
46	52	rg_wh_set_last_output($db, $ev['ui']['uid'],	rg_wh_set_last_output($db, $ev['ui']['uid'],
47		$ev['job']['hook_id'], $out);
	53		$req['hook_id'], $out);
48	54	$ret = array();	$ret = array();
49	55	break;	break;
50	56	}	}

File inc/wh/core.inc.php changed (mode: 100644) (index eca4d86..04b8606)
...	...	function rg_wh_list($db, $uid)
230	230	$r = rg_cache_get($key);	$r = rg_cache_get($key);
231	231	if (($r !== FALSE) && isset($r['LIST_LOADED'])) {	if (($r !== FALSE) && isset($r['LIST_LOADED'])) {
232	232	$ret['list'] = $r['list'];	$ret['list'] = $r['list'];
	233		rg_log_ml('list: ' . print_r($ret['list'], TRUE));
233	234	$ret['ok'] = 1;	$ret['ok'] = 1;
234	235	break;	break;
235	236	}	}

File inc/workers.inc.php changed (mode: 100644) (index 5cf3603..0bd3745)
2	2	$INC = isset($INC) ? $INC : dirname(__FILE__);	$INC = isset($INC) ? $INC : dirname(__FILE__);
3	3	require_once($INC . '/events.inc.php');	require_once($INC . '/events.inc.php');
4	4	require_once($INC . '/builder.inc.php');	require_once($INC . '/builder.inc.php');
	5		require_once($INC . '/keys.inc.php');
5	6
6	7	$rg_worker_error = '';	$rg_worker_error = '';
7	8

...	...	function rg_worker_add($db, $uid, $a)
332	333
333	334	/*	/*
334	335	* Updates 'last_connect' and 'last_ip', 'uname', 'host', 'arch', 'env'	* Updates 'last_connect' and 'last_ip', 'uname', 'host', 'arch', 'env'
	336		* and other fields.
	337		* It is called from builder.php when a worker connects.
335	338	*/	*/
336	339	function rg_worker_update($db, $uid, $id, $a)	function rg_worker_update($db, $uid, $id, $a)
337	340	{	{

...	...	function rg_worker_update($db, $uid, $id, $a)
348	351	$params['last_connect'] = time();	$params['last_connect'] = time();
349	352	$params['env'] = rg_serialize($a['env']);	$params['env'] = rg_serialize($a['env']);
350	353
	354		$ki = rg_keys_info($a['ssh_key']);
	355		if ($ki['ok'] != 1) {
	356		rg_worker_set_error('cannot parse ssh key: '
	357		. rg_keys_error());
	358		break;
	359		}
	360		$params['fingerprint_sha256'] = $ki['fingerprint_sha256'];
	361
	362		$r = rg_keys_weak($db, $ki);
	363		if ($r['ok'] != 1) {
	364		rg_worker_set_error('cannot detect weak ssh keys: '
	365		. rg_keys_error());
	366		break;
	367		}
	368		if ($r['weak'] == 1) {
	369		rg_worker_set_error('ssh key is weak');
	370		break;
	371		}
	372
351	373	$sql = 'UPDATE workers SET'	$sql = 'UPDATE workers SET'
352	374	. ' name = @@name@@'	. ' name = @@name@@'
353	375	. ', last_connect = @@last_connect@@'	. ', last_connect = @@last_connect@@'

...	...	function rg_worker_update($db, $uid, $id, $a)
357	379	. ', arch = @@arch@@'	. ', arch = @@arch@@'
358	380	. ', env = @@env@@'	. ', env = @@env@@'
359	381	. ', ssh_key = @@ssh_key@@'	. ', ssh_key = @@ssh_key@@'
	382		. ', fingerprint_sha256 = @@fingerprint_sha256@@'
360	383	. ' WHERE uid = @@uid@@'	. ' WHERE uid = @@uid@@'
361	384	. ' AND id = @@id@@';	. ' AND id = @@id@@';
362	385	$res = rg_sql_query_params($db, $sql, $params);	$res = rg_sql_query_params($db, $sql, $params);

...	...	function rg_worker_update($db, $uid, $id, $a)
366	389	}	}
367	390	rg_sql_free_result($res);	rg_sql_free_result($res);
368	391
	392		$event = array(
	393		'category' => 'worker_event_add',
	394		'prio' => 50,
	395		'ui' => array('uid' => $uid),
	396		'add' => 0,
	397		'id' => $id);
	398		$r = rg_event_add($db, $event);
	399		if ($r !== TRUE) {
	400		rg_worker_set_error('cannot add event'
	401		. ' (' . rg_event_error() . ')');
	402		break;
	403		}
	404
369	405	$key = 'workers' . '::' . $uid . '::' . 'list' . '::' . $id;	$key = 'workers' . '::' . $uid . '::' . 'list' . '::' . $id;
370	406	unset($params['uid']);	unset($params['uid']);
371	407	rg_cache_merge($key, $params, RG_SOCKET_NO_WAIT);	rg_cache_merge($key, $params, RG_SOCKET_NO_WAIT);

...	...	function rg_worker_add_high_level($db, $rg, $op, $paras)
522	558	while ($doit == 1) {	while ($doit == 1) {
523	559	if ($rg['worker']['id'] == 0) {	if ($rg['worker']['id'] == 0) {
524	560	$rg['worker']['id'] = rg_var_uint('worker::id');	$rg['worker']['id'] = rg_var_uint('worker::id');
525		$rg['worker']['key'] = rg_var_str('worker::key');
	561		$rg['worker']['key'] = trim(rg_var_str('worker::key'));
526	562	}	}
527		$rg['worker']['name'] = rg_var_str('worker::name');
	563		$rg['worker']['name'] = trim(rg_var_str('worker::name'));
528	564	$rg['worker']['cost'] = rg_var_uint('worker::cost');	$rg['worker']['cost'] = rg_var_uint('worker::cost');
529	565	$rg['worker']['workers'] = rg_var_uint('worker::workers');	$rg['worker']['workers'] = rg_var_uint('worker::workers');
530	566

File root/themes/default/builder/queue/line.html changed (mode: 100644) (index 02d1e56..96b5dd1)
7	7	<td>@@itime_nice@@</td>	<td>@@itime_nice@@</td>
8	8	<td>@@done_nice@@</td>	<td>@@done_nice@@</td>
9	9	<td>	<td>
10		env=@@request::env@@
11		<br />
12		url=@@request::url@@
13		<br />
14		head=@@request::head@@
15		<br />
16		hook_id=@@request::hook_id@@
17		<br />
	10		env=@@request::env@@<br />
	11		url=@@request::url@@<br />
	12		head=@@request::head@@<br />
	13		hook_id=@@request::hook_id@@<br />
	14		Machine started at @@status::vm_start_nice@@<br />
	15		Clone took @@status::clone_elap_nice@@<br />
	16		Script started at @@status::build_sh_start_nice@@<br/>
	17
	18		<details>
	19		<summary>DUMP</summary>
	20		<pre>
	21		@@DUMP@@
	22		</pre>
	23		</details>
18	24
19	25	@@status_packages@@	@@status_packages@@
20	26

File root/themes/default/doc/worker.html changed (mode: 100644) (index 970d766..9fed3bd)
...	...	env = fedora-server-24-x86_64
51	51	arch = x86_64	arch = x86_64
52	52	image = /var/lib/libvirt/images/rgw/fedora-server-24-x86_64.qcow2	image = /var/lib/libvirt/images/rgw/fedora-server-24-x86_64.qcow2
53	53	pkg_cmd = dnf -y install	pkg_cmd = dnf -y install
54		os-variant = fedora22
55	54	</div>	</div>
56	55	Now, you can add this worker into your account	Now, you can add this worker into your account
57	56	(Settings / Worker or Admin / Worker)<br />	(Settings / Worker or Admin / Worker)<br />

File root/themes/default/repo/log/header.html changed (mode: 100644) (index 78672e9..758c8dd)
6	6	<th>Subject</th>	<th>Subject</th>
7	7	<th>SHA-1 (click to jump to commit)</th>	<th>SHA-1 (click to jump to commit)</th>
8	8	<th>Author</th>	<th>Author</th>
9		<th>Date</th>
	9		<th>Date (UTC)</th>
10	10	</tr>	</tr>
11	11

File root/themes/default/user/settings/workers/add.html changed (mode: 100644) (index 9f2288e..dc3c8ff)
26	26	</p>	</p>
27	27
28	28	<p>	<p>
29		<label for="name">Max parallel workers</label><br />
	29		<label for="name">Max parallel workers (0 = let worker decide)</label><br />
30	30	<input type="text" name="worker::workers" id="name" value="@@worker::workers@@" size="80" />	<input type="text" name="worker::workers" id="name" value="@@worker::workers@@" size="80" />
31	31	</p>	</p>
32	32

File root/themes/default/user/settings/workers/hints.html changed (mode: 100644) (index 3dba743..5335b09)
2	2	Workers are used to process build and deployment jobs.<br />	Workers are used to process build and deployment jobs.<br />
3	3	Global workers are available for building at rocketgit.com, so	Global workers are available for building at rocketgit.com, so
4	4	just configure a build webhook.<br />	just configure a build webhook.<br />
	5		Key - it is used to authenticate the connection from worker.<br />
5	6	Cost - it is used to decide on which worker to send a job; bigger = less	Cost - it is used to decide on which worker to send a job; bigger = less
6	7	chances to choose it.<br />	chances to choose it.<br />
7		Key - it is used to authenticate the connection from worker.<br />
	8		Max parallel workers: you can limit how many simultaneously jobs are sent
	9		to the worker.<br />
8	10	<br />	<br />
9	11	To learn how to configure a build machine, go	To learn how to configure a build machine, go
10		<a href="/op/doc/worker" target="_blank">here</a>.<br />
	12		<a href="/op/doc/worker" target="_blank">here</a>.
11	13	The worker will connect automatically and will be ready to receive build jobs.	The worker will connect automatically and will be ready to receive build jobs.
12	14	<br />	<br />

File samples/worker-main.conf.sample changed (mode: 100644) (index 5a3b564..d1875b0)
...	...	state = /var/lib/rocketgit/worker
23	23	# This is used to be able to authenticate with the server	# This is used to be able to authenticate with the server
24	24	key = insert_your_pass_here	key = insert_your_pass_here
25	25
	26		# libvirt XML templates dir (example file names: x86_64.xml, aarch64.xml)
	27		templates = /var/lib/libvirt/images/rgw/templates
	28
	29		# disk images directory
	30		images = /var/lib/libvirt/images/rgw/images
	31
	32		# Allow network access for the build process (root always has access)
	33		# 1 = allow, else, do not allow
	34		net = 0
	35
	36		# The user/group libvirtd is running under, for environments of type 'libvirt'.
	37		# It is used to chown the images files.
	38		libvirtd_user = qemu
	39		libvirtd_group = qemu
	40
26	41	# Available build environments	# Available build environments
27		# os-variant can be set by 'osinfo-query os' command; it is used for virtio
28	42	#env = fedora23-server-x86_64	#env = fedora23-server-x86_64
29	43	# type = libvirt	# type = libvirt
	44		# # A file named $templates/$arch.xml must exist
30	45	# arch = x86_64	# arch = x86_64
31		# image = /var/lib/libvirt/images/rgw/fedora23-server-x86_64.qcow2
	46		# # You can specify a full path (if it starts with '/') or else
	47		# # is relative to 'images' global configuration
	48		# image = fedora23-server-x86_64.qcow2
32	49	# pkg_cmd = dnf -y install	# pkg_cmd = dnf -y install
33		# os-variant = fedora22
34	50
35	51	#env = debian-8.2-i386	#env = debian-8.2-i386
36	52	# type = libvirt	# type = libvirt
37	53	# arch = i386	# arch = i386
38		# image = /var/lib/libvirt/images/rgw/debian-8.2-i386
	54		# image = debian-8.2-i386
39	55	# pkg_cmd = apt-get -y install	# pkg_cmd = apt-get -y install
40		# os-variant = debian8
41	56
42	57	#env = arm-32-fedora-23	#env = arm-32-fedora-23
43	58	# type = libvirt	# type = libvirt
44	59	# arch = armv7l	# arch = armv7l
45		# image = /var/lib/libvirt/images/rgw/arm-32-fedora-23.qcow2
	60		# image = arm-32-fedora-23.qcow2
46	61	# paras = --boot kernel=/var/lib/libvirt/images/ARM-Server-23/vmlinuz-4.4.6-300.fc23.armv7hl,initrd=/var/lib/libvirt/images/ARM-Server-23/initramfs-4.4.6-300.fc23.armv7hl.img,kernel_args="console=ttyAMA0 rw root=LABEL=_/ rootwait"	# paras = --boot kernel=/var/lib/libvirt/images/ARM-Server-23/vmlinuz-4.4.6-300.fc23.armv7hl,initrd=/var/lib/libvirt/images/ARM-Server-23/initramfs-4.4.6-300.fc23.armv7hl.img,kernel_args="console=ttyAMA0 rw root=LABEL=_/ rootwait"
47	62	# pkg_cmd = dnf -y install	# pkg_cmd = dnf -y install
48		# os-variant = fedora22

File scripts/builder.TODO added (mode: 100644) (index 0000000..3868239)
	1	[ ] Add branch in 'build_jobs' table, to know what branch triggered the build.
	2	[ ] On success/error, allow the calling of another webhook?
	3	This way we can build a complex pipe system.
	4	[ ] Add an optional timeout for the commands.
	5	[ ] Look at some pipeline presentation to be inspired.
	6	[ ] Switch to an encrypted channel (https and/or ssh)
	7	[ ] If ANN is not received in a resonable time, close the connection.
	8	[ ]

File scripts/builder.php changed (mode: 100644) (index de4f9bf..1d6713a)
...	...	function xnew($key, $arg)
49	49	{	{
50	50	global $rg_conns;	global $rg_conns;
51	51	global $workers;	global $workers;
	52		global $features;
52	53
53	54	$s = &$rg_conns[$key];	$s = &$rg_conns[$key];
54	55	$s['func_data'] = 'xdispatch';	$s['func_data'] = 'xdispatch';

...	...	function xnew($key, $arg)
57	58	$s['auth'] = 0;	$s['auth'] = 0;
58	59	$s['is_master'] = 0;	$s['is_master'] = 0;
59	60	$workers++;	$workers++;
	61
	62		$f = array(
	63		'op' => 'FEATURES',
	64		'features' => $features
	65		);
	66		rg_conn_enq($key, json_encode($f) . "\n");
60	67	}	}
61	68
62	69	/*	/*

...	...	function xdispatch_one($key, $data)
66	73	{	{
67	74	global $rg_conns;	global $rg_conns;
68	75	global $jobs;	global $jobs;
	76		global $features;
69	77
70	78	rg_log($key . ': dispatching');	rg_log($key . ': dispatching');
71	79

...	...	function xdispatch_one($key, $data)
77	85	'errstr' => 'cannot decode JSON:' . json_last_error_msg()	'errstr' => 'cannot decode JSON:' . json_last_error_msg()
78	86	);	);
79	87	rg_log_ml('Cannot decode JSON: ' . json_last_error_msg());	rg_log_ml('Cannot decode JSON: ' . json_last_error_msg());
80		rg_conn_enq($key, json_encode($err));
	88		rg_conn_enq($key, json_encode($err) . "\n");
81	89	rg_conn_shutdown($key, 2);	rg_conn_shutdown($key, 2);
82	90	return;	return;
83	91	}	}

...	...	function xdispatch_one($key, $data)
152	160	$s['worker_uid'] = $worker_uid;	$s['worker_uid'] = $worker_uid;
153	161	$s['ann'] = $u;	$s['ann'] = $u;
154	162	$s['auth'] = 1;	$s['auth'] = 1;
155		$s['active_jobs'] = 0;
	163		$s['active_jobs'] = array();
156	164
157	165	$a = array();	$a = array();
158	166	$a['name'] = $u['name'];	$a['name'] = $u['name'];
159	167	$a['uname'] = $u['uname'];	$a['uname'] = $u['uname'];
160	168	$a['host'] = $u['host'];	$a['host'] = $u['host'];
161	169	$a['arch'] = $u['arch'];	$a['arch'] = $u['arch'];
162		$a['env'] = $u['env'];
	170		$a['env'] = empty($u['env']) ? array() : $u['env'];
163	171	$a['ssh_key'] = $u['ssh_key'];	$a['ssh_key'] = $u['ssh_key'];
164	172	$a['ip'] = rg_fix_ip($s['ip']);	$a['ip'] = rg_fix_ip($s['ip']);
165		rg_worker_update($s['db'], $worker_uid, $wi['id'], $a);
	173		$r = rg_worker_update($s['db'], $worker_uid, $wi['id'], $a);
	174		if ($r !== TRUE) {
	175		rg_log('cannot update worker: ' . rg_worker_error());
	176		$err = array('errstr' => rg_worker_info());
	177		rg_conn_enq($key, json_encode($err) . "\n");
	178		rg_conn_shutdown($key, 2);
	179		return;
	180		}
166	181
167	182	rg_log($key . ': peer [' . $u['name'] . '] announce processed.');	rg_log($key . ': peer [' . $u['name'] . '] announce processed.');
168	183	return;	return;

...	...	function xdispatch_one($key, $data)
180	195	$jobs[$jid]['worker'] = $key;	$jobs[$jid]['worker'] = $key;
181	196	$jobs[$jid]['worker_name'] = $s['ann']['name'];	$jobs[$jid]['worker_name'] = $s['ann']['name'];
182	197	$jobs[$jid]['worker_started'] = time();	$jobs[$jid]['worker_started'] = time();
183		$s['active_jobs'] += 1;
	198		if (!isset($s['active_jobs'][$jid]))
	199		$s['active_jobs'][$jid] = 1;
184	200	rg_log('Job started: ' . $jid);	rg_log('Job started: ' . $jid);
185	201	return;	return;
186	202	}	}
187	203
188	204	if (strcmp($u['op'], 'DON') == 0) {	if (strcmp($u['op'], 'DON') == 0) {
189		$s['active_jobs'] -= 1;
190
191	205	$jid = $u['id'];	$jid = $u['id'];
	206
	207		if (isset($s['active_jobs'][$jid]))
	208		unset($s['active_jobs'][$jid]);
	209
192	210	if (isset($u['error'])) {	if (isset($u['error'])) {
193	211	rg_log('job failed with error: ' . $u['error']);	rg_log('job failed with error: ' . $u['error']);
194	212	// Delay job and retry (on another worker)	// Delay job and retry (on another worker)

...	...	function xdispatch_one($key, $data)
208	226	return;	return;
209	227	}	}
210	228
	229		if (strcmp($u['op'], 'WORKER_STATS') == 0) {
	230		rg_log_ml('Worker stats received TODO');
	231		return;
	232		}
	233
211	234	rg_log('Unknown command [' . $u['op'] . ']!');	rg_log('Unknown command [' . $u['op'] . ']!');
212	235	$err = array('errstr' => 'unknown op');	$err = array('errstr' => 'unknown op');
213	236	rg_conn_enq($key, json_encode($err) . "\n");	rg_conn_enq($key, json_encode($err) . "\n");

...	...	function rg_process_job($db, &$job)
248	271
249	272	rg_log_ml('Processing job: ' . print_r($job, TRUE));	rg_log_ml('Processing job: ' . print_r($job, TRUE));
250	273
	274		if (!isset($job['request']))
	275		$req = $job;
	276		else
	277		$req = $job['request'];
	278
251	279	// Get the worker list, so we can sort it	// Get the worker list, so we can sort it
252		$workers_list = rg_worker_list_all($db, $job['uid']);
	280		$workers_list = rg_worker_list_all($db, $req['uid']);
253	281	if ($workers_list === FALSE) {	if ($workers_list === FALSE) {
254	282	rg_log('cannot load workers list: ' . rg_worker_error());	rg_log('cannot load workers list: ' . rg_worker_error());
255	283	$job['next_try'] = time() + 60;	$job['next_try'] = time() + 60;

...	...	function rg_process_job($db, &$job)
258	286	//rg_log_ml('DEBUG: workers list: ' . print_r($workers_list, TRUE));	//rg_log_ml('DEBUG: workers list: ' . print_r($workers_list, TRUE));
259	287
260	288	// Trying to find a worker in the list of connections	// Trying to find a worker in the list of connections
261		$found = FALSE;
262	289	foreach ($rg_conns as $key => $i) {	foreach ($rg_conns as $key => $i) {
263	290	if (strcmp($key, 'master') == 0)	if (strcmp($key, 'master') == 0)
264	291	continue;	continue;

...	...	function rg_process_job($db, &$job)
274	301	continue;	continue;
275	302	}	}
276	303
277		if (($i['worker_uid'] > 0) && ($i['worker_uid'] != $job['uid'])) {
278		//rg_log('uid does not match, try next');
	304		if (($i['worker_uid'] > 0) && ($i['worker_uid'] != $req['uid'])) {
	305		rg_log('uids do not match, try next');
279	306	continue;	continue;
280	307	}	}
281	308
282	309	$k = $i['worker_id'];	$k = $i['worker_id'];
	310		$name = $i['worker_name'];
283	311	if (!isset($workers_list[$k])) {	if (!isset($workers_list[$k])) {
284		rg_log('Worker ' . $k . ' not found in workers_list! Strange!');
	312		rg_internal_error('Worker ' . $name . ' not found'
	313		. ' in workers_list! Strange!');
285	314	continue;	continue;
286	315	}	}
287	316	$wi = $workers_list[$k];	$wi = $workers_list[$k];
288	317
289	318	if (isset($job['avoid'][$k])) {	if (isset($job['avoid'][$k])) {
290		rg_log('We must avoid worker ' . $k);
	319		rg_log('We must avoid worker ' . $name);
291	320	continue;	continue;
292	321	}	}
293	322
294		rg_log('DEBUG: selected worker ' . $k);
295
296	323	// If number of active jobs is == max workers, skip it	// If number of active jobs is == max workers, skip it
297		if ($wi['workers'] <= $i['active_jobs']) {
298		rg_log('DEBUG: workers=' . $wi['workers']
299		. ' active_jobs=' . $i['active_jobs']);
	324		$aj = count($i['active_jobs']);
	325		if ($wi['workers'] && ($aj >= $wi['workers'])) {
	326		rg_log('DEBUG: skip worker ' . $name . ' because'
	327		. ' active_jobs(' . $aj . ')'
	328		. ' >= workers(' . $wi['workers'] . ')');
300	329	continue;	continue;
301	330	}	}
302	331
303	332	foreach ($i['ann']['env'] as $env => $junk) {	foreach ($i['ann']['env'] as $env => $junk) {
304		if (strcasecmp($job['env'], $env) != 0) {
305		//rg_log('DEBUG job env [' . $job['env'] . ']'
306		// . ' != worker [' . $env . ']');
	333		if (strcasecmp($req['env'], $env) != 0) {
	334		rg_log('DEBUG: worker ' . $name
	335		. ': job env [' . $req['env'] . ']'
	336		. ' != worker [' . $env . ']');
307	337	continue;	continue;
308	338	}	}
309	339
310	340	// Send only what is really needed	// Send only what is really needed
311	341	$job2 = array();	$job2 = array();
312	342	$job2['op'] = 'BLD';	$job2['op'] = 'BLD';
313		$job2['cmds'] = $job['cmds'];
314		$job2['packages'] = $job['packages'];
315		$job2['hook_id'] = $job['hook_id'];
316		$job2['url'] = $job['url'];
317		$job2['head'] = $job['head'];
318		$job2['env'] = $job['env'];
	343		$job2['cmds'] = $req['cmds'];
	344		$job2['packages'] = $req['packages'];
	345		$job2['hook_id'] = $req['hook_id'];
	346		$job2['url'] = $req['url'];
	347		$job2['head'] = $req['head'];
	348		$job2['env'] = $req['env'];
319	349	$job2['id'] = $job['id'];	$job2['id'] = $job['id'];
320	350
321	351	rg_conn_enq($key, json_encode($job2) . "\n");	rg_conn_enq($key, json_encode($job2) . "\n");
322		// TODO: get a confirmation?
	352		// TODO: get a confirmation? We get 'STA'.
323	353	$job['worker'] = $key;	$job['worker'] = $key;
324	354	$job['worker_started'] = 0;	$job['worker_started'] = 0;
325	355	$job['worker_sent'] = time();	$job['worker_sent'] = time();

...	...	function rg_process_job($db, &$job)
329	359	// TODO: maybe the client must resync with server to	// TODO: maybe the client must resync with server to
330	360	// abort jobs already done on another host, to not	// abort jobs already done on another host, to not
331	361	// duplicate work	// duplicate work
332		$found = TRUE;
333	362	return;	return;
334	363	}	}
335	364	}	}
336	365
337		if (!$found)
338		rg_log('No workers found!');
	366		rg_log('No workers found!');
339	367	}	}
340	368
341	369

...	...	$rg_conns['master']['is_master'] = 1;
387	415
388	416	$jobs = array();	$jobs = array();
389	417
	418		// What features the builder supports
	419		$features = array('allow_stats' => 1);
	420
390	421	$workers = 0;	$workers = 0;
391	422	$original_mtime = @filemtime(__FILE__);	$original_mtime = @filemtime(__FILE__);
392	423	do {	do {

File scripts/cachec.php changed (mode: 100644) (index f51258f..af033a9)
...	...	do {
57	57	if ($r === FALSE)	if ($r === FALSE)
58	58	break;	break;
59	59	$r = socket_recv($master, $buf, 4096, 0);	$r = socket_recv($master, $buf, 4096, 0);
60		echo $buf;
	60		echo $buf . "\n";
61	61	} while (1);	} while (1);
62	62
63	63	socket_close($master);	socket_close($master);

File scripts/remote.php changed (mode: 100644) (index 9efc131..8ea2262)
...	...	if (isset($_SERVER['SSH_CONNECTION'])) {
163	163	$key_id = 0;	$key_id = 0;
164	164	$flags = '';	$flags = '';
165	165
166		$line = @fread(STDIN, 8000);
167		$line_len = strlen($line);
168		rg_log("line=[$line]");
169		if ($line_len < 4)
170		fatal("Line is too short!");
171		$len = @hexdec(substr($line, 0, 4));
172		if ($line_len < $len)
173		fatal("Too less data ($line_len/$len) received!");
	166		$line = ''; $line_len = 0; $len = 0;
	167		while (1) {
	168		$r = @fread(STDIN, 8000);
	169		if ($r === FALSE)
	170		fatal('Error in receive: ' . rg_php_err());
	171
	172		if (empty($r))
	173		fatal("Too less data ($line_len/$len) received!");
	174
	175		$line .= $r;
	176		$line_len += strlen($r);
	177		rg_log("line=[$line]");
	178		if ($line_len < 4)
	179		fatal('Line is too short [' . $line . ']!');
	180
	181		$len = @hexdec(substr($line, 0, 4));
	182		rg_log('Decoded len=' . $len . ' line_len=' . $line_len);
	183		if ($line_len >= $len)
	184		break;
	185		}
174	186
175	187	// parse something like: 002bgit-upload-pack /aa.git[0x00]host=localhost:9418[0x00]	// parse something like: 002bgit-upload-pack /aa.git[0x00]host=localhost:9418[0x00]
176	188	$line = substr($line, 4); // skip length	$line = substr($line, 4); // skip length

...	...	if (strstr($flags, 'W')) {
215	227	fatal('A worker can only clone!');	fatal('A worker can only clone!');
216	228	}	}
217	229
	230		$helper_flags = '';
	231		// bypass authentication because it is a global worker
	232		if (($login_uid == 0) && strstr($flags, 'W'))
	233		$helper_flags .= 'B';
	234
218	235	$r = rg_repo_fetch_push_helper($db, $host, $ip, $login_ui, $prefix, $user,	$r = rg_repo_fetch_push_helper($db, $host, $ip, $login_ui, $prefix, $user,
219		$repo, $cmd);
	236		$repo, $cmd, $helper_flags);
220	237	rg_log_ml('DEBUG: repo_fetch_push_helper: ' . print_r($r, TRUE));	rg_log_ml('DEBUG: repo_fetch_push_helper: ' . print_r($r, TRUE));
221	238	if (($r['ok'] !== 1) \|\| ($r['allow'] !== 1))	if (($r['ok'] !== 1) \|\| ($r['allow'] !== 1))
222	239	fatal($r['errmsg']);	fatal($r['errmsg']);

File scripts/worker.TODO changed (mode: 100644) (index 5638070..531353e)
4	4	<source dev="/dev/xxx"/>	<source dev="/dev/xxx"/>
5	5	<target dev="vdb" bus="virtio"/>	<target dev="vdb" bus="virtio"/>
6	6	</disk>	</disk>
7
8	7	# virsh attach-device vm1 dev1.xml	# virsh attach-device vm1 dev1.xml
9	8
10	9	[ ] Refuse jobs if the number of cpus/mem/etc. is too big for all workers.	[ ] Refuse jobs if the number of cpus/mem/etc. is too big for all workers.
11		[ ] Clean images after build.
	10		[ ] Clean images after build. Shred?
12	11	[ ] Add the templates to rocketgit git.	[ ] Add the templates to rocketgit git.
13	12	[ ] Add a secure channel for comunication between worker and server.	[ ] Add a secure channel for comunication between worker and server.
14	13	Done. Use it. For example, to detect if the VM started.	Done. Use it. For example, to detect if the VM started.

23	22	[ ] Activate SELinux with a custom profile.	[ ] Activate SELinux with a custom profile.
24	23	[ ] Activate AppArmor with a custom profile.	[ ] Activate AppArmor with a custom profile.
25	24	[ ] About resources availability: who is able to set them?	[ ] About resources availability: who is able to set them?
26		The web interface of the worker? I think the worker.
	25		The web interface or the worker? I think the worker.
27	26	[ ] Add notification channels to be able to notify if the build was ok/bad.	[ ] Add notification channels to be able to notify if the build was ok/bad.
28	27	At least notify the project owner.	At least notify the project owner.
29	28	[ ] Take load in consideration?	[ ] Take load in consideration?

31	30	[ ] Specify a global max pool size?	[ ] Specify a global max pool size?
32	31	[ ] When strting, load all $conf['state'] . '/job-*.ser' files and send a DON	[ ] When strting, load all $conf['state'] . '/job-*.ser' files and send a DON
33	32	command to the server.	command to the server.
	33		[ ] Add a build_job_history table to store the tries to show them on the web?
	34		[ ] At start, check that all templates are available.
	35		[ ] If the user never confirmed the e-mail address, maybe we should not allow
	36		network access to the build jobs? Or, maybe, do not allow build jobs?
	37		[ ] Report load of the worker to the server to be able to build a graph?
	38		[ ] Allow admin, when adding a worker, to disable network access.
	39		[ ] Should we validate we are talking with the correct server? I think yes.
	40		At least on tcp:// connections.
34	41	[ ]	[ ]

File scripts/worker.php changed (mode: 100644) (index fad4cd7..e0fff8d)
...	...	function reload_config()
46	46	global $conf_file;	global $conf_file;
47	47	global $conf;	global $conf;
48	48
	49		$r = @stat($conf_file);
	50		if ($r === FALSE) {
	51		rg_log('Cannot stat conf file ' . $conf_file . ': ' . rg_php_err());
	52		exit(1);
	53		}
	54
	55		if (($r['mode'] & 07) != 0) {
	56		$r = @chmod($conf_file, $r['mode'] & ~07);
	57		if ($r === FALSE) {
	58		rg_log('Error! Others can access the conf file and read the key!');
	59		exit(1);
	60		}
	61		}
	62
	63		rg_log('Loading configuration from ' . $conf_file);
49	64	$_conf = @file($conf_file);	$_conf = @file($conf_file);
50	65	if ($_conf === FALSE) {	if ($_conf === FALSE) {
51		// worker.conf not found
52		exit(0);
	66		rg_log('Error! Cannot open the conf file ' . $conf_file . ': ' . rg_php_err());
	67		exit(1);
53	68	}	}
54	69
55	70	$last_key = FALSE;	$last_key = FALSE;

...	...	function reload_config()
119	134	unset($conf['master']);	unset($conf['master']);
120	135	unset($conf['port']);	unset($conf['port']);
121	136
	137		// Create state dir
	138		@mkdir($conf['state'], 0771);
	139
	140		// allow libvirt user to access some stuff
	141		$r = @chmod($conf['state'], 0771);
	142		if ($r !== TRUE) {
	143		rg_log('Cannot chmod state dir [' . $conf['state']
	144		. ']: ' . rg_php_err());
	145		sleep(60);
	146		exit(1);
	147		}
	148
122	149	if (!file_exists($conf['state'] . '/key.pub')) {	if (!file_exists($conf['state'] . '/key.pub')) {
123	150	rg_log('Creating SSH key...');	rg_log('Creating SSH key...');
124	151	$cmd = 'ssh-keygen -t rsa -b 4096 -N \'\''	$cmd = 'ssh-keygen -t rsa -b 4096 -N \'\''

...	...	function reload_config()
138	165	exit(0);	exit(0);
139	166	}	}
140	167
	168		if (!isset($conf['templates'])) {
	169		$conf['templates'] = '/var/lib/libvirt/images/rgw/templates';
	170		rg_log('Warn: \'templates\' configuration line is missing'
	171		. '; I will assume to be ' . $conf['templates']);
	172		}
	173
	174		if (!isset($conf['images'])) {
	175		$conf['images'] = '/var/lib/libvirt/images/rgw/images';
	176		rg_log('Warn: \'images\' configuration line is missing'
	177		. '; I will assume the individual section specify full path');
	178		}
	179
	180		// Validate the path to the templates and images
	181		foreach ($conf['env'] as $name => &$i) {
	182		if (!isset($i['image'])) {
	183		rg_log('Warning! Environment ' . $name . ' is missing \'image\' declaration'
	184		. '; I will disable it.');
	185		unset($conf['env'][$name]);
	186		continue;
	187		}
	188
	189		if (strncmp($i['image'], '/', 1) != 0)
	190		$i['image'] = $conf['images'] . '/' . $i['image'];
	191
	192		if (!file_exists($i['image'])) {
	193		rg_log('Warning! Environment ' . $name . ' is missing ' . $i['image']
	194		. ' file; I will disable it.');
	195		unset($conf['env'][$name]);
	196		continue;
	197		}
	198
	199		if (!isset($i['arch'])) {
	200		rg_log('Warning! Environment ' . $name . ' is missing \'arch\' declaration'
	201		. '; I will assume to be x86_64.');
	202		$i['arch'] = 'x86_64';
	203		}
	204
	205		$f = $conf['templates'] . '/' . $i['arch'] . '.xml';
	206		if (!file_exists($f)) {
	207		rg_log('Warning! Environment ' . $name . ' is missing ' . $i['arch']
	208		. ' file (' . $f . '); I will disable it.');
	209		unset($conf['env'][$name]);
	210		continue;
	211		}
	212		}
	213		if (empty($conf['env'])) {
	214		rg_log('Fatal! No environments found!');
	215		sleep(60);
	216		exit(1);
	217		}
	218
	219		if (!isset($conf['net'])) {
	220		rg_log('\'net\' was not specified, so I will disable network'
	221		. ' access for the build user');
	222		$conf['net'] = 0;
	223		}
	224
	225		if (!isset($conf['libvirtd_user']))
	226		$conf['libvirtd_user'] = 'qemu';
	227
	228		if (!isset($conf['libvirtd_group']))
	229		$conf['libvirtd_group'] = 'qemu';
	230
141	231	rg_log_ml('conf: ' . print_r($conf, TRUE));	rg_log_ml('conf: ' . print_r($conf, TRUE));
142	232	}	}
143	233

...	...	function save_job($job)
177	267	return $ret;	return $ret;
178	268	}	}
179	269
	270		/*
	271		* Set correct libvirt rights on a file
	272		*/
	273		function rg_worker_libvirt_rights($f, $mode, $user, $group, &$reason, &$reason2)
	274		{
	275		$ret = FALSE;
	276		while (1) {
	277		// We need to allow libvirt access to the image
	278		$r = @chown($f, $user);
	279		if ($r !== TRUE) {
	280		$reason = 'cannot chown image to qemu user';
	281		$reason2 = rg_php_err();
	282		break;
	283		}
	284
	285		$r = @chgrp($f, $group);
	286		if ($r !== TRUE) {
	287		$reason = 'cannot chgrp image to qemu user';
	288		$reason2 = rg_php_err();
	289		break;
	290		}
	291
	292		$r = @chmod($f, $mode);
	293		if ($r !== TRUE) {
	294		$reason = 'cannot chmod image';
	295		$reason2 = rg_php_err();
	296		break;
	297		}
	298
	299		$r = rg_exec('/usr/sbin/restorecon '
	300		. escapeshellarg($f), '', FALSE, FALSE, FALSE);
	301		if ($r['ok'] !== 1) {
	302		rg_log('restorecon failed, trying to continue ('
	303		. $r['errmsg'] . ': ' . $r['stderr'] . ')');
	304		}
	305
	306		$ret = TRUE;
	307		break;
	308		}
	309
	310		return $ret;
	311		}
	312
180	313	/*	/*
181	314	* Starts an worker	* Starts an worker
182	315	*/	*/
183	316	function start_worker($job)	function start_worker($job)
184	317	{	{
185	318	global $conf;	global $conf;
186		global $php_errormsg;
	319
	320		rg_log_ml('DEBUG: start_worker: job: ' . print_r($job, TRUE));
187	321
188	322	$env = $conf['env'][$job['env']];	$env = $conf['env'][$job['env']];
189	323	rg_log_ml('DEBUG: start_worker: env: ' . print_r($env, TRUE));	rg_log_ml('DEBUG: start_worker: env: ' . print_r($env, TRUE));

...	...	function start_worker($job)
194	328	$name = 'rg-worker-' . $job['id'];	$name = 'rg-worker-' . $job['id'];
195	329	$ename = escapeshellarg($name);	$ename = escapeshellarg($name);
196	330	$master = escapeshellarg($env['image']);	$master = escapeshellarg($env['image']);
197		$img = $job['main'] . '/image.qcow2';
	331		$img = $job['main'] . '/image.qcow2'; // TODO: let admin control the image type
198	332	$eimg = escapeshellarg($img);	$eimg = escapeshellarg($img);
199	333	$img2 = $job['main'] . '/image2.raw';	$img2 = $job['main'] . '/image2.raw';
200	334	$eimg2 = escapeshellarg($img2);	$eimg2 = escapeshellarg($img2);

...	...	function start_worker($job)
217	351	break;	break;
218	352	}	}
219	353
220		$r = @mkdir($job['main'], 0700);
	354		$r = @mkdir($job['main'], 0770);
221	355	if ($r === FALSE) {	if ($r === FALSE) {
222	356	$reason = 'cannot create main dir';	$reason = 'cannot create main dir';
223	357	$reason2 = 'cannot create main dir (' . $job['main'] . '):'	$reason2 = 'cannot create main dir (' . $job['main'] . '):'

...	...	function start_worker($job)
225	359	break;	break;
226	360	}	}
227	361
	362		// We need to allow libvirt access to the image stored inside
	363		$r = rg_worker_libvirt_rights($job['main'], 0770,
	364		$conf['libvirtd_user'], $conf['libvirtd_group'], $reason, $reason2);
	365		if ($r !== TRUE)
	366		break;
	367
228	368	$r = save_job($job);	$r = save_job($job);
229	369	if ($r['ok'] !== 1) {	if ($r['ok'] !== 1) {
230	370	$reason = 'cannot save job';	$reason = 'cannot save job';

...	...	function start_worker($job)
232	372	break;	break;
233	373	}	}
234	374
235		// We need to allow libvirt access to the image
236		// TODO: 'qemu' must be read from the conf file (on Fedora it is qemu)
237		$r = rg_exec('chown qemu:qemu '
238		. escapeshellarg($job['main']), '', FALSE, FALSE, FALSE);
239		if ($r['ok'] !== 1) {
240		$reason = 'cannot chown build dir to qemu user';
241		$reason2 = $r['errmsg'] . ': ' . $r['stderr'];
242		break;
243		}
244
245	375	rg_exec('virsh destroy ' . $ename, '', FALSE, FALSE, FALSE);	rg_exec('virsh destroy ' . $ename, '', FALSE, FALSE, FALSE);
246	376	rg_exec('virsh undefine --nvram ' . $ename, '', FALSE, FALSE, FALSE);	rg_exec('virsh undefine --nvram ' . $ename, '', FALSE, FALSE, FALSE);
247	377

...	...	function start_worker($job)
254	384	break;	break;
255	385	}	}
256	386
257		// TODO: let the user specify the maximum disk space?
	387		$r = rg_worker_libvirt_rights($img, 0770,
	388		$conf['libvirtd_user'], $conf['libvirtd_group'], $reason, $reason2);
	389		if ($r !== TRUE)
	390		break;
	391
258	392	$r = rg_exec('qemu-img create -f raw ' . $eimg2	$r = rg_exec('qemu-img create -f raw ' . $eimg2
259	393	. ' ' . escapeshellarg($job['disk_size_gib'] . 'G'),	. ' ' . escapeshellarg($job['disk_size_gib'] . 'G'),
260	394	'', FALSE, FALSE, FALSE);	'', FALSE, FALSE, FALSE);

...	...	function start_worker($job)
264	398	break;	break;
265	399	}	}
266	400
	401		$r = rg_worker_libvirt_rights($img2, 0770,
	402		$conf['libvirtd_user'], $conf['libvirtd_group'], $reason, $reason2);
	403		if ($r !== TRUE)
	404		break;
	405
267	406	// Seems that mkfs is not in PATH when we are running from cron.	// Seems that mkfs is not in PATH when we are running from cron.
268	407	$path = getenv('PATH');	$path = getenv('PATH');
269	408	putenv('PATH=' . $path . ':/usr/sbin');	putenv('PATH=' . $path . ':/usr/sbin');

...	...	function start_worker($job)
295	434	// Clone repo	// Clone repo
296	435	$_env = 'GIT_SSH_COMMAND=ssh'	$_env = 'GIT_SSH_COMMAND=ssh'
297	436	. ' -o PasswordAuthentication=no'	. ' -o PasswordAuthentication=no'
298		. ' -o ControlMaster=yes'
	437		. ' -o ControlMaster=no'
299	438	. ' -o IdentitiesOnly=yes'	. ' -o IdentitiesOnly=yes'
300		. ' -o IdentityFile='
301		. escapeshellarg($conf['state'] . '/key');
302		rg_log('DEBUG: env: ' . $_env);
	439		. ' -o IdentityFile=' . escapeshellarg($conf['state'] . '/key');
303	440	putenv($_env);	putenv($_env);
304		$cmd = ' git clone --depth 1'
	441		$_s = time();
	442		$cmd = ' git clone'
305	443	. ' --recurse-submodules'	. ' --recurse-submodules'
306		. ' --shallow-submodules'
	444		//TODO . ' --shallow-submodules'
307	445	. ' --no-checkout'	. ' --no-checkout'
308	446	. ' ' . escapeshellarg($job['url'])	. ' ' . escapeshellarg($job['url'])
309	447	. ' ' . $emain . '/root/git';	. ' ' . $emain . '/root/git';
310	448	$r = rg_exec($cmd, '', FALSE, FALSE, FALSE);	$r = rg_exec($cmd, '', FALSE, FALSE, FALSE);
	449		@file_put_contents($job['main'] . '/root/T_clone', time() - $_s);
311	450	if ($r['ok'] !== 1) {	if ($r['ok'] !== 1) {
312		$reason = 'git clone error';
	451		$reason = 'git clone error; contact admin';
313	452	$reason2 = $r['errmsg'] . ': ' . $r['stderr'];	$reason2 = $r['errmsg'] . ': ' . $r['stderr'];
314	453	break;	break;
315	454	}	}

...	...	function start_worker($job)
317	456	// Build command list	// Build command list
318	457	// TODO: document how a user can add labels in configure or make	// TODO: document how a user can add labels in configure or make
319	458	$s = '#!/bin/bash' . "\n";	$s = '#!/bin/bash' . "\n";
	459		$s .= 'date +%s > /mnt/status/build.sh.start' . "\n\n";
320	460	$s .= 'export RG_LABELS=/mnt/status/RG_LABELS' . "\n\n";	$s .= 'export RG_LABELS=/mnt/status/RG_LABELS' . "\n\n";
321	461	$s .= 'cd /mnt/git' . "\n\n";	$s .= 'cd /mnt/git' . "\n\n";
322		$s .= 'git checkout -b rgw ' . escapeshellarg($job['head']) . "\n";
	462		$s .= 'git branch -f rgw ' . escapeshellarg($job['head']) . " &>/mnt/status/git.branch.log\n\n";
	463		$s .= 'git checkout rgw' . " &>/mnt/status/git.checkout.log\n\n";
323	464	foreach ($job['cmds'] as $_name => $i) {	foreach ($job['cmds'] as $_name => $i) {
324	465	if (empty($i['cmd']))	if (empty($i['cmd']))
325	466	continue;	continue;

...	...	function start_worker($job)
328	469	. escapeshellarg($_name);	. escapeshellarg($_name);
329	470
330	471	if (empty($i['label_ok']))	if (empty($i['label_ok']))
331		$lok = '';
	472		$lok = 'echo -n';
332	473	else	else
333	474	$lok = ' echo '	$lok = ' echo '
334	475	. escapeshellarg($i['label_ok'])	. escapeshellarg($i['label_ok'])
335	476	. ' >>/mnt/status/RG_LABELS' . "\n";	. ' >>/mnt/status/RG_LABELS' . "\n";
336	477
337	478	if (empty($i['label_nok']))	if (empty($i['label_nok']))
338		$lnok = '';
	479		$lnok = 'echo -n';
339	480	else	else
340	481	$lnok = ' echo '	$lnok = ' echo '
341	482	. escapeshellarg($i['label_nok'])	. escapeshellarg($i['label_nok'])

...	...	function start_worker($job)
348	489	. 'echo ${E} > ' . $prefix . ".status\n"	. 'echo ${E} > ' . $prefix . ".status\n"
349	490	. 'date +%s > ' . $prefix . '.done' . "\n"	. 'date +%s > ' . $prefix . '.done' . "\n"
350	491	. 'if [ "${E}" != "0" ]; then' . "\n"	. 'if [ "${E}" != "0" ]; then' . "\n"
351		. ' echo -n # just to not have an empty if branch' . "\n"
352	492	. $lnok	. $lnok
353	493	. ($i['abort'] ? ' exit 0' . "\n" : '')	. ($i['abort'] ? ' exit 0' . "\n" : '')
354	494	. 'else' . "\n"	. 'else' . "\n"
355		. ' echo -n # just to not have an empty if branch' . "\n"
356	495	. $lok	. $lok
357	496	. 'fi' . "\n\n";	. 'fi' . "\n\n";
358	497	}	}
359		rg_log_ml('DEBUG: build.sh: ' . $s);
	498		//rg_log_ml('DEBUG: build.sh: ' . $s);
360	499	$r = @file_put_contents($job['main'] . '/root/build.sh', $s);	$r = @file_put_contents($job['main'] . '/root/build.sh', $s);
361	500	if ($r === FALSE) {	if ($r === FALSE) {
362	501	$reason = 'cannot store build commands';	$reason = 'cannot store build commands';

...	...	function start_worker($job)
371	510	}	}
372	511
373	512	// Prepare packages - for now, we must list every package	// Prepare packages - for now, we must list every package
374		// on a single line to avoid not available packages
	513		// on a single line to avoid error if one is not available
375	514	if (!empty($job['packages'])) {	if (!empty($job['packages'])) {
376	515	rg_log('DEBUG: packages: ' . $job['packages'] . '.');	rg_log('DEBUG: packages: ' . $job['packages'] . '.');
377	516	$pkgs = explode(' ', $job['packages']);	$pkgs = explode(' ', $job['packages']);

...	...	function start_worker($job)
400	539	. ' sleep 1' . "\n"	. ' sleep 1' . "\n"
401	540	. 'done' . "\n"	. 'done' . "\n"
402	541	. 'date +%s > /mnt/T_NET_OK' . "\n\n"	. 'date +%s > /mnt/T_NET_OK' . "\n\n"
403		. '' . "\n"
	542		. "\n"
404	543	. $p_i_cmd	. $p_i_cmd
405	544	. 'date +%s > /mnt/T_PKGS_OK' . "\n\n"	. 'date +%s > /mnt/T_PKGS_OK' . "\n\n"
406		. '' . "\n"
	545		. "\n"
407	546	. '# Disabling further module loading.' . "\n"	. '# Disabling further module loading.' . "\n"
408	547	. 'echo 1 > /proc/sys/kernel/modules_disabled' . "\n"	. 'echo 1 > /proc/sys/kernel/modules_disabled' . "\n"
	548		. "\n"
	549		. '# Restricting dmesg' . "\n"
	550		. 'sysctl kernel.dmesg_restrict=1' . "\n"
	551		. "\n"
409	552	. '# Disabling root login' . "\n"	. '# Disabling root login' . "\n"
410	553	. 'chage -E 0 root' . "\n"	. 'chage -E 0 root' . "\n"
411	554	. 'if [ "${?}" != "0" ]; then' . "\n"	. 'if [ "${?}" != "0" ]; then' . "\n"
412	555	. ' ERR="cannot disable root account"' . "\n"	. ' ERR="cannot disable root account"' . "\n"
413	556	. 'fi' . "\n"	. 'fi' . "\n"
	557		. "\n"
	558		. '# Disable network access if needed' . "\n"
	559		. 'if [ "' . $conf['net'] . '" = "1" ]; then' . "\n"
	560		. ' echo -n # we allow network access' . "\n"
	561		. 'else' . "\n"
	562		. ' iptables -I OUTPUT -m owner --uid-owner build -j REJECT' . "\n"
	563		. ' if [ "${?}" != "0" ]; then' . "\n"
	564		. ' ERR="Cannot disable network access"' . "\n"
	565		. ' fi' . "\n"
	566		. 'fi' . "\n"
	567		. "\n"
414	568	. 'if [ "${ERR}" = "" ]; then' . "\n"	. 'if [ "${ERR}" = "" ]; then' . "\n"
415	569	. ' su - build -c "bash /mnt/build.sh" &>/mnt/status/build.log' . "\n"	. ' su - build -c "bash /mnt/build.sh" &>/mnt/status/build.log' . "\n"
416	570	. ' sync' . "\n"	. ' sync' . "\n"

...	...	function start_worker($job)
440	594	}	}
441	595	$do_umount = FALSE;	$do_umount = FALSE;
442	596
443		// We need to allow libvirt access to the image
444		$r = rg_exec('chown qemu:qemu ' . $eimg2, '', FALSE, FALSE, FALSE);
445		if ($r['ok'] !== 1) {
446		$reason = 'cannot chown image to qemu user';
447		$reason2 = $r['errmsg'] . ': ' . $r['stderr'];
448		break;
449		}
450
451		// TODO: store the path to the templates in the configuration file.
452		$_f = '/var/lib/libvirt/images/rgw/templates/' . $env['arch'] . '.xml';
	597		$_f = $conf['templates'] . '/' . $env['arch'] . '.xml';
453	598	$template = @file_get_contents($_f);	$template = @file_get_contents($_f);
454	599	if ($template === FALSE) {	if ($template === FALSE) {
455	600	$reason = 'cannot load template';	$reason = 'cannot load template';

...	...	function start_worker($job)
482	627	$reason2 = $r['errmsg'] . ': ' . $r['stderr'];	$reason2 = $r['errmsg'] . ': ' . $r['stderr'];
483	628	break;	break;
484	629	}	}
	630		$r = @file_put_contents($job['main'] . '/vm.start', time());
485	631
486	632	$err = FALSE;	$err = FALSE;
487	633	break;	break;

...	...	function xhandle_one($key, $data)
509	655	global $jobs;	global $jobs;
510	656	global $conf;	global $conf;
511	657	global $pid_to_jid;	global $pid_to_jid;
	658		global $features;
512	659
513		$job = @json_decode($data, TRUE);
514		if ($job === NULL) {
	660		$u = @json_decode($data, TRUE);
	661		if ($u === NULL) {
	662		rg_log('JSON: ' . $data);
515	663	rg_log_ml('Cannot decode JSON: ' . json_last_error_msg());	rg_log_ml('Cannot decode JSON: ' . json_last_error_msg());
516	664	$err = array('errstr' => 'cannot decode json');	$err = array('errstr' => 'cannot decode json');
517	665	rg_conn_enq('master', json_encode($err) . "\n");	rg_conn_enq('master', json_encode($err) . "\n");

...	...	function xhandle_one($key, $data)
519	667	return;	return;
520	668	}	}
521	669
522		$jid = $job['id'];
	670		if (isset($u['id']))
	671		$jid = $u['id'];
523	672
524		if (strcmp($job['op'], 'BLD') == 0) {
	673		if (strcmp($u['op'], 'BLD') == 0) {
525	674	// TODO: should we confirm quickly if the job is accepted,	// TODO: should we confirm quickly if the job is accepted,
526	675	// even if we could not fork?	// even if we could not fork?
527	676

...	...	function xhandle_one($key, $data)
531	680	return;	return;
532	681	}	}
533	682
534		$jobs[$jid] = $job;
	683		$jobs[$jid] = $u;
535	684	$jobs[$jid]['main'] = $conf['state'] . '/rocketgit-j-' . $jid;	$jobs[$jid]['main'] = $conf['state'] . '/rocketgit-j-' . $jid;
536	685	$jobs[$jid]['state'] = RG_JOB_INIT;	$jobs[$jid]['state'] = RG_JOB_INIT;
537	686
538		rg_log_ml('build job: ' . print_r($job, TRUE));
	687		rg_log_ml('build job: ' . print_r($u, TRUE));
539	688	$err = TRUE;	$err = TRUE;
540	689	while (1) {	while (1) {
541	690	// Fork	// Fork

...	...	function xhandle_one($key, $data)
568	717	}	}
569	718	$a['op'] = 'STA';	$a['op'] = 'STA';
570	719	rg_conn_enq('master', json_encode($a) . "\n");	rg_conn_enq('master', json_encode($a) . "\n");
571		} else if (strcmp($job['op'], 'DRE') == 0) { // DRE = done received
	720		} else if (strcmp($u['op'], 'DRE') == 0) { // DRE = done received
572	721	// So, we can clean up everything related to this job	// So, we can clean up everything related to this job
573	722	// TODO: do we clear the state file?	// TODO: do we clear the state file?
574	723	rg_log('DRE command');	rg_log('DRE command');

...	...	function xhandle_one($key, $data)
576	725	unset($pid_to_jid[$_job['pid']]);	unset($pid_to_jid[$_job['pid']]);
577	726	unset($jobs[$jid]);	unset($jobs[$jid]);
578	727	@unlink($conf['state'] . '/job-' . $jid . '.ser');	@unlink($conf['state'] . '/job-' . $jid . '.ser');
	728		} else if (strcmp($u['op'], 'FEATURES') == 0) { // what master suports
	729		rg_log_ml('FEATURES command: ' . print_r($u, TRUE));
	730		$features = $u['features'];
579	731	} else {	} else {
580		rg_log($key . ': cannot handle op: ' . $job['op']);
	732		rg_log($key . ': cannot handle op: ' . $u['op']);
581	733	}	}
582	734	}	}
583	735

...	...	function rg_job_extract_info(&$job)
627	779	break;	break;
628	780	}	}
629	781
	782		$r = @file_get_contents($job['main'] . '/error2.log');
	783		if ($r !== FALSE) {
	784		$job['error2'] = $r;
	785		break;
	786		}
	787
630	788	// Extract how much disk space was used	// Extract how much disk space was used
631		$r = stat($job['main'] . '/image2.raw');
	789		// TODO: Warn the user when the disk space is close to the limit?
	790		// TODO: or alloc a lot of space by default?
	791		$r = @stat($job['main'] . '/image2.raw');
632	792	if ($r === FALSE) {	if ($r === FALSE) {
633	793	$job['error'] = 'Missing image2 file';	$job['error'] = 'Missing image2 file';
634	794	break;	break;

...	...	function rg_job_extract_info(&$job)
637	797	// TODO - remove this	// TODO - remove this
638	798	$cmd = 'ln -f ' . $emain . '/image2.raw ' . $emain . '/..';	$cmd = 'ln -f ' . $emain . '/image2.raw ' . $emain . '/..';
639	799	$r = rg_exec($cmd, '', FALSE, FALSE, FALSE);	$r = rg_exec($cmd, '', FALSE, FALSE, FALSE);
	800		// TODO - remove this
	801		$cmd = 'ln -f ' . $emain . '/machine.xml ' . $emain . '/..';
	802		$r = rg_exec($cmd, '', FALSE, FALSE, FALSE);
640	803
641	804	$cmd = 'mount ' . $emain . '/image2.raw ' . $emain . '/root';	$cmd = 'mount ' . $emain . '/image2.raw ' . $emain . '/root';
642	805	$r = rg_exec($cmd, '', FALSE, FALSE, FALSE);	$r = rg_exec($cmd, '', FALSE, FALSE, FALSE);

...	...	function rg_job_extract_info(&$job)
646	809	}	}
647	810
648	811	$r = @file_get_contents($job['main'] . '/root/status/err');	$r = @file_get_contents($job['main'] . '/root/status/err');
649		if ($r !== FALSE) {
	812		if ($r !== FALSE)
650	813	$job['error'] = $r;	$job['error'] = $r;
651		break;
652		}
653	814
654	815	$labels = @file($job['main'] . '/root/status/RG_LABELS');	$labels = @file($job['main'] . '/root/status/RG_LABELS');
655	816	if ($labels === FALSE)	if ($labels === FALSE)

...	...	function rg_job_extract_info(&$job)
659	820	// Add worker name as label	// Add worker name as label
660	821	$labels[] = 'worker/' . $conf['name'] . '/color=fff';	$labels[] = 'worker/' . $conf['name'] . '/color=fff';
661	822
	823		$clone_elap = @file_get_contents($job['main'] . '/root/T_clone');
	824		if ($clone_elap === FALSE)
	825		$clone_elap = 'n/a';
	826
	827		$build_sh_start = @file_get_contents($job['main'] . '/root/status/build.sh.start');
	828		if ($build_sh_start === FALSE)
	829		$build_sh_start = 'n/a';
	830		else
	831		$build_sh_start = intval($build_sh_start);
	832
	833		$vm_start = @file_get_contents($job['main'] . '/vm.start');
	834		if ($vm_start === FALSE)
	835		$vm_start = 0;
	836		else
	837		$vm_start = intval($vm_start);
	838
662	839	$job['status'] = array(	$job['status'] = array(
	840		'vm_start' => $vm_start,
	841		'build_sh_start' => $build_sh_start,
663	842	'packages' => @trim(file_get_contents($job['main'] . '/root/packages.log')),	'packages' => @trim(file_get_contents($job['main'] . '/root/packages.log')),
	843		'clone_elap' => $clone_elap,
664	844	'start' => @trim(file_get_contents($job['main'] . '/root/T_START')),	'start' => @trim(file_get_contents($job['main'] . '/root/T_START')),
665	845	'net_ok' => @trim(file_get_contents($job['main'] . '/root/T_NET_OK')),	'net_ok' => @trim(file_get_contents($job['main'] . '/root/T_NET_OK')),
666	846	'pkgs_ok' => @trim(file_get_contents($job['main'] . '/root/T_PKGS_OK')),	'pkgs_ok' => @trim(file_get_contents($job['main'] . '/root/T_PKGS_OK')),

...	...	function rg_job_extract_info(&$job)
676	856	$sd = $job['main'] . '/root/status/' . $cmd;	$sd = $job['main'] . '/root/status/' . $cmd;
677	857	$job['status']['cmds'][$cmd] = array(	$job['status']['cmds'][$cmd] = array(
678	858	'cmd' => $i['cmd'],	'cmd' => $i['cmd'],
679		'start' => trim(file_get_contents($sd . '.start')),
680		'done' => trim(file_get_contents($sd . '.done')),
681		'status' => trim(file_get_contents($sd . '.status')),
	859		'start' => trim(@file_get_contents($sd . '.start')),
	860		'done' => trim(@file_get_contents($sd . '.done')),
	861		'status' => trim(@file_get_contents($sd . '.status')),
682	862	'log' => trim(rg_file_get_tail($sd . '.log', 4 * 4096))	'log' => trim(rg_file_get_tail($sd . '.log', 4 * 4096))
683	863	);	);
684	864	}	}

...	...	function vm_extract_info($name)
746	926	return $ret;	return $ret;
747	927	}	}
748	928
	929		/*
	930		* Send stats about the worker
	931		*/
	932		function send_worker_stats()
	933		{
	934		static $last_time = 0;
	935		global $features;
	936
	937		if (!isset($features['allow_stats']))
	938		return;
	939
	940		$load = rg_load();
	941		$now = time();
	942
	943		if ($last_time > $now - 30)
	944		return;
749	945
	946		$stats = array(
	947		'op' => 'WORKER_STATS',
	948		'ts' => $now,
	949		'load' => $load
	950		);
	951		rg_conn_enq('master', json_encode($stats) . "\n");
	952		$last_time = $now;
	953		}
	954
	955		umask(0007);
750	956	reload_config();	reload_config();
	957		rg_log('id is ' . $id);
751	958	$conf['id'] = $id;	$conf['id'] = $id;
752	959
	960		// What master supports
	961		$features = array();
	962
753	963	rg_log('Connecting to ' . $conf['master_host'] . '/' . $conf['master_port']	rg_log('Connecting to ' . $conf['master_host'] . '/' . $conf['master_port']
754	964	. ' with proto ' . $conf['master_proto']	. ' with proto ' . $conf['master_proto']
755	965	. ' with url [' . $conf['master_url'] . ']' . '...');	. ' with url [' . $conf['master_url'] . ']' . '...');

...	...	$ann['host'] = php_uname('n');
821	1031	$ann['arch'] = php_uname('m');	$ann['arch'] = php_uname('m');
822	1032	$ann['boot_time'] = time();	$ann['boot_time'] = time();
823	1033	$ann['sign'] = hash_hmac('sha512', $ann['boot_time'], $key);	$ann['sign'] = hash_hmac('sha512', $ann['boot_time'], $key);
824		$j_ann = json_encode($ann);
	1034		$j_ann = @json_encode($ann);
825	1035	if ($j_ann === FALSE) {	if ($j_ann === FALSE) {
826	1036	rg_log('Cannot encode json: ' . json_last_error_msg());	rg_log('Cannot encode json: ' . json_last_error_msg());
827	1037	exit(1);	exit(1);

...	...	$pid_to_jid = array();
833	1043	while(1) {	while(1) {
834	1044	rg_conn_wait(3);	rg_conn_wait(3);
835	1045
	1046		send_worker_stats();
	1047
836	1048	// Verify if the jobs are started	// Verify if the jobs are started
837	1049	while (1) {	while (1) {
838	1050	$pid = pcntl_waitpid(-1, $status, WNOHANG);	$pid = pcntl_waitpid(-1, $status, WNOHANG);

...	...	while(1) {
848	1060	$jobs[$jid]['state'] = RG_JOB_STARTED;	$jobs[$jid]['state'] = RG_JOB_STARTED;
849	1061	}	}
850	1062
851		// Verify if vms finished
	1063		// Verify if VMs finished
852	1064	$vms_loaded = FALSE;	$vms_loaded = FALSE;
853	1065	foreach ($jobs as $jid => &$job) {	foreach ($jobs as $jid => &$job) {
854	1066	if ($job['state'] != RG_JOB_STARTED)	if ($job['state'] != RG_JOB_STARTED)

...	...	while(1) {
893	1105	unset($xjob['debug']);	unset($xjob['debug']);
894	1106	unset($xjob['packages']);	unset($xjob['packages']);
895	1107	unset($xjob['main']);	unset($xjob['main']);
896		$j_xjob = json_encode($xjob);
	1108		$j_xjob = @json_encode($xjob);
897	1109	if ($j_xjob === FALSE) {	if ($j_xjob === FALSE) {
898	1110	rg_log('Cannot encode json: ' . json_last_error_msg());	rg_log('Cannot encode json: ' . json_last_error_msg());
899	1111	} else {	} else {

...	...	while(1) {
904	1116	$r = rg_exec($cmd, '', FALSE, FALSE, FALSE);	$r = rg_exec($cmd, '', FALSE, FALSE, FALSE);
905	1117	if ($r['ok'] != 1) {	if ($r['ok'] != 1) {
906	1118	// If error, probably the machine was not running, so, this is just a warning	// If error, probably the machine was not running, so, this is just a warning
	1119		// TODO: But we set an error!
907	1120	$job['error'] = 'Could not destroy: ' . $r['errmsg'];	$job['error'] = 'Could not destroy: ' . $r['errmsg'];
908	1121	rg_log('Error: ' . $job['error']);	rg_log('Error: ' . $job['error']);
909	1122	}	}

...	...	while(1) {
912	1125
913	1126	rg_prof_end('MAIN');	rg_prof_end('MAIN');
914	1127	rg_prof_log();	rg_prof_log();
915		?>

File tests/by_http.php changed (mode: 100644) (index e81baa6..9dc518c)
...	...	rg_log_set_file("by_http.log");
19	19	require_once("common.php");	require_once("common.php");
20	20
21	21	$_testns = 'by_http';	$_testns = 'by_http';
	22		$e_testns = escapeshellarg($_testns);
	23		//TODO $git_push = 'git push --verbose --push-option _testns=' . $e_testns;
	24		$git_push = 'git push --verbose';
22	25
23	26
24	27	prepare_http();	prepare_http();

...	...	if ($a['ok'] != 1) {
62	65	rg_log('');	rg_log('');
63	66	rg_log_enter('Trying to push to public (empty user)...');	rg_log_enter('Trying to push to public (empty user)...');
64	67	putenv('GIT_ASKPASS=' . dirname(__FILE__) . '/ask_pass_empty');	putenv('GIT_ASKPASS=' . dirname(__FILE__) . '/ask_pass_empty');
65		$r = rg_exec('cd .by_http && git push public_repo master',
	68		$r = rg_exec('cd .by_http'
	69		. ' && ' . $git_push . ' public_repo master',
66	70	'', FALSE, FALSE, FALSE);	'', FALSE, FALSE, FALSE);
67	71	if ($r['ok'] == 1) {	if ($r['ok'] == 1) {
68	72	rg_log_ml('r: ' . print_r($r, TRUE));	rg_log_ml('r: ' . print_r($r, TRUE));

...	...	rg_log_exit();
80	84	rg_log('');	rg_log('');
81	85	rg_log_enter('Trying to push to public (with guest)...');	rg_log_enter('Trying to push to public (with guest)...');
82	86	putenv('GIT_ASKPASS=' . dirname(__FILE__) . '/ask_pass_guest');	putenv('GIT_ASKPASS=' . dirname(__FILE__) . '/ask_pass_guest');
83		$r = rg_exec('cd .by_http && git push public_repo master',
	87		$r = rg_exec('cd .by_http && ' . $git_push . ' public_repo master',
84	88	'', FALSE, FALSE, FALSE);	'', FALSE, FALSE, FALSE);
85	89	if ($r['ok'] != 1) {	if ($r['ok'] != 1) {
86	90	rg_log_ml('r: ' . print_r($r, TRUE));	rg_log_ml('r: ' . print_r($r, TRUE));

...	...	rg_log_exit();
98	102	rg_log('');	rg_log('');
99	103	rg_log_enter('Trying to push to public (with user/pass)...');	rg_log_enter('Trying to push to public (with user/pass)...');
100	104	putenv('GIT_ASKPASS=' . dirname(__FILE__) . '/ask_pass');	putenv('GIT_ASKPASS=' . dirname(__FILE__) . '/ask_pass');
101		$r = rg_exec('cd .by_http && git push --verbose public_repo master',
	105		$r = rg_exec('cd .by_http && ' . $git_push . ' public_repo master',
102	106	'', FALSE, FALSE, FALSE);	'', FALSE, FALSE, FALSE);
103	107	if ($r['ok'] != 1) {	if ($r['ok'] != 1) {
104	108	rg_log_ml('r: ' . print_r($r, TRUE));	rg_log_ml('r: ' . print_r($r, TRUE));

...	...	rg_log_exit();
136	140	rg_log('');	rg_log('');
137	141	rg_log_enter('Trying to push to private (guest user)...');	rg_log_enter('Trying to push to private (guest user)...');
138	142	putenv('GIT_ASKPASS=' . dirname(__FILE__) . '/ask_pass_guest');	putenv('GIT_ASKPASS=' . dirname(__FILE__) . '/ask_pass_guest');
139		$r = rg_exec('cd .by_http && git push --verbose private_repo master',
	143		$r = rg_exec('cd .by_http && ' . $git_push . ' private_repo master',
140	144	'', FALSE, FALSE, FALSE);	'', FALSE, FALSE, FALSE);
141	145	if ($r['ok'] == 1) {	if ($r['ok'] == 1) {
142	146	rg_log_ml('r: ' . print_r($r, TRUE));	rg_log_ml('r: ' . print_r($r, TRUE));

...	...	rg_log_exit();
155	159	rg_log('');	rg_log('');
156	160	rg_log_enter('Trying to push to private (with user/pass)...');	rg_log_enter('Trying to push to private (with user/pass)...');
157	161	putenv('GIT_ASKPASS=' . dirname(__FILE__) . '/ask_pass');	putenv('GIT_ASKPASS=' . dirname(__FILE__) . '/ask_pass');
158		$r = rg_exec('cd .by_http && git push --verbose private_repo master',
	162		$r = rg_exec('cd .by_http && ' . $git_push . ' private_repo master',
159	163	'', FALSE, FALSE, FALSE);	'', FALSE, FALSE, FALSE);
160	164	if ($r['ok'] != 1) {	if ($r['ok'] != 1) {
161	165	rg_log_ml('r: ' . print_r($r, TRUE));	rg_log_ml('r: ' . print_r($r, TRUE));

...	...	rg_log('key: ' . $key);
188	192	rg_log('');	rg_log('');
189	193	rg_log_enter('Trying to push to public (with user/pass/login_token_missing)...');	rg_log_enter('Trying to push to public (with user/pass/login_token_missing)...');
190	194	putenv('GIT_ASKPASS=' . dirname(__FILE__) . '/ask_pass');	putenv('GIT_ASKPASS=' . dirname(__FILE__) . '/ask_pass');
191		$r = rg_exec('cd .by_http && git push --verbose public_repo master',
	195		$r = rg_exec('cd .by_http && ' . $git_push . ' public_repo master',
192	196	'', FALSE, FALSE, FALSE);	'', FALSE, FALSE, FALSE);
193	197	if ($r['ok'] !== 0) {	if ($r['ok'] !== 0) {
194	198	rg_log_ml('r: ' . print_r($r, TRUE));	rg_log_ml('r: ' . print_r($r, TRUE));

...	...	rg_log('');
215	219	rg_log_enter('Trying to push to public (with user/pass/login_token_ok)...');	rg_log_enter('Trying to push to public (with user/pass/login_token_ok)...');
216	220	putenv('RG_LOGIN_TOKEN=' . rg_totp_compute($key, (time() - 30) / 30, 6));	putenv('RG_LOGIN_TOKEN=' . rg_totp_compute($key, (time() - 30) / 30, 6));
217	221	putenv('GIT_ASKPASS=' . dirname(__FILE__) . '/ask_pass');	putenv('GIT_ASKPASS=' . dirname(__FILE__) . '/ask_pass');
218		$r = rg_exec('cd .by_http && git push --verbose public_repo master',
	222		$r = rg_exec('cd .by_http && ' . $git_push . ' public_repo master',
219	223	'', FALSE, FALSE, FALSE);	'', FALSE, FALSE, FALSE);
220	224	if ($r['ok'] != 1) {	if ($r['ok'] != 1) {
221	225	rg_log_ml('r: ' . print_r($r, TRUE));	rg_log_ml('r: ' . print_r($r, TRUE));

File tests/config.php changed (mode: 100644) (index dec64ef..9e6cb66)
...	...	if ($test_normal) {
58	58	$rg_no_db = TRUE;	$rg_no_db = TRUE;
59	59	$rg_cache_enable = TRUE;	$rg_cache_enable = TRUE;
60	60	}	}
	61
	62		putenv('GIT_CURL_VERBOSE=1');
	63		putenv('GIT_TRACE=1');
	64		putenv('GIT_TRACE_PACKET=1');

 rg_log_enter('Testing push with a lot of small objects (push)...');
 $r = rg_exec('cd temp_repos/git_small_obj'
 	. ' && git remote set-url origin ' . $remote
-	. ' && export GIT_CURL_VERBOSE=1 GIT_TRACE=1 GIT_TRACE_PACKET=1'
 	. ' && git push --force origin master',
 	'', FALSE, FALSE, FALSE);
 if ($r['ok'] != 1) {
 rg_log('');
 rg_log_enter('Testing push with a big file (push)...');
 $r = rg_exec('cd temp_repos/git_big_push'
-	. ' && export GIT_CURL_VERBOSE=1 GIT_TRACE=1 GIT_TRACE_PACKET=1'
 	. ' && git remote set-url origin ' . $remote
 	. ' && git push --force origin master',
 	'', FALSE, FALSE, FALSE);

File tests/keys.php changed (mode: 100644) (index 8b1cbe3..c68aa72)
...	...	$res = rg_sql_query($db, $sql);
28	28	rg_sql_free_result($res);	rg_sql_free_result($res);
29	29
30	30	// insert a key 1	// insert a key 1
31		$key1 = "ssh-dss AAAAB3NzaC1kc3MAAACBAJLkWtcoCUbWCRXecE907nO1gSh6IrfkD5bsyobrFOp6xYuJvfteKKpE79HUcbEpzIVFNk3mlQf/+k9cFP2Wy8F34UXFk8cXU4FU7z/TM1iHHOHnqFqvzv59LRaaMw4MaHm/4WKdfJy16KOLgosSBzWif3a1nKMdIZuYeIGso7qFAAAAFQC4JU7YoGu2nZQ0fEXFKaRhq+d9UQAAAIEAhgslkwwID6oBBdWx+mUuaXKt/bZcdCfNyjnejxlsZHPfDnayuqCKIgxlhYpiPS6LwiSK5feL55meF33HanCzX53z7ieoW6Je9z2H8/93sCvzk4LMj7XkeEy3G5UnRuL+uc6qrazF7Pu448cQH0pkh6N0zNueQlPpGL4/lHbIiVgAAACAQJup/h36aD9DprosVCQe40nalp7t4o/M75Y70sV7FNrL0azUQcn1ZL+J8F9l/dDRPG3rST2DABgba9pHGWa96vaNfTLnopy3po/296SYQl7/1nek0YtioEikoB+HQxk7eSwRI6bTpKEFNyUnp2/bcNjJYORKCeYwJJ2KDW6GJro= first_key";
	31		$key1 = "ssh-dss AAAA\nB3NzaC1kc3MAAACBAJLkWtcoCUbWCRXecE907nO1gSh6IrfkD5bsyobrFOp6xYuJvfteKKpE79HUcbEpzIVFNk3mlQf/+k9cFP2Wy8F34UXFk8cXU4FU7z/TM1iHHOHnqFqvzv59LRaaMw4MaHm/4WKdfJy16KOLgosSBzWif3a1nKMdIZuYeIGso7qFAAAAFQC4JU7YoGu2nZQ0fEXFKaRhq+d9UQAAAIEAhgslkwwID6oBBdWx+mUuaXKt/bZcdCfNyjnejxlsZHPfDnayuqCKIgxlhYpiPS6LwiSK5feL55meF33HanCzX53z7ieoW6Je9z2H8/93sCvzk4LMj7XkeEy3G5UnRuL+uc6qrazF7Pu448cQH0pkh6N0zNueQlPpGL4/lHbIiVgAAACAQJup/h36aD9DprosVCQe40nalp7t4o/M75Y70sV7FNrL0azUQcn1ZL+J8F9l/dDRPG3rST2DABgba9pHGWa96vaNfTLnopy3po/296SYQl7/1nek0YtioEikoB+HQxk7eSwRI6bTpKEFNyUnp2/bcNjJYORKCeYwJJ2KDW6GJro= first\n_key";
32	32	rg_state_set($db, 'ssh_key_allow_dsa', 1);	rg_state_set($db, 'ssh_key_allow_dsa', 1);
33	33	$key_id1 = rg_keys_add($db, $rg_ui, $key1);	$key_id1 = rg_keys_add($db, $rg_ui, $key1);
34	34	if ($key_id1 === FALSE) {	if ($key_id1 === FALSE) {
35	35	rg_log("Cannot add key 1 (" . rg_keys_error() . ")!");	rg_log("Cannot add key 1 (" . rg_keys_error() . ")!");
36	36	exit(1);	exit(1);
37	37	}	}
	38		// We should not have \n inside the key and comment
	39		$ki = rg_keys_info($key1);
	40		if (strstr($ki['key'], "\n")) {
	41		rg_log('\n is not allowed in a key body!');
	42		exit(1);
	43		}
	44		if (strstr($ki['comment'], "\n")) {
	45		rg_log('\n is not allowed in a key comment!');
	46		exit(1);
	47		}
38	48
39	49	// insert a key 2	// insert a key 2
40	50	$rg_ui['uid'] = 2;	$rg_ui['uid'] = 2;

...	...	if ($c === FALSE) {
60	70	rg_log("Cannot regenerate file (afile.txt not found)!");	rg_log("Cannot regenerate file (afile.txt not found)!");
61	71	exit(1);	exit(1);
62	72	}	}
	73		$key1b = str_replace("\n", '', $key1); $key1b = str_replace('first_key', 'first _key', $key1b);
63	74	$e = "command=\""	$e = "command=\""
64	75	. $rg_scripts . "/scripts/remote.sh 1 $key_id1 N\","	. $rg_scripts . "/scripts/remote.sh 1 $key_id1 N\","
65		. $rg_ssh_paras . ' ' . $key1 . "\n"
	76		. $rg_ssh_paras . ' ' . $key1b . "\n"
66	77	. "command=\""	. "command=\""
67	78	. $rg_scripts . "/scripts/remote.sh 2 $key_id2 N\","	. $rg_scripts . "/scripts/remote.sh 2 $key_id2 N\","
68	79	. $rg_ssh_paras . ' ' . $key2 . "\n";	. $rg_ssh_paras . ' ' . $key2 . "\n";

File tests/wh_cloud.php changed (mode: 100644) (index 5fca183..ecefd02)
...	...	rg_log('');
106	106	rg_log_enter('Testing if hook executed with success');	rg_log_enter('Testing if hook executed with success');
107	107	$key = 'DEBUG::' . $rg_ui['uid'] . '::webhooks::' . $wh_id;	$key = 'DEBUG::' . $rg_ui['uid'] . '::webhooks::' . $wh_id;
108	108	$r = test_wait_cache($key);	$r = test_wait_cache($key);
109		if (!isset($r['deploymentId']) && !strstr($r['message'], 'is already deploying')) {
110		rg_log_ml('r: ' . print_r($r, TRUE));
	109		$a = @json_decode($r, TRUE);
	110		if ($a === FALSE) {
	111		rg_log('Cannot decode JSON: ' . $r);
	112		exit(1);
	113		}
	114		if (!isset($a['deploymentId'])
	115		&& !(isset($a['message']) && strstr($a['message'], 'is already deploying'))) {
	116		rg_log_ml('a: ' . print_r($a, TRUE));
111	117	rg_log('Seems hook did not executed correctly!');	rg_log('Seems hook did not executed correctly!');
112	118	exit(1);	exit(1);
113	119	}	}

File tests/wh_lambda.php changed (mode: 100644) (index 3fa08ed..05f2e85)
...	...	rg_log('');
104	104	rg_log_enter('Testing if hook executed with success');	rg_log_enter('Testing if hook executed with success');
105	105	$key = 'DEBUG::' . $rg_ui['uid'] . '::webhooks::' . $wh_id;	$key = 'DEBUG::' . $rg_ui['uid'] . '::webhooks::' . $wh_id;
106	106	$r = test_wait_cache($key);	$r = test_wait_cache($key);
	107		$a = @json_decode($r, TRUE);
	108		if ($a === FALSE) {
	109		rg_log('Cannot decode JSON: '. $r);
	110		exit(1);
	111		}
107	112	$e = 'Hello from Lambda; key: aaa branch=master repo=wh-cloud';	$e = 'Hello from Lambda; key: aaa branch=master repo=wh-cloud';
108		if (strcmp($r, $e) != 0) {
109		rg_log('Seems hook did not executed correctly [' . $r . '] != [' . $e . ']!');
	113		if (strcmp($a, $e) != 0) {
	114		rg_log_ml('r: ' . print_r($r, TRUE));
	115		rg_log('Seems hook did not executed correctly [' . $a . '] != [' . $e . ']!');
110	116	exit(1);	exit(1);
111	117	}	}
112	118	rg_log_exit();	rg_log_exit();

Hints:
Before first commit, do not forget to setup your git environment:

git config --global user.name "your_name_here"
git config --global user.email "your@email_here"

Clone this repository using HTTP(S):

git clone https://rocketgit.com/user/catalinux/rocketgit

Clone this repository using ssh (do not forget to upload a key first):

git clone ssh://rocketgit@ssh.rocketgit.com/user/catalinux/rocketgit

Clone this repository using git:

git clone git://git.rocketgit.com/user/catalinux/rocketgit

You are allowed to anonymously push to this repository.
This means that your pushed commits will automatically be transformed into a merge request:

... clone the repository ...
... make some changes and some commits ...
git push origin main