benf_wspdigital / cpython (public) (License: PSF) (since 2017-05-12) (hash sha1)
Personal fork from https://github.com/python/cpython.git
Clone URLs: https://rocketgit.com/user/benf_wspdigital/cpython ssh://rocketgit@ssh.rocketgit.com/user/benf_wspdigital/cpython git://git.rocketgit.com/user/benf_wspdigital/cpython
master wip/issue/issue30181_parse-docstring-using-pydoc
List of commits:
Subject Hash Author Date (UTC)
bpo-29861: release references to multiprocessing Pool tasks (#743) 8988945cdc27ffa86ba8c624e095b51c459f5154 Antoine Pitrou 2017-03-24 12:52:11
bpo-19930: The mode argument of os.makedirs() no longer affects the file (#799) e304e33c16e060932d1e2cc8a030d42b02b429b5 Serhiy Storchaka 2017-03-24 11:27:42
doc: Fix small typos in library/multiprocessing (GH-698) 5619ab2db3a6c62ffaa55e8826cf67b7459fc484 Sylvain Bellemare 2017-03-24 08:26:07
`make tags` fixes (GH-717) 8a543c0bc7347d5b333f334d157bf4a7cd33c14a Alex Dzyoba 2017-03-24 08:23:43
faulthandler: Restore the old sigaltstack during teardown (#777) 20fbf8accd494fd15b0fc4c84928178c71ead4d1 Christophe Zeitouny 2017-03-23 17:14:29
Use NULL rather than 0. (#778) 0b3ec192259a65971001ce8f0de85a9c1e71d9c7 Serhiy Storchaka 2017-03-23 15:53:47
bpo-6532: Make the thread id an unsigned integer. (#781) aefa7ebf0ff0f73feee7ab24f4cdcb2014d83ee5 Serhiy Storchaka 2017-03-23 13:48:39
bpo-29728: Provide socket.TCP_NOTSENT_LOWAT (#477) 1e2147b9d75a64df370a9393c2b5b9d170dc0afd Nathaniel J. Smith 2017-03-23 03:56:55
Remove an outdated statement in execution model docs (GH-754) fad7f1560669af1766c583c7ef242c55d8c8de41 Ivan Levkivskyi 2017-03-23 00:54:53
doc: minor fix for library/profile (GH-761) bd3d8ba3b22da0bad018b53a3e6610ae03c5aa49 INADA Naoki 2017-03-22 07:56:36
fix function name in tabnanny documentation (GH-759) 75b6cf879fab58c0576d5415eaf38d92bfd678ab Jelle Zijlstra 2017-03-22 06:53:57
bpo-29859: Fix error messages from return codes for pthread_* calls (GH-741) d7fa6b259e00fca04dbf816bfcf4115fdda14bb7 Daniel Birnstiel 2017-03-21 13:06:06
bpo-29865: Use PyXXX_GET_SIZE macros rather than Py_SIZE for concrete types. (#748) fff9a31a91283c39c363af219e595eab7d4da6f7 Serhiy Storchaka 2017-03-21 06:53:25
Don't use Py_SIZE for dict object. (#747) c61ac1642d19f54c7b755098230967ad2e603180 Serhiy Storchaka 2017-03-21 06:52:38
bpo-27593: Revise git SCM build info. (#744) 554626ada769abf82a5dabe6966afa4265acb6a6 Ned Deily 2017-03-21 03:41:52
bpo-29849: fix a memory leak in import_from (GH-712) 4830f581af57dd305c02c1fd72299ecb5b090eca Xiang Zhang 2017-03-21 03:13:42
Fix "NotImplentedError" typo in constants documentation (#692) 05f53735c8912f8df1077e897f052571e13c3496 zertrin 2017-03-20 12:24:39
bpo-20087: Revert "make the glibc alias table take precedence over the X11 one (#422)" (#713) df8280838f52d6ec45ba03ef734b0dec8a9c43fb Benjamin Peterson 2017-03-20 06:49:43
bpo-28876: bool of large range raises OverflowError (#699) e46fb8611867fa3b407a813f53137929b7cb4a10 4kir4 2017-03-20 06:44:46
bpo-29856: Fix typo in curses documentation (GH-730) 64508780d72769e4c7afc67a511c057261c578f6 Mariatta 2017-03-20 03:48:04
Commit 8988945cdc27ffa86ba8c624e095b51c459f5154 - bpo-29861: release references to multiprocessing Pool tasks (#743)
* bpo-29861: release references to multiprocessing Pool tasks

Release references to tasks, their arguments and their results as soon
as they are finished, instead of keeping them alive until another task
arrives.

* Comments in test
Author: Antoine Pitrou
Author date (UTC): 2017-03-24 12:52
Committer name: GitHub
Committer date (UTC): 2017-03-24 12:52
Parent(s): e304e33c16e060932d1e2cc8a030d42b02b429b5
Signer:
Signing key:
Signing status: N
Tree: 3dd4f0619935cca2a0556f0a9cac10ff0aa6b678
File Lines added Lines deleted
Lib/multiprocessing/pool.py 6 1
Lib/test/_test_multiprocessing.py 28 0
Misc/NEWS 3 0
File Lib/multiprocessing/pool.py changed (mode: 100644) (index ffdf42614d..ae8cec4479)
... ... def worker(inqueue, outqueue, initializer=None, initargs=(), maxtasks=None,
128 128 util.debug("Possible encoding error while sending result: %s" % ( util.debug("Possible encoding error while sending result: %s" % (
129 129 wrapped)) wrapped))
130 130 put((job, i, (False, wrapped))) put((job, i, (False, wrapped)))
131
132 task = job = result = func = args = kwds = None
131 133 completed += 1 completed += 1
132 134 util.debug('worker exiting after %d tasks' % completed) util.debug('worker exiting after %d tasks' % completed)
133 135
 
... ... class Pool(object):
402 404 if set_length: if set_length:
403 405 util.debug('doing set_length()') util.debug('doing set_length()')
404 406 set_length(i+1) set_length(i+1)
407 finally:
408 task = taskseq = job = None
405 409 else: else:
406 410 util.debug('task handler got sentinel') util.debug('task handler got sentinel')
407 411
408
409 412 try: try:
410 413 # tell result handler to finish when cache is empty # tell result handler to finish when cache is empty
411 414 util.debug('task handler sending sentinel to result handler') util.debug('task handler sending sentinel to result handler')
 
... ... class Pool(object):
445 448 cache[job]._set(i, obj) cache[job]._set(i, obj)
446 449 except KeyError: except KeyError:
447 450 pass pass
451 task = job = obj = None
448 452
449 453 while cache and thread._state != TERMINATE: while cache and thread._state != TERMINATE:
450 454 try: try:
 
... ... class Pool(object):
461 465 cache[job]._set(i, obj) cache[job]._set(i, obj)
462 466 except KeyError: except KeyError:
463 467 pass pass
468 task = job = obj = None
464 469
465 470 if hasattr(outqueue, '_reader'): if hasattr(outqueue, '_reader'):
466 471 util.debug('ensuring that outqueue is not full') util.debug('ensuring that outqueue is not full')
File Lib/test/_test_multiprocessing.py changed (mode: 100644) (index b5f4782546..1d3bb0f8ba)
... ... import random
18 18 import logging import logging
19 19 import struct import struct
20 20 import operator import operator
21 import weakref
21 22 import test.support import test.support
22 23 import test.support.script_helper import test.support.script_helper
23 24
 
... ... def raise_large_valuerror(wait):
1738 1739 time.sleep(wait) time.sleep(wait)
1739 1740 raise ValueError("x" * 1024**2) raise ValueError("x" * 1024**2)
1740 1741
1742 def identity(x):
1743 return x
1744
1745 class CountedObject(object):
1746 n_instances = 0
1747
1748 def __new__(cls):
1749 cls.n_instances += 1
1750 return object.__new__(cls)
1751
1752 def __del__(self):
1753 type(self).n_instances -= 1
1754
1741 1755 class SayWhenError(ValueError): pass class SayWhenError(ValueError): pass
1742 1756
1743 1757 def exception_throwing_generator(total, when): def exception_throwing_generator(total, when):
 
... ... def exception_throwing_generator(total, when):
1746 1760 raise SayWhenError("Somebody said when") raise SayWhenError("Somebody said when")
1747 1761 yield i yield i
1748 1762
1763
1749 1764 class _TestPool(BaseTestCase): class _TestPool(BaseTestCase):
1750 1765
1751 1766 @classmethod @classmethod
 
... ... class _TestPool(BaseTestCase):
2000 2015 # check that we indeed waited for all jobs # check that we indeed waited for all jobs
2001 2016 self.assertGreater(time.time() - t_start, 0.9) self.assertGreater(time.time() - t_start, 0.9)
2002 2017
2018 def test_release_task_refs(self):
2019 # Issue #29861: task arguments and results should not be kept
2020 # alive after we are done with them.
2021 objs = [CountedObject() for i in range(10)]
2022 refs = [weakref.ref(o) for o in objs]
2023 self.pool.map(identity, objs)
2024
2025 del objs
2026 self.assertEqual(set(wr() for wr in refs), {None})
2027 # With a process pool, copies of the objects are returned, check
2028 # they were released too.
2029 self.assertEqual(CountedObject.n_instances, 0)
2030
2003 2031
2004 2032 def raising(): def raising():
2005 2033 raise KeyError("key") raise KeyError("key")
File Misc/NEWS changed (mode: 100644) (index 8003facf12..bd43227804)
... ... Extension Modules
287 287 Library Library
288 288 ------- -------
289 289
290 - bpo-29861: Release references to tasks, their arguments and their results
291 as soon as they are finished in multiprocessing.Pool.
292
290 293 - bpo-19930: The mode argument of os.makedirs() no longer affects the file - bpo-19930: The mode argument of os.makedirs() no longer affects the file
291 294 permission bits of newly-created intermediate-level directories. permission bits of newly-created intermediate-level directories.
292 295
Hints:
Before first commit, do not forget to setup your git environment:
git config --global user.name "your_name_here"
git config --global user.email "your@email_here"

Clone this repository using HTTP(S):
git clone https://rocketgit.com/user/benf_wspdigital/cpython

Clone this repository using ssh (do not forget to upload a key first):
git clone ssh://rocketgit@ssh.rocketgit.com/user/benf_wspdigital/cpython

Clone this repository using git:
git clone git://git.rocketgit.com/user/benf_wspdigital/cpython

You are allowed to anonymously push to this repository.
This means that your pushed commits will automatically be transformed into a merge request:
... clone the repository ...
... make some changes and some commits ...
git push origin main