Запуск фоновых задач через dramatiq
@dramatiq.actor(store_results=True)
def count_words(url):
try:
response = requests.get(url)
count = len(response.text.split(" "))
print(f"There are {count} words at {url!r}.")
except requests.exceptions.MissingSchema:
print(f"Message dropped due to invalid url: {url!r}")
result_backend = RedisBackend(host="172.17.0.2", port=6379)
result_broker = RabbitmqBroker(host="172.17.0.5", port=5672)
result_broker.add_middleware(Results(backend=result_backend))
dramatiq.set_broker(result_broker)
message = count_words.send('https://github.com/Bogdanp/dramatiq')
print(message.get_result(block=True))
1:C 22 Dec 2020 13:38:15.415 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 22 Dec 2020 13:38:15.415 # Redis version=6.0.9, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 22 Dec 2020 13:38:15.415 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
1:M 22 Dec 2020 13:38:15.417 * Running mode=standalone, port=6379.
1:M 22 Dec 2020 13:38:15.417 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 22 Dec 2020 13:38:15.417 # Server initialized
1:M 22 Dec 2020 13:38:15.417 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:M 22 Dec 2020 13:38:15.417 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo madvise > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled (set to 'madvise' or 'never').
1:M 22 Dec 2020 13:38:15.417 * Loading RDB produced by version 6.0.9
1:M 22 Dec 2020 13:38:15.417 * RDB age 1 seconds
1:M 22 Dec 2020 13:38:15.417 * RDB memory usage when created 0.80 Mb
1:M 22 Dec 2020 13:38:15.418 * DB loaded from disk: 0.000 seconds
1:M 22 Dec 2020 13:38:15.418 * Ready to accept connections
1:M 22 Dec 2020 14:38:16.082 * 1 changes in 3600 seconds. Saving...
1:M 22 Dec 2020 14:38:16.097 * Background saving started by pid 18
18:C 22 Dec 2020 14:38:16.107 * DB saved on disk
18:C 22 Dec 2020 14:38:16.108 * RDB: 4 MB of memory used by copy-on-write
1:M 22 Dec 2020 14:38:16.198 * Background saving terminated with success
1:M 25 Dec 2020 05:22:52.645 * 1 changes in 3600 seconds. Saving...
1:M 25 Dec 2020 05:22:52.808 * Background saving started by pid 19
19:C 25 Dec 2020 05:22:52.947 * DB saved on disk
19:C 25 Dec 2020 05:22:53.002 * RDB: 4 MB of memory used by copy-on-write
1:M 25 Dec 2020 05:22:53.020 * Background saving terminated with success
1:M 25 Dec 2020 06:22:54.160 * 1 changes in 3600 seconds. Saving...
1:M 25 Dec 2020 06:22:54.509 * Background saving started by pid 20
20:C 25 Dec 2020 06:22:54.629 * DB saved on disk
20:C 25 Dec 2020 06:22:54.644 * RDB: 4 MB of memory used by copy-on-write
1:M 25 Dec 2020 06:22:54.713 * Background saving terminated with success
1:M 25 Dec 2020 07:22:55.010 * 1 changes in 3600 seconds. Saving...
1:M 25 Dec 2020 07:22:55.110 * Background saving started by pid 21
21:C 25 Dec 2020 07:22:55.117 * DB saved on disk
21:C 25 Dec 2020 07:22:55.118 * RDB: 4 MB of memory used by copy-on-write
1:M 25 Dec 2020 07:22:55.210 * Background saving terminated with success
1:M 25 Dec 2020 08:22:56.082 * 1 changes in 3600 seconds. Saving...
1:M 25 Dec 2020 08:22:56.367 * Background saving started by pid 22
22:C 25 Dec 2020 08:22:56.771 * DB saved on disk
22:C 25 Dec 2020 08:22:56.857 * RDB: 2 MB of memory used by copy-on-write
1:M 25 Dec 2020 08:22:56.949 * Background saving terminated with success
1:M 25 Dec 2020 10:08:11.678 * 1 changes in 3600 seconds. Saving...
1:M 25 Dec 2020 10:08:11.973 * Background saving started by pid 23
23:C 25 Dec 2020 10:08:12.196 * DB saved on disk
23:C 25 Dec 2020 10:08:12.217 * RDB: 4 MB of memory used by copy-on-write
1:M 25 Dec 2020 10:08:12.274 * Background saving terminated with success
1:M 26 Dec 2020 19:23:59.445 * 1 changes in 3600 seconds. Saving...
1:M 26 Dec 2020 19:23:59.660 * Background saving started by pid 24
24:C 26 Dec 2020 19:23:59.890 * DB saved on disk
24:C 26 Dec 2020 19:23:59.905 * RDB: 4 MB of memory used by copy-on-write
1:M 26 Dec 2020 19:23:59.961 * Background saving terminated with success
Ошибка:
Traceback (most recent call last):
File "/opt/.pycharm_helpers/pydev/pydevd.py", line 1448, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "/opt/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/opt/var/jira-reports/run.py", line 44, in <module>
print(message.get_result(block=True))
File "/usr/local/lib/python3.6/dist-packages/dramatiq/message.py", line 147, in get_result
return backend.get_result(self, block=block, timeout=timeout)
File "/usr/local/lib/python3.6/dist-packages/dramatiq/results/backends/redis.py", line 81, in get_result
raise ResultTimeout(message)
dramatiq.results.errors.ResultTimeout: count_words('https://github.com/Bogdanp/dramatiq')
