Запуск фоновых задач через dramatiq


    @dramatiq.actor(store_results=True)
    def count_words(url):
        try:
            response = requests.get(url)
            count = len(response.text.split(" "))
            print(f"There are {count} words at {url!r}.")
        except requests.exceptions.MissingSchema:
            print(f"Message dropped due to invalid url: {url!r}")
    
    result_backend = RedisBackend(host="172.17.0.2", port=6379)
    result_broker = RabbitmqBroker(host="172.17.0.5", port=5672)
    
    result_broker.add_middleware(Results(backend=result_backend))
    
    dramatiq.set_broker(result_broker)
    message = count_words.send('https://github.com/Bogdanp/dramatiq')
    print(message.get_result(block=True))

RabbitMQ: введите сюда описание изображения


    1:C 22 Dec 2020 13:38:15.415 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
    1:C 22 Dec 2020 13:38:15.415 # Redis version=6.0.9, bits=64, commit=00000000, modified=0, pid=1, just started
    1:C 22 Dec 2020 13:38:15.415 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
    1:M 22 Dec 2020 13:38:15.417 * Running mode=standalone, port=6379.
    1:M 22 Dec 2020 13:38:15.417 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
    1:M 22 Dec 2020 13:38:15.417 # Server initialized
    1:M 22 Dec 2020 13:38:15.417 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
    1:M 22 Dec 2020 13:38:15.417 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo madvise > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled (set to 'madvise' or 'never').
    1:M 22 Dec 2020 13:38:15.417 * Loading RDB produced by version 6.0.9
    1:M 22 Dec 2020 13:38:15.417 * RDB age 1 seconds
    1:M 22 Dec 2020 13:38:15.417 * RDB memory usage when created 0.80 Mb
    1:M 22 Dec 2020 13:38:15.418 * DB loaded from disk: 0.000 seconds
    1:M 22 Dec 2020 13:38:15.418 * Ready to accept connections
    1:M 22 Dec 2020 14:38:16.082 * 1 changes in 3600 seconds. Saving...
    1:M 22 Dec 2020 14:38:16.097 * Background saving started by pid 18
    18:C 22 Dec 2020 14:38:16.107 * DB saved on disk
    18:C 22 Dec 2020 14:38:16.108 * RDB: 4 MB of memory used by copy-on-write
    1:M 22 Dec 2020 14:38:16.198 * Background saving terminated with success
    1:M 25 Dec 2020 05:22:52.645 * 1 changes in 3600 seconds. Saving...
    1:M 25 Dec 2020 05:22:52.808 * Background saving started by pid 19
    19:C 25 Dec 2020 05:22:52.947 * DB saved on disk
    19:C 25 Dec 2020 05:22:53.002 * RDB: 4 MB of memory used by copy-on-write
    1:M 25 Dec 2020 05:22:53.020 * Background saving terminated with success
    1:M 25 Dec 2020 06:22:54.160 * 1 changes in 3600 seconds. Saving...
    1:M 25 Dec 2020 06:22:54.509 * Background saving started by pid 20
    20:C 25 Dec 2020 06:22:54.629 * DB saved on disk
    20:C 25 Dec 2020 06:22:54.644 * RDB: 4 MB of memory used by copy-on-write
    1:M 25 Dec 2020 06:22:54.713 * Background saving terminated with success
    1:M 25 Dec 2020 07:22:55.010 * 1 changes in 3600 seconds. Saving...
    1:M 25 Dec 2020 07:22:55.110 * Background saving started by pid 21
    21:C 25 Dec 2020 07:22:55.117 * DB saved on disk
    21:C 25 Dec 2020 07:22:55.118 * RDB: 4 MB of memory used by copy-on-write
    1:M 25 Dec 2020 07:22:55.210 * Background saving terminated with success
    1:M 25 Dec 2020 08:22:56.082 * 1 changes in 3600 seconds. Saving...
    1:M 25 Dec 2020 08:22:56.367 * Background saving started by pid 22
    22:C 25 Dec 2020 08:22:56.771 * DB saved on disk
    22:C 25 Dec 2020 08:22:56.857 * RDB: 2 MB of memory used by copy-on-write
    1:M 25 Dec 2020 08:22:56.949 * Background saving terminated with success
    1:M 25 Dec 2020 10:08:11.678 * 1 changes in 3600 seconds. Saving...
    1:M 25 Dec 2020 10:08:11.973 * Background saving started by pid 23
    23:C 25 Dec 2020 10:08:12.196 * DB saved on disk
    23:C 25 Dec 2020 10:08:12.217 * RDB: 4 MB of memory used by copy-on-write
    1:M 25 Dec 2020 10:08:12.274 * Background saving terminated with success
    1:M 26 Dec 2020 19:23:59.445 * 1 changes in 3600 seconds. Saving...
    1:M 26 Dec 2020 19:23:59.660 * Background saving started by pid 24
    24:C 26 Dec 2020 19:23:59.890 * DB saved on disk
    24:C 26 Dec 2020 19:23:59.905 * RDB: 4 MB of memory used by copy-on-write
    1:M 26 Dec 2020 19:23:59.961 * Background saving terminated with success

Ошибка:


    Traceback (most recent call last):
      File "/opt/.pycharm_helpers/pydev/pydevd.py", line 1448, in _exec
        pydev_imports.execfile(file, globals, locals)  # execute the script
      File "/opt/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
        exec(compile(contents+"\n", file, 'exec'), glob, loc)
      File "/opt/var/jira-reports/run.py", line 44, in <module>
        print(message.get_result(block=True))
      File "/usr/local/lib/python3.6/dist-packages/dramatiq/message.py", line 147, in get_result
        return backend.get_result(self, block=block, timeout=timeout)
      File "/usr/local/lib/python3.6/dist-packages/dramatiq/results/backends/redis.py", line 81, in get_result
        raise ResultTimeout(message)
    dramatiq.results.errors.ResultTimeout: count_words('https://github.com/Bogdanp/dramatiq')


Ответы (0 шт):